alpine-ec2-ami merge requestshttps://gitlab.alpinelinux.org/alpine/cloud/alpine-ec2-ami/-/merge_requests2020-09-03T22:04:31Zhttps://gitlab.alpinelinux.org/alpine/cloud/alpine-ec2-ami/-/merge_requests/84udhcpc hooks for ENI IPv6 & secondary IPv42020-09-03T22:04:31ZJake Buchholz Göktürkudhcpc hooks for ENI IPv6 & secondary IPv4Automatically sets up any IPv6 and secondary IPv4 on instance ENIs when DHCP leases are bound or renewed on that interface.
Resolves #70
Freshly instantiated instance...
```
~ % ssh -i AWS/tomalok.pem alpine@2600:1f13:224:d501:c9...Automatically sets up any IPv6 and secondary IPv4 on instance ENIs when DHCP leases are bound or renewed on that interface.
Resolves #70
Freshly instantiated instance...
```
~ % ssh -i AWS/tomalok.pem alpine@2600:1f13:224:d501:c953:a14a:7b8:3909
Warning: Permanently added '2600:1f13:224:d501:c953:a14a:7b8:3909' (ECDSA) to the list of known hosts.
Welcome to Alpine!
The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <http://wiki.alpinelinux.org/>.
You can setup the system with the command: setup-alpine
You may change this message by editing /etc/motd.
alpine@ip-172-30-30-85:~$ cat /var/log/messages
Aug 30 02:00:58 localhost syslog.info syslogd started: BusyBox v1.32.0
Aug 30 02:00:58 localhost daemon.info init: starting pid 2346, tty '': '/sbin/openrc default'
Aug 30 02:00:59 localhost daemon.notice udhcpc/post-bound[2406]: eth0 add 2600:1f13:224:d501:c953:a14a:7b8:3909 - success
Aug 30 02:01:00 localhost daemon.info chronyd[2517]: chronyd version 3.5.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP -SCFILTER +SIGND +ASYNCDNS -SECHASH +IPV6 -DEBUG)
Aug 30 02:01:00 localhost daemon.warn chronyd[2517]: Could not read valid frequency and skew from driftfile /var/lib/chrony/chrony.drift
Aug 30 02:01:05 localhost daemon.info chronyd[2517]: System's initial offset : 0.428769 seconds slow of true (slew)
Aug 30 02:01:07 localhost auth.info sshd[2551]: Server listening on 0.0.0.0 port 22.
Aug 30 02:01:07 localhost auth.info sshd[2551]: Server listening on :: port 22.
Aug 30 02:01:07 localhost user.info : password for 'root' changed
Aug 30 02:01:07 localhost user.info : password for 'alpine' changed
Aug 30 02:01:08 localhost daemon.info init: starting pid 2632, tty '/dev/ttyS0': '/sbin/getty -L ttyS0 115200 vt100'
Aug 30 02:01:11 localhost daemon.info chronyd[2517]: Selected source 169.254.169.123
Aug 30 02:03:32 localhost auth.info sshd[2638]: Accepted publickey for alpine from 2601:602:8f80:75b9:a567:6cb7:c10e:41bb port 57388 ssh2: RSA SHA256:NXpuX150N2HDjZ8bBv9UnjC1gi52KLrvtSyLiHqW8FA
alpine@ip-172-30-30-85:~$ ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 02:5e:2d:89:26:ed brd ff:ff:ff:ff:ff:ff
inet 172.30.30.85/20 brd 172.30.31.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 2600:1f13:224:d501:c953:a14a:7b8:3909/128 scope global
valid_lft forever preferred_lft forever
inet6 fe80::5e:2dff:fe89:26ed/64 scope link
valid_lft forever preferred_lft forever
```Mike Crutemike@crute.usMike Crutemike@crute.ushttps://gitlab.alpinelinux.org/alpine/cloud/alpine-ec2-ami/-/merge_requests/83New Release Tool2020-12-12T02:02:15ZMike Crutemike@crute.usNew Release ToolHere's the new release tool. It uses the identity broker to acquire credentials for all activated regions and copy the AMI, including tags, to that region. For future accounting it also adds a `source_ami` tag. Everything should also wor...Here's the new release tool. It uses the identity broker to acquire credentials for all activated regions and copy the AMI, including tags, to that region. For future accounting it also adds a `source_ami` tag. Everything should also work if the user isn't using the identity broker, so long as AWS credentials are accessible by the SDK and `--region` is passed (one or more times) to specify target regions. The tool will try not to copy the AMI more than once to a region, using the `source_ami` tag, so if it's run multiple times for the same source AMI it will copy to new regions and fix permissions if needed.
The flip to public permissions is designed to happen all at once, post-copy, in the linear flow so that a release looks like it happens approximate at the same time.
I've tested the identity broker path as well as the permissions fix path but have not tested the standalone path since that isn't a use-case I have and I don't have a test account handy right now; any testing there would be appreciated. I also haven't tested sharing with separate accounts `--allow-accounts` instead of `--public`, but that should work as well.
To use the identity broker, grab the API key from the broker homepage and export it as the environment variable `IDENTITY_BROKER_API_KEY`. Everything else should just work from there. The token is valid for 6 hours. Note that there are pretty aggressive rate limits on the broker for getting credentials so if you're doing a lot of testing in a row you'll end up waiting for the timeouts, but the script should handle it gracefully.
Any feedback would be appreciated. This should unblock the 3.12 release. There's more stuff coming with tools to prune AMIs and build the catalog from the tag metadata instead of YAML files; but I'll follow up with those a little later.Jake Buchholz GöktürkJake Buchholz Göktürkhttps://gitlab.alpinelinux.org/alpine/cloud/alpine-ec2-ami/-/merge_requests/66Tooling updates2020-05-30T22:10:09ZMike Crutemike@crute.usTooling updatesThis is kind of a big one.
This change combines all of the various python scripts into one builder script and uses argparse subcommands to handle dispatch to the other commands. It also removes a lot of hand-rolled command line manipu...This is kind of a big one.
This change combines all of the various python scripts into one builder script and uses argparse subcommands to handle dispatch to the other commands. It also removes a lot of hand-rolled command line manipulation in favor of using argparse. It also tries to be resilient to path locations but always finding the git root before operating on paths. The rationale behind the major refactor is that we need to share logic between a bunch of the scripts now and that's easier to do if they're all in one file. That also seemed easier than making an importable python library for less than 700 lines of code.
The `make-amis` script was converted from shell to python. In the process identity broker integration was added so authentication happens in the region the build is happening, which is passed through to packer. This should eliminate needing to export credentials from the identity broker directly. Building will still require exporting `IDENTITY_BROKER_API_KEY` (which is available on the identity broker home page). In the future I hope to be able to use GitHub actions to do the builds and releases so we don't have to run them by hand, this will be a lot easier with the new python implementation.
There's a new identity broker client that can handle multi-regional credentials and provides a list of all regions, including the ability to get a credential for regions that are opted-in for an account. Any region iteration logic should use the broker going forwards instead of the EC2 `describe_regions` function from boto3 (the broker uses similar logic behind the scenes but with more checks).
The runtime services (`svcs`) was remodelled to use a nested map in the profiles. I think this is a little easier to read through at a glance and it greatly simplified the transform of that structure into a flat list.
The `resolve-profiles` script was very heavily modified. The goal was to make the logic as declarative as possible and eliminate some of the unused code there. I think that was accomplished and it made adding some new key transforms pretty easy.
Additionally I removed all region manipulation logic. There will be a PR forthcoming after the ARM release that adds a release tool that handles AMI replication and permission updates to make them public. This will also handle the opt-in regions case. I'll link that work to #53
The rest of the scripts remain largely unchanged, just shuffled around to fit into classes.
The final major change I hope to make to this tooling is an update to the `prune-amis` subcommand to resolve #23 but we can talk about that over there.Mike Crutemike@crute.usMike Crutemike@crute.us