Crib-Notes: Validating Consistent ENA and SRIOV Support in AMIs

One of the contracts I work on, we're responsible for producing the AMIs used for the entire enterprise. At this point, the process is heavily automated. Basically, we use a pipeline that leverages some CI tools, Packer and a suite of BASH scripts to do all the grunt-work and produce not only an AMI but artifacts like AMI configuration- and package-manifests.

When we first adopted Packer, it had some limitations on how it registered AMIs (or, maybe, we just didn't find the extra flags back when we first selected Packer – who knows, it's lost to the mists of time at this point). If you wanted the resultant AMIs to have ENA and/or SRIOV support baked in (we do), your upstream AMI needed to have it baked in as well. This necessitated creating our own "bootstrap" AMIs as you couldn't count on these features being baked in – not even within the upstream vendor's (in our case, Red Hat's and CentOS's) AMIs.

At any rate, because the overall process has been turned over from the people that originated the automation to people that basically babysit automated tasks, the people running the tools don't necessarily have a firm grasp of everything that the automation's doing. Further, the people that are tasked with babysitting the automation differe from run-to-run. While automation should see to it that this doesn't matter, sometimes it pays to be paranoid. So a quick way to assuage that paranoia is to run quick reports from the AWS CLI. The following snippet makes for an adequate, "fifty-thousand foot" consistency-check:

aws ec2 describe-images --owner <AWS_ACCOUNT_ID> --query \
      'Images[].[ImageId,Name,EnaSupport,SriovNetSupport]' \
      --filters 'Name=name,Values=<SEARCH_STRING_PATTERN>' \
      --out text | \
   aws 'BEGIN {
         printf("%-18s%-60s%-14s-10s\n","AMI ID","AMI Name","ENA Support","SRIOV Support")
      } {
         printf("%-18s%-60s%-14s-10s\n",$1,$2,$3,$4)
      }'

• There's a lot of organizations and individuals publishing AMIs. Thus, we use the --owner flag to search only for AMIs we've published.
• We produce a couple of different families of AMIs. Thus, we use the --filter statement to only show the subset of our AMIs we're interested in.
• I really only care about four attributes of the AMIs being reported on: ImageId, Name, EnaSupport and SriovNetSupport. Thus, the use of the JMSE --query statement to suppress all output except for that in which I'm interested.
• Since I want the output to be pretty, I used the compound awk statement to create a formatted header and apply the same formatting to the output from the AWS CLI (using but a tiny bit of the printf routine's many capabilities).

This will produce output similar to:

   AMI ID                 AMI Name                                        ENA Support  SRIOV Support
   ami-187af850f113c24e1  spel-minimal-centos-7-hvm-2019.03.1.x86_64-gp2  True         simple
   ami-91b38c446d188643e  spel-minimal-centos-7-hvm-2019.02.1.x86_64-gp2  True         simple
   ami-22867cf08bb264ac4  spel-minimal-centos-7-hvm-2019.01.1.x86_64-gp2  True         simple
   [...elided...]
   ami-71c3822ed119c3401  spel-minimal-centos-7-hvm-2018.03.1.x86_64-gp2  None         simple
   [...elided...]
   ami-8057c2bf443dc01f5  spel-minimal-centos-7-hvm-2016.06.1.x86_64-gp2  None         None

As you can see, not all of the above AMIs are externally alike. While this could indicate a process or personnel problem, what my output actually shows is evolution in our AMIs. Originally, we weren't doing anything to support SRIOV or ENA. Then we added SRIOV support (because our AMI users were finally asking for it). Finally, we added ENA support (mostly so we could use the full range and capabilities of the fifth-generation EC2 instance-types).

At any rate, running a report like the above, we can identfy if there's unexpected differences and, if a sub-standard AMI slips out, we can alert our AMI users "don't use <AMI> if you have need of ENA and/or SRIOV".