Crib-Notes: Manifest Deltas

Each month, the group I work for publishes new CentOS and Red Hat AMIs (and Azure templates and Vagrant boxes). When we complete the publication-event, we post a news announcement to our user-portal so that subscribers can receive an alert of the new publication. Included in that news announcement is a "what's changed" section.

In prior months, figuring out "what changed" was left as a manual step for the team-member charged with running the automation for a given month's publication event. This month, no one generated that news article and there were several updated and new RPMs included in the new image. So, I set about figuring out "how to extract this information programmatically so as to more-easily suss-out what to include in the announcement posting." The following does so (though, presumably, in a not-particularly-optimized) fashion:

git diff $(
      git log --pretty='%H' --follow -- <PATH_TO_MANIFEST_FILE> | \
      head -2 | \
      tac | \
      sed 'N;s/\n/../'
   ) -- <PATH_TO_MANIFEST_FILE> | \
grep -E '(amazon|aws|ec2)-' | \
sed 's/^./& /' | \
sort -k 2

To explain:

1. Use `git log` to output the commit-hashes for all the commits for the target file (in this case, the project's manifest-file)
2. Use `head -2` to grab only the two most-recent commit hashes from the output-stream
3. Use the `tac` command to invert the order of the two lines returned from the `head` command
4. Use the `sed` command to join the two lines, replacing the first line's line-ending newline character with ".."
5. Use `git diff` against the output created in steps 1-4, and constrain the diff-activity to just the manifest-file.
6. Pipe that output through `grep` to suppress all information other than the bits containing the `amazon-`, `aws-` and `ec2-` substrings.
7. Pipe that through `sed` so that the +/- that `git diff` uses to show new and removed files, respectively, becomes an easily-tokenized substring.
8. Sort the remaining output-stream (with `sort`) so that the lines are groups by manifest-element (the second key/token in the sorted output)

Taking that output and converting to a news article is still manual, but it at least makes it a lot easier to do than either hand-diffing two files or having to "just know" what's changed.

Notes

Because Red Hat has placed EL6 is in its final stage of de-support, we've stopped publishing CentOS6 and RHEL6. We did this to discourage our subscribers from doing new deployments on EL6 (since the underlying platform will go into final de-support come November of this year).

Similarly, due to current lack of CentOS offering for EL8, lack of security-related build- or hardening-guidance for EL8 and associated lack of subscriber-demand for an EL8 build, we don't yet include builds for CentOS8 or RHEL8 in our process. Thus, for the time being, we only need to provide a "whats changed" for EL7 builds. Given this, we currently only need to do change-queries against the "manifests/spel-minimal-centos-7-hvm.manifest.txt" file.