Why, Palo Alto, WHY??

Yesterday, I was doing up some SaltStack based automation to help a customer automate the installation of the Cortex XDR Agent on RHEL-based Linux hosts. The vendor delivers the agent in the form of a ZIP-archived RPM. Yeah, I was a bit unimpressed by them deciding an RPM needed to be encapsulated in a ZIP-archive.

When you read the installation documentation, there's a link in the page they tell you to download. Yesterday, the embedded link was:

https://docs-cortex.paloaltonetworks.com/v/u/cortex-xdr-agent.zip

This URL was actually set up as an HTTP 302 (redirect) to:

https://docs-cortex.paloaltonetworks.com/api/khub/documents/Im1wc74y4HN15mXxBu3nYQ/content?Ft-Calling-App=ft%2Fturnkey-portal&Ft-Calling-App-Version=5.2.49&download=true&locationValue=viewer

SaltStack didn't care for trying to use file.managed to try to download from a redirect. I had to whip up some logic to chase the redirect and stuff it into a (Jinja) variable. Worked well once I got it in place.

Today, I was attempting to continue with the refactoring that I'd started, yesterday. This means launching an EC2 with the new automtion. I was surprised to find that the automation — unaltered since yesterday's day-ending push — was failing. When I checked the logs, SaltStack was complaining that I was trying to pass a null value to file.managed's source parameter. Perplexed, I started troubleshooting.

Ultimately, what I found was that the URL I was doing redirection-chasing on was no longer redirecting. Using curl like:

curl -Ls -o /dev/null 
  -w %{url_effective} "https://docs-cortex.paloaltonetworks.com/v/u/cortex-xdr-agent.zip"

Was returning null. So, I opted to try to make it provide more-definitive output like:

curl -Ls -o /dev/null \
  -w "Status: %{http_code}\nEffective URL: %{url_effective}\n" \
  "https://docs-cortex.paloaltonetworks.com/v/u/cortex-xdr-agent.zip"

This returned:

Status: 200
Effective URL: https://docs-cortex.paloaltonetworks.com/v/u/cortex-xdr-agent.zip

Which is to say, the reference URL is no longer returning an HTTP 302. Thus receiving a null-value when looking for a "url_effective" value. 

So, I revisited the documentation. I scrolled down to the where the document called out the signature-file's link. When I hovered over the link, it's value had changed since yesterday. I guess they pushed out a documentation-portal change and, along with that, nuked the previous URL's redirect action. This meant that the URL I was expecting was now returning a data-stream. Looking at the data-stream's first and last five lines:

curl -Ls https://docs-cortex.paloaltonetworks.com/v/u/cortex-xdr-agent.zip | \
sed -n '1,5p; :a;$p;N;11,$D;ba'

It was obvious from the output that the HTTP data-stream was now sending me JavaScript, presumably substituting the prior HTTP redirect with JS-based navigation-aids. Unfortunately, those kinds of navigation-aids work well enough for graphical browsers but not at all well for curl-type methods. 

None of this would have been necessary if they just maintained a file-repository of the RPMs and their signing-keys. Or, since they were ZIPing up the RPMs, include the damned signing-key in the archive-file. But, no, that would be too fucking easy and way too sensible.

I wish I could say this was surprising. However, I've had to integrate enough tooling — particularly security tooling — that I've gotten sort of used to (especially security) vendors  doing things in absolutely baffling ways. At least this vendor wasn't doing things in ways that required reducing system-security to allow installation of their tool: that is a hallmark of the brain-damage I frequently witness with security vendors' tooling.