Why I Default to The Old Ways

I work with a growing team of automation engineers. Most are purely dev types. Those that have lived in the Operations world, at all, skew heavily towards Windows or only had to very lightly deal with UNIX or Linux.

I, on the other hand, have been using UNIX flavors since 1989. My first Linux system was the result of downloading a distribution from the MIT mirrors in 1992. Result, I have a lot of old habits (seriously: some of my habits are older than some of my teammates). And, because I've had to get deep into the weeds with all of those operating systems many, many, many times, over the years, those habits are pretty entrenched ("learned with blood" and all that rot).

A year or so ago, I'd submitted a PR that included some regex-heavy shell scripts. The person that reviewed the PR had asked "why are you using '[<space><TAB>]*' in your regexes rather than just '\s'?". At the time, I think my response was a semi-glib, "A) old habits die hard; and, B) I know that the former method always works".

That said, I am a lazy-typist. Typing "\s" is a lot fewer keystrokes than is "[<space><TAB>]*". Similarly, "\s" takes up a lot less in the way of column-width than does "[<space><TAB>]*" (and I/we generally like to code to fairly standard page-widths). So, for both laziness reasons and column-conservation reasons, I started to move more towards using "\s" and away from using "[<space><TAB>]*".  I think in the last 12 months, I've moved almost exclusively to  "\s".

Today, that move bit me in the ass. Well, yesterday, actually, because that's when I started receiving reports that the tool I'd authored on EL7 wasn't working when installed/used on EL6. Ultimately, I traced the problem to an `awk` invocation. Specifically, I had a chunk of code (filtering DNS output) that looked like:

awk '/\sIN SRV\s/{ printf("%s;%s\n",$7,$8)}'

Which worked a treat on EL7 but on EL6, "not so much." When I altered it to the older-style invocation:

awk '/[  ]*IN[  ]*SRV[  ]*/{ printf("%s;%s\n",$7,$8)}'

It worked fine on both EL7 and EL6. Turns out the ancient version of `awk` (3.1.7) on EL6 didn't know how to properly interpret the "\s" token. Oddly (my recollection from writing other tooling) is that EL6's version of `grep` understands the "\s" token just fine.

When I Slacked the person I'd had the original conversation with a link to the PR with a "see: this is why" note, he replied, "oh: I never really used awk, so never ran into it".

Commercial UNIX Background vs Linux DIYism

I've been managing UNIX systems for over 18 years - 14 of them on a professional basis. In that time I've had the pleasure to support systems running:

• Solaris
• IRIX
• AIX
• HP/UX
• RedHat Linux
• SuSE Linux
• SlackWare Linux
• Coherent
• Xenix
• DG/UX
• TRU-64
• OSF
• UNICOS
• NeXTSTEP
• Several different flavors of BSD

Things have changed a lot in that 14-18 years of UNIX. Especially in recent years, where most of my work was on Solaris and AIX, I got kind of used to things being (relatively) easy. The enterprise-grade commercial UNIX solutions included lots of tools to facilitate working with complex storage and networking environments. Over the last year-and-change, I've had to switch gears and go to the Linux world (specifically, RedHat).

Now, the I've made use of Linux, off-and-on, since 1992. While Linux has improved greatly in that time, it hasn't improved uniformly or in the same directions that enterprise-grade, commercial UNIX systems have. Specifically, a lot of the tools that are just sort of a given in commercial UNIX operating systems aren't immediately at hand in Linux.

Where, on a commercial UNIX operating system, you might have a tool or a suite of tools to accomplish a given task, in Linux you frequently only have the capability to do those tasks but not bundled tools. Frequently, especially when working with complex storage or networking tasks, you have to write your own tools leveraging all of the lower-level hooks that the Linux developers have made available. You end up writing a lot of scripts to interact with structures in the /proc and /sys hierarchies.

Unfortunately, it seems like so many of the online Linux resources are dedicated towards the Linux hobbyist. When you're trying to support Linux in a production environment, you frequently end up trying to re-engineer methods and tools are "standard equipment" on commercial UNIX releases. Because so many of the forums and other resources are geared towards hobbyists and neophytes, the questions you have aren't frequently broached on the forums (or, if they are, they aren't answered). So, there's a lot of mucking about to do.

Fortunately, while a lot of the tools you might be used to on commercial UNIXes aren't in Linux, the Linux maintainers have at least provided the building blocks for recreating those tools. I just wish there were more and better resources documenting those building-blocks. It would make my job a lot easier. Re-engineering a tool is hard enough - having to do research to find the appropriate building-blocks is just a real time-suck.

And, yes, having experience with all of those systems over all of those years, I realize that life as a UNIX SA/engineer/etc. has always been part DIY. It's just that, you get used to only having to know a given OSes toolset rather than having to write things on your own. You take for granted the advances that the mature, commercial UNIX OSes have habituated you to and are simply absent in Linux.

Oh well, live and learn, I suppose. (cue The Timewarp music).