In a Funny Twist, IT Problems Hobble Everybody But Southwest

The airline industry has been given another harsh lesson in IT reliance after thousands of flights were canceled and delayed amid the downstream effects of a bungled software update.

An IT issue caused at least 2,500 flight cancellations across the USA, while delaying north of 8,000 more, all thanks to a botched software update by cybersecurity firm CrowdStrike. Locked-up computers around the country confounded flight ops personnel as they struggled to keep up the routine, but without a solid backbone of scheduling, airlines quickly admitted defeat against the tide, preferring to hand out travel waivers for affected passengers. It wasn't just the customer-facing systems that got hit, either, flight planning, internal corporate communications, and even the weight & balance systems crews depend on started their Friday morning completely seized.

Some carriers managed to avoid the issue, though, thanks to some unspecified luck or acute IT attention. It's a fun stroke of fate if that for once, Southwest's continued use of their archaic computer system actually helped them avoid a scheduling meltdown for a change. (At least, that's the water cooler theory for those who recall mentions of their in-house Crew Web Access or SkySolver software, software put into service in the early nineties.) Either way, Southwest Airlines managed to come out of the fracas smelling like roses, a refreshing change of pace for the brand after years of high-profile scheduling fiascoes. American, Delta, and United all saw delays and cancellations for a brief time, as well as the big cargo brands UPS and FedEx.

Software updates are the industry's favorite crutch these days, used to preempt growing digital threats as they arise - or so the theory goes. Salty IT folks in the trenches tend to have a less rosy view of custom, believing that modern software updates are just as often used to correct sloppy work and even force users to restart their applications once in a while. In this case, CrowdStrike, a 3rd party cybersecurity operator, pushed an update to Windows machines using its Falcon Sensor that inadvertently caused an issue with a core system32 file, preventing affected machines from booting until they got some personal attention from support staff. CrowdStrike systems ultimately caused a whole host of issues in just about every internet-connected industry in the country, from TV stations going off air to hospitals entering a holding pattern until they could chart appointments electronically again. In the end, CrowdStrike rolled back the change, which should prevent further failures, but some machines may remain boot-looped and blue-screened until they get some hands-on attention to reboot in safe mode and delete the System32 file preventing startup.

While it's not the biggest fiasco the industry's ever seen, it's an operational hiccup that should teach some lessons about how interdependent operations are on software systems entirely out of their control. Will anyone apply lessons learned? Probably not, but...well, probably not is all.

FMI: www.crowdstrike.com