Lipstick on pigs

"I first said this some years ago but was forcefully reminded of it this week: Any sufficiently advanced negligence is indistinguishable from malice." – Deb Chachra

I have a running joke that Microsoft Teams and Microsoft 365 are lipstick on the SharePoint pig. It always gets a laugh from a room full of IT people, because they know it is only half a joke.

Behind the shiny interfaces, the Copilot buttons and the keynote demos, many of the big software-as-a-service operations sit on systems that are decades old. Those systems have accreted layer upon layer over the years. Acquisitions get bolted on, migrations are left half finished, workarounds turn into features, and the people who wrote the original code left long ago. Every year the stack gets taller, and fewer people understand the whole thing.

That is a recipe for fragility. And I think we are starting to see it.

A word on this week

I want to be clear up front. I am not claiming to know what caused this week's Microsoft 365 outage. Microsoft has not yet published its post-incident report, and I have no inside knowledge. Anyone who tells you they know the root cause right now is guessing.

What we do know is what Microsoft has said publicly. The outage began on 12 November and stretched past 27 hours as Microsoft's fixes kept failing. Microsoft has said it was "a technical issue not related to cyber or AI/agents", introduced during an earlier network capacity upgrade and surfacing after a later network change. Along the way, its explanation shifted several times, which is normal when engineers are working through a complex failure in real time.

There has also been commentary from people with good industry sources. For example, Gergely Orosz of The Pragmatic Engineer reported that the fix involved moving parts of the legacy Exchange system onto Azure. Microsoft has not confirmed that, so I would treat it with caution.

So this post is not a diagnosis of one incident. It is about a pattern that this week's outage reminded me of, and that I have been thinking about for some time.

The pattern

When you read the published post-incident reports from the big providers, the same story keeps turning up. There is a latent defect that has sat quietly for months or years. Then an ordinary change arrives, a hidden dependency kicks in, and the failure cascades.

Microsoft's own review of the October 2025 Azure Front Door outage is a good example. Valid customer configuration changes made across two different control plane build versions produced incompatible metadata and exposed a latent bug. Microsoft says the defect escaped detection because not every feature was tested across versions. In other words, two versions of the same thing were running side by side and interacting in a way nobody had anticipated.

That is what accretion looks like in practice.

Salesforce has its pigs too

Salesforce is a useful second example, because so much of its estate came through acquisition.

• Heroku, June 2025. Salesforce's root cause analysis says an unsanctioned process ran an automatic operating system update in production, and when networking restarted the routes were not re-applied. One analysis notes that the networking service relied on a legacy script that only applied routing rules on first boot. An old assumption sat in a script for years, waiting for the wrong restart.
• Heroku, October 2025. Heroku's own write-up says the AWS-triggered disruption was compounded by Heroku's single-region control plane architecture and automation failures. Architecture choices from an earlier era turned someone else's bad day into theirs.
• Pardot, 2019. A database script deployment for Pardot, a marketing tool purchased in 2012, gave users read and write access to all of their company's data. Salesforce shut down more than 100 instances to contain it, and the restore script sometimes went backwards and regranted full permissions.
• 30 October 2026. Salesforce logged four incidents in a single day. The longest was a 9.5-hour file storage disruption caused by network permissions not being in place before traffic was moved onto a new proxy path.

None of these were attacks. In every case the system tripped over itself.

Oracle and the Cerner inheritance

When Oracle bought Cerner in 2022, it inherited an electronic health record platform whose Millennium architecture relied on on-premises databases first engineered in the late 1990s.

• In April 2025, Oracle Health engineers inadvertently deleted patient information during routine maintenance at a data centre, contributing to a five-day EHR outage at Community Health Systems hospitals.
• The 2025 Cerner breach was a cyber incident, so it sits a little outside my main argument. However, the weak point was legacy. Oracle's own notice referred to data on an old legacy server not yet migrated to the Oracle Cloud, and the tally now stands at nearly 20 million people. Old systems do not only fall over. They also sit there, unwatched, waiting to be found.
• Oracle Cloud Infrastructure's US East region had a roughly 20-hour network infrastructure incident in March 2026, and Oracle has not published a root cause.

The hyperscalers are not immune

It is tempting to assume the younger cloud platforms escape this. They do not, because they are now old enough to have accreted layers of their own.

• AWS, October 2025. The us-east-1 outage came from a latent race condition in the DynamoDB DNS management system that left an empty DNS record the automation could not repair. It cascaded into EC2 and the load balancers because so much of AWS's back end depends on DynamoDB.
• Google Cloud, June 2025. The global outage was a null pointer in code that had been deployed but never exercised until a policy change with blank fields arrived.
• Cloudflare, November 2025. A database permission change doubled the size of an automatically generated configuration file, crashing the software that handles traffic.

The Uptime Institute's 2025 analysis found outages from IT and networking issues rose to 23% of impactful outages, likely driven by increased IT and network complexity leading to change management problems and misconfigurations. It also warns that the software-based resilience tools we add to cope can introduce new risks and blur responsibility for failures.

Thus, complexity is now a primary cause of failure, and our usual answer to complexity is to add more of it.

Cutting the people who know where the bodies are buried

Legacy systems run on institutional knowledge as much as on code. The people who know why a particular script exists, or which service quietly depends on another, are the ones who keep the old plumbing working. Much of that knowledge is not written down anywhere.

That brings me to the layoffs. Again, I am not saying they caused this week's outage. Nevertheless, I find it hard to believe that shedding a very large number of experienced engineers has made these systems easier to run.

The numbers at Microsoft alone are significant:

• Microsoft cut about 15,000 jobs in 2025, and another 4,800 in July 2026.
• In its May 2025 round, software engineering was by far the largest single job category cut in Washington state, making up more than 40% of the roles there. The Seattle Times reported that later rounds included firmware engineers and cloud network engineers.
• Product research and development headcount has fallen to 77,000 from a peak of 81,000 in 2024.
• Last month Microsoft offered voluntary buyouts to about 60,000 employees, roughly a quarter of its workforce, as it spends heavily on AI.

Microsoft is hardly alone. Oracle said this year it would cut 20,000 to 30,000 employees, and Amazon announced roughly 30,000 corporate job cuts, with both citing the need to fund AI infrastructure. After the AWS outage last year, Corey Quinn argued in The Register that Amazon's brain drain had finally caught up with AWS.

Voluntary buyouts worry me in particular. The people most likely to accept them are often the long-serving engineers who are close to retirement or confident they can find work elsewhere. They are frequently the same people who carry the history of the system in their heads.

You can replace headcount. You cannot easily replace twenty years of knowing why the system is the way it is.

Negligence and malice

This is where Deb Chachra's line comes in. It is a riff on Arthur C. Clarke, and she develops the idea in her book How Infrastructure Works, where she writes: "Neglect the system badly enough or long enough or incautiously enough and it will be at risk of failure."

Nobody at these companies sets out to break their own platforms. But when you know the systems are old and complex, keep piling new features on top, and cut the people who understand the foundations, the outcome is foreseeable. From the customer's side, when the email is down for a day, the difference between negligence and malice does not matter very much.

There is a lot of anxiety right now about AI agents going rogue and nation-state attackers. Those risks are real, and I have written about them here. However, if I had to bet on the cause of the next big outage at a major SaaS provider, I would bet on the boring stuff. An old component, an undocumented assumption, a change that interacts with something nobody remembered was there, and fewer people around who might have remembered.

The irony is that the investment goes to the lipstick while the pig keeps the lights on. Analysts made a similar point this week, noting that email and Teams, treated as legacy revenue next to AI, still carry most enterprise and public-sector work.

What this means for organisations

As I argued in AI is now infrastructure, when critical services depend on a handful of providers, outages stop being local IT problems. In my view, boards and executives should be asking four questions:

1. Dependency. Which of our critical services depend on a single SaaS provider, and what would we do if it disappeared for 27 hours?
2. Vendor risk. When we assess a vendor, are we asking about the age and architecture of the platform under the product, and about the stability of the teams who run it, not just its certifications?
3. Continuity. Are our business continuity plans tested against vendor outages, not just cyber incidents?
4. Sovereignty. For Australian organisations with obligations under the Security of Critical Infrastructure Act, is our dependency on offshore platforms visible in our risk register?

If you cannot see inside the platform, and the vendor cannot quickly explain why it fell over, then your resilience is borrowed from someone else's technical debt.

You can put as much lipstick on a pig as you like. It is still a pig, and that pig is getting older every single day. 🐷