The Azure Well-Architected Operational Excellence Pillar: What Most Teams Skip

Can you change this system safely, and will you know what happened when you did?

That is the whole operational excellence pillar. One question, two halves. And the answer nearly always arrives as a gap: the part of the environment that lives in code, versus the part that exists because somebody clicked through the portal one afternoon and never came back to it.

No diagram shows that gap. Nobody can fully account for it.

Teams skip this pillar because nothing forces the issue. Security has breaches. Reliability has outages. Cost has a bill at the end of the month, with somebody’s name on the approval. Operational excellence has none of that. So the work that makes a system operable (infrastructure as code, safe deployment, real observability, written runbooks) slides into next quarter, then the quarter after. Then something breaks at three in the morning and the entire absence comes due at once.

The four design principles, in plain terms

Adopt infrastructure as code: the environment is defined in version-controlled templates, not assembled by hand. Use safe deployment practices: changes roll out progressively and you can halt or roll back. Implement observability: collect the signals that say the system is healthy, and act on them. Automate and document operations: the knowledge lives in runbooks and code instead of in someone’s head.

None of that is controversial. Nobody argues with it in a design review.

The findings come from the distance between the principle and how environments actually grow. Incrementally. Under delivery pressure. With the portal one click away every single time the template would have taken twenty minutes longer.

The portal is always one click away

A manual change creates a version of the truth the code has never heard of. The next deployment either ignores it or fights it. That is the operational face of configuration drift, and it is the finding I hit most consistently.

The story barely varies. The estate is defined in Bicep or Terraform, mostly. A production incident lands at an inconvenient hour. An engineer opens the portal, changes a network security group rule, scales up a database, adds an app setting, whatever it takes to stop the bleeding. Incident resolved, everyone goes back to bed. The change never makes it back into the code, and nobody wrote down where it went.

Then it compounds. The next terraform apply may quietly revert the emergency fix and hand you the same incident a second time. Or the drift just sits there, invisible, until a region rebuild or a DR event redeploys from code that no longer matches reality. The assessment catches it by comparing what Azure Resource Graph reports as deployed against what the IaC would produce: resources with no matching template, settings that diverge from what the code declares.

Here is the part I think people miss. “We use Bicep” almost never means the whole estate is in Bicep. It means the greenfield bits are, and the rest accumulated.

The deeper problem sits one layer down anyway. No policy stops portal changes in the first place. Azure Policy can deny resource modification outside the deployment identity, and when that guardrail is missing, the door is not just unlocked, it is off its hinges.

At that point drift is not a risk. It is a schedule.

Green dashboards that tell you nothing

Defaults-only monitoring is a wall of green that means nothing at all. Azure Monitor collecting platform metrics nobody chose, firing no alerts anybody would act on. And it shows up more often than having no monitoring, for a boring reason: an empty monitoring page is obviously wrong, while a busy useless one sails through inspection.

The components are almost always present. Azure Monitor is on. A Log Analytics workspace exists. Application Insights is instrumented. What is missing is the configuration that turns telemetry into awareness, and it fails the same four ways.

Diagnostic settings on critical resources are absent. Azure Monitor grabs platform-level metrics on its own, but resource logs (what a Key Vault, an Application Gateway, or a firewall is actually doing) only flow when you configure diagnostic settings to stream them to the workspace. The assessment finds critical resources with none configured, which means the data that would have explained the last incident was never collected in the first place.

Then there are no alert rules worth the name. Metrics get gathered. Nothing is defined against them. No rule fires when latency blows past its budget, when the failed-request rate climbs, when a dependency starts timing out. The signal is sitting right there. Nothing is watching it.

Where an alert does exist, it often routes nowhere. No action group, no on-call rotation, no escalation path. An alert that fires into an empty inbox is theatre.

And Application Insights sits at defaults. Instrumented, but with no custom availability tests, no defined health signal, no distributed tracing across the dependencies that actually fail. Same observability gap the reliability pillar surfaces, seen from the health-modelling angle instead.

If you want to test your own environment in the next five minutes, open a critical resource and look at its diagnostic settings blade. Go on. I’ll wait.

The deployment process lives in one person’s head

Ask a plain question. If the one engineer who deploys is on a plane, can anybody else ship a change and recover a failure?

In most environments I assess, the honest answer is no. The process is undocumented, the runbooks do not exist, and both live in the working memory of one or two people. Those people are now a single point of failure with a pulse and a holiday allowance.

A documented deployment process spells out how a change moves from commit to production: the pipeline stages, the gates, the approval, the rollback. Safe deployment adds progressive exposure on top: deployment slots in App Service for staged swaps, canary or ring-based rollout, automated rollback when something fails. What I usually find instead is a pipeline that deploys straight to production, no staging gate, and no rollback beyond “redeploy the last good version and hope.”

Runbooks are the other half, the recovery half. Step-by-step responses to failures you can already name: the database failover, the certificate rotation, the dependency outage. Architecture Decision Records sit next to them and capture why the environment is built the way it is. Both go missing together, always. When they do, the architecture documentation describes a system that no longer exists, and the recovery knowledge walks out the door the day its owner does.

I have been that single point of failure, by the way, and I quite enjoyed it at the time. Being the only one who knows how the deployment works feels a lot like being important. It is not the same thing.

Design principle	Finding in nearly every assessment	The artifact that should exist
Adopt infrastructure as code	Resources created in the portal, not in code	Full estate in version-controlled Bicep/Terraform
Use safe deployment practices	Direct-to-production pipeline, no rollback	Staged deployment with automated rollback
Implement observability	Diagnostic settings absent on critical resources	Resource logs streaming to Log Analytics
Implement observability	Metrics collected, no alert rules	Actionable alerts routed to an action group
Document operations	Deployment and recovery knowledge in one person’s head	Runbooks and ADRs in version control

NIS2 does not ask whether you have controls

NIS2 Article 21(2)(f) asks for policies to assess the effectiveness of cybersecurity risk-management measures. Read that slowly. It is not asking whether the controls exist. It is asking whether you know they work.

And you cannot know what you never observed.

In regulatory terms, operational excellence is the measurement layer sitting underneath everything else. The link is concrete, not conceptual. Knowing a control works needs telemetry: diagnostic logs streaming to a workspace, alerts that fire when the control degrades, a deployment process that records what changed and when. An environment on defaults-only monitoring cannot demonstrate effectiveness, because it is not measuring effectiveness at all. The diagnostic settings, the alert history, the deployment audit trail: those are what turn the assessment from a policy PDF into something that happened.

“But our security controls are fine, this is just an ops thing, right?”

Right up until someone asks you to prove the controls are fine. Then it turns out the proving apparatus is the ops thing you skipped.

Evidence you never had to assemble

A well-operated environment produces its evidence by construction. A version-controlled IaC repository that matches the running estate. A deployment pipeline with an audit trail. Diagnostic settings on every critical resource, alert rules with a firing history, runbooks in source control. That is the material an auditor accepts as proof that measures are assessed for effectiveness rather than assumed to work.

Here is the part that actually matters. None of it gets built for the audit. The IaC repository exists because the team deploys from code. The audit trail exists because the pipeline records every run. The diagnostic logs exist because somebody configured observability on purpose, on a Tuesday, when nothing was on fire.

A well-run environment throws off evidence as exhaust. The absence of that exhaust is itself the finding.

The reason I ended up building Platform Architecture Authority is that I got tired of reconstructing this by hand. Every assessment started the same way: exporting resource lists, diffing them against a repo, opening blade after blade to see which resources had diagnostic settings and which did not, then writing the same NIS2 mapping I had written the month before. That is not architecture work, it is clerical work with a consulting day rate attached. So I automated the clerical half and kept the judgement half, because deciding which alerts genuinely warrant waking someone at 3am is still a senior engineer’s call and always will be.

What actually sticks

• Manual changes made outside infrastructure as code are the most common finding in this pillar. They create drift the code cannot describe, and the next deployment either reverts them silently or collides with them.
• “We use Bicep” rarely means the whole estate is in Bicep. The assessment finds the resources created in the portal and never brought back under code.
• Most environments have monitoring deployed and configured at defaults. Azure Monitor collecting platform metrics, no custom alerts, no thresholds worth acting on, nobody owning the signal.
• Deployment processes are usually undocumented. The person who knows how to ship is the process.
• The evidence maps to NIS2 Article 21(2)(f), policies to assess the effectiveness of risk-management measures. You cannot assess effectiveness you never measured.

This is the pillar with no crisis to force it. Which is exactly why it is worth doing before the crisis shows up and asks why nobody saw it coming.

FAQ

What counts as infrastructure as code for this pillar? A version-controlled definition of the environment (Bicep or Terraform) that recreates the resources deterministically. The pillar checks not just that IaC exists but that it matches the running estate. Resources created by hand in the portal, with no matching template, are the gap the assessment surfaces as drift.

Is Azure Monitor enough for observability? No, not on its own. At defaults it collects baseline metrics and stops there. Real observability needs diagnostic settings streaming resource logs to a Log Analytics workspace, alert rules against thresholds that mean something, action groups routing those alerts to an owner, and Application Insights configured with health signals. The platform is necessary. The configuration is what makes it useful.

What is the fastest improvement to make? Configure diagnostic settings on critical resources (Key Vault, firewall, Application Gateway, gateways) to stream to a Log Analytics workspace. Low effort, closes the most common observability gap, and produces the telemetry that both incident response and NIS2 effectiveness assessment lean on. Alerting on that telemetry is the next step.