Azure Already Told You. You Parsed Four Fields.

Here is a Resource Graph query. It is the one our scanner uses to read the AKS clusters in an Azure estate.

Resources
| where type =~ 'microsoft.containerservice/managedclusters'
| project id, name, resourceGroup, location, properties

That last column hands back the cluster’s properties object as Resource Graph holds it: node pools, API server configuration, network profile, identity, add-ons, upgrade channel, security profile. One column, already paid for.

We parsed four.

enableRBAC. apiServerAccessProfile.enablePrivateCluster. apiServerAccessProfile.authorizedIPRanges. networkProfile.networkPlugin. The rest of what Azure returned was deserialised, held in memory, and dropped on the floor.

How you find out

Not by auditing the collector. Nobody audits a collector that works.

You find out when somebody asks for a new check. In our case: does this cluster run a network policy engine? The reflex answer is a cost estimate — new collector, new query, new permission for the customer to approve, new consent conversation, two weeks.

The actual answer was that networkProfile.networkPolicy sits directly beside networkProfile.networkPlugin in an object we had been fetching since the collector was written. One property read. No new query, no new permission, no conversation with the customer at all.

Worse, our own backlog had recorded the opposite. The item describing this gap listed networkPolicy as in-cluster-only — something you would need a cluster credential to see. That is wrong, and it had been wrong on the record long enough that work got scoped out on the strength of it.

I gave that estimate. Two weeks, a new permission, a conversation with the customer about consent. The answer was one property read, sitting beside one we were already fetching.

Why it happens, structurally

Carelessness has nothing to do with it. The shape of how collectors get written does.

You write a collector to answer the checks you have today. You project the whole object because projecting individual fields in KQL is fiddly and brittle, so properties is the pragmatic choice. Then you parse exactly what the current checks need, because parsing more would be speculative work with no reader.

Both decisions are correct. The bug is that nothing routinely revisits them, so the gap between fetched and parsed widens as Azure adds properties. Azure adds them. Your parse list doesn’t.

And the gap costs you a mis-priced decision rather than a missing field: somebody asks for a capability, gets told it needs new access, and the capability gets deferred — or the customer gets asked for more permission than the work requires.

One caveat on “the whole object”, because it cuts both ways: Microsoft notes that Resource Graph serves a particular resource-provider API version, so a property you expect can be absent from the response even though the resource has it. Read what came back rather than what the schema promises.

The uncomfortable second half

Projecting the whole object is not free of consequences either.

Everything in that object crosses the network, lands in your process, and ends up in whatever you persist. If you store collection snapshots, you are storing cluster configuration nobody reads and nobody reviewed, from a customer’s estate, under whatever retention your platform happens to have.

So “parse everything” is the wrong lesson to draw. The useful one: know the difference between what you fetch and what you read, and be able to state it. Those are two different numbers, and most tools can only tell you the second.

Three things to check in your own tooling

1. Take one collector and diff the two sets. What does the query project, and what does the code read? If the first is properties and the second is a short list of named paths, you have the gap.
2. Before you estimate a “new data” request, grep the raw response for the field. az graph query -q "Resources | where type =~ '<type>' | project properties | limit 1" and read it. This costs a minute. It is how the two-week estimate above turned into one property read.
3. Check your backlog for claims about where data lives. Ours asserted a field was in-cluster-only when it was sitting in ARM. A wrong claim about where something lives is more expensive than a missing feature, because it prices the fix wrong and nobody re-checks a settled scoping decision.

The query was never the constraint. The parse list was, and nobody owned it.