Metrics That Mean Something

Insider risk programmes are reported on badly, and the standard metrics actively misinform the people reading them.

The metrics to stop using

Alert volume. Falls by ninety percent during tuning, which is success and looks like collapse. Rises when you add a channel, which is coverage and looks like deterioration. It measures configuration, not risk.

Blocked events. Sounds like prevention. Mostly counts legitimate business activity that a policy was too broad to distinguish. A rising block count usually means a tuning problem, not a threat.

Incidents detected. Tempting and treacherous. If the number is low, is the programme working or blind? Nobody can tell, and the metric creates an incentive to classify minor events as incidents.

Coverage percentage. Endpoints with an agent installed. Necessary hygiene, unrelated to whether anything is being caught.

What is worth reporting

Time from event to review. How long between something happening and a human looking at it. This is the measure of whether the programme functions operationally, and it is the one that degrades first when the team is stretched.

Proportion of alerts reviewed. If you generate more than the team can look at, the ones that matter are invisible. A programme reviewing 100% of a small volume is stronger than one reviewing 20% of a large volume, and the metric makes that visible.

False positive rate on acted-upon policies. Only for policies that trigger a user warning or a block — those are the ones that cost the business. Track it per policy; an aggregate hides the one policy that is causing all the complaints.

Categories of data covered. Against the inventory. "Three of our seven confidential categories are instrumented" is an honest statement of coverage that alert counts cannot express.

Approved workflow exclusions. How many legitimate processes had to be excluded. A rising number means either the business is changing or the policies are too blunt, and both are worth knowing.

Repeat behaviour after intervention. Of the people who received a warning, how many repeated within ninety days. This measures whether the intervention works, which is the actual question for the negligence category.

Time to revoke access on departure. Sits outside the DLP tool and predicts more risk than anything inside it.

Reporting to executives

They are asking one question: are we exposed, and is this working.

Answering with alert counts invites the follow-up you cannot survive — "so is that good or bad?"

A better shape: what we can see and what we cannot, against the data categories that matter. What we caught and what it would have cost. What we changed as a result. What we still cannot observe and what closing that gap would take.

The last item is the one that builds credibility. A programme that states its blind spots is trusted about the rest.

The measurement problem nobody solves

You cannot measure incidents prevented, and you cannot measure incidents missed.

A quiet quarter might mean strong controls or complete blindness. This is not a reporting failure to be engineered around; it is a property of the domain, and the honest response is to say so rather than substituting a proxy that implies otherwise.

What you can measure is whether the machinery works: are the right things instrumented, is output reviewed, do interventions change behaviour, does access get removed on time.

Those are process metrics, they are less satisfying than a threat count, and they are the only ones that are true.