Testing Your Own Controls

A policy that is configured is not a policy that works. Agents fail, exclusions overreach, upgrades reset settings, and a rule that fired last year may be silently dead.

Testing is straightforward, almost nobody does it, and everyone who starts finds something broken in the first hour.

What to test

That the detection fires at all. Move a file matching your policy through the channel and confirm an alert appears. This is the basic test and it fails more often than anyone expects.

That it reaches a human. An alert that fires and lands in an unreviewed queue is not detection. Test the whole path, including the notification.

Coverage across the estate. Agents missing from a subset of machines is the commonest gap. Test on the oldest hardware, on machines that were offline during a rollout, and on any platform that is a minority in your estate.

That exclusions are not too broad. An exclusion written for one legitimate workflow frequently covers more than intended. Test that the excluded path does not also suppress the thing you care about.

Alternative routes. If email is covered, is the same file blocked via cloud sharing, browser upload and removable media? Most programmes cover one channel well and assume the rest.

How to run it

Use synthetic data that matches your policies, not real sensitive data. Generate records in the right format, and mark them so they are identifiable if they escape.

Announce it to the reviewing team afterwards, not before. Otherwise you are testing whether they respond to an expected event.

Document what you tried and what happened. This is the record that lets you show coverage to an auditor and, more usefully, tells you what changed since last time.

Run it quarterly, and after every product upgrade. Upgrades reset configuration surprisingly often.

The scope boundary

Testing your own detection is verification. It stays firmly on the defensive side.

Two rules keep it there:

Use synthetic data. Never move real sensitive information to test a control. The test should not create the incident it is checking for.

Test in a controlled, authorised way. Written authorisation, defined scope, a stated window, and the security leadership informed. An unannounced test that triggers an investigation of a real person wastes everyone's time and damages trust.

The purpose is to confirm your instrumentation works, not to demonstrate that controls can be circumvented. Everyone already knows they can.

What to do with the results

Fix coverage gaps first. A channel with no working detection is a larger problem than a rule with poor precision.

Record which routes remain uncovered, deliberately. Every programme has them. Writing them down converts an unknown into an accepted risk, which is the difference between a gap and a surprise.

Report honestly. A test result showing that two of five channels failed is more valuable to an executive than a dashboard showing green, and it is the kind of finding that gets budget.

The related exercise

Once a year, walk an incident through end to end on paper: detection fires, someone reviews it, it escalates, HR and legal are engaged, access is suspended, evidence is preserved.

Every organisation that does this finds at least one broken link — usually that nobody knows who can authorise access suspension out of hours, or that the preservation step depends on a person who has left.

Finding it in a rehearsal costs an afternoon.