← CYBERNETICSINTERN.COMCYBERNETIC INTERN
Detection EngineeringFOUNDATION

If the source samples, silence proves nothing

Sampled telemetry answers questions about volume and pattern well. It cannot tell you whether one specific short event happened.

Some telemetry is complete. Some is a sample, and it does not always say so.

Flow records are frequently sampled at the device, one packet in a thousand or one in ten thousand, because full capture at line rate is expensive. Some cloud providers sample high-volume audit events. Some agents drop events under load rather than queue them, which is sampling by another name and usually invisible.

The consequence is specific. On a sampled source, absence is not evidence of absence. A brief connection to a rare destination may simply never have been picked. You can still detect volume and pattern, because sampling preserves those. What you cannot do is conclude that a single event did not happen, which is exactly the question an investigation tends to ask.

Analysts get this wrong in one direction. Somebody will say that no flow records show the connection and treat it as a clean result, when the honest statement is that the connection was not sampled, if it was short.

For each source feeding your investigations, find out whether it samples and at what rate, then write it somewhere the person running the query will actually see it. After that, check whether your agents drop events under load. That is the sampling nobody documented.

CHECK YOURSELF

An investigation needs to know whether a host made a single short connection to a suspicious address. Your only network data is flow records sampled at one in a thousand. You find nothing. What can you honestly conclude?

SHOW THE ANSWER

Nothing conclusive, because a short single flow would probably never be sampled

Why. At one in a thousand, most individual flows are never recorded, so their absence carries almost no information. Sampled data still answers volume and pattern questions well, because those survive sampling. It cannot answer whether one specific short-lived event occurred. Raising the rate helps future investigations and does nothing for this one, and stating the limit is far more useful to the case than presenting an empty result as a clean one.

← ALL DETECTION ENGINEERING