Testing and tuning 1.0

Testing and tuning

Last updated 10 Nov 2026
On this page

Every rule carries its own tests. Write them well, read a failed check, and tune a rule on real hosts before you trust it.

The fidelity contract

Every rule must carry at least one record it must match and one look-alike it must not. A look-alike is the benign thing a careless version of the rule would catch: notrclone.exe, a -ExecutionPolicy switch, "Temp" in an installed program's folder name.

Test key

Meaning

evidence

One of the rule's evidence types.

fields

The record's fields (only fields that type carries); log.event_id also sets the event's ID.

expect

match or no_match.

note

Why this case is here.

Tests check which records a rule matches; min_count is applied over a run.

Reading a failed check

Output 
FAIL  EXEC-MY-RULE                         C:\signal-rules\mine.json
        test 1: must match, did not - condition "editor"; editor: false (process.name endswith "\notepad.exe"; the record's is "C:\Windows\write.exe")
        test 2 (a look-alike): must not match, did - condition "editor"; editor: true

Each selection is shown as:

  • true;

  • false, with the first field that did not match and what the record holds there;

  • unknown, when the record lacks a field that depends on a policy.

An unknown modifier lists the valid ones; a missing key is named; a JSON mistake gives its line and column.

Tuning on real hosts

  1. Run it on clean hosts first

    Run your rule across hosts you know are clean - a few workstations, a server, a domain controller. Every match there is a false positive.

  2. Narrow it on what Windows records

    Exact paths anchored at the drive, a SID instead of a name, a logon session, a signature. Never a broad exclusion an attacker could use.

  3. Add each false positive as a look-alike test

    So the rule can never regress.

  4. Re-grade if needed

    If benign matches remain, the rule is medium or low - or it is context, a column of a table, not a signal.

See the rules a run would use

PowerShell 
signal rules -m Executions -rules C:\signal-rules

Lists every rule with its origin: built-in, or your file. In Signal Console, the Rules page shows the same, with each rule's outcome on each host.