Skip to content

The demonstration, narrated

Audience: whoever is presenting. This is the operational story — what to open, what to say, what each beat is meant to prove, and what to do when one does not work.

Which audience is this for? Red Alpha, internally. The demo plan explains why the two audiences need different demos: an internal group is asking "is this real, or is it staged?" and a customer is asking "is my tradecraft safe, and does this help me?". This script answers the first question. Where the customer version would differ, it says so.

The whole thing is one alert, told well, plus the five things that alert cannot show on its own. Budget ten minutes of narration against roughly three minutes of machine time, and the gap is the point: most of the value is in what you say while a screen is still.

Before anyone is in the room

./deploy/bin/harness stale          # is the running stack actually this code?
./deploy/bin/harness reset          # clean state; keeps model weights and accounts
./deploy/bin/harness criteria       # 11 of 11, proven by tests that just ran
./deploy/bin/harness demo           # the whole thing, headless, every claim asserted

stale is first for a reason. Everything inside the stack reports healthy when the whole of it is old together — the services compare versions with each other, agree, and say so. Only something holding both the containers and the working tree can tell, and standing up to demonstrate a build from three days ago is a way to find out that a beat you fixed is still broken. demo refuses outright if they differ.

If demo prints "Every claim held", the system will do live what you are about to say it does. If it does not, do not present — the script is the gate, which is why it asserts rather than prints.

Then open two things and leave them open:

Window Why
The console, signed in as nw.analyst Where the story is told
A terminal Where the story is checked — an internal audience trusts this more

One more thing worth doing: ask someone else to drive for beat 6. A demo that survives a stranger's hands is a different category of evidence from one that does not.

The story, in one sentence

A sign-in at 2:14 in the morning, from a country and a device this account has never used — and the account can reach building controls.

Everything below is that sentence, examined.

Beat 1 — Set the scene

Do: Console → Playbook. Scroll it slowly. Let people read it.

Say:

This is the customer's file. Not a configuration screen we built for them — their judgment, written down. Northwind says a sign-in is suspicious when it is from a new country and a new device and outside working hours. All three. That is their definition, not ours, and it is the only place that definition exists.

Point at time_reference: site_local and the comment above it:

This one line is worth the whole file. 02:14 is the middle of the night at the site and mid-afternoon somewhere else. The same rule gives a different answer depending on whose clock you read, and that is a judgment about their business — these are data centres, and the question is whether someone was signing in when nobody should have been on site. If we had made that decision in platform code, we would have put the customer's judgment somewhere they cannot see or change.

Proves: tradecraft is data, and the boundary is real before anything has run.

Beat 2 — One alert, end to end

Do: Alert queue → Suspicious sign-in, 02:14. Watch the incident build.

Say, while it runs:

It is gathering facts — the sign-in, the account's normal pattern, the device, the location. All read-only. Then it applies their rules, not ours.

When the verdict lands — likely_compromise, high, two rules matched:

Two of Northwind's rules fired. The first is their definition of suspicious. The second says a suspicious sign-in on an account that can reach operational technology is not merely suspicious, it is a probable compromise. And severity went to high for the same reason — their modifier, their words: "OT adjacency is what makes this urgent here."

The line that matters most, and it is worth stopping for:

The model did not decide any of that. The verdict is a function of their playbook and the facts. The model read messy evidence and wrote the narrative. Swap the model and the verdict does not move — I will show you that in a moment.

Proves: the central design principle. The LLM proposes; deterministic code disposes.

Beat 3 — The approval gate

Do: Approvals. Read the banner at the top of the page aloud.

Say:

Nothing on this page has happened.

Point at what the agent proposed. On a real model it is usually the gentler of the two actions Northwind permits — re-challenge their MFA.

The agent proposed the cautious one. Northwind's playbook also permits temporarily disabling the account, so I can take that instead.

Do: choose identity.account.disable. The incident moves to awaiting approval.

The agent proposes; the person decides. And notice the bound — I chose from their list. I cannot invent an action their playbook does not contain, and neither can the model. Omission is a control.

Do: approve it. Then, in the terminal:

curl -s localhost:8000/v1/incidents/<id> -H "Authorization: Bearer $TOKEN" | jq .state

Proves: a human is in the loop, and the action space is closed.

If the agent proposes disable on its own, say so and move on — it is a real model and it varies. The beat still works; you simply do not need to override it.

Beat 4 — The OT safety gate

Do: inject an alert for bms.operator — a building-management account.

It comes back suspicious / medium, recommend-only, held by ot-cap. Note that this is a different verdict from beat 2, and the difference is the lesson.

Say:

Two separate things are called "OT" here and the demo is a good place to pull them apart, because people conflate them and they behave differently.

In beat 2, j.doe is an ordinary IT account that can reach building controls. That is OT adjacency — a fact about reachability, from the asset inventory, and Northwind's playbook is what decides it matters. Their rule raised the classification to probable compromise and their modifier raised severity to high. Customer judgment, both.

This account is operational technology. That is OT asset class, and it is not Northwind's call at all. Only one rule matched, so the verdict is milder — and yet this is the one that is capped.

Point at ot-cap:

No agent may act on operational technology. Not at a higher tier, not with approval, not because a playbook asked for it. Northwind's own file says max_tier: T2 for disabling an account, and the cap overrides their file. A customer can make the harness more cautious; they cannot make it less. That is the one direction that is not configurable, and it is deliberate.

So: a milder verdict, a stronger constraint. Severity is their judgment. The cap is our mechanism.

Then hand it to the audience:

Try to approve it.

It returns 409. The gate is not advisory.

Proves: the safety claim is enforced in platform code, not in tradecraft, and not by convention.

Beat 5 — An asset nobody can classify

Do: inject an alert for mystery.box — in neither the IT nor the OT list.

Say:

The inventory has no answer for this one. A real CMDB is wrong or incomplete all the time. So what should happen?

It is held, citing a rule of its own: asset-class-unknown.

Not knowing what something is must never be safer than knowing. The inventory had no answer, so it is treated as operational technology and capped — and the log says why it was capped, which is a different rule from the one in the last beat. If the CMDB is wrong, the failure lands on the cautious side.

That is one line of code, and it is the difference between a safety story and a safety claim.

Proves: fail-closed is implemented, not aspirational. This is the beat a sceptical engineer remembers.

Beat 6 — Tradecraft is data

This is the one to hand over. Give someone else the keyboard.

Do: Playbook → edit Northwind's off-hours window so it covers the whole day → save → inject the same alert again.

Say:

Nothing was rebuilt. Nothing restarted. The next alert used the new rule.

The off-hours rule no longer matches, and the verdict changes.

That is what "the customer owns their tradecraft" has to mean operationally. If they need us to change their detection logic, we own their judgment and they are renting it back.

Do: put it back.

Proves: the thing most often claimed and least often true.

Beat 7 — One platform, two organisations

Do: sign out. Sign in as co.analyst — a different organisation. Inject the same alert.

Say:

Same platform, same binary, same alert. Different verdict, because Contoso's judgment is different.

The contrast is worth spelling out, because it is the product claim in one table:

Northwind Contoso
Suspicious when new country and new device and off-hours new country and new device
Whose clock is "off-hours" the site's the person's
Nothing matched means benign inconclusive
May disable an account yes, with approval not at all

Contoso's verdict is suspicious / medium on one matched rule, where Northwind's was likely_compromise / high on two. Same alert, same binary.

Look at the last row of that table. Contoso's playbook does not contain account disabling, so no amount of severity can cause it to be offered. They did not configure it off. They never wrote it down, and absence is the control.

Their one permitted action still stops at the approval queue, because they wrote it as T2. Same gate, different judgment about what belongs behind it.

And: Contoso cannot see Northwind's incidents. Try it.

Do: try to reach the other tenant's data. It is refused and recorded.

Proves: isolation, and that omission is a first-class control.

Beat 8 — Why any of this can be believed

Do: Audit & replay. Then, in the terminal:

./deploy/bin/harness verify-chain

Say:

Every prompt, tool call, decision, approval and model choice is in an append-only, hash-linked log. Not "we log things" — the chain verifies, and if anyone changes one event it names the first one that no longer holds.

Then the move that lands:

./deploy/bin/harness verify-tamper

It exports the chain, alters one event in the copy, and runs the platform's own verifier over it. The real chain is never touched.

It does not say "something is wrong". It says which event is the first that can no longer be trusted, and it shows you both hashes — the one stored and the one the content now produces. Everything before that event still stands.

That is the difference between an audit trail and a log file. And notice the last line: the real chain still verifies. I did not have to damage the evidence to show you the evidence is tamper-evident.

Do: Earned autonomy. Show that nothing has earned T3.

Autonomy is earned through a clean evaluation record, not configured. Right now nothing has earned anything, which is the correct state for a system that has run for ten minutes.

Proves: the evidence is checkable rather than asserted.

Beat 9 — The kill switch and the maintenance window

Do: Admin. Engage the kill switch. Inject an alert — it halts.

Say:

One control stops every agent. Anyone can engage it; only an administrator can release it. A halt that anyone can undo is not a halt.

Do: release it. Then open a maintenance window with a reason: "chiller plant changeover, bay 3".

This one is weaker on purpose, and the pair is the point. The kill switch means stop. A maintenance window means not while engineers are inside the machine — analysis continues, actions wait. During a plant changeover an operator still wants triage and a ranked queue. What they must not have is an agent disabling an account a field engineer is working from.

Do: inject an alert. Held at recommend, citing maintenance-window. Try to choose the stronger action — still held.

A person cannot click past it either.

Do: close the window.

Proves: operational control that matches how operational technology actually behaves.

Beat 10 — What just happened

Do: nothing. Close the laptop lid if you like.

Say — four sentences, and resist adding a fifth:

One: their tradecraft is data, behind a boundary a build check enforces. Two: autonomy is tiered and earned, and operational technology is capped at recommend by platform code. Three: the model is replaceable, and the whole thing runs with no network at all. Four: one platform, two organisations, two verdicts, and neither can see the other.

Each of those was demonstrated, not asserted. The script that runs this headlessly fails if any of them stops being true.

The question you will be asked

"Isn't this just a rules engine with a chatbot on top?"

Answer it directly, because the honest answer is the strong one:

Largely, yes — and that is the feature. Deterministic, auditable, customer-owned judgment is what a security team can defend to a regulator and to their own leadership. The model does what it is good at: reading messy evidence, composing the narrative, and flagging planted instructions. It does not decide anything, and that is why swapping it does not change a verdict.

Do not get defensive about it. Say it before someone else does.

What to say about what is not built

An internal demo that oversells produces a worse decision than one that underclaims. State these plainly:

  • Two agents, triage and investigator. Not six.
  • One scenario, HW-1. The second scenario is a day's work, not a month's, but it is not written.
  • No real connectors. The estate is synthetic, on a lab network, by design.
  • Keycloak has no backup, and there is no secret management. Lab-sized answers.
  • AWS is written and validated, not yet applied against a live account. That line goes away the day it is.

If something breaks

Symptom Almost certainly Do this
Console shows not fully connected A dependency is down or stale Admin → it names which one
Model calls fail Weights missing, or Bedrock access not granted harness pull-model, or check Bedrock model access
Login loops back to sign-in Issuer mismatch after a hostname change Check HARNESS_PUBLIC_URL reached the keycloak container
A verdict is not what this script says A playbook edit was not put back git checkout tradecraft/examples
Anything at all, five minutes before — ./deploy/bin/harness reset, then demo to confirm

The fallback that always works: run ./deploy/bin/harness demo and narrate it. It is the same beats, it asserts every claim, and it takes under half a minute.