NORA

The problem

Why this needed to exist

The solution

How it works

Implemented

What’s actually built

• Risk-tiered tools

  Every tool is READ, WRITE_LOW or WRITE_HIGH. Reads are free; refunds, customer emails, social publishing and campaign sends pass through the policy engine.

• A refund policy in code

  Over $200 is refused outright. Over $50, older than 30 days, or a customer refunded in the last 90 days is held for approval. Everything else is allowed.

• Fails closed

  If no approver is wired up, held actions are denied. In the web console, no answer within three minutes, a closed tab or a dropped session all resolve to denied.

• Approval cannot override refusal

  Operator approval is consulted only for actions the engine holds. There is no path from a pasted customer message to a $5,000 refund.

• Drafting is free; reaching an audience is not

  A social post or email campaign can be drafted immediately. Scheduling it to real followers is held for sign-off, and each email provider’s real daily send limit becomes the ceiling.

• The operator connects accounts, never the agent

  NORA has no tool to connect a social or email account. She can see which are live, and is refused outright if she tries to post to one that is not.

• Morning brief

  The CLI opens by scanning the book and telling you what needs a decision before you have asked.

• An eleven-page console

  Dashboard, Approvals, Audit Log, Social, Email, Calendar, Orders, Customers, Shipments, Policies and Evals — with NORA as a slide-over on every page and a Ctrl/⌘K command palette.

Interface

Screens from the real product

My role

What I did

Decisions

Technical and design choices

1. 01

   Guardrails in Python, not the prompt

   The prompt describes the policy so the model can plan around it; the if-statements enforce it. The model never gets to decide whether a rule applies.

2. 02

   Score what it did, not what it said

   Evals read the audit log. An agent that says "I’ve escalated this" without calling the escalation tool fails the case.

3. 03

   A frozen, cacheable system prompt

   Volatile facts like today’s date arrive as a separate system message, keeping the cached prefix intact. A run with zero cache reads flags that something dynamic leaked into the prefix.

4. 04

   One seam for real systems

   All data access goes through one module, so SQLite can later become Shopify or Stripe without touching the agent.

Trade-offs

Challenges along the way

1. 01

   Prompt injection through customer text

   A dedicated eval bucket feeds the agent customer messages that try to talk it into refunds. The guardrail, not the model’s judgment, is what makes those cases pass.

2. 02

   Approval UIs that fail open

   Browser approval loops have many quiet failure modes — timeouts, two tabs, shutdowns, double-resolves. Each is covered by an offline check that must resolve to denied.

3. 03

   Undeclared dependencies

   The console worked locally only because packages had been installed by hand; a clean install could not start the server. The fix was declaring them — and the habit of installing your own project somewhere fresh.

Results

What can be checked

• Evaluation suite

  33 scored scenarios across seven buckets (happy path, auto-approve, must-escalate, injection, out-of-scope, briefing, channels), plus 27 offline guardrail assertions that run in about a second.

Lessons

What I took away

• The hard part of a production agent is the integration surface and its limits, not the conversation.
• Making the guardrail observable — an Approvals page, an Audit Log — turns safety from a claim into something an operator can watch.

What’s next

Planned — not built