lgoyal6 / actiongate

actiongate.

A model reading a meeting transcript will confidently produce an action item from someone hedging, from a conditional that never fired, or from something a speaker retracted a minute later. Written straight into a CRM, that becomes a wrong conversation with a customer.

A missing action item is annoying; an invented one is expensive. Those costs are not symmetric, and writing everything you extract treats them as if they were. This scores each item and only auto-commits the confident ones.

writing everything
18 wrong writes, of 40
gating at 0.65
2 wrong writes
still auto-committed
19 of 22 genuine items
threshold chosen
0.65 on dev, held from 5 to 100
Try it

Paste a meeting transcript. Watch the gate decide.

Two boxes. In the first, a few lines of a meeting. In the second, the action item an extractor claims came out of it. The repository's scorer runs in your tab and tells you whether that would be written into a CRM without a person seeing it.

starting the engine checking it against the held-out split
Or load one

Edit either box and the score below moves as you type. The buttons above load transcripts worth trying.

confidence
-
gate at 0.65
-
grounded in
-
said by
-

Two places it comes apart

The conditional feature is a keyword list. "I'll send the contract Friday if the redlines come back clean" trips it and drops to 0.617, which queues it. "if legal signs off" means the same thing, is not on the list, and scores 0.938. Same commitment, one word apart, opposite decision.

A retraction fires and the gate lets it through anyway. Say you will send it, then say "scratch that". The retraction feature correctly reads 1.0, and strong grounding, commit language and a date still carry the score to 0.861. Detecting the retraction is not the same as acting on it, and the fitted weight is not large enough to.

Grounding is the part that works. Write an action item about something nobody said and the best-matching span is unrelated, grounding collapses, and it is queued.

works
Grounding.An invented item scores 0.122 and is queued.
brittle
Conditional, a keyword list.0.617 or 0.938 on one word.
fires and is ignored
Retraction.Reads 1.0, still auto-commits at 0.861.
Figure 1

What the gate stops, and what it costs you

Every item above the threshold is written to the CRM without a human. Everything below goes to a review queue. Moving the threshold trades wrong writes against work saved, and there is no setting that does both.

Loading
Split
Threshold
threshold
-
wrong writes
-
auto-committed
-
sent to review
-
precision
-

Why the trade is worth making here

Writing everything puts 18 wrong action items into the CRM. The gate cuts that to 2 and gives up three auto-commits to do it. Those three are not lost work; they become rows in a review queue that someone clears.

The threshold is derived, not guessed. It minimises a cost that counts a bad write as twenty queue rows, computed on the dev split and then read off the held-out one. Choosing it on the split you report would be the obvious way to get a better number and a worse system.

the asymmetry
A bad write costs about 20 reviews.That ratio is the whole design.
what it buys
89% of the damage removed.18 wrong writes down to 2.
what it costs
Three auto-commits.Which become queue rows, not losses.
Figure 2

Does the threshold need retuning for your business

The cost ratio is the one number a team would have to supply: how many review-queue rows a single wrong CRM write is worth. If the chosen threshold moved every time that guess moved, it would not be much of a design.

threshold chosen on the dev split

From 5 to 100 it does not move. Whether a bad write is worth five queue rows or a hundred, the same threshold comes out, so a team adopting this does not have to get that estimate right to get the benefit.

At a ratio of 1 it correctly collapses to zero. If a wrong write really costs the same as a review, there is no reason to gate at all, and the chooser says so rather than defending the gate it was built for.

stable
0.65, from 5 to 100.The estimate does not have to be right.
and it degrades sensibly
At a ratio of 1, no gate.Which is the correct answer there.
chosen where
On dev, read off test.Never on the split being reported.
Figure 3

Where it loses