Tools that score whether an inbound lead matches your ideal customer profile ship without any measurement of how often they are right, because building the labeled set is the tedious part. So the labeled set is the point here, not the classifier.
120 real YC companies, hand-labeled on three axes against a rubric written before anything was scored. Twenty-one of them are borderline on purpose. You cannot tune a filter you have never measured.
This runs classify.py out of the repository, in your tab, on whatever you
type. Not a reimplementation and not a lookup: the same function that produced every
number below.
Four fields, which is everything a signup form gives you. The B2B axis trusts the industry tag when there is one and falls back to vocabulary. The stage axis is headcount and years since the batch. The SaaS axis is the one that fails, and you can make it fail: describe a real software company without using a software word and watch it say no.
The engine self-checks at load. It re-runs all 120 labelled rows through the copy in your browser and reports the score it gets, which should be the same 0.829 and 0.967 the repository publishes.
Every classifier here is scored on the same 120 rows against the same hand labels. Enriching every lead is the thing most teams do, and it is on the list so the others have something honest to beat.
The strict rule is right 93.8% of the time it says yes, and it says no to half the real customers. The relaxed one is right 82.9% of the time and finds 29 of 30. Which is better depends entirely on whether a wasted enrichment call costs more than a missed customer, and that is a question about your business rather than about the model.
What is not defensible is shipping either one without knowing the numbers, which is the state of every tool in this category.
Every predicted positive gets an enrichment call and only the true positives were worth making. The price per call is an assumption, not a measurement, so it is swept rather than asserted.
The composite label is B2B and SaaS and stage. Going in, stage was expected to dominate the loss, because it is the one measured by proxy. It does not.
is_saas is the weakest axis, at 0.632 F1, and stage_fit is not far off its best. Nearly every missed customer has the same shape: a real B2B software company whose one-line description never says software. "Interactive meetings and presentations" is SaaS. "Cybersecurity awareness for fast growing companies" is SaaS. A keyword rule reading one sentence cannot see it.
That is the finding worth having. The axis that looked risky was fine, and the one that looked obvious was not, which is only visible because the set was labeled first.