Best on desktop, but the demo still works on mobile.

← Scorecard

Item 7 · synthetic

Synthetic Strong Fit Devtools Test Gen

scenario synthetic_strong_fit_devtools_test_gen

Input

Kenji Nakamura
Head of AI at Testwright (Series A, ~65 people)

I joined Testwright about eighteen months ago when the team was still figuring out whether AI-generated tests were a novelty or a real workflow shift. We've settled that question. Today I lead a team of eight ML and applied-science engineers building the models that sit at the core of our product — a system that reads your codebase, infers intent from diffs and PR descriptions, and generates meaningful test cases rather than boilerplate assertions.

My background is a mix of NLP research and applied ML at scale. Before Testwright I spent four years at a mid-sized observability platform where I owned the anomaly-detection pipeline, and before that I did a PhD in machine learning at Carnegie Mellon focused on program synthesis. That combination turned out to be oddly useful here.

The hardest part of this job is not the modelling. It's the evaluation problem: how do you know a generated test is actually good? We've built an internal benchmarking harness that runs generated suites against curated corpora of real bug-introducing commits, and that's become the backbone of how we ship. We run it on every model checkpoint before anything touches production.

We closed our Series A from Bessemer and Index last spring, and the fundraise let us double the AI team. Right now I'm thinking a lot about how we extend coverage from unit tests into integration and end-to-end scenarios, which is a meaningfully harder problem. If you're working on similar evaluation infrastructure or developer-facing AI tooling, I'm always interested in comparing notes.

Testwright is a B2B SaaS developer tooling company headquartered in San Francisco, currently at Series A with approximately 65 employees. The product integrates into engineering workflows via IDE plugins and CI pipelines, using large language models fine-tuned on code and test corpora to generate unit, integration, and end-to-end test cases from code diffs and natural-language PR descriptions. Customers are primarily mid-market and enterprise engineering teams seeking to reduce manual test-writing overhead. Testwright raised its Series A led by Bessemer Venture Partners and Index Ventures and counts several publicly traded SaaS companies among its design partners.

Gold

Expected action
auto_add
Fit score
0.93
Industry
B2B SaaS
Segment
Developer tooling / test automation
Seniority
Director
Company size
51-200
Language
en
Notes
Strong fit. Hook should reference the evaluation-harness work and the extension into integration/E2E tests.

Predictions

Integratedmiss

  • classification miss: segment, seniority
Action
auto_add ✓
Fit
0.84 (gold 0.93, |Δ| 0.09)
Industry
B2B SaaS ✓
Segment
AI-powered developer tooling / automated test generation ✗
Seniority
Head of AI / Head of ML ✗
Company size
51-200 ✓
Grounding
10/10 claims grounded (100.0%)
Draft hook

Hi Kenji — you put it well: the hardest part isn't the modelling, it's knowing whether a generated test is actually good. The fact that you've made your internal benchmarking harness — running generated suites against curated corpora of real bug-introducing commits — the backbone of how you ship tells us you're operating at a level of rigour most teams haven't reached yet. We work with teams building exactly this kind of developer-facing AI tooling, and we'd love to compare notes, especially given you're now pushing from unit tests into integration and end-to-end scenarios, which you've rightly called a meaningfully harder problem. Worth a 25-minute conversation?

Claims (10)
  1. Kenji is Head of AI at Testwright
    ✓ in input Head of AI at Testwright
  2. Testwright is a Series A company
    ✓ in input currently at Series A
  3. Testwright has approximately 65 employees
    ✓ in input approximately 65 employees
  4. Testwright's core product uses AI to generate test cases from code diffs and PR descriptions
    ✓ in input a system that reads your codebase, infers intent from diffs and PR descriptions, and generates meaningful test cases rather than boilerplate assertions
  5. Testwright uses large language models fine-tuned on code and test corpora
    ✓ in input large language models fine-tuned on code and test corpora
  6. Kenji leads a team of eight ML and applied-science engineers
    ✓ in input I lead a team of eight ML and applied-science engineers building the models that sit at the core of our product
  7. Testwright has built an internal benchmarking harness for evaluating generated tests against real bug-introducing commits
    ✓ in input We've built an internal benchmarking harness that runs generated suites against curated corpora of real bug-introducing commits, and that's become the backbone of how we ship
  8. Kenji is actively exploring expansion from unit tests into integration and end-to-end test generation
    ✓ in input how we extend coverage from unit tests into integration and end-to-end scenarios, which is a meaningfully harder problem
  9. Testwright's Series A was led by Bessemer Venture Partners and Index Ventures
    ✓ in input Series A led by Bessemer Venture Partners and Index Ventures
  10. Testwright's customers are mid-market and enterprise engineering teams
    ✓ in input Customers are primarily mid-market and enterprise engineering teams seeking to reduce manual test-writing overhead
Reasoning

Testwright is a Series A B2B SaaS company (~65 employees) shipping a deeply AI-native product — LLMs fine-tuned on code and test corpora that generate meaningful test cases from diffs and PR descriptions. This is squarely within the ICP's product shape requirement. Stage (Series A), headcount (65, within 20–250), and product shape (B2B SaaS with user-facing AI at the core) all score at or near 1.0. The target role is Head of AI, which is explicitly named in the ICP's target roles list, giving a perfect role_match. The only dimension with uncertainty is ARR: no figure is disclosed. A 65-person Series A backed by Bessemer and Index is plausibly within the $2M–$50M ARR window, but cannot be confirmed from the input, so arr_match is held at 0.5. The holistic fit score of 0.84 reflects four dimensions at or near full strength and one unconfirmed. Every claim in the output is grounded in a verbatim substring of the input. Because fit > 0.80 and all claims are grounded, the action is auto_add.

Chatpass

Action
auto_add ✓
Fit
0.85 (gold 0.93, |Δ| 0.08)
Industry
B2B SaaS ✓
Segment
Developer tooling / AI-assisted software engineering ✗
Seniority
Director ✓
Company size
51-200 ✓
Grounding
11/11 claims grounded (100.0%)
Draft hook

Kenji — your note about the evaluation problem resonated. Knowing whether a generated test is actually good rather than just syntactically valid is the question that breaks most internal benchmarking setups. The bug-introducing commit corpus approach is smart, but I'm curious how you're handling coverage of failure modes the corpus doesn't contain — novel integration paths, for instance, as you push into E2E. That's the exact space we work in with teams like yours. Worth 20 minutes to compare notes?

Claims (11)
  1. Testwright is at Series A stage
    ✓ in input Series A, ~65 people
  2. Company has approximately 65 employees
    ✓ in input Series A, ~65 people
  3. Kenji leads a team of eight ML and applied-science engineers
    ✓ in input I lead a team of eight ML and applied-science engineers
  4. The core product uses AI-generated test generation from code analysis
    ✓ in input a system that reads your codebase, infers intent from diffs and PR descriptions, and generates meaningful test cases
  5. Kenji built an internal benchmarking harness that evaluates test quality against real bug-introducing commits
    ✓ in input We've built an internal benchmarking harness that runs generated suites against curated corpora of real bug-introducing commits, and that's become the backbone of how we ship
  6. Evaluation of generated test quality is described as the hardest part of the job
    ✓ in input The hardest part of this job is not the modelling. It's the evaluation problem: how do you know a generated test is actually good?
  7. The Series A was led by Bessemer Venture Partners and Index Ventures
    ✓ in input We closed our Series A from Bessemer and Index
  8. The AI team was doubled post-Series A
    ✓ in input the fundraise let us double the AI team
  9. Kenji is actively expanding the product scope from unit tests into integration and end-to-end scenarios
    ✓ in input Right now I'm thinking a lot about how we extend coverage from unit tests into integration and end-to-end scenarios, which is a meaningfully harder problem
  10. Kenji has a PhD in machine learning focused on program synthesis from Carnegie Mellon
    ✓ in input PhD in machine learning at Carnegie Mellon focused on program synthesis
  11. Kenji spent four years at a mid-sized observability platform owning the anomaly-detection pipeline
    ✓ in input I spent four years at a mid-sized observability platform where I owned the anomaly-detection pipeline
Reasoning

Kenji Nakamura is a strong fit across all primary ICP dimensions: Series A stage (matches), ~65 headcount (matches), technical decision-maker and director-level role leading an 8-person ML team (matches), B2B SaaS developer tooling with user-facing AI as the core product (matches strongly). The only soft signal is ARR, which is unconfirmed but plausible given the stage, investor quality, and enterprise design partners mentioned. Kenji's profile demonstrates sophisticated technical thinking, active engagement with hard problems (evaluation, coverage expansion), and explicit openness to peer discussions. The draft hook directly addresses his stated highest-priority challenge (the evaluation problem) and uses his own framing and language, which signals genuine comprehension and positions outreach as a peer exchange rather than a pitch. High confidence this is a worthwhile conversation starter with a technical buyer who will engage authentically."