XMarts room

Summer 2025

TraceRoot.AI

Open Source AI Agent Improvement Platform

Visit website

What they do

open-source tool that turns agent traces into feedback and evals

How close to us

5, very close

watches agent runs and feeds improvements back to a coding agent; direct match for seeing what agents do

Wedges

  • W2 Seeing whether your AI agents actually produce direct
  • W1 Trusted 24/7 coding for small startup teams adjacent
  • W8 The other 90% of an AI skill adjacent
L1 Company brain (knows the organisation) 1, light
L2 Agent team (runs agents, tools, approvals) 1, light
L3 Does and learns (finishes, checks, measures, updates the brain) 2, clear

Who buys

Firm size
sme
Buyer type
developer
Role
engineer or head of engineering who runs AI agents in production
UK
not known
Their own description (from YC)

TraceRoot is the open-source AI agent improvement platform. Turn production traces into actionable feedback and evals, closing the improvement loop with your coding agent.

Full teardown

TraceRoot.AI (YC Summer 2025): teardown vs XMarts

In one line: TraceRoot is an open-source tool that records what a company's AI agents do in production, flags the runs that went wrong, explains why against the code, and hands the finding to a coding agent to fix [site] [blog]. Threat: 2, because it sits on our W2 ground (seeing whether agents really produce) for a similar developer buyer, but it is tiny, young, and stops at "you review and act" [blog] [yc].

1. Fact file

Item Finding Source
What they say "Your agent passed the tests." "Production found the gaps." Tagline: "Open Source AI Agent Improvement Platform." [site]
Founded, team, funding Founded 2025, YC S25, San Francisco, 4 people, YC partner Jon Xu. Funding: no primary announcement found. Aggregators say about $500K (UNVERIFIED; likely the standard YC deal, inference). [yc] [press]
Founders Xinwei He (CEO): ex Research Scientist at Meta, ex Founding Engineer at Kumo.AI, Stanford CS, core contributor to PyTorch Geometric [yc]. Zecheng Zhang co-wrote the Show HN post and is listed as an author, but YC shows one founder only [hn] [press]. What YC likely saw: a technical founder with open-source credibility, building in public (inference). [yc] [hn]
Customers Logos on the site: Vector Legal, Perfectly, DINQ, Trident. No case studies found. They claim 40+ integrations, 73+ contributors and about 800 GitHub stars (808 stars, 278 forks seen on GitHub). Earlier YC launch claimed the SDK passed 10,000 downloads. [site] [github] [yc]
Price Public. Free $0 (50k events a month, 15-day retention). Starter $30 a month. Pro $200 a month. Enterprise custom. Overage billed per 50k events or per 100 AI investigation runs. Bring your own model keys on all plans. [site]
Path Show HN (2025): "open-source agentic debugging for distributed services" (MIT) → YC launch: "AI agents that automatically fix your production bugs" → open-sourced SDK, LangGraph and LangChain support, Product Hunt launch (Dec 2025 to Feb 2026, dates from a tracker) → Sept to Oct 2026: "AI Agent Improvement Platform" with Detectors, Signals and a CLI for coding agents. Licence on GitHub now Apache 2.0 with a separate enterprise licence for ee folders. [hn] [yc] [press] [blog] [github]

2. Their proposition, reverse engineered

Part What they do
Who buys Engineering teams that already run AI agents in production and use alerting; the blog does not name a buyer, so this is (inference) [blog]. Job ads show a founding GTM hire, so they are still finding the buyer [jobs].
The pain Too many traces and no idea which runs were wrong. In commenters' words: "collecting was never the hard part." Evals only cover failures you already thought of [blog].
The promise Turn production traces into feedback and evals, and close the improvement loop with your coding agent [site] [yc].
How it works SDKs (Python, JS/TS, OpenTelemetry) record model and tool calls. "Detectors" are AI judges that sample 25% of runs by default and flag failures that look like successes. Flagged runs group into "Signals". A background agent reads the trace plus the code at the production commit and recommends a change down to file and line. A CLI lets Claude Code or Codex pull the same data. Offline evals compare versions to a baseline [site] [blog].
The output A grouped list of recurring failures with a root cause, a suggested code change, and optionally a pull request to review. Eval datasets from confirmed failures [blog].
Proof style Counts of downloads, stars, integrations and contributors. No customer money or time saved [site] [yc].
Who sells it Self-serve with a free tier, plus a book-a-call link and a founding GTM role [site] [jobs].
Price model Seats and event volume, tiers at $0, $30, $200 a month; run-based overage [site].
The story Their enemy is failure that looks like success: a tool call returns 200 and the answer is still wrong. Their belief is that the fix should be made where the code lives, in your coding agent, by a human who reviews it [blog].

3. Positioning

  • Category they claim: Open source AI agent improvement platform [site].
  • Category a YC partner would put them in: open-source LLM and agent observability (the Langfuse, Arize, Braintrust space), with an agent that suggests fixes (inference).
  • What they want to be known for: the open-source way to turn production agent failures into fixes and tests.
  • Where they sit on the map: layers L1/L2/L3 = 1/1/2, buyer = sme, developer.

4. Three layers vs ours

Layer Them (score + evidence) Us (Marton's logs show)
L1 Company brain 1. Uses traces and the repo at the production commit as context for each investigation [blog]. No lasting memory of the company. The 2025 Show HN listed "persistent memory of past incidents" as a plan; no shipped sign found [hn]. A vault of about 1,400 pages, 112 call transcripts and about 8,700 customer quotes, searchable by meaning.
L2 Agent orchestration 1. One in-app investigator agent plus AI judges. It can open a pull request but cannot merge one [blog]. About 5,600 jobs run by agents from three makers, each checked by a different AI, then automatic tests.
L3 Execution and learning 2. Does part of the work (suggests the change, can open the PR) and measures with offline evals [blog]. But the blog says verification is manual for now, nothing ships automatically, and it does not say the agent learns from accepted or rejected changes [blog]. Every failure gets a written record and an automatic check (about 160 records). Offer feedback lines fell from 44 to 18 as rules were added. Only about 15% of jobs reached the main code; merges were unreliable in October.

5. Same and different

Topic Same or different Them Us
Core belief Same Agents make work that looks done and is not; you need a second check [blog]. Same: Marton's logs show a different AI checks every job.
Problem Same Find silent failures in agent runs [blog]. Marton's logs show a heartbeat reports what is stuck, what silently stopped and what it costs.
Whose agents Different Customers' own product agents, in production [site]. Coding agents working on the code of the business (inventory).
Where it checks Different Reads traces after the run, with AI judges [blog]. Checks before the work lands: review by another AI, then tests, then merge.
Who fixes Different A human reviews, a coding agent proposes [blog]. The system finishes the job and lands it; Marton approves rules.
Learning Different Failures become eval cases; no sign the system updates itself [blog]. Marton's logs show rule changes added from failures and feedback.
Delivery Different Open-source software, cloud or self-hosted [github]. A system run by the founders. Nobody else has run it.
Proof Different Stars, downloads, integrations [site]. No customer number yet. Own-business logs only.
Price Different $0 to $200 a month public tiers [site]. Not set yet.
Size Same 4 people [yc]. Two founders.

6. Wedge overlap

Wedge Fit Why
W2 Seeing whether your AI agents actually produce direct Same problem: what failed silently, what it cost. Same kind of buyer: engineers running agents. They trace Claude Code sessions and have a terminal tool for coding agents [site] [changelog].
W1 Trusted 24/7 coding adjacent They sit beside coding agents, feeding them findings. They do not run a coding fleet [blog].
W8 The other 90% of an AI skill adjacent Closest to their loop: production failure → finding → eval → fix. They do not mention skills or upkeep of skills in the posts and changelog read (UNVERIFIED that none exists) [blog] [changelog].

7. Whitespace (where they stop)

  • Layer gap: they watch and suggest. The human merges, and the system does not change itself from accepted or rejected fixes [blog]. To own it we need a log where accepted and rejected work changes the next run, and a falling rework rate.
  • Buyer gap: they watch the agents inside a customer's product. They do not watch a team's coding agents as a fleet (queue, merge, land) [site]. To own it we need one startup team running coding agents and saying "we cannot tell what is stuck."
  • Number gap: no money or time result is public [site]. To own it we need one before/after number such as merged work per week.
  • Price gap: none. Their free tier and $30 start undercut anything we would charge. Do not compete on price.
  • Speed gap: not clear. Self-serve signup suggests minutes to first trace, but first useful finding depends on traffic (inference) [site].

8. For the YC application

  • Proven today: Marton's logs show his system already tracks what is stuck, what silently stopped and what it costs across coding agents, and every failure has a record with an automatic check (about 160 records). It runs on his own business only.
  • Draft answer to "how are you different from TraceRoot?": DRAFT. "TraceRoot watches your product's agents and suggests fixes for a human to review. Our system watches and runs the coding agents themselves, and each failure becomes a check that stops it coming back." Founders say it their own way.
  • Future claim (needs proof): "Accepted and rejected work makes the next run better." Needs one team's agent work run for 4+ weeks, with rework falling and a log of rule changes tied to checked runs.
  • What we understand that they don't: the pain is agents that look done and are not, and the fix is a loop that lands the work and updates itself, not a better dashboard. needs proof on a customer; Marton's logs show it on his own work.
  • Copyability (M3): easy. They already trace Claude Code and ship a feature roughly every 1 to 3 weeks [changelog]. Adding a "stuck, silent, cost" view for coding agents is a feature for them and fits their developer buyer (inference). The harder part (queue, review and merge state) needs data they do not hold. Not hard, so M3 can not score above 3 against TraceRoot.
  • Why now (M4): None found in this teardown. Do not use "agents are everywhere now": that is general AI progress.
  • Do not claim: "first" or "only" tool for watching agents (this space is crowded and TraceRoot is open source). That we beat them on any number (no customer run). That our loop learns from rejected work at scale (Marton's logs show thin loop history).

9. Ideas worth taking

  1. Findings that group into one recurring problem, with one root cause each ("Signals"). It cuts alert noise and is easy to explain [blog].
  2. Hand data to the coding agent through a terminal tool, so the fix happens where the code lives [blog].
  3. Turn each confirmed failure into a test case, and write up buyer pain from public forum comments (their 1,085-comment Reddit study) as cheap research [blog].

10. Unknowns

  • Real funding and round date: aggregators disagree; look for a founder post or a Form D.
  • What the four named customers pay or use. Look for case studies on their blog or the customers' sites.
  • Whether they plan persistent memory or skill upkeep: the 2025 Show HN said memory was planned [hn]. Check the roadmap in the GitHub issues.
  • UK or European customers: not shown anywhere read.
  • Whether "Jev" in their detector post is the same product as the Jev model in Marton's system. The post names TypeSafe AI as its maker [blog]; Marton's logs show a "Jev" typed decision model in his own system (inventory), but whether they are one product is UNVERIFIED. If it is, TraceRoot is a possible partner or integration, not just a rival.

Sources

Technical detail
Slug
traceroot-ai
Lens hash
179096c479
Pass
deep
Stale
no