The news
Gorgias, an ecommerce customer support software company, is publishing AI customer service evals that rank its AI agent against rivals, including where it loses, SaaStr reported on September 26, 2026; the benchmark's public GitHub repository dates to July 3. Separately, AppDirect announced on September 28 that it had acquired Soul Machines, which makes AI avatars for customer interactions.
According to SaaStr founder Jason Lemkin, whose SaaStrFund led Gorgias's seed round and who says SaaStr pushed Gorgias to publish these evals, the benchmark covered 8,356 live conversations on 212 storefronts across 18 vendors as of September 26. In a September 15 Gorgias blog interview, senior vice president of product Max Pruvost described blind scoring, 26 yes-or-no checks and a trailing 90-day window ranking 12 vendors; the live board now shows 18 vendors tested and 13 ranked. SaaStr and the board say results refresh weekly; the Gorgias post says daily.
Gorgias counts a conversation as automated only when the AI finished it without a person stepping in and without sending the customer to another channel. The public GitHub project weights shopping-assistant scores 40% automation, 35% quality and 25% speed, and support scores 50% automation, 40% quality and 10% speed.
Lemkin wrote on September 26 that Gorgias ranked first in support but Yuma resolved more conversations, and that Envive led Gorgias in pre-sale shopping 72 to 65 on composite score; Gorgias scored 76 on answer quality but averaged 18.4 seconds per answer against Envive's 7.9, with 28% of its shopping answers taking over 20 seconds. The board has since moved: on September 29 it ranked Gorgias first overall, second in support behind Decagon (77 to 69, with Yuma at 67) and third in shopping, where Envive led 72 to 63. SaaStr put Gorgias at about $100 million in annual recurring revenue, roughly 80% of it from AI support.
AppDirect, a business-to-business subscription commerce platform, did not disclose what it paid for Soul Machines. The company said Soul Machines' Digital Brain and Autonomous Animation technology powers digital teammates that respond in real time across text, voice and video, and that it will run as a standalone business. AppDirect plans to embed the avatars in Devs.ai, its app-building platform, for uses including customer support, onboarding and employee help desks. AppDirect said its ecosystem includes more than 1,000 providers, 14,000 advisors and 16 million subscribers.
The numbers
- Live conversations in Gorgias benchmark (per SaaStr, September 26)
- 8,356
- Pre-sale composite score, Envive vs Gorgias (per SaaStr, September 26)
- 72 to 65
- Shopping composite score, Envive vs Gorgias (live board, September 29)
- 72 to 63
- Average answer time, Gorgias vs Envive (per SaaStr, September 26)
- 18.4 seconds vs 7.9 seconds
- Gorgias annual recurring revenue (per SaaStr, a Gorgias investor)
- About $100 million
- AppDirect advisors (company figure)
- 14,000
Why CEOs should care
For customer experience leaders, the Gorgias benchmark offers a template for vendor selection. Ask every shortlisted vendor to run the same scripted conversations on your own storefront or help center, and score them on automation, answer quality and speed with weights that match the job. A support deployment should weight resolution heavily; a shopping assistant needs fast answers. Demos rarely show what happens on the tenth turn of a messy conversation.
For CFOs, the definition of automation drives the business case. Gorgias counts a conversation as automated only if no person stepped in and the customer was not sent elsewhere, and buyers should insist on that standard in contracts. If a vendor charges per resolution, its definition of a resolution determines the bill, so get it in writing and audit a sample of transcripts each quarter.
For CIOs and boards considering avatar-based service after AppDirect's deal, the same evidence rules apply. Ask whether video and voice avatars resolve more issues than text agents, what they add in latency and cost, and how customers are told they are dealing with AI. Pilot avatars on a narrow use case, such as onboarding or an internal help desk, before putting them in front of every customer.
The bigger picture
Lemkin, whose fund is a Gorgias investor, rests his case for publishing evals on a change in how software gets bought: buyers discount marketing claims, and he wrote that more software evaluations now start with a question to an AI assistant such as Claude or ChatGPT. Public, reproducible results give those assistants something concrete to cite. The caveat is that Gorgias designs the rubric and runs the tests; its open-source harness lets rivals and buyers check the method, but it is still a vendor-run benchmark.
What’s next
AppDirect said it would share more on its AI strategy at its Thrive conference in Los Angeles, running September 29 to October 1, 2026. Watch whether rivals named in the Gorgias results, such as Decagon, Yuma and Envive, dispute the numbers or publish their own tests, and whether larger customer service software vendors follow with open benchmarks.
What “Fact-checked” means
Fact-checking means testing a story’s facts against the evidence before it is published. This story went through at least two separate checks before this version was published.
- What we checked
- Its names, figures, dates, job titles, quotes and who said what were checked against the story’s sources, including its main source where it could be opened. The headline was checked for accuracy and overstatement.
- How
- A first check reviewed the whole story. If it passed, a second, skeptical check went back to the sources to look for mistakes in the most important facts. If a check flagged the story, it was edited to fix the problems found, and a separate re-check then reviewed the whole story again.
- Who
- The checks are made by our newsroom, as steps kept separate from the writing, under rules set by our editor, Hussein Mukhtar. A story the checks still flag is held for the editor, who decides whether it is fixed, published or dropped.
- If something is wrong
- “Fact-checked” does not mean error-free. If a material error is found after publication, we correct the story and add a note saying what changed. Report an error
Companies in this story
