Evals and benchmarks

Vendors that test models and agents with benchmarks and graded tasks.

154companies
$21.8Bpublished funding, 95 companies
2024median founding year
7with a partner program

Our coverage

Companies (154), by published funding

Figures as published by each company or the source on its profile. n/a means no source found.

CompanyWhat it doesBased inFundingPartner program
ScaleData-partnerships program for company data. Use the public program page, not outdated deal rumors.United States$15.9BYes
SnorkelSnorkel AI. Expert-built datasets, agent environments, benchmarks and evals, sold as products and custom services.United States$587Mn/a
MercorExpert-data vendor. Alternate bidder at $7.5 million in the Spirit Airlines de-identified data auction, per Business Insider.United States$483.6MYes
ArenaModel evaluation business built on large-scale human preference voting, sold to labs, enterprises and developers.United States$450Mn/a
HandshakeEmployer data-partnerships program; licenses professional documents, per the public page.United States$435.5MYes
LightwheelSells physics-accurate simulation assets, demonstration data and validation tools to robotics teams.United States$293Mn/a
AliceThe former ActiveFence. Sells safety and security datasets, RL environments and red teaming to model labs, plus guardrail and moderation software.Israel$280Mn/a
TaskUsOutsourcing firm running AI data operations: collection, labeling, RLHF, evals, red teaming and robotics data.United States$279Mn/a
BugcrowdCrowdsourced security testing company that also offers AI developers cyber training environments and an exploitation benchmark.United States$230Mn/a
TuringTraining data, evals and RL environments for coding, knowledge work and STEM, plus enterprise agent deployment.United States$225MYes
LabelboxLabeling platform company that now sells RL environments, expert preference data and evals to frontier labs.United States$189MNo
ClaruSupplies custom and licensed video, robot demonstrations and annotation for robotics and world model work.United States$170Mn/a
LatchLatchBio sells biotech cloud software and bioinformatics workflows, and builds evals for biology AI agents.United States$163Mn/a
Prime IntellectOpen stack covering RL environments, evals, hosted training and inference, sandboxes and GPU rental.United States$150M+n/a
Invisible TechnologiesSupplies expert training data, fine-tuning, evals and custom RL environments, plus enterprise AI services.United States$144Mn/a
micro1Expert network and company-data program (Data Payout). Take rates, when quoted, come from its public program page.United States$135M+Yes
EncordMultimodal data platform with managed collection, labeling and eval services for vision and physical AI teams.United States$110Mn/a
Defined.aiData marketplace with a public partnership program. Partner-revenue figures on that page are company claims.United States$85M+No
SamaHuman-in-the-loop labeling, validation and evaluation services for vision and generative AI teams.United States$84.84Mn/a
David AIDesigns, records, labels and licenses human audio datasets and evaluates speech models.United States$80.5Mn/a
TolokaHuman expert and synthetic data, agent environments, evals and red teaming for labs and developers.Netherlands$72Mn/a
XDOFCollects robot teleoperation data, runs real-world policy evals, and supplies robot systems and tooling.United States$70Mn/a
SuperAnnotateAI data platform plus managed expert services for labeling, training data, evals and RL environments.United States$67.5MNo
ProtegeReal-world data platform (healthcare and other modalities). a16z announced a $30 million round on January 8, 2026, and said the company already had a $25 million Series A.United States$65Mn/a
CentificProvides multimodal training data, annotation, evals and simulated enterprise RL environments to labs and corporate AI teams.United States$60MNo
FleetSells simulated practice environments and evaluation tools for AI agents to labs, cloud providers and enterprises.United States$60Mn/a
MeckaTurns recorded human activity into robotics training data and offers data tools and evaluation support.United States$60Mn/a
Mind LabBuilds custom benchmarks, task environments and post-training for pharma, finance, public sector and gaming clients.Singapore$60Mn/a
e2fRenamed TrustScale in 2026. Sells custom datasets, expert labeling, evals and RL feedback through its ArgusRL platform.United States$50Mn/a
Gray Swan AIAI security firm offering pre-release adversarial testing plus runtime filtering and monitoring.United States$50Mn/a
NewtonXVetted professional network that runs model evals, builds post-training data and audits labeling quality.United States$47Mn/a
ReveloDeveloper talent firm selling code-focused training data, evals, benchmarks and RL environments.United States$46Mn/a
Vals AIEvaluation software, expert benchmarks and review tools for testing models on practical domain tasks.United States$45Mn/a
Contra LabsUses professional creatives to judge model output and to build benchmarks, feedback data and recorded design workflows.United States$44.5Mn/a
Bespoke LabsBuilds company-scale RL environments and agent training and evaluation infrastructure for frontier labs and enterprises.United States$40Mn/a
HalluminateSells finance-focused computer-use RL environments, simulated business apps, data and agent benchmarks.United States$38.5Mn/a
ProlificParticipant and expert panel platform for collecting human feedback, training data and evals.United Kingdom$33.4Mn/a
Centaur AIBoston labeling company that scores competing human and AI annotators, sells evals, and licenses finished labeled datasets.United States$32.54Mn/a
LuelLicenses multimodal datasets and runs paid contributor campaigns to collect rights-cleared audio, video and sensor data.United States$31.7Mn/a
AfterQuerySells AI labs and enterprises a mix of expert-made and synthetic datasets, RL environments and custom evals.United States$30.5MYes
Moonlake AITurns text prompts or real-world captures into 3D simulation environments for robotics and autonomy research.United States$28Mn/a
Deccan AISupplies expert post-training data, human feedback, RL environments and evals, plus enterprise AI operations services.United States$25Mn/a
iMeritExpert annotation and human feedback vendor that also sells the Ango Hub workflow platform.United States$24.5Mn/a
Taste LabsPreference and reasoning data and eval environments for design quality, plus a Brand API for developers.United States$18.5Mn/a
ParetoSpecialist-made training data, evals and RL environments for frontier labs.United States$17.8Mn/a
DatacurveExpert coding data vendor offering RL environments, benchmarks and evals to model labs and developer tool companies.United States$17.7Mn/a
Perle AISpecialist-generated multimodal data, labeling, RLHF and evals through expert-led programs.United States$17.5Mn/a
HUDDeveloper kit and cloud service for creating, running and scoring RL environments, plus a marketplace where vendors sell them.United States$16Mn/a
SapienHuman-created and expert-reviewed data, labeling and output quality checks for AI developers.Canada$15.5Mn/a
MillionResearch lab building front-end coding data, RL environments and evals, including the ReactBench benchmark.per its Y Combinator profile and SEC filing$14.15M over 3 rounds, per CB Insightsn/a
General ReasoningHosts RL environments behind an API, plus infrastructure to deploy them and to train or test agents at scale.United Kingdom$10.9Mn/a
Arga LabsBuilds stateful copies of business software so agent developers can train and test against realistic workflows safely.United States$10.5Mn/a
Chakra LabsWrites expert task sets and graders, including computer-use environments, for model training and testing in finance, law and other fields.United States$10.1Mn/a
BioStack PlatformsSells clinical and preclinical data, healthcare training environments and eval tools to medical and drug discovery AI teams.United States$10Mn/a
CollinearSells simulated environments, agent evals and synthetic training data to labs and enterprise teams building agents.United States$10MNo
IdlerSells RL environments, benchmarks and eval sets built from real production work.United States$9.5Mn/a
MechanizeBuilds simulated software engineering tasks and graded benchmarks for frontier labs training coding agents.United States$9.1Mn/a
IncalmoBuilds RL cyber ranges and security evals for training and testing cybersecurity-capable models.United States$8.7Mn/a
Veris AISimulation and RL platform that mimics enterprise systems and the people who use them, for agent training and testing.United States$8.5Mn/a
Gradient HealthMedical data company whose Atlas product offers de-identified scans and clinical records to healthcare AI developers.United States$8.37Mn/a
PolymathBuilds environments, tasks and verifiers for long-horizon agents, including the Horizon-SWE benchmark.United States$8Mn/a
IntelligenceOperates DesignArena, a voting site for AI-made sites, games and media, and sells model evals to labs.United States$7.9Mn/a
RunloopDevelopment sandboxes, orchestration and benchmarking tools for building and tuning coding agents.United States$7Mn/a
ApturaProduces expert training and eval data, benchmarks and RL environments for finance, healthcare and legal AI.United Kingdom$6.7Mn/a
Cortex AIProvides workplace video, robot trajectories and supervised robot rollouts that show recovery from failure, for embodied AI labs.United States$6.5Mn/a
Huzzle LabsSells RL environments, expert data, evals and custom models for agents that handle real business software.United Kingdom$5.98Mn/a
Vetto AISells labs expert-written post-training data, human evaluation work, custom RL environments and verifier-scored benchmarks.United States$3.64Mn/a
Good Start LabsRL environments built on video games, plus gameplay training data and public benchmarks.United States$3.6Mn/a
Paper InstrumentsResearch shop that fine-tunes language models for knowledge-heavy work and builds the evals, tooling and agent software around them.United States$3.6Mn/a
Besimple AISupplies licensed conversational audio, expert labeling and benchmarks to teams building speech and voice models.United States$3.5Mn/a
PlatoBuilds simulated software environments from real data for training and scoring browser and computer-use agents.United States$2.5Mn/a
Vibrant LabsRL-ready simulated environments, tasks and benchmarks for browser, computer and enterprise agents.United States$2.5Mn/a
QuesmaPublishes coding agent benchmarks, builds RL environments, and sells agent cost and workflow analytics.Poland$2.3Mn/a
Synthetic SciencesAI research workbench for literature review, experiments and drafting, plus scientific evals.United States$1.9Mn/a
BenchFlowBuilds simulated workflow environments and benchmarks for agents, and also runs post-training as a managed service on its own runtime.United States$1Mn/a
StickerBuilds RL environments and benchmarks for procurement work such as backorders and invoice mismatches.United States$1Mn/a
EmulatedProvides post-training data, production-style engineering environments and benchmarks for coding and ops agents.United States$850Kn/a
AbundantBuilds reinforcement learning environments, benchmarks and datasets for labs and companies working on AI agents.United States$500Kn/a
Andon LabsBuilds custom evals and benchmarks that put AI agents through simulated and real business, spatial and robotics tasks.United States$500Kn/a
ArchalSells resettable sandboxes that imitate third-party software APIs so teams can train and test agents without live accounts.United States$500Kn/a
AviroSells RL environments, post-training data and evals for long-running agents across coding, computer use and office work.United States$500Kn/a
CoarenaHosts a public site where people compare two computer-use agents on the same task, and sells the vote data to AI labs.per its site (San Francisco per its Y Combinator profile)$500Kn/a
CuaAgent infrastructure company offering desktop sandboxes and runtimes for computer use, plus RL environments, benchmarks and trajectory data.USA$500Kn/a
CueBenchBuilds RL environments and benchmarks on hard science reasoning and performance engineering problems.United States$500Kn/a
EdotEnvOffers RL environments and benchmarks built on financial markets for quantitative research agents.United States$500Kn/a
EnactFinds where robot models fail on real hardware, records recovery demos, and fine-tunes and evaluates the models.United States$500Kn/a
MaingenBuilds industrial operations simulations and RL environments, such as running solar portfolios or factories.United States$500Kn/a
MarkovSells computer-use datasets and environments built from expert CAD, office, coding and gaming workflows.United States$500Kn/a
MonteInfrastructure for continual learning and post-training, used to test models on work tasks and improve specialized agents.United States$500Kn/a
Ooak DataY Combinator company that describes turning internal company data into a training ground for AI agents.France$500KNo
RefreshSimulated software environments, verifiable evals and real-workflow coding data for agent training.United States$500Kn/a
TensorTestPrivate evals, RL environments and expert data for creative agents, starting with video editing (TimelineBench).United States$500Kn/a
ThetaBuilds custom simulations of business workflows and helps labs and enterprises train specialized agents on them.United States$500Kn/a
VmaxMarkets Campaign, a self-play platform that builds training environments from a developer's own data and evals.United States$250Kn/a
AnthromindRuns managed projects that deliver curated preference and training data, custom model evals and fine-tuning support, using experts.United States$200,000n/a
AI AstrolabeData and evaluation vendor centered on Arabic dialects and other underserved languages, covering text, speech, images and safety data.United Statesn/an/a
AIChampRuns robot policies through simulated environments and physics-checked tests, then reports scores, recordings and failure analysis to robotics teams.United Statesn/an/a
AITraining.jobsExpert data shop run by O-mega Enterprise that staffs US professionals on evals, rubrics, preference data and RL tasks for AI developers.Delaware)n/an/a
Align AISupplies expert data, simulated RL environments, ongoing evals and runtime checks for agents built for regulated work.United Statesn/an/a
AndromedeTurns real-world data into tasks, verifiers and RL environments used to train and test agents on long jobs.Switzerlandn/an/a
Apex Data SciencesAI unit of Apex CoVantage, started in 2024, selling expert RLHF and SFT data, benchmarks and multimodal annotation.United Statesn/an/a
AppenHuman data vendor. Labs and product teams buy its labeling, expert review, model evaluation and RL environment work.AustraliaPublicNo
Archangel HealthMedical AI data vendor offering EHR sandbox RL environments, physician-written answers and rubrics, de-identified clinical data and labeling.per its site (Y Combinator lists San Francisco)n/an/a
ARIMLABSCreates simulated workspaces and verifiable benchmarks for agents doing security, incident response and software tasks.Polandn/an/a
CNTXT AIAbu Dhabi data shop sourcing, labeling and evaluating training data, including Arabic speech and ranked model responses.United Arab Emiratesn/an/a
Consequence LabsProduces custom human data such as workflow demos, speech, expert judgments and labels, recruiting and managing contributors.n/an/an/a
DARE LabsSells Cerebro, software that turns company operational data into training sets, RL data and outcome-graded environments.United Statesn/an/a
DataOcean AISells stock datasets and custom collection, labeling and evaluation for speech, vision, LLM and driving models.United Statesn/an/a
DeepLLMDataXlairs Inc. brand making expert training and eval data, including repository-level coding trajectories with tests.per its site; it also lists Mumbai and Delhin/an/a
DisseiFinance-focused vendor selling RL environments, benchmarks, transaction data and analyst trajectories.n/an/an/a
Eigent AIOpen-source multi-agent desktop software maker that also builds custom RL environments and benchmarks for enterprises.United Kingdomn/an/a
exponentialSells multi-agent security environments, security training data and adaptive red team testing.Germanyn/an/a
FasterLabelingVira Hub LLC brand that builds made-to-order RL environments with programmatic graders, shipped as Docker images.is a Wyoming limited liability company)n/an/a
Foundry AICreates expert-reviewed coding and technical workflow data, RL task suites and private evals.Indian/an/a
G2iRuns managed coding data projects and places contract engineers inside lab teams for post-training work.United StatesSelf-fundedn/a
GLGExpert network whose GLG Frontier practice, launched in 2026, recruits credentialed professionals for AI lab data work.United Statesn/an/a
GroundTruthPune company licensing benchmarks, including the BFSI-Bench Indian finance regulation test, and running custom evals.Indian/an/a
Humyn LabsCollects and annotates first-person video, motion signals and speech for AI labs and robot makers.United Statesn/an/a
HyperPrior ResearchSmall London firm building environments, graders and agent test suites on fixed-scope or retainer terms.United KingdomSelf-fundedn/a
InnodataData engineering firm supplying training data, human feedback, RL environments, evals and red teaming.United Statesn/an/a
LightWrkProduces expert reasoning labels, preference signals and eval workflows for robotics and world model teams.n/an/an/a
LogitlyHong Kong firm that generates agent training traces with AI and checks them automatically, including DPO trace pairs.per its siten/an/a
LXTCustom and stock training data, annotation, evaluation and RLHF across text, speech, images and video.Canadan/an/a
MaxinsightsRecords and labels first-person video and body motion of people at work for physical AI and robotics customers.United Statesn/an/a
MedcaseMatches licensed clinicians to label medical imaging, waveforms, video, audio and records for AI training.per its siten/an/a
MetaphiBuilds simulated workplaces, custom or off the shelf, along with datasets and evals for agent training.United Statesn/an/a
NavadayManages specialists who write examples, fix data and compare model answers across professional fields.n/an/an/a
NormalPublishes CAD benchmarks and datasets for hardware design AI, including CAD Arena.United Statesn/an/a
Olam LabsBuilds multi-agent simulations, behavioral evals and game-based data for studying social and safety behavior.United Statesn/an/a
OpenTrain AIMarketplace connecting AI teams with vetted experts for RLHF, evals, red teaming and labeling.United Statesn/an/a
PanoptiveMakes clinical trial operations agents and publishes ClinOpsBench, a simulated trial oversight environment.per its YC pagen/an/a
ParsewaveWrites custom expert datasets, solution traces and evals for coding and professional domain models.United Statesn/an/a
Poindexter LabsExpert labeling and eval services plus syncronus, a platform for expert-led model training.n/an/an/a
pre.devSells graded coding environments and data built from licensed private code repositories.United Statesn/an/a
ProvisionReal-world multimodal data, physical-world evals and deployment support for robotics teams.n/an/an/a
ProximalBuilds coding tasks, data and RL environments for training software engineering agents.United Statesn/an/a
RaycasterEval programs, data and reproducible environments for agents working in documents and spreadsheets.United Statesn/an/a
ReadySolveLicenses MartechBench, a marketing workflow environment for RL and evaluation with deterministic checks.n/an/an/a
ReasonCoreExpert-curated datasets, RL environments and adaptive evals for labs and enterprises.n/an/an/a
Rise Data LabsHuman and synthetic data production, labeling and evaluation services for generative AI teams.United Statesn/an/a
Sepal AIExpert training and eval data, benchmarks and RL environments for frontier and enterprise model teams.United Statesn/an/a
Sherpa LabsSan Francisco builder of coding and finance RL environments, with tasks written by senior engineers and private equity professionals.United Statesn/an/a
SieveSupplies curated or freshly collected video, audio, images and interaction data, with labels, eval sets and environments.United Statesn/an/a
SurgeHuman-data and RLHF vendor.United Statesn/an/a
SymbalMultimodal post-training data, private evals and human-in-the-loop data for labs, plus enterprise agents.United Statesn/an/a
Tacit LabsExpert-written life science evals, agent environments and datasets for biology and chemistry work.United Statesn/an/a
TELUS DigitalTELUS unit supplying expert data, labeling, RL environments, red teaming, evals and physical AI datasets.CanadaPublicn/a
The General Data CompanyBrokers anonymized operating records and workflow data from European companies to AI labs.USAn/aYes
Tiki AICustom post-training data for frontier teams: coding RL environments, computer-use fine-tuning sets and tool-use environments.per its siten/an/a
Trajectory LabsSafety evals, expert red teaming and RL environments for finding and fixing model vulnerabilities.United Statesn/an/a
Uber AI SolutionsUber's AI data arm: collection, labeling, specialist RLHF, red teaming and evals for labs and robotics firms.United Statesn/an/a
UlamMath and STEM focused vendor of reasoning data, expert-checked solution traces, benchmarks and RL environments with verifiers.Polandn/an/a
Verita AISupplies vetted domain experts and project operations for labeling, feedback, eval and RL environment work.United Statesn/an/a
Xplore AIExpert-designed RL environments and evals for mechanical, electrical and aerospace engineering, including RWE-bench.n/an/an/a