Vendor shortlist
Top RLHF vendors in 2026
The companies labs and enterprises shortlist for RLHF, preference and expert demonstration data, compared on the things that decide a contract: who does the work, how it is priced, and where the data lives.
RLHF vendors sell three different things under one name: a managed crowd for high-volume preference labels, an expert network for demonstrations and evaluations in a specialist field, and tooling you staff yourself. The right pick depends on which of the three you need, and most serious buyers run two vendors against a gold set rather than betting on one.
The list below covers the vendors that come up in 2026 shortlists. Facts are taken from the vendors' own material and widely cited public sources; enterprise pricing is mostly not published, so get quotes.
vendors on the shortlist
with an EU entity
with published pricing
that tests experts on your data before they join
Vendor facts come from the vendors' own material and widely cited public sources as of October 2026; pricing is per quote unless the vendor publishes it. Pathwize is EU-native: all data stays within the EU.
How we compared
01
Workforce model
Open crowd, vetted crowd, curated expert network, or software only. This decides quality ceiling and price floor.
02
Pricing model
Per task, per hour, platform fee plus reward, or SaaS subscription. Hourly with time tracking makes cost per item visible; per task hides it.
03
EU data residency
Whether the vendor has an EU entity and can keep storage, processing and annotator access inside the EU. An EU parent alone is not residency.
04
Verification
How the vendor proves the data was produced by qualified humans: tests, recorded interviews, time per item, audit trail.
The shortlist
| Vendor | Model | Best for | Pricing | EU residency |
|---|---|---|---|---|
Scale AI San Francisco, US | Managed workforce plus expert programs, full-stack lab engagements Meta took a minority stake in 2025; several labs diversified away as a result, according to public reporting. | Large lab programs, government, physical AI | Per quote | Partial |
Surge AI San Francisco, US | Curated, vetted contractor workforce with careful task design Selective about clients; positioned at the top of the quality and price range. | Premium RLHF and expert feedback for frontier labs | Per quote; expert work reported at $85 to $200+ per hour | Partial |
Mercor San Francisco, US | Expert marketplace recruiting specialists in coding, law, medicine and more Reported revenue run rate above $1B in early 2026; a security incident was reported in March 2026. | Expert demonstrations and evaluations at scale | Per quote | Partial |
Toloka Amsterdam, NL | Crowd platform with separate expert tracks Dutch entity (Toloka AI BV), part of Nebius Group; Yandex origin is a perception question for some buyers. | High-volume preference data, pay-as-you-go | Pay-as-you-go, from cents per microtask to about $90 per hour on expert tracks | Yes |
Prolific London, UK | Verified research participants, 300k+ with prescreening filters Research-participant model rather than a managed delivery team. | Human preference studies and evaluations with demographic control | Participant reward plus platform fee; at least $12 per hour recommended | Partial |
Labelbox / Alignerr San Francisco, US | Software platform with an attached expert crowd (Alignerr) Strong tooling; you decide how much of the workforce to bring yourself. | Teams that want tooling plus a workforce on demand | SaaS subscription; Alignerr advertises $20 to $120 per hour to experts | Partial |
Pathwize Cologne, Germany | Credentialed experts and students from a 200k+ network, qualification test on your own data, paid per hour Smaller network than the US vendors; strongest when the answer depends on domain knowledge. | Expert feedback and evaluation data in regulated fields, EU residency required | Per hour, quoted per task; free pilot on 50 items | Yes |
Scale AI
San Francisco, US
Managed workforce plus expert programs, full-stack lab engagements
Best for: Large lab programs, government, physical AI
Pricing: Per quote
Meta took a minority stake in 2025; several labs diversified away as a result, according to public reporting.
Surge AI
San Francisco, US
Curated, vetted contractor workforce with careful task design
Best for: Premium RLHF and expert feedback for frontier labs
Pricing: Per quote; expert work reported at $85 to $200+ per hour
Selective about clients; positioned at the top of the quality and price range.
Mercor
San Francisco, US
Expert marketplace recruiting specialists in coding, law, medicine and more
Best for: Expert demonstrations and evaluations at scale
Pricing: Per quote
Reported revenue run rate above $1B in early 2026; a security incident was reported in March 2026.
Toloka
Amsterdam, NL
Crowd platform with separate expert tracks
Best for: High-volume preference data, pay-as-you-go
Pricing: Pay-as-you-go, from cents per microtask to about $90 per hour on expert tracks
Dutch entity (Toloka AI BV), part of Nebius Group; Yandex origin is a perception question for some buyers.
Prolific
London, UK
Verified research participants, 300k+ with prescreening filters
Best for: Human preference studies and evaluations with demographic control
Pricing: Participant reward plus platform fee; at least $12 per hour recommended
Research-participant model rather than a managed delivery team.
Labelbox / Alignerr
San Francisco, US
Software platform with an attached expert crowd (Alignerr)
Best for: Teams that want tooling plus a workforce on demand
Pricing: SaaS subscription; Alignerr advertises $20 to $120 per hour to experts
Strong tooling; you decide how much of the workforce to bring yourself.
Pathwize
Cologne, Germany
Credentialed experts and students from a 200k+ network, qualification test on your own data, paid per hour
Best for: Expert feedback and evaluation data in regulated fields, EU residency required
Pricing: Per hour, quoted per task; free pilot on 50 items
Smaller network than the US vendors; strongest when the answer depends on domain knowledge.
Where Pathwize fits
Pathwize is the EU option on this list. The company is based in Cologne, data stays in the EU, and experts work from Europe under clean contractor classification.
Every task starts with a qualification test on your own data, not on a generic sample. You see each candidate's answers, notes and time per item, and you approve who joins. Items can get a second rater on a sample, disagreements go to a reviewer.
Provenance is built in: each label carries who did it, when, and how long it took; AI interviews are recorded and checked for undisclosed AI use; exports come as CSV, JSON or COCO, with an API and webhooks for pipelines.
It is not a fit for everything. If you need a million crowd labels by Friday, a crowd platform is cheaper. Pathwize is for work where the answer depends on domain knowledge: medicine, engineering, law, finance, the sciences.
FAQs
For frontier-lab RLHF most shortlists name Surge AI, Scale AI and Mercor. For high-volume preference data, Toloka and Prolific. Labelbox is the tooling pick. Pathwize is the EU-based option for expert feedback in regulated domains.
Enterprise vendors rarely publish prices. Publicly reported figures range from cents per microtask on crowd platforms to $85 to $200 per expert hour at the premium end. Pathwize quotes per hour per task and shows cost per item in the dashboard.
Ask for the evidence per item: who labeled it, when, how long it took, and how the person was qualified. Pathwize records a qualification test on your data, time per item and recorded AI interviews with an integrity check.
Usually within a few days. You send the task, the classes or rubric and a few examples; qualified experts take the test on your data; you approve who continues. A 50-item pilot is free.
Per hour of expert work, quoted per task depending on the domain and the complexity. Time is tracked automatically per item, so you see cost per item and per class in the dashboard and in the export.
In the EU. Pathwize is based in Cologne, Germany; storage and processing run on EU infrastructure, and experts work from Europe. Sensitive data can be worked in a no-download environment.
Related