Skip to main content
Put two AI outputs in front of a person and ask which one is better. That single judgment, collected carefully and at scale, answers most of the hard questions about a generative system:
  • Benchmark: which model, prompt, or checkpoint wins, and with what confidence.
  • Optimize: which generation settings produce the best output, and which parameters actually matter.
  • Train: every vote is a preference pair. Export them as training data for RLHF or DPO.
  • Ship safely: confirm a new version beats the one in production before you release it.
HeyBee runs that loop end to end. Define what you are comparing, share a voting link with anyone, and watch judgments turn into a clear answer: what wins, by how much, and what to do next. It works with text, images, audio, and video.

Quickstart

Go from a question to your first evidence-backed decision.

Create an evaluation

Define what you are comparing and how voters should judge it.

Collect votes

Share voting links and see exactly what voters see.

Read your results

Understand the verdict, the confidence score, and the next action.

​
Build with HeyBee

Run the whole lifecycle from code: create evaluations, upload outputs, open collection, and export results.

Python SDK

Typed sync and async clients.

CLI

Every workflow from your shell, with stable JSON output.

Agent Skills

Let coding agents run evaluations for you.