Skip to main content
An evaluation is one question you want answered with human judgments. HeyBee supports two kinds:
  • Which option wins? Compare named candidates: models, prompts, checkpoints, or anything else you can produce outputs from. Best when you have a fixed set of alternatives.
  • Which settings win? Explore generation parameters, such as temperature or guidance scale, and let HeyBee recommend the best configuration.
Create one from Evaluations with New evaluation on a tablet or desktop. The wizard walks through four steps: basics, grouping, voting requirements (settings to test, in a settings evaluation), and review.

​
Basics

Pick a name, an optional note on what you are comparing, and the output type: text, image, audio, or video.
Video must be MP4 with H.264 video and AAC audio, the format nearly every editor and ffmpeg produce by default. HeyBee checks files on upload and tells you exactly what to re-export, but it does not convert them for you.
Then choose the workspace: Personal keeps the evaluation private to you, while a team workspace shares it with teammates according to their roles.

​
Group your uploads

The shared input, by default prompt, is how HeyBee knows which outputs compete against each other. Two outputs are only ever compared when they share the same input value. For example, if you upload variant_a and variant_b outputs for 50 prompts, voters always compare A against B on the same prompt, never across different prompts.
The shared input field, the output type, and the criteria cannot be changed after creation. The name, notes, and voting requirements stay editable until the evaluation is completed or archived.

​
Criteria

By default, voters answer one question: which output is better overall. Add up to five criteria when you want separate answers for different dimensions, for example Motion, Detail, and Prompt adherence in a video evaluation. Each criterion has:
  • a name voters see
  • a weight used for the combined ranking
  • optional guidance for voters, such as “Which one moves more smoothly and naturally?”
Each criterion gets its own leaderboard in Results. Voters judge all criteria in a single pass over each pair, so three criteria still cost one credit per comparison.
Two or three criteria is the sweet spot. Beyond that, each vote takes noticeably longer and voter attention drops.

​
Voting requirements

Control how carefully voters must look before they can submit:
  • Minimum viewing seconds holds the submit button until enough time has passed.
  • Minimum playback percent (audio) requires listening to both clips.
  • Allow playback speed control lets voters speed up long clips.
  • Require reviewer notes asks for a short written justification with every vote.

​
Readiness and lifecycle

An evaluation moves through Draft, Ready, Collecting, Paused, Complete, and Archived. Collection can start once three checks pass:
  • at least two outputs exist
  • every output has a shared input value
  • at least one input has outputs from two different candidates
Setup & collect lists anything missing, each with its fix. You can pause and resume collection at any time. Completed and archived evaluations become read-only but stay fully readable and exportable.