Skip to main content
An RLHF evaluation makes HeyBee the human side of a training loop: collect preferences on a checkpoint, train, bring the next checkpoint, repeat. Create an evaluation with Build an RLHF training loop as its objective. In Feedback, choose Compare outputs or Score outputs, then choose Overall quality or Multiple criteria. Turn on Mix response formats only when individual criteria need different response types. Then set the round policy: decisive pairs per criterion, a credit budget, and the queue strategy. HeyBee verifies trainability before the first round opens, so you find gaps before spending votes. Each round tracks its checkpoint, its evidence target, and its completion state. If a round fails, retry it; collected preferences and scores are never lost. Before export, consistency and reliability diagnostics flag votes that should be excluded, so you see exactly what goes into the training set and why. Exports are training-ready and reproducible: raw preferences, score observations, exclusions with their reasons, and the policy that produced them. Run the whole loop from the app, the Python SDK, the CLI (heybee rlhf status, policy, export), or the heybee-run-rlhf Agent Skill.