---
title: "RLHF and preference feedback"
description: "Ayush Rai on RLHF-shaped work: human review and preference signals for model outputs, scoped to the experience record — not reward-model training."
canonical: "https://www.ayushrai.site/ai-engineering/rlhf/"
last-updated: "2026-10-16"
---

# RLHF and preference feedback

Source: https://www.ayushrai.site/ai-engineering/rlhf/
Markdown: https://www.ayushrai.site/ai-engineering/rlhf.md

Ayush Rai on RLHF-shaped work: human review and preference signals for model outputs, scoped to the experience record — not reward-model training.

Evidence level: Mixed: shipped pieces and explicit limits.

## Scope

RLHF, reinforcement learning from human feedback, is a family of training methods that turn human judgments into a learning signal. The public part of my work sits on the human-judgment side: reviewing outputs, applying criteria, and working on dataset quality for LLM training workflows.

I use 'RLHF-shaped' on purpose. The experience record says evaluation and model-reliability work. It does not say I designed a reward model, ran PPO, or owned an alignment research agenda.

## Where preference data goes wrong

Preference pairs are only as good as the instruction the reviewer was given. If the rubric rewards fluency, the signal teaches fluency. If two reviewers disagree and the disagreement is averaged away, the dataset hides the ambiguity.

For code and STEM tasks, a preference is a poor substitute for a failing test when a failing test exists. Human feedback still matters for the cases tests do not name: insecure-but-passing code, a right answer with a wrong explanation, a solution that ignores the constraint in the prompt.

## What the roles were

FoCDoT, April 2024 to August 2025: AI engineering and technical evaluation, including RLHF-oriented evaluation. Outlier, from August 2025: contract evaluation for frontier-lab programs, including dataset-quality and review loops. I do not list the labs as employers.

Older portfolio copy that implied direct collaboration with a lab, as opposed to work delivered through a vendor platform, was the thing the fact-check log told this site to stop saying. This page keeps that distinction.

## How I would evaluate a feedback loop

If I were publishing a loop of my own, I would want the guideline, a small set of double-reviewed items, and a written rule for what happens on disagreement. I do not have that packet for the contract work, so I do not invent the numbers.

## Evidence

- [FoCDoT role](/experience#focdot-sde-ai): RLHF-oriented evaluation is the phrase the experience record uses.
- [Outlier contract](/experience#outlier-genai): Dataset quality and review. Not a keynote role and not lab employment.
- [LLM evaluation](/ai-engineering/llm-evaluation): The rubric side of the same work.

## Related

- [LLM evaluation](/ai-engineering/llm-evaluation)
- [Evaluation is the job](/notes/eval-is-the-job)
- [AI engineering hub](/ai-engineering)


## Further reading

- [Deep reinforcement learning from human preferences](https://arxiv.org/abs/1706.03741): Christiano et al. Foundational paper. Not my experiment.
- [InstructGPT](https://arxiv.org/abs/2203.02155): The widely cited RLHF-for-language-models write-up.

## Gaps

- No reward-model training logs, preference UI, or policy-update results.
- No description of proprietary task content, guidelines, or customer names beyond what the experience page already says.
- Do not read this page as a research paper or as an alignment credential.

Contact: /contact

_Unverified award metrics are omitted. Wording matches the public site._