---
title: "AI research engineering"
description: "A personal research-engineering notebook. The Harbor Meter retrieval comparison uses prepared fictional rows and deterministic checks. It is not a model run and not a published result."
canonical: "https://www.ayushrai.site/pow/ai-research-engineering/"
last-updated: "2026-10-19"
---

# AI research engineering

Source: https://www.ayushrai.site/pow/ai-research-engineering/
Markdown: https://www.ayushrai.site/pow/ai-research-engineering.md

A personal research-engineering notebook. The Harbor Meter retrieval comparison uses prepared fictional rows and deterministic checks. It is not a model run and not a published result.

I turn AI research questions into reproducible experiments and working systems.

I work at the intersection of AI research, model evaluation, and engineering—translating hypotheses into experiments, implementing methods, and investigating what the evidence actually supports.

This is a prepared demonstration. No model was called. Harbor Meter Co. is fictional.

## Question

Does adding a fixed bundle of retrieved evidence change exact-match accuracy and prepared evidence labels on this question set?

Hypothesis: Under a fixed evaluation set and comparable generation settings, adding relevant retrieved evidence may improve evidence-supported answer accuracy.

Null hypothesis: On this set, the retrieval condition does not increase the exact-match rate relative to the baseline.

The hypothesis is not assumed true. The prepared rows are a methodology demonstration. They are not a test of the hypothesis on real model outputs.

## Prepared results

Rows: 12. Exact-match denominator: 9. Excluded: 3.

Baseline exact match: 6 of 9.

Retrieval exact match: 3 of 9.

Retrieval helps on 2 scorable rows, hurts on 5, and ties on 2.

Retrieval recall, micro-average over non-duplicate rows with a relevance set: 8 of 9 relevant ids.

These counts are descriptive. They are not a significance test and not a claim that retrieval works.

## Inspected work

- [OpenAI-RLHF](https://github.com/ayushrai-hub/OpenAI-RLHF): It is preference-artifact and unit-test evaluation work. It is not a reward-model experiment.
- [RLHF-tasks](https://github.com/ayushrai-hub/RLHF-tasks): Related to evaluation engineering. The benchmark proof-of-work page is the closer home for task design. It is listed here because the repository is part of the RLHF-named work I was asked to inspect.
- [Adaptive RAG](https://github.com/ayushrai-hub/adaptive-rag): It is a retrieval system I can read. It is not the Harbor Meter experiment, and it is not a result I am reporting.
- [Revenue Intelligence Engine](https://github.com/ayushrai-hub/REVENUE-INTELLIGENCE-ENGINE): Useful as an example of a deterministic score and a run id. It is a systems demo, not a scientific result about models.
- [AGENYRA](https://github.com/ayushrai-hub/AGENYRA_product): Related as structured representation and claim labels. It is not an AI research experiment.
- [SpiderWeb](https://github.com/ayushrai-hub/SpiderWeb): Included because I was asked to inspect it. It is a data product. It is not an AI research study, and I do not describe it as one.

RLHF-tas (https://github.com/ayushrai-hub/RLHF-tas) has no README. The tree looks like exported task bundles. I do not describe it as a system.

Paper used as a reading example, not as my result: [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401), Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela, NeurIPS 2020.

I have not published this experiment. The ORCID link on the site is an identity record, not a publication list.

_Unverified award metrics are omitted. Wording matches the public site._