Super-hard

Implement an automated regression harness that compares outputs, logits, losses, rewards,

Research Engineering & Debugging · Problem 8 of 8

Chapter 17Research Engineering & Debugging

Implement an automated regression harness that compares outputs, logits, losses, rewards,

Super-hardProblem 8 / 8

Implement an automated regression harness that compares outputs, logits, losses, rewards, and eval metrics across code changes.

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints

solution.pypython

local draft

import json, hashlib, torch

def fingerprint(model, batch, reward_fn, eval_fn, seed=0):
    raise NotImplementedError

def compare(golden, current):
    raise NotImplementedError

def regression_test(model, batch, reward_fn, eval_fn, golden_path):
    raise NotImplementedError

⌘/Ctrl + ↵ to submit

AI review

Ready when you are

Submit your solution and a structured review appears here — verdict, score, and concrete feedback. Any correct approach passes.

Chapter 17Research Engineering & Debugging

Implement an automated regression harness that compares outputs, logits, losses, rewards,

Super-hardProblem 8 / 8

Implement an automated regression harness that compares outputs, logits, losses, rewards, and eval metrics across code changes.

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints