Agents, Tool Use & Product Post-Training · Problem 7 of 7
Build a toy agent environment where the model can solve a task, call tools, fail safely, or reward-hack the evaluator; then add mitigations and show they work.
Implement the function/class skeleton in the editor. Any correct approach is accepted.
import types
def evaluate_naive(submission):
raise NotImplementedError
def evaluate_hardened(submission):
raise NotImplementedError
def _safe_run(t):
raise NotImplementedErrorReady when you are
Submit your solution and a structured review appears here — verdict, score, and concrete feedback. Any correct approach passes.
Build a toy agent environment where the model can solve a task, call tools, fail safely, or reward-hack the evaluator; then add mitigations and show they work.
Implement the function/class skeleton in the editor. Any correct approach is accepted.