Build a dataset-audit tool that flags duplicated prompts, suspicious templates, tool-call

Super-hard

SFT, Instruction Tuning & PEFT6 / 6

Chapter 116 / 6

SFT, Instruction Tuning & PEFT

Build a dataset-audit tool that flags duplicated prompts, suspicious templates, tool-call

Super-hard

Build a dataset-audit tool that flags duplicated prompts, suspicious templates, tool-call leakage, and evaluation-set contamination.

Target interface

_norm(t)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import hashlib, re
from collections import Counter

def _norm(t):
    raise NotImplementedError

def _hash(t):
    raise NotImplementedError

def audit(dataset, eval_prompts):
    raise NotImplementedError

⌘/Ctrl↵to submit

AI review

No review yet

Submit your solution for a verdict, a score out of 100 and concrete feedback. Any correct approach passes.

Chapter 116 / 6

SFT, Instruction Tuning & PEFT

Build a dataset-audit tool that flags duplicated prompts, suspicious templates, tool-call

Super-hard

Build a dataset-audit tool that flags duplicated prompts, suspicious templates, tool-call leakage, and evaluation-set contamination.

Target interface

_norm(t)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.