Super-hard

Implement an IsoFLOP analysis: from loss curves at several fixed-FLOP budgets, extract the

Pretraining Objectives & Scaling Laws · Problem 4 of 4

Chapter 04Pretraining Objectives & Scaling Laws

Implement an IsoFLOP analysis: from loss curves at several fixed-FLOP budgets, extract the

Super-hardProblem 4 / 4

Implement an IsoFLOP analysis: from loss curves at several fixed-FLOP budgets, extract the compute-optimal frontier and fit $N^{\star}(C)$ , $D^{\star}(C)$ .

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints

solution.pypython

local draft

import numpy as np

def synth_isoflop(C, Ns, coeffs=(1.69, 406.4, 410.7, 0.34, 0.28), noise=0.0, rng=None):
    raise NotImplementedError

def isoflop_minimum(Ns, losses):
    raise NotImplementedError

def fit_frontier(Cs, N_stars, D_stars):
    raise NotImplementedError

def run_isoflop_analysis(Cs, N_range, seed=0):
    raise NotImplementedError

⌘/Ctrl + ↵ to submit

AI review

Ready when you are

Submit your solution and a structured review appears here — verdict, score, and concrete feedback. Any correct approach passes.

Chapter 04Pretraining Objectives & Scaling Laws

Implement an IsoFLOP analysis: from loss curves at several fixed-FLOP budgets, extract the

Super-hardProblem 4 / 4

Implement an IsoFLOP analysis: from loss curves at several fixed-FLOP budgets, extract the compute-optimal frontier and fit $N^{\star}(C)$ , $D^{\star}(C)$ .

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints