Hard

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Pretraining Objectives & Scaling Laws · Problem 3 of 4

All problems

Chapter 04Pretraining Objectives & Scaling Laws

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

HardProblem 3 / 4

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least squares in log space.

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints

solution.pypython

local draft

import numpy as np
from scipy.special import logsumexp
from scipy.optimize import least_squares

def make_synthetic(seed=0):
    raise NotImplementedError

def fit_scaling_law(N, D, L):
    raise NotImplementedError

⌘/Ctrl + ↵ to submit

AI review

Ready when you are

Submit your solution and a structured review appears here — verdict, score, and concrete feedback. Any correct approach passes.

Chapter 04Pretraining Objectives & Scaling Laws

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

HardProblem 3 / 4

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least squares in log space.

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints

Fit L(N,D)=E+A/Nα+B/DβL(N,D)=E+A/N^{\alpha}+B/D^{\beta}L(N,D)=E+A/Nα+B/Dβ to a synthetic (N,D,loss)(N,D,\text{loss})(N,D,loss) grid via least

Fit L(N,D)=E+A/Nα+B/DβL(N,D)=E+A/N^{\alpha}+B/D^{\beta}L(N,D)=E+A/Nα+B/Dβ to a synthetic (N,D,loss)(N,D,\text{loss})(N,D,loss) grid via least

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least