Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Hard

Pretraining Objectives & Scaling Laws3 / 4

Chapter 053 / 4

Pretraining Objectives & Scaling Laws

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Hard

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least squares in log space.

Target interface

make_synthetic(seed=0)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import numpy as np
from scipy.special import logsumexp
from scipy.optimize import least_squares

def make_synthetic(seed=0):
    raise NotImplementedError

def fit_scaling_law(N, D, L):
    raise NotImplementedError

⌘/Ctrl↵to submit

AI review

No review yet

Submit your solution for a verdict, a score out of 100 and concrete feedback. Any correct approach passes.

Chapter 053 / 4

Pretraining Objectives & Scaling Laws

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Hard

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least squares in log space.

Target interface

make_synthetic(seed=0)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.