Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Hard

Pretraining Objectives and Scaling Laws3 / 4

Chapter 053 / 4

Pretraining Objectives and Scaling Laws

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Hard

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least squares in log space.

Target interface

make_synthetic(seed=0)

Complete the starter code in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import numpy as np
from scipy.special import logsumexp
from scipy.optimize import least_squares

def make_synthetic(seed=0):
    raise NotImplementedError

def fit_scaling_law(N, D, L):
    raise NotImplementedError

Local draft

⌘/Ctrl↵to submit

AI review

Ready when you are

Submit your solution for a score and feedback on correctness, edge cases, and clarity. Any correct approach is accepted.

Chapter 053 / 4

Pretraining Objectives and Scaling Laws

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least

Hard

Fit $L(N,D)=E+A/N^{\alpha}+B/D^{\beta}$ to a synthetic $(N,D,\text{loss})$ grid via least squares in log space.

Target interface

make_synthetic(seed=0)

Complete the starter code in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.