Implement SimPO (length-normalized, reference-free) and KTO losses and unit-test on toy da

Medium

Direct Alignment: DPO, Rejection Sampling & Distillation2 / 5

Chapter 132 / 5

Direct Alignment: DPO, Rejection Sampling & Distillation

Implement SimPO (length-normalized, reference-free) and KTO losses and unit-test on toy da

Medium

Implement SimPO (length-normalized, reference-free) and KTO losses and unit-test on toy data.

Target interface

simpo_loss(pi_logps_w, pi_logps_l, len_w, len_l, beta=2.0, gamma=1.0)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import torch
import torch.nn.functional as F

def simpo_loss(pi_logps_w, pi_logps_l, len_w, len_l, beta=2.0, gamma=1.0):
    raise NotImplementedError

def kto_loss(pi_logps, ref_logps, labels, beta=0.1, lam_d=1.0, lam_u=1.0):
    raise NotImplementedError

def _test():
    raise NotImplementedError

⌘/Ctrl↵to submit

AI review

No review yet

Submit your solution for a verdict, a score out of 100 and concrete feedback. Any correct approach passes.

Chapter 132 / 5

Direct Alignment: DPO, Rejection Sampling & Distillation

Implement SimPO (length-normalized, reference-free) and KTO losses and unit-test on toy da

Medium

Implement SimPO (length-normalized, reference-free) and KTO losses and unit-test on toy data.

Target interface

simpo_loss(pi_logps_w, pi_logps_l, len_w, len_l, beta=2.0, gamma=1.0)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.