Implement the GRPO group-normalized advantage and the clipped token-level objective

Medium

RLHF & Preference Optimization3 / 7

Chapter 123 / 7

RLHF & Preference Optimization

Implement the GRPO group-normalized advantage and the clipped token-level objective

Medium

Implement the GRPO group-normalized advantage and the clipped token-level objective. [DeepSeek]

Target interface

grpo_advantage(rewards)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import torch
import torch.nn.functional as F

def grpo_advantage(rewards):
    raise NotImplementedError

def grpo_loss(logp_new, logp_old, advantages, mask, eps=0.2):
    raise NotImplementedError

⌘/Ctrl↵to submit

AI review

No review yet

Submit your solution for a verdict, a score out of 100 and concrete feedback. Any correct approach passes.

Chapter 123 / 7

RLHF & Preference Optimization

Implement the GRPO group-normalized advantage and the clipped token-level objective

Medium

Implement the GRPO group-normalized advantage and the clipped token-level objective. [DeepSeek]

Target interface

grpo_advantage(rewards)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.