Implement the GRPO group-normalized advantage and the clipped token-level objective

Medium

RLHF: Reward Models, PPO & GRPO3 / 7

Chapter 123 / 7

RLHF: Reward Models, PPO & GRPO

Implement the GRPO group-normalized advantage and the clipped token-level objective

Medium

Implement the GRPO group-normalized advantage and the clipped token-level objective. [DeepSeek]

Target interface

grpo_advantage(rewards)

Complete the starter code in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import torch
import torch.nn.functional as F

def grpo_advantage(rewards):
    raise NotImplementedError

def grpo_loss(logp_new, logp_old, advantages, mask, eps=0.2):
    raise NotImplementedError

Local draft

⌘/Ctrl↵to submit

AI review

Ready when you are

Submit your solution for a score and feedback on correctness, edge cases, and clarity. Any correct approach is accepted.

Chapter 123 / 7

RLHF: Reward Models, PPO & GRPO

Implement the GRPO group-normalized advantage and the clipped token-level objective

Medium

Implement the GRPO group-normalized advantage and the clipped token-level objective. [DeepSeek]

Target interface

grpo_advantage(rewards)

Complete the starter code in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.