Implement a small end-to-end masked diffusion LM sampler with confidence-based remasking a

Super-hard

Diffusion & Non-Autoregressive Language Models4 / 4

Chapter 224 / 4

Diffusion & Non-Autoregressive Language Models

Implement a small end-to-end masked diffusion LM sampler with confidence-based remasking a

Super-hard

Implement a small end-to-end masked diffusion LM sampler with confidence-based remasking and compare its output/latency to a greedy AR baseline on a toy vocabulary.

Target interface

target(L)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.

solution.pyPython

Local draft

import time
import torch
import torch.nn.functional as F

def target(L):
    raise NotImplementedError

class Oracle:

    def __call__(self, x):
        raise NotImplementedError

@torch.no_grad()
def diffusion_sample(model, L, steps, mask_id=MASK):
    raise NotImplementedError

@torch.no_grad()
def ar_sample(model, L, mask_id=MASK):
    raise NotImplementedError

⌘/Ctrl↵to submit

AI review

No review yet

Submit your solution for a verdict, a score out of 100 and concrete feedback. Any correct approach passes.

Chapter 224 / 4

Diffusion & Non-Autoregressive Language Models

Implement a small end-to-end masked diffusion LM sampler with confidence-based remasking a

Super-hard

Implement a small end-to-end masked diffusion LM sampler with confidence-based remasking and compare its output/latency to a greedy AR baseline on a toy vocabulary.

Target interface

target(L)

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints0 / 2

Reference solution

Your own code stays in the editor.