Easy

Implement sliding-window causal attention in PyTorch

Attention Efficiency & Long Context · Problem 1 of 5

Chapter 03Attention Efficiency & Long Context

Implement sliding-window causal attention in PyTorch

EasyProblem 1 / 5

Implement sliding-window causal attention in PyTorch.

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints