Transformer Architecture Internals · Problem 6 of 7
Implement a single MLA layer (down-proj to latent, up-proj, decoupled RoPE).
Implement the function/class skeleton in the editor. Any correct approach is accepted.
import torch
import torch.nn as nn
import torch.nn.functional as F
class MLA(nn.Module):
def __init__(self, d, n_heads, d_c, d_c_q, dh_nope, dh_rope):
raise NotImplementedError
def forward(self, x, cos, sin):
raise NotImplementedErrorReady when you are
Submit your solution and a structured review appears here — verdict, score, and concrete feedback. Any correct approach passes.
Implement a single MLA layer (down-proj to latent, up-proj, decoupled RoPE).
Implement the function/class skeleton in the editor. Any correct approach is accepted.