Multimodal & Vision-Language Models3 / 3
Multimodal & Vision-Language Models
Implement cross-attention adapter layers letting text tokens attend to frozen vision features with gated residual injection.
class GatedCrossAttnComplete the starter code in the editor. Any correct approach is accepted.
import torch
import torch.nn as nn
class GatedCrossAttn(nn.Module):
def __init__(self, d_model, n_heads, d_vision):
raise NotImplementedError
def forward(self, x, vision, vision_mask=None):
raise NotImplementedError⌘/Ctrl↵to submit
Submit your solution for a score and feedback on correctness, edge cases, and clarity. Any correct approach is accepted.
Multimodal & Vision-Language Models
Implement cross-attention adapter layers letting text tokens attend to frozen vision features with gated residual injection.
class GatedCrossAttnComplete the starter code in the editor. Any correct approach is accepted.