SFT, Instruction Tuning & PEFT2 / 6
SFT, Instruction Tuning & PEFT
Implement a LoRA-wrapped linear layer (frozen trainable scaled by ). [HuggingFace]
class LoRALinearImplement the function/class skeleton in the editor. Any correct approach is accepted.
import math
import torch
import torch.nn as nn
class LoRALinear(nn.Module):
def __init__(self, base: nn.Linear, r: int=8, alpha: int=16, dropout: float=0.0):
raise NotImplementedError
def forward(self, x):
raise NotImplementedError
@torch.no_grad()
def merge(self):
raise NotImplementedError⌘/Ctrl↵to submit
No review yet
Submit your solution for a verdict, a score out of 100 and concrete feedback. Any correct approach passes.
SFT, Instruction Tuning & PEFT
Implement a LoRA-wrapped linear layer (frozen trainable scaled by ). [HuggingFace]
class LoRALinearImplement the function/class skeleton in the editor. Any correct approach is accepted.