Easy

Given a merge list, implement a BPE tokenizer for a string

Tokenization & Embeddings · Problem 1 of 4

Chapter 02Tokenization & Embeddings

Given a merge list, implement a BPE tokenizer for a string

EasyProblem 1 / 4

Given a merge list, implement a BPE tokenizer for a string.

Implement the function/class skeleton in the editor. Any correct approach is accepted.

Hints