OPEN SOURCE, OPEN TO EVERYONE

Open source. Open possibilities.

Discover quality open-source projects, submit projects anonymously, and claim and edit your own project.

Human-curated · Discover open source4109discovered

A little curiosity. A world of open source.

THE FIRST COLLECTION
Topic: tokenizer清除

Official code repository for 'Build a Large Language Model (From Scratch)': step-by-step PyTorch implementation of GPT-like models, covering pretraining and finetuning with notebooks and exercises.

AI & MLTraining & evaluation
swift-tokenizersDePasqualeOrg
ADDED

Swift Tokenizers is a high-performance Swift wrapper around Hugging Face's Rust tokenizers crate, focused solely on tokenization without Hub dependencies. It supports macOS, iOS and Linux, offering encoding, decoding, streaming detokenization, chat templates and tool calling.

AI & MLDeveloper toolsModel runtime