cantonese-llm-lab

Cantonese-LLM-Lab is an open-source project centered on Cantonese large language model fine-tuning, evaluation, and adaptation to domestic Ascend hardware, primarily aimed at researchers and engineers who want to reproduce or improve Cantonese question-answering models. The project does not rely on LLM-based review; instead, it provides a reusable objective evaluation framework: full HKMMLU official-benchmark evaluation, official Cantonese-Mandarin mutual translation tasks, written Cantonese purity, degeneration detection, and perplexity, along with per-item paired McNemar exact tests for all questions. The project consists of six main parts: first, the evaluation framework described above; second, a prompt ablation method that uses a 2×2 (language × format scaffold) design to test whether rankings are affected by prompt choice; third, a release gate employing non-inferiority tests and δ sensitivity checks; fourth, LoRA diagnostic tools that compute ‖ΔW‖/‖W‖ module by module; fifth, multi-source Cantonese SFT data reconstruction scripts, including simplified-traditional normalization, two-level deduplication, and split-before-training; and sixth, Ascend 910C adaptation records, covering migration from CUDA requiring only 4 lines of changes, native W8A8 quantization, negative results from graph-mode optimization, and cases where error messages do not match the actual root cause. The repository provides complete evaluation, training, and deployment scripts, including multiple evaluation and statistical tools under eval/, the data/build_yue_sft.py data construction script, LoRA training and merging scripts under train/, and CUDA and Ascend deployment guides under deploy/. The results documentation records detailed comparisons of the 8B and 30B-A3B model lines, prompt ablation, data ablation, and quantization experiments. Model weights are released in a gated repository on Hugging Face, including a bf16 merged version, an Ascend-native W8A8 dynamic quantization version, an int4 version, an ablation version trained only on public data, and LoRA adapters. The code is MIT-licensed, and the datasets used for evaluation follow their respective licenses.