About this project

MoziAI-35B-V3.8 is a locally deployable open-source multimodal large language model developed by the Chen Yumo team. It is built on the Ornith-1.5-35B-A3B base (Qwen3.5/3.6-35B-A3B architecture, MoE with 256 routing experts plus 1 shared expert, 8 experts active per token) and is distributed as a GGUF file for use with llama.cpp, Ollama, LM Studio, and Jan. The headline feature is the self-developed MoziSmartBit hybrid quantization, which compresses the 35B-parameter MoE model to roughly 15.9 GB — about 30% smaller than a standard Q4_K_M build (~22 GB) — while reportedly retaining around 99% of FP16 accuracy. The model supports a 256K (262,144 token) context window, multimodal vision via a separate mmproj projector file, and native tool calling. The README reports inference speeds of 140+ tok/s on an AMD R9700 GPU and 70+ tok/s on an AMD MAX+395 iGPU when speculative decoding is enabled. Two reasoning mechanisms are described. The first is a dynamic seven-dimensional thinking framework that expands from a two-dimension quick answer for simple Q&A up to a full seven-dimension deep reasoning pass for complex development and strategy tasks, triggered by a moziAI-Think marker. The second is an Agent LOOP iteration mechanism that runs a two-round execute-then-verify cycle on complex tasks, automatically skipped for simple queries. Both are injected through a custom Jinja chat template that must be loaded alongside the model weights. The model is positioned as a financial vertical assistant, with documented capabilities in market analysis, earnings and research interpretation, risk and compliance review, quant strategy design, and pluggable tool calling into live market data, databases, and research retrieval. It also advertises general programming, writing, and multilingual support across 201 languages, plus an uncensored output mode inherited from the base model. Deployment requires three files: the main GGUF model at the repository root, the vision projector under mmproj/35B/, and the chat template under V3.8/. The README provides minimal and full llama-server launch commands, recommended sampling parameters (temperature 0.6–0.8, top_p 0.95, top_k 20, min_p 0.024), and VRAM guidance — 20 GB for 150K context with vision, 24 GB for full 256K with vision, and 32 GB+ for maximum headroom. Ollama deployment is shown via a Modelfile, though the README notes Ollama's support for mmproj and chat templates is limited compared to llama.cpp. Benchmarks are provided against Ornith-1.0-35B-A3B, Qwen3.6-35B-A3B, Gemma-4-31B, Muse-Glimmer-30B, and Qwen3.5-397B across coding (Terminal-Bench, SWE-bench variants, NL2Repo), reasoning (HLE, GPQA Diamond), and agentic (MCP-Atlas, Toolathlon, WideSearch, BrowseComp, ClawEval) suites. The license is a custom MoziAI Model License with Apache-2.0 upstream components from Qwen3.5/3.6 and Gemma 4, and the model is stated to be free for commercial use.