limix

LimiX is an open-source project providing large foundation models for structured (tabular) data, described as LDMs. The repository releases inference code and pretrained checkpoints for the LimiX model family, including LimiX-2 (400M parameters), LimiX-2M (2M) and LimiX-16M (16M). Core capability: a single pretrained model performs classification, regression and missing-value imputation in one forward pass, without task-specific parameter updates. LimiX-2 uses a contextual mechanism network (CMN) paradigm pretrained with context-conditional masked modeling, learning a joint representation p(x, y | D_context) rather than the target-centric p(y | x, D_context) objective of conventional tabular prior-fitted networks. Pretraining data is synthetic, generated from structural causal models spanning varied graph structures, functional mechanisms and observation processes. The project states that the CMN design also yields causal awareness, with feature attention encoding direct causal relationships. Usage: Python >= 3.12 is required. Installation is via `pip install -e .` after cloning, with optional CUDA-matched PyTorch (torch==2.9.1 recommended) and flash-attn wheels. The public entry point is `inference.predictor.LimiXPredictor`, which routes to v1_0 or v2_0 based on checkpoint architecture version; configs must match the model (LimiX-2 / V2.0 uses `*_v2.json`). A command-line tool `LimiX-infer` is provided, requiring `--task_type`, `--data_dir` and `--model_path`. Task types include Classification, Regression and Feature_imputation, with aliases cls / reg / imputation. `--data_dir` expects a benchmark root with one subdirectory per dataset containing train/test CSVs (and a masked CSV for imputation), where the last column is the target. Optional flags include device (cuda or cpu), gpuid, gpu_num_per_predictor, autobatch, show_progress and seed. The Python API exposes `LimiXPredictor(device, model_path, inference_config, ...)` with options such as mix_precision, outlier_remove_std, softmax_temperature, average_before_softmax, categorical_features_indices, inference_with_DDP, use_data_cache and seed. Its `predict(x_train, y_train, x_test, task_type, unique_dataset_name)` returns class probabilities for classification, predictions on the original target scale for regression, or an imputed feature matrix for missing-value imputation. Example scripts cover classification (breast cancer dataset), regression (diabetes dataset) and missing-value imputation. Packaged inference configs distinguish retrieval and no-retrieval variants: V2.0 configs for LimiX-2, and retrieval/no-retrieval configs for LimiX-16M and LimiX-2M, where retrieval is described as improving accuracy and no-retrieval as faster. The README reports benchmark evaluations on TabArena, TALENT and BCCO, plus a scaling study across 12.5M to 406.2M parameter configurations. These are the project's own reported results; independent verification is not provided here. Licensing: the code is released under the Stable AI Technology Co., Ltd. License, Version 1.0 (September 2026), derived from Apache License 2.0, and model usage is subject to a non-commercial license. Links are provided to arXiv papers, technical reports, a project page, HuggingFace and ModelScope model/dataset hosting, and BCCO classification and regression benchmark datasets.