hiyouga/llamafactory
Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)
How llamafactory is put together
LlamaFactory is a Python framework for unified fine-tuning of 100+ large language models (SFT, DPO, PPO, KTO, reward modeling, multimodal) built on PyTorch/Hugging Face (transformers, peft, trl, datasets, accelerate). It exposes three frontends sharing one core: a CLI entrypoint (src/llamafactory/cli.py dispatching to train/eval/export/chat/api/webui launchers), a FastAPI OpenAI-compatible inference server, and a Gradio WebUI (LLaMA Board). Data flows from YAML/CLI args through the hparams parser into the data pipeline (dataset_info.json registry, templates, converters, processors), then into model loading/adaptation (LoRA/quantization patches) and Trainer-based tuning; trained models are served back through pluggable chat engines (HF, vLLM, SGLang). A next-generation 'v1' subsystem (launcher, plugin-style model/data/kernel plugins) coexists with the classic pipeline.
Languages
Frameworks
Infrastructure
Major components
cli / launcher
Entry points (llamafactory-cli, lmf) that dispatch commands to training, evaluation, export, chat, API server, and WebUI launchers.
hparams
Argument models (model_args, data_args, finetuning_args, generating_args, training_args) and the parser that merges CLI args with YAML config files.
data
Dataset loading from data/dataset_info.json, prompt templates for dozens of model families, converters/collators, and per-task processors (supervised, pairwise, feedback, pretrain, unsupervised) plus multimodal mm_plugin.
model
Model/tokenizer loading, adapter (LoRA etc.) attachment, patching (attention, checkpointing, longlora, MoE, liger kernel, kv_cache) and quantization integration.
train
Training orchestration via tuner.py wrapping HF/TRL trainers, callbacks, fp8 utilities, and trainer_utils for alignment methods.
chat
ChatModel abstraction over pluggable inference engines: hf_engine, vllm_engine, sglang_engine with a common base_engine.
api
OpenAI-style FastAPI server (/v1/models, /v1/chat/completions, /v1/score/evaluation) with SSE streaming, Bearer-token auth, and request media handling.
webui
Gradio-based LLaMA Board GUI (interface, runner, chatter, manager, locales) for zero-code fine-tuning and inference.
A quiet week focused on documentation of NPU Docker images.
5 weeksWeek of 2026-08-24
Week of 2026-08-24latest
A quiet week focused on documentation of NPU Docker images.
Week of 2026-08-17
New training capabilities landed for advanced attention setups and vision-language models via KTransformers.
Week of 2026-08-10
Big week for efficient MoE training, faster attention, and refreshed NPU software stacks.
Week of 2026-08-03
Bug fixes for Gemma model crashes plus a new vision-language model and NPU kernel cleanup.
Week of 2026-07-27
Multimodal training arrived in the v1 engine alongside Megatron-Bridge integration and loss-scaling fixes.
Over the past five weeks, LLaMA-Factory has been pushing hard on large-scale and multimodal training: the v1 engine gained multimodal data support, MoE efficiency plugins, and sequence-parallelism features, while KTransformers-based fine-tuning was introduced and hardened. Hardware coverage also expanded with NPU image updates and kernel refactors for Ascend devices. Alongside these big-ticket items came steady bug fixes, including a crash fix for Gemma models during data packing.
Week by week
2026-08-24A quiet week focused on documentation of NPU Docker images.latest1 change
Docs
NPU Docker image tag history
Added a record of past version tags for the NPU Docker images so users can find and reuse older builds.
2026-08-17New training capabilities landed for advanced attention setups and vision-language models via KTransformers.2 changes
Feature
GDN Ulysses context parallelism in v1
The v1 training engine can now split very long sequences across multiple GPUs using Ulysses-style context parallelism for GDN models.
Feature
KTransformers VLM fine-tuning
Users can now fine-tune vision-language models through the KTransformers backend, which runs big models on limited hardware.
2026-08-10Big week for efficient MoE training, faster attention, and refreshed NPU software stacks.4 changes
Refactor
FSDPTurbo plugin for MoE training
Added a new v1 plugin that speeds up mixture-of-experts training by combining expert parallelism with FSDP sharding.
Fix
Hardened KTransformers MoE LoRA fine-tuning
Made the KTransformers path for LoRA fine-tuning of mixture-of-experts models more robust and reliable.
Feature
FlashAttention 3 support
Training now supports FlashAttention 3, a newer, faster attention implementation on supported GPUs.
Chore
NPU images upgraded to CANN 9.1 and PyTorch 2.10
The prebuilt Docker images for Huawei Ascend NPUs were updated to newer versions of the CANN toolkit and PyTorch.
2026-08-03Bug fixes for Gemma model crashes plus a new vision-language model and NPU kernel cleanup.4 changes
Fix
Fixed rotary embedding crash when packing data
Fixed a crash that happened when packing training data without FlashAttention 2, which affected Gemma-3 and Gemma-4 models.
Docs
Fixed broken README links
Repaired links in the README that pointed to pages that no longer worked.
Refactor
NPU kernel matching by model type
Cleaned up how the v1 engine picks optimized kernels on Ascend NPUs so it matches them by model type instead of ad-hoc checks.
Feature
MOSS-VL model support
Added support for training and fine-tuning the MOSS-VL vision-language model.
2026-07-27Multimodal training arrived in the v1 engine alongside Megatron-Bridge integration and loss-scaling fixes.4 changes
Feature
Multimodal data training in v1
The new v1 training engine can now handle datasets that mix text with images or other modalities.
Fix
CI Python version alignment
Made the automated test workflows use the same minimum Python version that the project officially requires.
Fix
Hyper parallel tail loss scaling fix
Corrected how loss is scaled at the end of each training pass under hyper-parallel setups, improving training accuracy.
Feature
Megatron-Bridge support for PT/SFT
Pretraining and supervised fine-tuning can now run through Megatron-Bridge, enabling very large-scale distributed training.
Dependencies and code review
Dependency advisories
Security Watch
No known advisories across 0 scanned dependencies.
No known advisories in the scanned dependencies.
Code review
No concrete code-level issues confirmed.
Get this report every week for your repos.
GitZoid learns each repo, reports what changed, and flags what needs attention. One flat price for the whole team.
$19 a month, flat · First 10 outputs free · No card required