Post-training Large Language Models with MinT

1. What Is Post-training? What Is MinT?

1.1 Post-training

The construction of a large language model proceeds in two stages.

The first stage is pre-training. A model reads a large corpus of text and learns to predict the next token. The result is a generalist: it has absorbed grammar, factual associations, and broad reasoning patterns. This generalist has a defined limitation. It has no knowledge of a specific task, a required output format, or a set of domain rules, because none of these were present as a training objective.

The second stage is post-training. A pre-trained base model is trained further on a smaller and focused dataset so that its behavior matches a target task. Three forms are common.

• SFT (Supervised Fine-Tuning). The data consists of prompt -> response pairs with known correct responses. The objective teaches the model to reproduce the correct output. A math question paired with its correct numerical answer is a typical example.
• DPO (Direct Preference Optimization). The data consists of answer pairs, one preferred and one rejected. The objective shifts probability mass toward the preferred answer.
• RL (Reinforcement Learning). No single fixed answer is provided. Instead, a reward signal scores each generated output, supplied by a verifier, a test suite, or an environment. The model samples outputs, receives scores, and updates toward higher reward. GRPO and PPO are two algorithms in this class.

A frequent composition applies SFT first to establish baseline behavior, followed by RL to optimize a scalar objective.

1.2 MinT

MinT (Mind Lab Toolkit) is the model service platform from Mind Lab, a model company. It exposes the training and serving stack for large language models as a managed service. Its operating model can be stated in one sentence.

A short Python script runs on a CPU machine and specifies the data, the loss function, and the RL environment. MinT executes the resulting computation on a remote GPU cluster.

The user never addresses the GPUs directly. A call to the mint Python SDK travels through an API gateway to a set of core services, which schedule model loading, forward and backward passes, optimizer steps, and sampling on the cluster.

Two properties of this design are relevant when reading the code.

• The training loop remains explicit. Every forward_backward and every optim_step is visible and editable in user code. The loss function, learning rate, and reward are chosen by the user. The distributed-systems work is hidden while the algorithm remains under user control.
• The base model is selected by a single argument. Moving from Qwen, Deepseek-4to a GLM-5.2 model changes one string in the script.

The system is organized in four layers.

1.3 The core idea: a resident base plus many adapter revisions

The technical report frames MinT around one design decision. The base model stays resident in GPU memory, loaded once. Each trained behavior is represented by a small LoRA adapter revision that sits beside the shared base. Task variants, product branches, experimental versions, and rollback points all become different adapters over the same resident base.

This decision changes what has to cross the boundary between training and serving. Compare three ways to send a trained behavior to an inference server.

Under full fine-tuning, every trained variant produces a complete checkpoint that must be stored and shipped. LoRA merge trains a small adapter but folds it back into the base before serving, so it still moves a full-size checkpoint. MinT exports the adapter alone, checks that it matches the resident base, and loads it into an inference engine that already holds that base. The artifact that crosses the boundary is a LoRA revision, which stays small.

MinT tracks two separate objects for each trained behavior.

• Adapter revision. A fixed, exported snapshot of the LoRA weights in serving tensor layout. It is the executable payload that rollout, evaluation, serving, and rollback select.
• Policy record. The durable service entry that names the base version, the LoRA rank and target modules, the latest training checkpoint, and the exported revisions. It makes the payload reproducible, reloadable, and reversible.

Separating these two objects lets one resident base support many LoRA policies at once, without turning every policy into another full checkpoint or another full-model server.

2. Why Post-training? Why MinT?

2.1 Why post-training

A pre-trained base model is general by construction. A deployed product typically requires narrower behavior, and post-training supplies it along three axes.

• Task adaptation. A base model can converse, yet it may fail to follow a required output format, invoke specified tools, or solve a domain problem with acceptable reliability. Post-training raises reliability on the target task.
• Alignment. Instruction following and helpfulness are shaped by RL and DPO, which move the model toward outputs that human evaluators prefer.
• Domain knowledge. Legal, medical, and financial tasks use vocabulary and reasoning that appear infrequently in general text. SFT on domain data reduces this gap.

Post-training costs a fraction of pre-training and grants control over final behavior. For these reasons nearly every deployed model passes through it.

2.2 Why MinT

A self-built LLM training stack imposes several costs that are unrelated to the learning algorithm itself.

• Distributed engineering. Sharding a model across GPUs, managing memory, partitioning batches, and coordinating inter-machine communication consume most of the implementation effort while contributing nothing to the algorithm.
• The RL sample-train loop. RL requires the model to generate, receive scores, and update weights in sequence. The sampling service and the training service must stay synchronized on the current weights.
• Model swapping. A change of model size usually forces a rewrite of the parallelism configuration.

The report organizes MinT's capabilities along three scaling axes. Each axis carries a concrete measurement.

• Scale Up. LoRA RL runs on frontier-scale dense and Mixture-of-Experts (MoE) models. Model-parallel training and serving paths are validated beyond one trillion total parameters, including a 235B-A22B GRPO run and a 1T-class MoE countdown-task path.
• Scale Down. The training-serving handoff moves only the exported adapter, which can fall below 1% of the base-model size at rank 1. Adapter-only handoff reduces the measured handoff step by 18.3x on a 4B dense model and by 2.85x on a 30B MoE model. Under one resident base, running three GRPO policies concurrently shortens wall time by 1.77x (4B) and 1.45x (30B) with no increase in peak memory.
• Scale Out. One serving layer exposes a million-scale adapter namespace. The report materializes 1,000,000 adapter revisions with zero build errors and audits 256 adapters across all 100 shards, while each engine keeps a bounded CPU cache and GPU-batch working set. Packing MoE LoRA tensors improves live engine loading by 8.5x to 8.7x.

2.3 Why LoRA

Full fine-tuning updates every parameter. For a 30B model this requires storing gradients and optimizer state for 30 billion values, which sets a high memory and storage cost.

LoRA (Low-Rank Adaptation) freezes the original weight matrix W and trains a small low-rank term beside it.

The reduction produces three consequences.

• Lower training cost. One to two orders of magnitude fewer trainable parameters reduce both memory and time.
• Small artifact. A trained adapter occupies tens of MB. The report measures a Qwen3-4B rank-32 adapter at 252 MiB, about 3.3% of the 8.0 GB base-weight floor; the same target pattern at rank 1 falls to roughly 7.9 MiB, about 0.10% of the base. This small artifact is what makes the Scale Down handoff cheap.
• Flexible deployment. An adapter can be mounted on the base model at inference time and share its memory, which yields fast startup and allows many adapters to be served from one base. An adapter can also be merged into the base to produce a standalone model for offline use.

Immediately after training, MinT returns a sampling client backed by the new adapter, so the trained behavior can be evaluated without a separate export step.

3. How to Post-train with MinT

The appropriate path depends on the user's coding skill and available infrastructure. Three profiles are treated below.

3.1 For Trainers with Coding Ability (Community Edition)

A user who writes Python uses the Community Edition through the mint SDK against the hosted endpoint at mint.macaron.xin. No local GPU is involved.

Installation and configuration, apply your API key from MinT.

pip install git+https://github.com/MindLab-Research/mindlab-toolkit.git
   export MINT_API_KEY=sk-your-api-key-here
   export MINT_BASE_URL=https://mint.macaron.xin/

Step 1. Create a LoRA training client.

import mint
from mint import types
service_client = mint.ServiceClient()
training_client = service_client.create_lora_training_client(
    base_model="GLM/GLM-5.2",
    rank=16,             # LoRA rank; larger rank increases capacity and memory use
    train_mlp=True,
    train_attn=True,
    train_unembed=True,
)
tokenizer = training_client.get_tokenizer()

Step 2. Construct a training example (SFT).

An SFT example is a token sequence paired with a per-token loss weight. Prompt tokens carry weight 0 and produce no gradient; answer tokens carry weight 1. A teacher-forcing shift aligns inputs and targets: the input is all_tokens[:-1] and the target is all_tokens[1:].

def process_sft_example(prompt, completion, tokenizer):
    prompt_tokens     = tokenizer.encode(prompt, add_special_tokens=True)   # e.g. 12 tokens
    completion_tokens = tokenizer.encode(f" {completion}", add_special_tokens=False)
    completion_tokens.append(tokenizer.eos_token_id)                        # e.g. 3 tokens
    all_tokens  = prompt_tokens + completion_tokens        # len = 15
    all_weights = [0.0]*len(prompt_tokens) + [1.0]*len(completion_tokens)   # 12 zeros + 3 ones
    return types.Datum(
        model_input=types.ModelInput.from_ints(tokens=all_tokens[:-1]),     # shape (14,)
        loss_fn_inputs={
            "target_tokens": all_tokens[1:],     # shape (14,)
            "weights":       all_weights[1:],    # shape (14,); first 11 are 0, last 3 are 1
        },
    )

The three arrays share one length, equal to len(all_tokens) - 1.

Step 3. Run the training loop.

for step in range(num_steps):
    fb = training_client.forward_backward(sft_data, loss_fn="cross_entropy").result()
    training_client.optim_step(types.AdamParams(learning_rate=5e-5)).result()

Two conventions apply. The loss_fn argument belongs to forward_backward, not to AdamParams. A zero_grad call is unnecessary, since the server clears gradients between steps.

Step 4. Refine with RL (GRPO).

RL runs a generate-score-update loop. Several answers are sampled per question, each is scored, and the scores are converted to advantages.

# (a) obtain a sampling client from the current weights
sampling_client = training_client.save_weights_and_get_sampling_client(name="rl-step-0")
# (b) sample a group of answers for one question
res = sampling_client.sample(
    prompt=types.ModelInput.from_ints(tokens=prompt_tokens),
    num_samples=8,
    sampling_params=types.SamplingParams(max_tokens=16, temperature=0.7,
                                         stop_token_ids=[tokenizer.eos_token_id]),
).result()
# res.sequences: 8 answers, each with .tokens and .logprobs
# (c) score each answer: 1 for correct, 0 for wrong
rewards = [1.0 if extract_answer(tokenizer.decode(s.tokens)) == answer else 0.0
           for s in res.sequences]                       # length 8
# (d) advantage = reward minus the group mean (the central GRPO step)
mean_r     = sum(rewards) / len(rewards)
advantages = [r - mean_r for r in rewards]                # length 8

The advantages are zero-centered within each group. When every answer in a group scores equally, all advantages equal 0 and the group carries no learning signal, so it is skipped.

Each answer becomes an RL Datum. Relative to SFT it carries two additional arrays: logprobs (log-probabilities recorded during sampling) and advantages.

full   = prompt_tokens + response_tokens
prefix = len(prompt_tokens) - 1              # the prompt segment carries no loss
datum = types.Datum(
    model_input=types.ModelInput.from_ints(tokens=full[:-1]),
    loss_fn_inputs={
        "target_tokens": full[1:],
        "weights":       [0.0]*prefix + [1.0]*len(response_tokens),
        "logprobs":      [0.0]*prefix + response_logprobs,
        "advantages":    [0.0]*prefix + [adv]*len(response_tokens),
    },
)
training_client.forward_backward(datums, loss_fn="importance_sampling").result()
training_client.optim_step(types.AdamParams(learning_rate=2e-5)).result()

The RL learning rate (2e-5) is smaller than the SFT rate (5e-5). The importance_sampling objective is sensitive to the divergence between the current policy and the sampling policy, so small steps preserve stability. The ppo objective, which clips the update, offers a stronger stability guarantee.

Step 5. Save and deploy.

sampling_client = training_client.save_weights_and_get_sampling_client(name="my-run-v1")

The call returns an OpenAI-compatible endpoint, reached by switching the base URL in the application. One caveat applies: a sampling client created before a save continues to serve the earlier weights, so a fresh client should be created from each new checkpoint.

3.2 For Trainers without Coding Ability (Enterprise Edition)

Some teams require a trained model without authoring the training loop themselves. The Enterprise Edition addresses this case. It runs the same mint SDK underneath and adds a managed layer around it.

Managed onboarding. Access uses a whitelisted account, and an administrator assigns a workspace and a compute quota. In place of self-service setup, an onboarding session pairs the team with MinT staff, who assist with data organization, objective definition, and the first training run. Sales and support are reachable through a scheduled demonstration or by email.

Operational features provided by the platform. Enterprise supplies capabilities that a production deployment requires and that carry a high cost to build independently.

The AI-assistant path. A training script can be produced without hand-writing it, by describing the objective to an AI coding assistant such as Cursor or Claude Code. MinT publishes its full documentation as a single machine-readable file at /en/llms-full.txt. The assistant is pointed at that file once and then instructed in plain language.

Read https://mint-doc.macaron.xin/en/llms-full.txt for the full MinT documentation,
then write a GRPO training loop on Qwen3-30B with a custom reward for my task.

The assistant emits a loop that uses the same SDK described in Section 3.1. Because both editions share one SDK, the generated script runs on Enterprise once the endpoint and key are set to the Enterprise values. Recording the endpoint requirement in the project's CLAUDE.md or .cursorrules file keeps the assistant consistent across sessions.

A security note: any sk-* API key should be removed from text before it is pasted into an assistant prompt.

3.3 For Trainers with Infrastructure Ability (Self-hosted Open-source Runtimes)

A team that owns GPUs and requires data to remain inside its own cluster can run MinT itself. Two open-source runtimes expose the same MinT HTTP API, so a client script written for the hosted service runs against a self-hosted deployment after one change: the base_url.

Both runtimes share an architecture. A FastAPI control plane receives MinT requests, routes training work to a training engine and sampling work to an inference engine, and returns a future that the client polls. The runtimes differ in which engines they use.

3.3.1 verl-mint (built on veRL)

verl-mint runs MinT-style workloads on a veRL + Ray cluster. It assumes an existing Ray cluster and does not start Ray or provision GPUs.

• Models. Qwen 0.6B, 4B, and 30B, which suits small and mid-size validation.
• Algorithms. SFT and GRPO; DPO is planned.
• LoRA initialization. Six methods are offered through lora_config.init_method: random, olora_head, olora_tail, pissa, milora, and custom. The SVD-based methods initialize the adapter from a meaningful subspace of the base weights instead of from noise, which can shorten convergence. Which subspace matters under RL. The scaling study finds that methods anchored in the principal singular directions, pissa and olora_head, can destabilize RL with verifiable rewards: under the DAPO objective, olora_head reward collapses around step 100 and its KL divergence from the reference model climbs by several orders of magnitude, and pissa shows a similar collapse. The olora_tail method instead uses the minor singular directions and drops the singular-value scaling that milora applies, which keeps early updates inside the trust region that on-policy RL needs. It stays stable across 500 steps and reaches a higher average on math benchmarks than standard LoRA (58.3% vs 56.3%); at rank 1 it holds roughly +20% over the base across batch sizes where standard LoRA degrades. A helper script builds these bundles from a Hugging Face model.

Start the service on a cluster node.

uv run --extra smoke --extra ray \
  verl-mint serve \
  --backend verl \
  --host 0.0.0.0 --port 8000 \
  --storage-root "$VERL_MINT_STORAGE_ROOT" \
  --model-path Qwen/Qwen3-0.6B \
  --verl-config-dir "$VERL_CONFIG_DIR"

Connect from any machine with network access.

import mint
client = mint.ServiceClient(base_url="http://<cluster-node-ip>:8000", api_key="sk-local-smoke")
training = client.create_lora_training_client("GLM/GLM-5.2", rank=16)
print(training.get_info())

The training, sampling, and checkpoint calls match Section 3.1.

3.3.2 AReaL-MinT (built on AReaL)

AReaL-MinT targets larger models on a single 8-GPU node. Training uses AReaL (FSDP2) and inference uses SGLang, with LoRA hot-reload between the two engines.

• Models. Qwen3.6-27B (dense) and Qwen3.6-35B-A3B (MoE).
• Algorithms. SFT and GRPO.
• GPU layout. 4 GPUs for training and 4 GPUs for inference on one node.

Install and launch.

git clone https://github.com/areal-project/AReaL-MinT.git
cd AReaL-MinT
python3.12 -m venv --system-site-packages .venv
source .venv/bin/activate
pip install -e '.[gpu]' --no-deps
# plus the runtime dependencies listed in the README
export PATH=$PWD/scripts:$PATH
mintctl start 27b        # or: mintctl start 35b

mintctl is a management script. start loads the model (about 3 minutes, including CUDA graph capture) and keeps it resident. stop terminates the whole process group, which prevents orphaned GPU memory. status and logs -f report on the running service.

Verify end to end.

bash examples/smoke_e2e.sh      # SFT -> GRPO -> RL convergence, about 15 min for 27B

Connect a client.

import mint
sc = mint.ServiceClient(base_url="http://<your-gpu-host>:8000", timeout=600)
tc = sc.create_lora_training_client(base_model="Qwen/Qwen3.6-27B", rank=16,
                                    train_mlp=True, train_attn=True, train_unembed=True)

A current limitation. In the present version, save_weights takes about 74 seconds for 27B and 93 seconds for 35B per synchronization, because it writes the full model to disk and reloads it into the inference engine. Under RL this cost recurs every step and dominates wall-clock time. The project lists it as an optimization target for the next version, through LoRA-only delta save or direct GPU-to-GPU transfer.

4.Own Your Intelligence

MinT is built for one purpose: post-training. Every design choice in this article follows from that focus. The resident base with small adapter revisions, the shared serving path, the single lifecycle across SFT, preference optimization, and RL. These are the mechanics of turning a general model into a specific one, done cheaply enough to repeat.

We build MinT because we believe post-training is where general capability becomes personal capability. A pretrained model holds broad priors. Post-training is the step that turns those priors toward one person's task, one team's data, one domain's rules. The measurements in this article show the cost of that step falling by one to two orders of magnitude. When a step gets that much cheaper, who gets to take it changes.

The goal MinT works toward is straightforward to state. Training a model should not require a cluster, a systems team, or a research budget. It should require a dataset and an objective. When that holds, the ability to train is no longer held by a few labs. Each person can adapt a model to their own work, keep the adapter as their own artifact, and own the intelligence they shape.

That is the direction. A general base, shared by many. A specific adaptation, owned by each. MinT is the infrastructure that makes the second part reachable.

5.Q & A

How much does MinT cost?

Two components set the bill. Training cost scales with GPU time, and LoRA reduces the trainable-parameter count by one to two orders of magnitude against full fine-tuning, which lowers both the memory footprint and the wall-clock time per run. Serving cost scales with how the base and adapters are placed: one base stays resident in GPU memory and is shared across many adapters, so each additional personal model adds the cost of a small revision instead of a full model replica. For a concrete quote, contact the sales team at sales@mindlab.ltd. New Community Edition accounts currently come with 5M free tokens to cover a first round of training and evaluation.

Can I use MinT with my own GPUs?

Yes. Section 3.3 describes the self-hosted path. A team that owns GPUs and needs data to stay inside its own cluster can run one of two open-source runtimes, verl-mint or AReaL-MinT, both of which expose the same MinT HTTP API. Client code written against the managed service transfers to the self-hosted deployment without change. The hosted Community and Enterprise editions run on Mind Lab's cluster; the open-source runtimes run on yours.

Do I need to know how to code to use MinT?

It depends on the edition. The Community Edition expects a short Python script that names the data, the loss function, and the base model, so it assumes basic Python. The Enterprise Edition covers the case where a team wants a trained model without authoring the training loop: onboarding is managed, and a training script can be generated by pointing an AI coding assistant at the MinT documentation and describing the objective in plain language. Both editions run on the same SDK underneath, so a script produced either way uses the identical training path.

Which post-training method should I use: SFT, DPO, or RL?

Match the method to the signal you can provide. Choose SFT when you hold example outputs that demonstrate the target behavior directly. Choose DPO when labeling a preferred answer against a rejected one is easier than writing the ideal answer yourself. Choose RL (GRPO or PPO) when correctness is checkable by a program or a scalar reward, such as a unit test or a math grader. A common composition applies SFT first to set baseline behavior, then RL to optimize the scalar objective. All three run through the same LoRA lifecycle in MinT, so switching between them changes the loss function and the data format, while the train-export-serve path stays the same.

Can MinT serve many trained models at once?

Yes, and this is the property MinT is built around. Every trained behavior is a small LoRA adapter over one shared resident base. The base weights stay in GPU memory once, and thousands of adapters can be attached to that same base without loading a full copy of the model per adapter. The report materializes a catalog of one million adapter revisions, and the serving layer routes each request to the requested adapter over the common base. For a student, the mental model is one large read-only object in memory plus many small patches selected per request.