Trajectory’s concurrent multi-LoRA stack reports a 2.81× experiment-throughput gain over single-tenant RL, with all code in the NovaSky-AI/SkyRL GitHub repository.
Most language models improve in discontinuous jumps. A team collects data, trains, and ships a new version. This takes months and produces remarkable or catastrophic behavior for users. Trajectory wants to replace that cycle with continual learning.
The Trajectory team published a field report describing how. It built a concurrent, multi-LoRA training platform for continuously learning workloads. The work was done with UC Berkeley Sky Lab and Anyscale. All training code is open-sourced in the NovaSky-AI/SkyRL repository.
The result is a 2.81× end-to-end experiment-throughput improvement. The comparison is against a single-tenant training framework. Trajectory reports no regression on any training rewards.
What Multi-LoRA Training Actually Is
Continual learning requires models to update from live feedback and production interactions. A coding agent could learn engineering patterns as developers correct its work. A support agent could resolve hard tickets as operators intervene on difficult cases.
Most training infrastructure still assumes a linear lifecycle. Teams allocate GPUs, initialize the model, run a job, then spin down. Continual learning revises that relationship. When production interactions become training inputs, training becomes part of a live system.
Modern RL training reduces to three core primitives. The Sampler generates trajectories from the current policy model. The Trainer computes gradients and updates the policy weights. Parameter synchronization broadcasts updated weights back to inference workers.
Trajectory calls its approach Continuous Multi-LoRA Training, or C-LoRA. Each experiment maps to a dedicated LoRA adapter on a warm, multi-tenant engine.
The Problems It Targets
The Trajectory team identifies four inefficiencies in traditional stacks:
(1) Cold starts are slow: Every serial job reloads checkpoints, initializes the distributed runtime, and warms inference engines. For large models, this step alone can exceed 30 minutes per run.
(2) RL is memory intensive: Frontier models often exceed 100B parameters. Qwen3.5-397B can require up to eight H200 nodes to fit into memory. LoRA cuts memory usage by an order of magnitude. It freezes the base model and trains only small adapter weights.
(3) Traditional stacks are single-tenant: They run one experiment at a time. Multi-LoRA maps each experiment to one adapter, multiplexing throughput by a factor of N.
(4) Job utilization is low: Trainers and inference engines stall while waiting for each other. Multi-LoRA load balances across jobs to fill idle capacity.
Inside the Architecture
Most throughput wins come from inference. In vLLM, all adapters are hot-loaded in GPU memory. Decode steps can then mix tokens from different adapters in the same batch. The key enabler is the SGMV decode kernel. It fuses per-adapter matrix-vector work into one GPU launch per decode step.
After each optimization step, updated LoRA weights load in-place into the inference engine. The scheduler does not freeze, so other tenants keep decoding.
Training works differently. One active LoRA adapter trains on the GPU. The rest sit in pinned CPU memory. Each tenant’s state lives in an AdapterStore. It holds LoRA parameters, FP32 master weights, optimizer moments, and gradient buffers.
The engine swaps one tenant’s state onto the GPU, runs a single forward_backward pass, then swaps it back. This training path is still single-adapter. The inference concurrency gains do not yet apply to training.
The Numbers
Trajectory tested on a single H200 node with Qwen3-4B-Instruct-2507. It ran sync RL on GSM8K in an agentic setting. The Trajectory team reframed GSM8K as a tool use learning task. The model decides when to call a Calculator and a Final Answer tool. Reward is 1.0 only when Final Answer is called with the correct answer.
The policy starts near 40% accuracy at step 0. With the right learning algorithm, it climbs past 90% by step 9.
The Trajectory team scaled to eight concurrent multi-LoRA runs. Final Experiment Time hit 5433s at N=8, a 2.81× speedup. Eight concurrent experiments finished before three serial runs back-to-back. Mean Experiment Time also improved, peaking at N=4 with a 1.88× speedup. Every concurrency level reached reward_accuracy above 90% by step 9.
The Tradeoffs
Higher throughput costs per-step latency. As N grows, First Experiment Time and Step Time degrade. At N=8, the first serial experiment finishes 1.97× faster. Mean step time rises from 191s to 500s, only 2.62× slower.
Most of that increase is rollout time. Rollout grows from 162s to 401s, roughly 77% of the increase. At N=2, doubling the load adds only 15% rollout time. That is the ideal case for multi-LoRA.
The pattern held on a harder workload. On τ-bench retail with the NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 MoE model, N=2 finished 10 steps 1.28× faster. Per-tenant step time rose 1.57×.
Strengths and Weaknesses
Strengths:
- 2.81× end-to-end experiment-throughput gain at eight concurrent runs
- No accuracy regression; runs tracked the serial baseline within ±1σ in the final steps
- LoRA cuts memory by an order of magnitude versus full fine-tuning
- Fully open-sourced in NovaSky-AI/SkyRL for the community to build on
Weaknesses:
- Per-step latency and First Experiment Time degrade as N grows
- Training remains serialized across tenants; only inference is multiplexed
- Tested mainly on mid-sized models, not frontier-scale parameters
- Setup requires an 8× H100/H200 node and a Megatron build
Key Takeaways
- Trajectory built a concurrent, multi-LoRA RL training stack for continual learning, open-sourced in NovaSky-AI/SkyRL.
- It reports a 2.81× end-to-end experiment-throughput gain over a single-tenant baseline, with no reward regression.
- Each experiment maps to a dedicated LoRA adapter on an always-hot engine, multiplexing throughput by N.
- Most gains come from vLLM multi-LoRA inference via the SGMV decode kernel; training stays single-adapter.
- The tradeoff is per-step latency: at N=8, step time rises from 191s to 500s.
Marktechpost’s Visual Explainer
Where (Inferences) to Run
Inference & compute providers
Where to access the Qwen3-4B-Instruct-2507 base model, the SkyRL training stack, and the NVIDIA GPUs used in the experiments.
Check out the Repo and Technical Details. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.
