VLA Model Architectures Compared

Vision-language-action models, VLAs for short, now sit at the center of robot learning research, and the reason comes down to three engineering decisions: how the model turns pixels into features, how its language backbone reasons over those features, and how it turns that reasoning into motor commands. Every architecture on the market, from RT-2 to π₀ to GR00T N1, is a different answer to those three questions, and the differences aren't cosmetic. Those differences appear directly in latency, in smoothness of motion, and in how badly errors compound over a long manipulation sequence.

The dominant framework for robot learning

The old way of building a robot brain was modular, with a perception stack, a planner, and a separate controller bolted on at the end, each piece trained or tuned on its own. VLA models scrap that setup. Vision, language, and motor control all live inside one foundation model, trained together so the boundaries between "seeing," "understanding," and "acting" mostly disappear. That's the structural break that matters, and the field has noticed. ICLR 2026 received 164 VLA-related submissions, up from just 9 at ICLR 2025, an 18-fold jump in a single year.

The literature is racing to keep up with its own output. A 2025 systematic review in Information Fusion catalogs more than 100 distinct VLA architectures, and a separate arXiv survey covers over 80 models published in just the prior three years. When a field needs two competing taxonomies just to organize what already exists, that's usually a sign the underlying approach has moved past the experimental stage and into the stage where everyone is trying to make sense of the sprawl.

The Sa et al. review published in Drones in 2026 identifies the hardest test case as bimanual manipulation. review published in Drones in 2026, is bimanual manipulation: two arms, each with seven or more degrees of freedom, moving in coordination toward a shared goal. The same review points to UAV control as a structurally similar problem, one where latency and payload limits force many of the same design tradeoffs. Whatever architecture wins in either domain will have earned it under real constraints, not benchmark convenience.

The three structural decisions that define every VLA family

Strip away the marketing names and every VLA is built from the same three choices, made in sequence.

The first is how vision gets encoded. Almost every model leans on a pretrained vision encoder, CLIP's ViT, SigLIP, DINOv2, or some combination, to turn a camera frame into a set of feature vectors the rest of the model can work with. That choice isn't neutral. A semantic encoder like SigLIP is good at knowing "this is a mug," while a spatial encoder like DINOv2 is better at knowing where the mug's handle is relative to the gripper. OpenVLA runs both SigLIP and DINOv2 at once, combining their complementary representations.

The second decision is how the language backbone reasons over those visual tokens once it has them. Most VLAs bolt visual tokens onto a pretrained LLM, LLaMA, Gemma, PaLM, or a Flamingo-style architecture, and let it reason over vision and language jointly. Whether that reasoning happens through causal attention (token by token, only looking backward) or bidirectional attention (looking both ways at once) changes the latency profile and how coherent the output ends up being. Some designs go a step further and add a separate action expert, a distinct set of weights that never touches the shared LLM parameters. That's not a stylistic flourish. It changes which part of the network ends up specializing in control versus which part keeps hold of general world knowledge, and π₀'s design leans on exactly that split.

The third decision, and the one this article spends the most time on, is how the action head actually generates output. Four families fall out of this single choice: autoregressive models that emit discrete action tokens, flow-matching models that generate continuous trajectories, diffusion-based approaches, and hybrids that mix elements of more than one. This decision alone governs how often the model can act per second, how smooth the resulting motion looks, and how badly small errors early in a sequence snowball into failure later on.

Autoregressive models: how discretizing actions into tokens started the VLA paradigm

RT-2 is where this all started in earnest. Built on Google's PaLI-X, a 55-billion-parameter vision-language model, RT-2 was fine-tuned on real robot demonstrations layered on top of internet-scale pretraining. Its trick was treating actions like words: each action gets chopped into 256 discrete bins, so the same autoregressive machinery that predicts the next word in a sentence can predict the next joint command instead.

The tradeoff appeared immediately. A model that size can't run locally on a robot's onboard compute, so RT-2 operated as a cloud service, closing the loop at only 1 to 5 Hz. Slow, but it worked in a way that mattered: across 6,000 evaluation trials, RT-2 generalized to objects and commands it had never seen paired together during robot training. That result is what convinced a lot of the field that the whole approach was worth building on. Google never released the weights or training code, though, so nobody outside the original team could fine-tune it or dig into what actually drove the generalization.

OpenVLA answered that gap directly. At a parameter count far below that of its predecessor, built on the Prismatic VLM stack (LLaMA 2 paired with SigLIP and DINOv2), and fine-tuned on 970,000 robot demonstrations pulled from the Open X-Embodiment dataset, it used the same 256-bin discretization scheme as RT-2, yet outperformed the 55-billion-parameter RT-2-X. A model roughly an eighth the size beat its predecessor; scale alone was never the thing doing the work. Fine-tuning OpenVLA on a new task takes something like 50 to 200 demonstrations and 6 to 12 hours on a single high-end accelerator card, and because the weights and code are fully open, it's become the default baseline everyone else compares against.

Then came OpenVLA-OFT, and the update it made says something important about the whole family. The "Optimized Fine-Tuning" recipe swaps out discrete token prediction for continuous L1 regression, and the payoff is 25 to 50 times faster inference, better task success, and the ability to handle multiple camera inputs plus high-frequency bimanual control. On the LIBERO benchmark, OpenVLA-OFT hit 97.1% average success, the highest single-model score in the snapshot reported in arXiv:2512.04952. What that really shows is that the autoregressive backbone was never the bottleneck. The discrete action tokens were, and once the community was willing to drop them, performance jumped.

That's the standing tradeoff for this whole family. Token-by-token decoding is powerful for transfer and generalization, since it borrows the same machinery that makes large language models good at handling novel combinations of familiar concepts. But it introduces latency, and small errors in one token can compound into the next, so smooth motion usually requires extra engineering, chunked prediction, continuous output heads, something to paper over the seams.

Flow-matching models: continuous action generation and the action expert's role

Physical Intelligence's π₀ (pi-zero) took a different route entirely, and its design is worth walking through carefully because the architecture itself encodes a philosophy. The backbone is PaliGemma, a 3-billion-parameter pretrained vision-language model. Sitting alongside it is a separate action expert: 300 million additional parameters dedicated entirely to proprioceptive state and action tokens, using full bidirectional attention. Total parameter count comes to 3.3 billion, but the number matters less than the split. Pretrained world knowledge stays locked in the backbone, undisturbed, while the action expert specializes in control. Nothing has to fight over the same weights.

Action generation itself runs through flow matching, producing smooth continuous trajectories at 50 Hz. The training data behind it consists of 903 million timesteps, roughly 10,000 hours of manipulation data, spanning 68 tasks across 7 robot platforms. On LIBERO, π₀ averages 94.2%. On SimplerEnv-Bridge, that drops to 66.7%, a gap that says more about cross-embodiment generalization being harder than LIBERO alone would suggest than it does about any flaw in the model. Weights remain closed, though Physical Intelligence released openpi, an inference framework compatible with π₀.5-class models; the training data itself stays proprietary.

π₀.5 builds on the same base architecture but co-trains on a wider mix: web multimodal data, robot action data pulled from multiple environments, and cross-embodiment examples. Structurally, it's a hybrid within a hybrid, using discrete autoregressive decoding for high-level subtask prediction and continuous flow matching to produce the low-level motor output. On LIBERO it reaches 96.8% average, with better long-horizon generalization and faster inference than the original π₀.

The same lab produced FAST, an action tokenizer that takes a different angle on the same problem. It compresses robot actions using a discrete cosine transform followed by byte-pair encoding, cutting down the correlation between adjacent tokens so autoregressive VLAs can train on high-frequency, dexterous data without choking on redundancy. Trained on a million real trajectories spanning different embodiments, action spaces, and control rates, FAST is meant to work as a universal tool usable across specific models. It's a reminder that the autoregressive family isn't standing still while flow matching gets the attention; FAST is an attempt to bring frequency-domain compression into that older paradigm rather than abandon it.

Flow matching's advantage over standard diffusion, for real-time control specifically, comes down to needing fewer denoising steps and offering a more deterministic path through the transport function, both of which fit neatly inside a 50 Hz control loop. That's also why the field, broadly, keeps drifting toward continuous, chunked action generation for bimanual tasks: it sidesteps the quantization bottleneck baked into discrete tokens and avoids the multi-step latency tax that comes with standard diffusion sampling.

Dual-system architectures: separating reasoning from control at the structural level

Some teams decided the cleanest fix wasn't a better single model, but two models talking to each other. The idea borrows loosely from dual-process theories of cognition: a slow "System 2" that reasons over vision and language to break a goal into subtasks, and a fast "System 1" that executes the low-level motor commands those subtasks imply. This is a structural choice.

NVIDIA's GR00T N1 is one implementation. Built as a 2-billion-parameter model, it's positioned as an open foundation model for humanoid robots generally, not one specific platform. Its System 1 runs fast diffusion policies at 10 millisecond latency for immediate control, while System 2 handles high-level task decomposition through a vision-language reasoning module. Training draws on over 20,000 hours of human egocentric video combined with synthetic data generated in NVIDIA's Isaac simulator. An updated GR00T N1.5 has since followed.

Google's Gemini Robotics, and its newer Gemini Robotics 1.5 release, takes the same dual-system intuition and puts the emphasis somewhere else. The design philosophy is explicit: the model thinks in language first, then acts, so high-level reasoning happens before any action gets generated. Gemini Robotics 1.5 adds what Google calls embodied reasoning and Motion Transfer, a mechanism meant to unify data collected across different robot platforms into one shared representation space. It has not been released publicly at the time of writing.

Setting the two side by side shows the contrast is instructive rather than a matter of one being better. NVIDIA's design foregrounds the latency of the execution layer, the 10ms System 1 loop is the headline number. Google's foregrounds language-mediated reasoning as the thing that should happen first. Same underlying architecture pattern, different bet on which half matters more.

The open problem neither one has fully solved is the interface itself: how subtasks get passed from the slow reasoning layer down to the fast control layer, and at what frequency that handoff should happen. The System 2 layer adds reasoning overhead that System 1 can't afford to wait on, and getting that handoff right, without either stalling the robot or feeding it stale instructions, remains unresolved in both published models.

Lightweight and efficient VLAs: what happens when compute is the binding constraint

Not every team has datacenter-scale compute, and a separate branch of VLA design starts from that constraint rather than treating it as an afterthought.

Octo is the clearest example. It's a transformer-based diffusion policy pretrained on 800,000 robot episodes from Open X-Embodiment, and notably, there's no pretrained LLM backbone at all: the transformer is trained from scratch. It comes in two sizes, Octo-Small and Octo-Base, the latter several times larger than the former, both small enough to fine-tune on modest hardware. The architecture is modular by design: input tokenizers feed a transformer backbone, which feeds readout heads, and the model supports natural language instructions, goal images, observation histories, and multimodal action distributions through diffusion decoding. It's been validated across multiple robot platforms. On LIBERO, Octo scores 75.1% average, lower than the larger flow-matching models, but that number needs to be read next to the fact that it doesn't require anything close to datacenter hardware to train or run. The cost is limited language understanding, since there's no internet-scale pretraining backing it up, and that's a tradeoff the Octo team made on purpose rather than something that crept in by accident.

SmolVLA, released under the LeRobot framework, pushes the same idea further: a compact model built to run on laptops and hobby-grade robots, with an integrated data loading and training pipeline meant as an entry point for people without access to a cluster of specialized accelerators.

Quantization is the other lever. BitVLA applies 1-bit quantization, cutting memory use to 30% of the original footprint while holding onto most of the performance. QuantVLA takes a related but distinct approach, introducing scale-calibrated post-training quantization as a standalone framework applicable across different VLA systems. MoLe-VLA goes a third direction, using a mixture-of-experts router to route computation through a mixture-of-experts mechanism, cutting computational cost by 40%.

None of these approaches beat the flagship models on raw benchmark numbers, and none are trying to. They're solving for a different constraint: getting a working VLA onto hardware that doesn't have a data center behind it.

Emerging hybrid and world-model designs that blur the family boundaries

The cleanest categorization, autoregressive versus flow-matching versus dual-system, is already breaking down at the edges, and Discrete Diffusion VLA is a good example of why. It folds vision, language, and action modeling into a single transformer trained jointly. Each action dimension gets discretized into tokens, and the discrete diffusion framework operates over that tokenized action space. The result borrows diffusion's parallelism and the autoregressive family's token vocabulary at the same time, rather than picking one. Discrete diffusion VLAs represent one of the more active emerging directions in current architectural exploration.

WorldVLA pushes the boundary in a different direction, building an autoregressive action-world model that unifies image, language, and action within one modeling framework, incorporating predictions about future states alongside action generation. That's a meaningfully different objective than anything covered above, closer to model-based reinforcement learning's long-standing bet that a system which can predict consequences will act more reliably than one that only maps observations to outputs. It's an early sign that the boundaries between action generation and world modeling in VLA design may continue to blur and act as a byproduct.