Qwen-Drive-1.0-4B: An Open, Modular Driving Vision-Language Model

Qwen-Drive-1.0-4B is Alibaba Qwen’s open-weight vision-language model for autonomous-driving research, developed with Huazhong University of Science and Technology. Rather than retraining the underlying Qwen3.5-4B vision-language model into an opaque end-to-end driving stack, the project retains that shared multimodal backbone and adds external modules for bird’s-eye-view (BEV) perception and motion planning. The weights and accompanying software are released under Apache 2.0. 101115

The architecture: one shared VLM, two attachable driving modules

The shared Qwen3.5-4B model handles images and language. It supports visual question answering, including driving-scene questions, while providing representations for the added driving components. Qwen’s stated design goal is to add driving capability without replacing the base model’s general multimodal architecture. 1011

Two external modules extend it:

• BEV perception head: A BEVFormer-style head processes surround-camera representations and produces 3D detection, semantic occupancy, and BEV map-segmentation outputs. The published perception documentation describes up to 300 3D boxes across seven unified classes, a 200 × 200 × 16 occupancy grid over 10 classes, and a 60 m × 30 m map raster at 0.15 m resolution over six classes. 18
• Planning Expert: A separate module generates future ego-vehicle trajectories from the shared representations. The project standardizes trajectories into 50 ego-frame poses—position and heading—at 10 Hz across five seconds, allowing heterogeneous public driving datasets to use a common planning target. 111

This division is important for research. A team can probe perception separately from planning, compare direct planning with language-mediated planning, or swap and ablate modules without changing the core VLM.

planner-sft vs. planner-rl

Qwen releases two planning variants built around the same backbone:

• planner-sft is the supervised or imitation-trained Planning Expert. It can generate trajectories directly or after the model produces a textual rationale. 117
• planner-rl is further optimized with reward-based training against benchmark-oriented objectives. The project documentation recommends using it in reasoning-conditioned planning because its optimization rollouts were conditioned on sampled reasoning. 117

That distinction helps explain an important evaluation tradeoff. Reward optimization can improve preference-alignment or pseudo-closed-loop-oriented scores while slightly worsening open-loop displacement error against a recorded human trajectory. These measures answer different questions; a lower trajectory error is not automatically the same as a better preference or closed-loop score. 1

Three ways to run the model

The public cookbook documents three inference modes: 17

Mode	What runs	Output
VQA	The VLM only	Text answer
DIRECT_PLANNING	One VLM pass followed by the Planning Expert	Trajectories
REASONING_PLANNING	The VLM produces a rationale, then the expert uses that context	Reasoning and trajectories

The modes use the same underlying network, but differ in whether a rationale is generated and whether the planner receives representations that include that generated context. 17

Training strategy and unified annotations

Qwen describes staged training that combines driving-specific 3D perception, driving VQA, and planning supervision with general-purpose vision-language data. The pipeline maps heterogeneous perception labels to a shared taxonomy, revises driving-VQA answers for consistency, and converts trajectories from public datasets into the common waypoint format. 111

The approach is intended to balance specialization with retention: the model should gain driving-scene understanding and planning ability without simply discarding the base VLM’s general image-language capabilities. The available materials support that this was a design and evaluation goal, but they do not by themselves establish comprehensive independent performance across all general-knowledge or spatial benchmarks. 111

What the reported results show—and what they do not

Qwen’s reported planning results show higher NAVSIM v1.1 PDMS for the RL planner than the SFT planner: 90.7 versus 88.2, rising to 91.4 and 89.3 respectively with best-of-six sampling. The project also reports Waymo Open Dataset end-to-end RFS values of 7.91 for RL and 7.78 for SFT. On NVIDIA PhysicalAI open-loop evaluation, it reports a three-second minADE of 0.38 m for RL and 0.34 m for SFT. 1

The repository also reports driving-VQA improvements over a listed Qwen3.5-4B baseline on several driving-oriented benchmarks, including LingoQA, VLAD, SURDS, WaymoQA, and DriveLM-style measures. 1

These are project-reported benchmark results, not an independent certification of safety or deployment readiness. Planning scores, open-loop similarity to logged trajectories, preference-based metrics, and real-world safety validation are distinct claims. Qwen itself frames the release as an initial, research-oriented step rather than a production autonomous-driving system. 511

What developers can access

The Hugging Face model repository hosts Qwen-Drive-1.0-4B weights and configurations, and directs users to the public QwenLM/Qwen-Drive-1.0 repository for the codebase, demo data, and documentation. 1012

The GitHub project includes modular inference and evaluation materials. Developers can use the release to:

• run image-text and driving-scene VQA;
• run supplied driving demos;
• generate trajectories through direct or reasoning-conditioned planning;
• inspect BEV perception outputs;
• work with the released scene and evaluation formats; and
• experiment with public datasets and custom scene inputs. 111718

The release does not, on its own, reproduce every part of training. Non-released training corpora, annotations, preference labels, and proprietary data remain outside what can be reconstructed solely from public weights and code. 15

Why the modular release matters

Most useful as a research platform, Qwen-Drive-1.0-4B makes the boundaries between language-and-vision understanding, 3D perception, and planning explicit. That can make ablations more informative: researchers can investigate whether a change improves scene representation, trajectory generation, language reasoning, or a benchmark-specific reward objective.

The project also leaves room for extensions, including replacement heads, custom scene construction, and alternative camera or sensor layouts. Its value is therefore less a claim that a 4B model solves autonomous driving than an open starting point for testing how a shared VLM can support interpretable driving components. 1118