What APXInf does for on-robot AI inference

A robot using an AI policy must turn observations into actions quickly enough to be useful. APXInf addresses the model-inference part of that process: it is an open-source engine released by Infinigence AI with Tsinghua University and Shanghai Jiao Tong University to run embodied-AI models on or near the robot. It is not the motor controller, and faster inference alone does not establish how quickly an entire robot senses, decides and acts. 144

How the engine is designed to help

APXInf pairs a small Rust runtime, which manages execution and memory, with Python bindings that let developers call a policy from familiar robotics code. Its model-porting workflow begins with a fixed reference checkpoint and test inputs, checks the port against reference outputs and intermediate values, and then applies optimizations. That verification step matters: a faster deployment is only useful if it still produces the intended model outputs. 35

The optimizations address different sources of delay. Pipeline scheduling targets the full inference path; graph capture and reusable buffers reduce repeated setup; tuned GPU kernels speed up costly operations; and FP8 or INT8 quantization can lower computation and memory traffic. Together, these measures aim to make action generation faster and more predictable under on-device constraints. 85

What the Jetson Thor numbers actually say

In a published launch comparison for π0.5 at FP8 precision on Jetson Thor, end-to-end inference latency falls from 278 milliseconds for an unoptimized baseline to under 26 milliseconds—about a 10.7× reduction using 26 ms as the comparison point. The reported 38.46 inferences per second is consistent with roughly one inference every 26 ms; 278 ms implies about 3.6 per second. Some coverage also says 46 Hz, but the 26 ms and 38.46 Hz figures agree with each other. These are inference figures, not a demonstrated 38.46 Hz rate for every robot’s complete sensor-to-actuator loop. 862

There is a further qualification: the project repository’s benchmark table lists Jetson AGX Thor FP8 at 41.16 ms, or 24.3 Hz, rather than under 26 ms. The supplied figures do not establish that the two results use identical benchmark conditions, so they should not be treated as interchangeable. Anyone assessing a deployment should check the test configuration and reproduce the measurement for their own model and device. 58

Models, integration and what comes next

Launch coverage names π0.5 and WALL-OSS as initial models and RTX 4090, Jetson Orin and Jetson Thor as supported platforms. The repository more specifically describes a highly optimized first-release π0.5 path on Thor and Orin, with BF16, FP8 and INT8 precision options. APXInf sits on the inference and deployment side of the RLinf ecosystem; its robot-facing package also provides an OpenPI-compatible WebSocket serving interface. Interface compatibility does not mean every OpenPI checkpoint has already been ported. 451

The stated roadmap extends to additional vision-language-action (VLA), vision-language (VLM) and world models, with Qwen and GR00T being adapted, as well as more chip backends and platform work. Those are plans, not guarantees of present support. Likewise, neither the launch result nor the repository table can be carried over to a different checkpoint, precision, device or power limit without testing. Inference latency by itself does not prove closed-loop task success or sustained performance within a robot’s thermal and power envelope. 95