pi0 Architecture

Camera images are processed through a SigLIP vision encoder, producing a grid of visual tokens that capture spatial features for the robot.

Visual tokens merge with language instruction tokens inside PaliGemma, producing fused multimodal embeddings that understand both scene and task.

Noisy action samples are iteratively denoised through learned velocity fields, producing smooth robot trajectories via flow matching.

Three token blocks are assembled into a single sequence and processed through block-wise causal attention inside the transformer.