Youtu-Parsing-Omni: One Encoder, One Schema, All Modalities
A single 5-billion-parameter model turns documents, images, audio and video into structured records that locate evidence in space and time.
The authors report 96.96 on OmniDocBench v1.6 and a leading open-weight score of 75.08 on OmniParsingBench, with the latter advantage concentrated in audio and video. Shared encoder weights and a common output schema support ten parsing tasks, while fusion layers align video frames with concurrent audio. During post-training, the model learns from its own outputs using token-level targets supplied by a reference-conditioned copy of itself. Code and model weights are publicly available.
HF Daily Papers' note
The authors report 96.96 on OmniDocBench v1.6 and a leading open-weight score of 75.08 on OmniParsingBench, with the latter advantage concentrated in audio and video. Shared encoder weights and a common output schema support ten parsing tasks, while fusion layers align video frames with concurrent audio. During post-training, the model learns from its own outputs using token-level targets supplied by a reference-conditioned copy of itself. Code and model weights are publicly available.
HF Daily Papers' note
score 5