Concurrent Image Understanding and Generation: Self-Correcting Coupled Markov Jump Processes
The paper proposes a sampler that lets text and image decisions revise each other during the same generation step.
SC-CMJP ties each modality’s transition rates to the other modality’s confidence, weighted through cross-modal attention. Its `CO2Jump` sampler adds remasking, so earlier text or image commitments can be pulled back when new cross-modal evidence contradicts them. The authors say it is training-free, single-pass, and performs best on their joint image editing, understanding, maze, and nonogram benchmarks. They also introduce three planned-release corpora: JEdit-1M, JMaze-200K, and JNono-200K. HF Daily Papers' note
SC-CMJP ties each modality’s transition rates to the other modality’s confidence, weighted through cross-modal attention. Its `CO2Jump` sampler adds remasking, so earlier text or image commitments can be pulled back when new cross-modal evidence contradicts them. The authors say it is training-free, single-pass, and performs best on their joint image editing, understanding, maze, and nonogram benchmarks. They also introduce three planned-release corpora: JEdit-1M, JMaze-200K, and JNono-200K. HF Daily Papers' note
score 5