Megadose The AI curve, compressed.

StreamEmbed: Real-Time Indexing from Pixels for Screenshot Retrieval

· ArXiv · AI/CL/LG ·
StreamEmbed encodes screenshots up to 3.7 times faster without sacrificing retrieval accuracy in the authors’ benchmark.

It skips repeated image blocks and reuses cached computations for unchanged regions, with a small learned adapter preserving embedding fidelity. Tests on an M2 Pro GPU used 10,426 questions over 184 hours of screen histories from two humans and a computer-use agent. At matched latency, it beat OCR-based indexing by up to 6.3 Recall@10 points. Its memory-efficient vector index held a simulated year of screenshots in 0.13 GB of RAM.

ArXiv · AI/CL/LG's note

score 5

Categories: Research