GeoPair: Geometry-Preserving Cross-Layer Factorization for Training-Free Transformer Compression
GeoPair compresses transformers by pairing compatible layers and preserving their separate activation geometry without retraining.
The paper argues that many compression pipelines either treat layers alone or group them with heuristics. GeoPair instead optimizes cross-layer weight pairings and shared-dictionary factorizations in sequence. The authors say this preserves functional fidelity better than independent structured decompositions and other pairwise methods across architectures, scales, and modalities. HF Daily Papers' note
The paper argues that many compression pipelines either treat layers alone or group them with heuristics. GeoPair instead optimizes cross-layer weight pairings and shared-dictionary factorizations in sequence. The authors say this preserves functional fidelity better than independent structured decompositions and other pairwise methods across architectures, scales, and modalities. HF Daily Papers' note
score 5