Unifying Conformal Language Tasks with In-Context Ensembles
The paper proposes a conformal relevance framework that uses curated in-context examples and ensembles to keep coverage guarantees while making retrieved text shorter.
The method targets NLP tasks that can be framed as pulling relevant content from documents, including summarization and extractive QA. It replaces hand-engineered LLM importance prompts with an in-context ensembling approach meant to reduce task-specific prompt work. The authors test it across seven NLP tasks and add theory on when diversity in conformal score ensembles helps, including limits on that improvement. Source: HF Daily Papers' note
The method targets NLP tasks that can be framed as pulling relevant content from documents, including summarization and extractive QA. It replaces hand-engineered LLM importance prompts with an in-context ensembling approach meant to reduce task-specific prompt work. The authors test it across seven NLP tasks and add theory on when diversity in conformal score ensembles helps, including limits on that improvement. Source: HF Daily Papers' note
score 4