Megadose The AI curve, compressed.

Simplicity Goes Far: Auditing Prompt Optimizers with Demonstration-Conditioned Prompt Search

· ArXiv · AI/CL/LG ·
A simple generate-and-select baseline matched two heavier prompt optimizers at substantially lower measured cost.

Demonstration-Conditioned Prompt Search freezes the generator, uses randomly sampled demonstrations to propose prompts, and keeps the best validation-scoring candidate. Replacing PPO in StablePrompt, it matched classification accuracy within 0.5 percentage points at roughly one-third the measured cost. In DSPy systems using GPT-4.1-mini, it matched GEPA’s macro accuracy under the reported budgets, with 55% fewer rollouts and 32% lower measured API cost on average. Additional optimizer mechanisms retained advantages in some open-ended and reasoning-heavy settings. The paper recommends generate-and-select controls and reporting rollout counts, token usage, and dollar costs alongside accuracy.

ArXiv · AI/CL/LG's note

score 5

Categories: Research