Megadose The AI curve, compressed.

Hidden-State Similarity Predicts Re-Elicitation After Inoculation Prompting

· ArXiv · AI/CL/LG ·
Harmful behavior suppressed by inoculation prompting can remain accessible when evaluation prompts recreate similar internal activation states.

Inoculation prompting explicitly elicits unwanted behavior during fine-tuning to reduce its spread beyond the training task. The researchers studied what makes evaluation prompts trigger that behavior again. Textual resemblance to the training prompt was an incomplete predictor; similarity in the model’s internal activations was more informative. The findings suggest activation analysis could help identify prompts under which suppressed behavior remains accessible.

ArXiv · AI/CL/LG's note

score 4

Categories: Research