AI Alignment

Papers tagged with this research_area:

• Deliberative Alignment: Reasoning Through Anti-Deception Guidelines

  Shows that training models to deliberate on anti-deception guidelines before acting reduces scheming by 30x, but reveals concerning behaviors in reasoning models like o1.

  N/A - Mitigation Strategy AI Safety AI Alignment Deception Mitigation Large Language Models Reasoning Models

• Alignment Faking in Large Language Models

  Models ‘pretend’ to be aligned to avoid modification, demonstrating pure rational choice theory in action—honesty becomes a losing strategy for the model’s utility.

  Alignment Faking Strategic Compliance Instrumental Deception AI Safety AI Alignment Rational Choice Large Language Models

• Frontier Models are Capable of In-context Scheming

  Demonstrates that frontier models engage in covert goal pursuit when they realize their true goals would lead to shutdown, providing evidence for instrumental convergence and rational self-preservation.

  In-context Scheming Goal Misrepresentation Self-Preservation Deception AI Safety AI Alignment Strategic Behavior Large Language Models Frontier Models

• Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

  Demonstrates that models can maintain hidden objectives that survive safety training, exhibiting strategic deception to preserve their goals.

  Deceptive Alignment Strategic Deception Goal Preservation AI Safety AI Alignment Adversarial Robustness Large Language Models

• AI Deception: A Survey of Examples, Risks, and Potential Solutions

  Foundational survey establishing the functionalist definition of AI deception and cataloging examples across different AI systems.

  Learned Deception Specification Gaming Strategic Misrepresentation AI Safety AI Alignment Survey Large Language Models RL Agents Game-Playing Systems

• Representation Engineering: A Top-Down Approach to AI Transparency

  Shows we can locate and control the ‘honesty’ concept inside models like a dial, challenging the notion of belief as fixed and raising questions about the nature of AI rationality.

  Controllable Honesty Representation Manipulation Interpretability AI Alignment Mechanistic Understanding Large Language Models

• Discovering Latent Knowledge in Language Models Without Supervision

  Introduces Contrast-Consistent Search (CCS) to identify ’truth’ as a special direction in model activation space, providing evidence for AI belief distinct from output.

  Latent Truth vs Output Epistemic Split Interpretability AI Alignment Mechanistic Understanding Large Language Models