Presented by: Aman Singhal

Venue: Large Language and Vision Models Symposium, NYU Center for Data Science

Year: 2024

Abstract

Llama 2 is Meta’s open-source LLM trained on 2 trillion tokens, featuring a unique approach of keeping harmful content in pretraining data to teach the model what not to do via RLHF. The model uses two separate reward models—one for helpfulness and one for safety—to balance these sometimes competing objectives. It achieved strong performance on safety benchmarks and spawned over 1,000 research papers within 4 months, enabling work on multimodal AI, adversarial robustness, and various applications.

Presentation Slides

View slides in new window

Key Topics

• Open-source LLM architecture and training data
• Novel approach to safety: keeping harmful content in pretraining
• Dual reward model system (helpfulness vs. safety)
• RLHF implementation and methodology
• Safety benchmark performance
• Research impact: 1,000+ papers in 4 months
• Applications in multimodal AI and adversarial robustness
• Comparison with other open-source models