• Time: 14:01
• Type: Panel
• Speaker(s):
  • Dev Rishi, CEO and Co-founder, Predibase
  • Margaret, Head of Product, Mistral AI
  • Pablo, Distinguished Scientist and Research Manager, NVIDIA
  • Luna, Lead of the Small Language Model team, Hugging Face
  • Diego, Head of Generative AI Partnerships, Meta
• Session Goal: Discuss the future of generative AI, focusing on the training and serving of small language models

Defining SLMs:

• Models running on laptops/mobile devices with low latency
• Typically less than 3-4 billion parameters
• Optimized through quantization and compression
• Suited for specific tasks not requiring extensive world knowledge:
  • Rephrasing
  • Summarization
  • Dialogue generation

Implementation Strategies:

• Hybrid deployment combining small and large models
• Task-based model selection
• Fine-tuning with synthetic data from larger models
• Focus on agentic workflows
• Development of reasoning engines

Hamba Language Model:

• 1.5 billion parameters
• MMLU score: 50
• Designed for:
  • On-device deployment
  • Rephrasing
  • Summarization
  • Dialogue generation

2025 Predictions:

• Advanced generative AI capabilities
• Sophisticated agentic workflows
• Improved reasoning engines
• Enhanced deployment strategies

Industry Direction:

• Investment in open-source development
• Focus on device-deployable models
• Growing agentic workflow adoption
• Widespread industry implementation

The panel established core definitions and characteristics of SLMs while highlighting their role in the future of AI deployment. Key emphasis was placed on the practical advantages of small models and their complementary relationship with larger systems.