• Time: 14:01
• Type: Panel
• Speaker(s):
  • Dev Rishi, CEO and Co-founder, Predibase
  • Margaret, Head of Product, Mistral AI
  • Pablo, Distinguished Scientist and Research Manager, NVIDIA
  • Luna, Lead of the Small Language Model team, Hugging Face
  • Diego, Head of Generative AI Partnerships, Meta
• Session Goal: Discuss the future of generative AI, focusing on the training and serving of small language models.

Definition and Characteristics:

• Small language models can run on laptops and mobile phones with low latency
• Typically less than 3-4 billion parameters
• Optimized through quantization and compression techniques
• Best suited for tasks not requiring extensive world knowledge:
  • Rephrasing
  • Summarization
  • Dialogue generation

Implementation Strategies:

• Hybrid approaches combining small and large models
• Small models for simpler tasks
• Large models for complex queries
• Fine-tuning on synthetic data from larger models
• Focus on agentic workflows for task automation

Hamba Language Model:

• Specifications:
  • 1.5 billion parameters
  • MMLU score: 50
• Use Cases:
  • On-device deployment
  • Rephrasing
  • Summarization
  • Dialogue generation

Technology Shifts:

• Movement toward efficient, device-deployable models
• Growing focus on agentic workflows and automation
• Heavy investment in open-source model development
• Expected acceleration of adoption across industries

Future Outlook (2025):

• Significant advancements in generative AI
• More sophisticated agentic workflows
• Better reasoning engines
• Deeper understanding of workflow construction

The panel established foundational definitions for small language models while highlighting the industry's shift toward more efficient, task-specific implementations. The discussion emphasized the complementary role of small and large models in creating effective AI systems.