-   Name: SmallCon
-   Date: December 11, 2024
-   Focus: Small Language Models (SLM)
-   Format: In-person


# 👥 Session Details

-   Time: 14:01
-   Type: Panel
-   Speaker(s):
    -   Dev Rishi, CEO and Co-founder, Predibase
    -   Margaret, Head of Product, Mistral AI
    -   Pablo, Distinguished Scientist and Research Manager, NVIDIA
    -   Luna, Lead of the Small Language Model team, Hugging Face
    -   Diego, Head of Generative AI Partnerships, Meta
-   Session Goal: Discuss the future of generative AI, focusing on the training and serving of small language models.


# 💡 Key Technical Insights

Definition and Characteristics:

-   Small language models can run on laptops and mobile phones with low latency
-   Typically less than 3-4 billion parameters
-   Optimized through quantization and compression techniques
-   Best suited for tasks not requiring extensive world knowledge:
    -   Rephrasing
    -   Summarization
    -   Dialogue generation

Implementation Strategies:

-   Hybrid approaches combining small and large models
-   Small models for simpler tasks
-   Large models for complex queries
-   Fine-tuning on synthetic data from larger models
-   Focus on agentic workflows for task automation


# 🤖 Technical Announcements

Hamba Language Model:

-   Specifications:
    -   1.5 billion parameters
    -   MMLU score: 50
-   Use Cases:
    -   On-device deployment
    -   Rephrasing
    -   Summarization
    -   Dialogue generation


# 📈 Industry Trends

Technology Shifts:

-   Movement toward efficient, device-deployable models
-   Growing focus on agentic workflows and automation
-   Heavy investment in open-source model development
-   Expected acceleration of adoption across industries

Future Outlook (2025):

-   Significant advancements in generative AI
-   More sophisticated agentic workflows
-   Better reasoning engines
-   Deeper understanding of workflow construction

The panel established foundational definitions for small language models while highlighting the industry's shift toward more efficient, task-specific implementations. The discussion emphasized the complementary role of small and large models in creating effective AI systems.