-   Name: SmallCon
-   Date: December 11, 2024
-   Focus: AI Agents, LLMs, Fine-tuning, and Agent Development Lifecycle
-   Format: Virtual


# 👥 Session Details

-   Time: 14:50
-   Type: Technical Talk
-   Speaker: Manji, Leader on AI Platforms for Agent Force team at Einstein (Salesforce)
-   Session Goal: Share lessons learned from building LLM-based AI agents and workflows at scale


# 🤖 Platform Overview

Salesforce Agent Force:

-   Low-code/no-code platform for AI agents
-   Studio and builder interface
-   Workflow integration capabilities
-   Focus on trust and responsible AI
-   Comprehensive testing and monitoring tools

Use Cases:

-   Service automation
-   Sales assistance
-   Marketing content generation
-   Customer interaction management


# 💡 Key Technical Insights

Agent Evolution:

-   Progression from rule-based chatbots to reasoning agents
-   Integration beyond conversational interfaces
-   Enterprise knowledge grounding
-   Action-taking capabilities

Implementation Best Practices:

-   Define evaluation metrics (eVALs) upfront
-   Conduct thorough batch testing
-   Assess knowledge quality
-   Optimize retrieval/generation pipelines
-   Fine-tune for specific tasks

Key Challenges:

-   Hallucination mitigation
-   Cost/performance optimization
-   Complex system evaluation
-   Knowledge silos
-   Trust building


# 📈 Technical Architecture

Platform Components:

-   Agent builder
-   Testing suite
-   Deployment tools
-   Monitoring systems

Performance Metrics:

-   Accuracy
-   Precision
-   Latency
-   Token count efficiency


# 💬 Notable Insights

Key Quotes:
> "Don't just think about the conversational interface because these agents are going to be embedded into any workflow you can think of."

> "Building the flashy demo is only 10% of the work, and the rest 90% of the hard work is actually building the trust that this solution works."

> "Teams that are doing very fast iterations… are moving fast and that's why I think this ability to tune things, make changes and test is super super important."


# 📋 Industry Impact

Technology Shifts:

-   Movement toward reasoning-capable agents
-   Emphasis on fine-tuned applications
-   Focus on comprehensive toolchains
-   Integration across workflows

The session emphasized the importance of systematic development approaches and trust-building in AI agent deployment, highlighting the evolution from simple chatbots to sophisticated workflow automation tools.