• Time: 14:50
• Type: Technical Talk
• Speaker: Manji, Leader on AI Platforms for Agent Force team at Einstein (Salesforce)
• Session Goal: Share lessons learned from building LLM-based AI agents and workflows at scale

Salesforce Agent Force:

• Low-code/no-code platform for AI agents
• Studio and builder interface
• Workflow integration capabilities
• Focus on trust and responsible AI
• Comprehensive testing and monitoring tools

Use Cases:

• Service automation
• Sales assistance
• Marketing content generation
• Customer interaction management

Agent Evolution:

• Progression from rule-based chatbots to reasoning agents
• Integration beyond conversational interfaces
• Enterprise knowledge grounding
• Action-taking capabilities

Implementation Best Practices:

• Define evaluation metrics (eVALs) upfront
• Conduct thorough batch testing
• Assess knowledge quality
• Optimize retrieval/generation pipelines
• Fine-tune for specific tasks

Key Challenges:

• Hallucination mitigation
• Cost/performance optimization
• Complex system evaluation
• Knowledge silos
• Trust building

Platform Components:

• Agent builder
• Testing suite
• Deployment tools
• Monitoring systems

Performance Metrics:

• Accuracy
• Precision
• Latency
• Token count efficiency

Key Quotes: > "Don't just think about the conversational interface because these agents are going to be embedded into any workflow you can think of."

> "Building the flashy demo is only 10% of the work, and the rest 90% of the hard work is actually building the trust that this solution works."

> "Teams that are doing very fast iterations… are moving fast and that's why I think this ability to tune things, make changes and test is super super important."

Technology Shifts:

• Movement toward reasoning-capable agents
• Emphasis on fine-tuned applications
• Focus on comprehensive toolchains
• Integration across workflows

The session emphasized the importance of systematic development approaches and trust-building in AI agent deployment, highlighting the evolution from simple chatbots to sophisticated workflow automation tools.