• Time: 16:53
• Type: Technical Presentation
• Speaker: Arnav Garg, ML Engineering Lead, Predibase
• Session Goal: Discuss strategies for updating machine learning models in production using data collected from production.

Training Strategies:

• Continuous model quality improvement for production LLMs
• Incremental fine-tuning for cost-effective updates
• Rehearsal learning for performance enhancement
• Hybrid approach combining:
  • Incremental updates
  • Periodic full retraining
  • Performance/cost balance

Predibase Platform:

• SDK and UI components
• 100+ base models for LoRA fine-tuning
• Incremental training via continue_from_version
• Configurable retraining interface

Deployment Options:

• SDK integration
• UI-based configuration
• LoRA parameter customization
• Learning configuration flexibility

Efficiency Gains:

• Improved precision and accuracy
• Reduced training costs
• Faster update cycles
• Better data utilization

Production Advantages:

• Continuous model improvement
• Cost-effective updates
• Rapid knowledge incorporation
• User feedback integration

Implementation Strategy:

1. Start with Predibase platform exploration
2. Experiment with incremental training
3. Implement rehearsal learning
4. Develop hybrid training approach
5. Monitor performance metrics
6. Optimize cost efficiency

Resources:

• Predibase SDK documentation
• Platform guidelines
• Integration examples
• Training configurations

The session highlighted how continuous model updates can be practically implemented in production environments, with particular emphasis on balancing performance improvements with operational costs through incremental training approaches.