Advanced CLI Integration
The CLI is just the beginning of a powerful deployment and experimentation ecosystem. Here’s how CLI deployment integrates with Inworld Portal for complete graph lifecycle management.Complete Deployment + Experimentation Pipeline
1. CLI Development & Deployment- Develop and test locally with
inworld serve - Deploy to cloud with
inworld deploy - Register variants with
inworld graph variant register
- Configure traffic splits in Graph Registry
- Set targeting rules based on user attributes
- Enable experiments for live traffic
- View real-time metrics and dashboards
- Debug with traces and logs
- Analyze experiment performance
- Create new variants based on experiment results
- Deploy winning variants
- Repeat the cycle
Observability Integration
CLI-deployed graphs automatically integrate with Portal’s observability tools:Automatic Metrics Collection
- Default metrics: Execution count, latency, token usage automatically collected
- Custom metrics: Add your own business metrics using telemetry API
- Dashboard visualization: View trends and performance in Portal
Tracing and Debugging
- Execution traces: Every graph execution creates detailed traces
- Error tracking: Failed executions highlighted with error details
- Performance analysis: Identify bottlenecks across graph nodes
Setup for Maximum Observability
Playground Integration
Before deploying to production, use Inworld Portal’s Playground to test and refine your graph components:Pre-Deployment Testing Workflow
-
LLM Playground: Test prompts, models, and parameters
- Experiment with different language models
- Test system prompts and user messages
- Use variables for dynamic content testing
- Validate responses before implementing in code
-
TTS Playground: Test voice synthesis (for LLM+TTS template users)
- Try different AI voices for your use case
- Adjust speech parameters (speed, tone)
- Generate audio samples for quality validation
- Test text-to-speech integration
-
Export to CLI: Once satisfied with Playground results
- Risk Reduction: Validate approach before development
- Faster Iteration: Test ideas quickly without code changes
- Parameter Optimization: Find optimal settings interactively
- Voice Selection: Choose best voices for your template needs
Complete End-to-End Workflow
Here’s the complete workflow from initial development to production monitoring and optimization:Phase 1: Development & Validation
- Use LLM Playground to test prompts and models
- Use TTS Playground to select optimal voices (if using LLM+TTS template)
- Validate core functionality before development
Phase 2: Local Development & Testing
Phase 3: Cloud Deployment & Graph Registry
- Navigate to Graph Registry
- Register your graph ID:
production-graph - Verify baseline variant appears
Phase 4: Experimentation & Optimization
CLI: Create Experimental Variants- Create targeting rules in Graph Registry
- Set traffic distribution (70% baseline, 30% experiment-v1)
- Enable experiment
- Ensure proper user context in production:
Phase 5: Monitoring & Analysis
Portal Dashboards:- Monitor default metrics: execution count, latency, errors
- Create custom dashboards for business metrics
- Track experiment performance by variant
- Traces: Analyze execution flow and performance bottlenecks
- Logs: Debug errors and unexpected behavior
- Metrics: Compare variant performance in real-time
Phase 6: Optimization & Iteration
Based on experiment results:- Gradually increase traffic to winning variants
- Retire underperforming variants
- Update default variant for new users
- Plan next iteration cycle
Phase 7: Production Scaling
- Regular Portal dashboard reviews
- Monthly experiment planning based on metrics
- Quarterly graph architecture optimization
- Regular template and model optimization
Success Metrics
Technical Health:- P99 latency < 2 seconds
- Error rate < 0.1%
- 99.9% uptime
- Weekly variant deployments via CLI
- Monthly A/B test conclusions
- Quarterly major feature rollouts
- Custom metrics tracking business KPIs
- User engagement improvements
- Cost optimization through model efficiency
Production Best Practices
Monitoring & Alerting
Set up alerts for:- Error rate > 1%
- P99 latency > 5 seconds
- Deployment failures
- Variant performance degradation
- Daily dashboard reviews
- Weekly experiment performance analysis
- Monthly architecture optimization review
- Quarterly business impact assessment
Development Workflow
Recommended Git workflow:- Automated testing with
inworld runin CI pipeline - Staging deployment validation
- Production deployment with monitoring
- Automated variant registration for approved changes
Security Considerations
API Key Management:- Use environment-specific keys
- Rotate keys regularly
- Never commit keys to version control
- Use secure secret management systems
- Limit CLI access to authorized developers
- Use role-based permissions in Portal
- Monitor CLI usage and deployments
- Regular access reviews and cleanup
Advanced CLI Tips
Productivity Shortcuts
Aliases for common commands:Bulk Operations
Deploy multiple graphs:Getting Help
Documentation Resources
- CLI Setup & Getting Started - Installation and basic usage
- Deployment Guide - Cloud deployment workflows
- Variants Guide - A/B testing and experimentation
- Portal Integration - Complete Portal features
Community & Support
When encountering issues:- For all troubleshooting: Check the comprehensive CLI Troubleshooting Guide covering setup, development, and production issues
- For workflows and best practices: Review Portal logs/traces and the guidance in this advanced integration guide
- Use
inworld auth statusand--infoflags for debugging - Check for CLI updates: compare
inworld --versionwith latest releases