Benchmarking

​

Overview

• End-to-end latency
• Audio generation latency per second
• Streaming metrics (first chunk, 4th chunk, average chunk latencies)
• Request success rates
• Server performance under different load conditions

​

Quick start

# Install the load test tool
pip install -e .

# Basic load test with streaming
python load-test.main \
    --host http://localhost:8081 \
    --stream \
    --min-qps 1.0 \
    --max-qps 7.0 \
    --qps-step 2.0 \
    --number-of-samples 300

​

Parameters

​

Required

​

Load configuration

​

TTS configuration

​

Output and analysis

​

Examples

​

Streaming vs non-streaming comparison

# Non-streaming test
python load-test.main \
    --host http://localhost:8081 \
    --min-qps 10.0 \
    --max-qps 50.0 \
    --qps-step 10.0 \
    --number-of-samples 500 \
    --benchmark_name non-streaming-test

# Streaming test
python load-test.main \
    --host http://localhost:8081 \
    --stream \
    --min-qps 10.0 \
    --max-qps 50.0 \
    --qps-step 10.0 \
    --number-of-samples 500 \
    --benchmark_name streaming-test

​

Plot-only mode

./scripts/tts-load-test \
    --plot_only \
    --benchmark_name prod-stress-test

​

Understanding results

​

Latency metrics

• E2E Latency: Complete request-response time
• Audio Generation Latency: Time per second of generated audio
• First Chunk Latency: Time to first audio chunk (streaming only)
• 4th Chunk Latency: Time to 4th audio chunk (streaming only)
• Average Chunk Latency: Mean time between chunks (streaming only)

​

Percentiles

​

Output files

• result.json — Raw performance data
• {benchmark_name}_*.png — Performance charts

​

Burstiness parameter

​

Performance tips

1. Start small — Begin with low QPS and small sample sizes
2. Use appropriate text samples — Match your production text length distribution
3. Monitor server resources — Watch CPU, memory, and network during tests
4. Consider burstiness — Real-world traffic is often bursty (try 0.7–0.9)
5. Test both modes — Compare streaming vs non-streaming for your use case

​

Troubleshooting

​

Common issues

​

Debug mode

./scripts/tts-load-test --verbose --host ... # other params

​

Architecture

• Async/await: Efficient concurrent request handling
• Pausable timers: Accurate server-only timing measurements
• Multiple protocols: gRPC, HTTP REST API support
• Configurable clients: Pluggable client architecture
• Real-time progress: Live progress bars and status updates