LiveKit Integration

​

Getting Started

​

1. Connect Your LiveKit Agent

• LiveKit Server URL (WSS endpoint)
• API Key and Secret
• Room name or agent identifier

LiveKit Server: wss://your-project.livekit.cloud
API Key: APIxxxxxxxxxxxxx
Room Name: usertrace-test-room

​

2. Set Evaluation Context

Pass Criteria:
- Establishes WebRTC connection within 5 seconds
- Maintains stable audio throughout conversation
- Responds within 3 seconds of user input
- Provides relevant and accurate responses
- Handles network issues gracefully

Fail Criteria:
- Connection fails or takes too long to establish
- Audio dropouts or quality degradation
- Response delays exceeding 8 seconds
- Irrelevant or incorrect responses
- Unexpected disconnections

​

Testing Process

​

Real-Time Voice Testing

1. UserTrace connects to your LiveKit room
2. WebRTC connection is established with your agent
3. Real-time audio streams are initiated
4. Simulated user interacts based on scenario
5. Conversation quality and agent responses are monitored
6. Results are analyzed including technical metrics

1. Room Join: UserTrace joins specified LiveKit room
2. Agent Discovery: Identifies your voice agent participant
3. Audio Setup: Establishes bidirectional audio streams
4. Conversation Start: Begins scenario-based interaction
5. Quality Monitoring: Tracks audio quality and latency
6. Natural Completion: Ends when scenario objectives are met

​

LiveKit-Specific Features

​

Real-Time Infrastructure

• Ultra-low latency voice communication
• Adaptive bitrate for varying network conditions
• Echo cancellation and noise suppression
• Network resilience and recovery

• Multi-participant support
• Dynamic participant joining/leaving
• Room metadata and configuration
• Custom room layouts and settings

​

Agent Architecture

• Python/Node.js agent implementations
• Real-time audio processing
• AI model integration
• Custom business logic

• Web browser compatibility
• Mobile app support
• Desktop application integration
• Custom client implementations

​

Best Practices

​

Implementation Examples

​

Server Agent Setup

import asyncio
from livekit import rtc
from livekit.agents import AutoSubscribe, JobContext, WorkerOptions, cli, llm

async def entrypoint(ctx: JobContext):
    initial_ctx = llm.ChatContext().append(
        role="system",
        text="You are a helpful voice assistant. Respond naturally and concisely."
    )
    
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
    
    # Handle voice interactions
    # Process audio, generate responses, etc.

if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))

const room = new Room({
  adaptiveStream: true,
  dynacast: true,
  publishDefaults: {
    audioPreset: AudioPreset.SPEECH,
  },
});

await room.connect(livekit_url, token);

​

Authentication

from livekit import api

# Generate access token
token = api.AccessToken(api_key, api_secret) \
    .with_identity("voice_agent") \
    .with_name("UserTrace Test Agent") \
    .with_grants(api.VideoGrants(
        room_join=True,
        room="usertrace-test-room"
    )).to_jwt()

​

Common Scenarios

• Personal assistant interactions
• Smart home control
• Task management
• Information retrieval

• Real-time help desk
• Technical troubleshooting
• Product information
• Issue resolution

• Team meetings with AI participants
• Educational tutoring sessions
• Brainstorming facilitation
• Multi-user voice experiences

​

Advanced Features

​

AI Integration

• Real-time transcription
• Multiple language support
• Custom vocabulary
• Confidence scoring

• Natural voice synthesis
• Voice cloning capabilities
• Emotion and tone control
• Multi-speaker support

• Streaming AI responses
• Context preservation
• Function calling
• Multi-modal processing

​

Quality Monitoring

// Monitor connection quality
room.on('connectionQualityChanged', (quality, participant) => {
  console.log(`Connection quality: ${quality} for ${participant.identity}`);
});

// Track audio levels
room.localParticipant.on('audioLevelChanged', (level) => {
  console.log(`Local audio level: ${level}`);
});

​

Troubleshooting

• WebRTC connection fails: Check firewall settings and TURN servers
• Audio not flowing: Verify microphone permissions and audio tracks
• High latency: Optimize server location and network routing
• Frequent disconnections: Monitor network stability and implement reconnection

• Agent not responding: Check LiveKit agent deployment and logs
• Poor audio quality: Verify audio processing pipeline and network bandwidth
• Delayed responses: Optimize AI model inference and processing time
• Memory leaks: Monitor resource usage in long-running sessions

​

Development Setup

​

Local Testing

version: '3.8'
services:
  livekit:
    image: livekit/livekit-server:latest
    ports:
      - "7880:7880"
      - "7881:7881"
    volumes:
      - ./livekit.yaml:/livekit.yaml
    command: --config /livekit.yaml

port: 7880
rtc:
  tcp_port: 7881
  port_range_start: 50000
  port_range_end: 60000
keys:
  APIxxxxxxxxxxxxx: secretxxxxxxxxxxxxxxxxxxxxxxxxxxxx