update livekit inference model to match latest #993
Conversation
🦋 Changeset detectedLatest commit: 5c736a8 The changes in this PR will be included in the next version bump. This PR includes changesets to release 18 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📝 WalkthroughWalkthroughThis PR updates the LiveKit Agents inference model types across LLM, STT, and TTS services. It introduces new model variants (GPT-5.1/5.2, Deepgram/Aura, Moonshot, AssemblyAI, ElevenLabs), removes deprecated options, and modernizes existing model unions to reflect current provider APIs. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In `@agents/src/inference/llm.ts`:
- Around line 85-90: The LLMModels union type is split across multiple lines
causing Prettier to fail; update the declaration of LLMModels so the entire
union is on a single line (e.g., collapse the multi-line union of OpenAIModels,
GoogleModels, MoonshotModels, DeepSeekModels, and AnyString into one line) to
conform with the repo's Prettier rules and re-run formatting/linting for the
file containing the LLMModels type.
In `@agents/src/inference/stt.ts`:
- Around line 36-40: Prettier flags the AssemblyaiModels union formatting;
update the type declarations so each union member is on its own line —
specifically reformat the AssemblyaiModels type (and keep consistent style with
CartesiaModels and ElevenlabsSTTModels) so the two members
'assemblyai/universal-streaming' and
'assemblyai/universal-streaming-multilingual' are split across separate lines to
satisfy Prettier.
In `@agents/src/inference/tts.ts`:
- Around line 58-62: Replace the three empty interface declarations
DeepgramTTSOptions, RimeOptions, and InworldOptions with equivalent type aliases
(e.g. "type DeepgramTTSOptions = {}") so they are not flagged as empty
interfaces by CI; update the declarations for DeepgramTTSOptions, RimeOptions,
and InworldOptions in the tts module accordingly, keeping names unchanged and
preserving exported visibility.
- Around line 64-72: Prettier flagged formatting on the _TTSModels / TTSModels
union; reformat these type declarations to match the project's Prettier rules
(consistent line breaks and pipe placement) — specifically update the _TTSModels
and export type TTSModels declarations (symbols: _TTSModels and TTSModels) so
the union members are arranged and indented according to the repo style, then
run Prettier (or your editor's format) to ensure CI passes.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
.changeset/kind-coins-add.mdagents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Add SPDX-FileCopyrightText and SPDX-License-Identifier headers to all newly added files with '// SPDX-FileCopyrightText: 2025 LiveKit, Inc.' and '// SPDX-License-Identifier: Apache-2.0'
Files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
**/*.{ts,tsx}?(test|example|spec)
📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
When testing inference LLM, always use full model names from
agents/src/inference/models.ts(e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')
Files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
**/*.{ts,tsx}?(test|example)
📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Initialize logger before using any LLM functionality with
initializeLogger({ pretty: true })from '@livekit/agents'
Files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
🧠 Learnings (2)
📓 Common learnings
Learnt from: CR
Repo: livekit/agents-js PR: 0
File: .cursor/rules/agent-core.mdc:0-0
Timestamp: 2026-01-16T14:33:39.551Z
Learning: Applies to **/*.{ts,tsx}?(test|example|spec) : When testing inference LLM, always use full model names from `agents/src/inference/models.ts` (e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')
📚 Learning: 2026-01-16T14:33:39.551Z
Learnt from: CR
Repo: livekit/agents-js PR: 0
File: .cursor/rules/agent-core.mdc:0-0
Timestamp: 2026-01-16T14:33:39.551Z
Learning: Applies to **/*.{ts,tsx}?(test|example|spec) : When testing inference LLM, always use full model names from `agents/src/inference/models.ts` (e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')
Applied to files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
🧬 Code graph analysis (2)
agents/src/inference/stt.ts (1)
agents/src/inference/tts.ts (1)
CartesiaModels(25-29)
agents/src/inference/tts.ts (3)
agents/src/inference/utils.ts (1)
AnyString(8-8)plugins/deepgram/src/tts.ts (1)
TTSOptions(21-29)plugins/cartesia/src/tts.ts (1)
TTSOptions(30-45)
🪛 Biome (2.1.2)
agents/src/inference/tts.ts
[error] 58-58: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
[error] 60-60: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
[error] 62-62: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
🪛 GitHub Actions: Build
agents/src/inference/llm.ts
[error] 85-85: Prettier: Replace ⏎··|·OpenAIModels⏎··|·GoogleModels⏎··|·MoonshotModels⏎··|·DeepSeekModels with OpenAIModels|GoogleModels|MoonshotModels|DeepSeekModels [Error/prettier/prettier]
agents/src/inference/stt.ts
[error] 38-38: Prettier: Replace ·'assemblyai/universal-streaming' with ⏎··|·'assemblyai/universal-streaming'⏎· [Error/prettier/prettier]
agents/src/inference/tts.ts
[warning] 58-58: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[warning] 60-60: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[warning] 62-62: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[error] 64-64: Prettier: Replace CartesiaModels|DeepgramTTSModels|ElevenlabsModels|RimeModels with CartesiaModels|DeepgramTTSModels|ElevenlabsModels|RimeModels [Error/prettier/prettier]
🔇 Additional comments (9)
agents/src/inference/llm.ts (3)
19-32: LGTM!The new OpenAI GPT-5.x model variants follow the established naming convention and integrate correctly into the existing type union.
34-41: LGTM!The Google models have been updated from preview to stable variants (
gemini-3-pro,gemini-3-flash), which aligns with the PR objective of matching the latest inference models.
43-45: LGTM!New MoonshotModels and expanded DeepSeekModels types are well-defined and follow the
provider/modelnaming convention.agents/src/inference/stt.ts (2)
27-34: LGTM!DeepgramModels appropriately expanded with
flux-generalandnova-2-conversationalaivariants, following the established naming convention.
80-84: LGTM!The
_STTModelsunion correctly includes all the new model types (ElevenlabsSTTModels, AssemblyaiModels) alongside existing ones.agents/src/inference/tts.ts (3)
25-31: LGTM!CartesiaModels and DeepgramTTSModels are well-defined. The Cartesia sonic variants and Deepgram aura models follow the established naming convention.
40-46: LGTM!InworldModels and updated RimeModels types are correctly defined with the
provider/modelnaming convention.
76-86: LGTM!The
TTSOptions<TModel>conditional type correctly maps each model type to its corresponding options interface, with proper nesting and fallback toRecord<string, unknown>..changeset/kind-coins-add.md (1)
1-5: LGTM!The changeset correctly identifies this as a patch release for
@livekit/agents. A patch bump is appropriate since the changes are additive (new model types) without breaking existing functionality.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| export type LLMModels = | ||
| | OpenAIModels | ||
| | GoogleModels | ||
| | QwenModels | ||
| | KimiModels | ||
| | MoonshotModels | ||
| | DeepSeekModels | ||
| | AnyString; |
There was a problem hiding this comment.
Fix Prettier formatting to pass CI.
The pipeline indicates a Prettier formatting error. The union type should be on a single line per Prettier's configuration.
Proposed fix
-export type LLMModels =
- | OpenAIModels
- | GoogleModels
- | MoonshotModels
- | DeepSeekModels
- | AnyString;
+export type LLMModels = OpenAIModels | GoogleModels | MoonshotModels | DeepSeekModels | AnyString;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export type LLMModels = | |
| | OpenAIModels | |
| | GoogleModels | |
| | QwenModels | |
| | KimiModels | |
| | MoonshotModels | |
| | DeepSeekModels | |
| | AnyString; | |
| export type LLMModels = OpenAIModels | GoogleModels | MoonshotModels | DeepSeekModels | AnyString; |
🧰 Tools
🪛 GitHub Actions: Build
[error] 85-85: Prettier: Replace ⏎··|·OpenAIModels⏎··|·GoogleModels⏎··|·MoonshotModels⏎··|·DeepSeekModels with OpenAIModels|GoogleModels|MoonshotModels|DeepSeekModels [Error/prettier/prettier]
🤖 Prompt for AI Agents
In `@agents/src/inference/llm.ts` around lines 85 - 90, The LLMModels union type
is split across multiple lines causing Prettier to fail; update the declaration
of LLMModels so the entire union is on a single line (e.g., collapse the
multi-line union of OpenAIModels, GoogleModels, MoonshotModels, DeepSeekModels,
and AnyString into one line) to conform with the repo's Prettier rules and
re-run formatting/linting for the file containing the LLMModels type.
| export type CartesiaModels = 'cartesia/ink-whisper'; | ||
|
|
||
| export type AssemblyaiModels = 'assemblyai/universal-streaming' | 'assemblyai/universal-streaming-multilingual'; | ||
|
|
||
| export type AssemblyaiModels = 'assemblyai' | 'assemblyai/universal-streaming'; | ||
| export type ElevenlabsSTTModels = 'elevenlabs/scribe_v2_realtime'; |
There was a problem hiding this comment.
Fix Prettier formatting to pass CI.
The pipeline indicates a Prettier formatting error on line 38. The AssemblyaiModels union should be formatted across multiple lines.
Proposed fix
export type CartesiaModels = 'cartesia/ink-whisper';
-
-export type AssemblyaiModels = 'assemblyai/universal-streaming' | 'assemblyai/universal-streaming-multilingual';
+
+export type AssemblyaiModels =
+ | 'assemblyai/universal-streaming'
+ | 'assemblyai/universal-streaming-multilingual';
export type ElevenlabsSTTModels = 'elevenlabs/scribe_v2_realtime';📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export type CartesiaModels = 'cartesia/ink-whisper'; | |
| export type AssemblyaiModels = 'assemblyai/universal-streaming' | 'assemblyai/universal-streaming-multilingual'; | |
| export type AssemblyaiModels = 'assemblyai' | 'assemblyai/universal-streaming'; | |
| export type ElevenlabsSTTModels = 'elevenlabs/scribe_v2_realtime'; | |
| export type CartesiaModels = 'cartesia/ink-whisper'; | |
| export type AssemblyaiModels = | |
| | 'assemblyai/universal-streaming' | |
| | 'assemblyai/universal-streaming-multilingual'; | |
| export type ElevenlabsSTTModels = 'elevenlabs/scribe_v2_realtime'; |
🧰 Tools
🪛 GitHub Actions: Build
[error] 38-38: Prettier: Replace ·'assemblyai/universal-streaming' with ⏎··|·'assemblyai/universal-streaming'⏎· [Error/prettier/prettier]
🤖 Prompt for AI Agents
In `@agents/src/inference/stt.ts` around lines 36 - 40, Prettier flags the
AssemblyaiModels union formatting; update the type declarations so each union
member is on its own line — specifically reformat the AssemblyaiModels type (and
keep consistent style with CartesiaModels and ElevenlabsSTTModels) so the two
members 'assemblyai/universal-streaming' and
'assemblyai/universal-streaming-multilingual' are split across separate lines to
satisfy Prettier.
| export interface DeepgramTTSOptions {} | ||
|
|
||
| export interface RimeOptions {} | ||
|
|
||
| export interface InworldOptions {} |
There was a problem hiding this comment.
Convert empty interfaces to type aliases to pass CI.
The pipeline flags these empty interfaces. Use type aliases instead, which is the recommended pattern for placeholder option types.
Proposed fix
-export interface DeepgramTTSOptions {}
-
-export interface RimeOptions {}
-
-export interface InworldOptions {}
+export type DeepgramTTSOptions = Record<string, unknown>;
+
+export type RimeOptions = Record<string, unknown>;
+
+export type InworldOptions = Record<string, unknown>;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export interface DeepgramTTSOptions {} | |
| export interface RimeOptions {} | |
| export interface InworldOptions {} | |
| export type DeepgramTTSOptions = Record<string, unknown>; | |
| export type RimeOptions = Record<string, unknown>; | |
| export type InworldOptions = Record<string, unknown>; |
🧰 Tools
🪛 Biome (2.1.2)
[error] 58-58: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
[error] 60-60: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
[error] 62-62: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
🪛 GitHub Actions: Build
[warning] 58-58: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[warning] 60-60: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[warning] 62-62: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
🤖 Prompt for AI Agents
In `@agents/src/inference/tts.ts` around lines 58 - 62, Replace the three empty
interface declarations DeepgramTTSOptions, RimeOptions, and InworldOptions with
equivalent type aliases (e.g. "type DeepgramTTSOptions = {}") so they are not
flagged as empty interfaces by CI; update the declarations for
DeepgramTTSOptions, RimeOptions, and InworldOptions in the tts module
accordingly, keeping names unchanged and preserving exported visibility.
| type _TTSModels = CartesiaModels | DeepgramTTSModels | ElevenlabsModels | RimeModels | InworldModels; | ||
|
|
||
| export type TTSModels = CartesiaModels | ElevenlabsModels | RimeModels | InworldModels | AnyString; | ||
| export type TTSModels = | ||
| | CartesiaModels | ||
| | DeepgramTTSModels | ||
| | ElevenlabsModels | ||
| | RimeModels | ||
| | InworldModels | ||
| | AnyString; |
There was a problem hiding this comment.
Fix Prettier formatting to pass CI.
The pipeline indicates a Prettier formatting error on the _TTSModels union type definition.
Proposed fix
-type _TTSModels = CartesiaModels | DeepgramTTSModels | ElevenlabsModels | RimeModels | InworldModels;
+type _TTSModels =
+ | CartesiaModels
+ | DeepgramTTSModels
+ | ElevenlabsModels
+ | RimeModels
+ | InworldModels;📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| type _TTSModels = CartesiaModels | DeepgramTTSModels | ElevenlabsModels | RimeModels | InworldModels; | |
| export type TTSModels = CartesiaModels | ElevenlabsModels | RimeModels | InworldModels | AnyString; | |
| export type TTSModels = | |
| | CartesiaModels | |
| | DeepgramTTSModels | |
| | ElevenlabsModels | |
| | RimeModels | |
| | InworldModels | |
| | AnyString; | |
| type _TTSModels = | |
| | CartesiaModels | |
| | DeepgramTTSModels | |
| | ElevenlabsModels | |
| | RimeModels | |
| | InworldModels; | |
| export type TTSModels = | |
| | CartesiaModels | |
| | DeepgramTTSModels | |
| | ElevenlabsModels | |
| | RimeModels | |
| | InworldModels | |
| | AnyString; |
🧰 Tools
🪛 GitHub Actions: Build
[error] 64-64: Prettier: Replace CartesiaModels|DeepgramTTSModels|ElevenlabsModels|RimeModels with CartesiaModels|DeepgramTTSModels|ElevenlabsModels|RimeModels [Error/prettier/prettier]
🤖 Prompt for AI Agents
In `@agents/src/inference/tts.ts` around lines 64 - 72, Prettier flagged
formatting on the _TTSModels / TTSModels union; reformat these type declarations
to match the project's Prettier rules (consistent line breaks and pipe
placement) — specifically update the _TTSModels and export type TTSModels
declarations (symbols: _TTSModels and TTSModels) so the union members are
arranged and indented according to the repo style, then run Prettier (or your
editor's format) to ensure CI passes.
lukasIO
left a comment
lukasIO
left a comment
There was a problem hiding this comment.
coderabbitai is right about the formatter still needing to run.
Looks good to me.
A general question: should we surface the fact that some of these are deprecated now? Right now users don't get any info on this, wondering if we should collect the deprecated ones in an Array and log a warning if they're still being used?
Summary by CodeRabbit
New Features
Deprecations
✏️ Tip: You can customize this high-level summary in your review settings.