For clean Markdown of any page, append .md to the page URL. For a complete documentation index, see For full documentation content, see For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at

Speaker Diarization

Add speaker labels to your transcript

For the complete documentation index, see llms.txt

Supported models: Universal-3 Pro (universal-3-pro), Universal-2 (universal-2). Supported languages: en, en_au, en_uk, en_us, es, fr, de, it, pt, nl, hi, ja, zh, fi, ko, pl, ru, tr, uk, vi. Supported regions: US and EU. Key parameters: Set "speaker_labels": true. Optionally set "speakers_expected" (1-20) to hint the expected number of speakers. The response includes an utterances array with speaker, text, start, and end for each speaker turn.

cURL quickstart:

curl  \
  --header "Authorization: YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "audio_url": "",
    "speech_models": ["universal-3-pro", "universal-2"],
    "speaker_labels": true
  }'

US & EU

Learn how to identify who said what in your audio files by adding speaker labels to your transcript.

The Speaker Diarization model lets you detect multiple speakers in an audio file and what each speaker said.

If you enable Speaker Diarization, the resulting transcript will return a list of utterances, where each utterance corresponds to an uninterrupted segment of speech from a single speaker.

Speaker Diarization assigns generic labels like "Speaker A" and "Speaker B" to distinguish between speakers. If you want to replace these labels with actual names or roles (e.g., "John Smith" or "Customer"), use Speaker Identification. Speaker Identification analyzes the conversation content to infer who is speaking and transforms your transcript from generic labels to meaningful identifiers.

Quickstart

To enable Speaker Diarization, set speaker_labels to True in the POST request body:

import requests
import time

base_url = ""

headers = {
    "authorization": "<YOUR_API_KEY>"
}

with open("./my-audio.mp3", "rb") as f:
  response = requests.post(base_url + "/v2/upload",
                          headers=headers,
                          data=f)

upload_url = response.json()["upload_url"]

data = {
    "audio_url": upload_url, # You can also use a URL to an audio or video file on the web
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": True,
    "speaker_labels": True
}

url = base_url + "/v2/transcript"
response = requests.post(url, json=data, headers=headers)

transcript_id = response.json()['id']
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

while True:
  transcription_result = requests.get(polling_endpoint, headers=headers).json()

  if transcription_result['status'] == 'completed':
    print(f"Transcript ID: {transcript_id}")
    break

  elif transcription_result['status'] == 'error':
    raise RuntimeError(f"Transcription failed: {transcription_result['error']}")

  else:
    time.sleep(3)

for utterance in transcription_result['utterances']:
  print(f"Speaker {utterance['speaker']}: {utterance['text']}")

To enable Speaker Diarization, set speaker_labels to True in the transcription config.

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

# You can use a local filepath:
# audio_file = "./example.mp3"

# Or use a publicly-accessible URL:
audio_file = (
    ""
)

config = aai.TranscriptionConfig(
  speech_models=["universal-3-pro", "universal-2"],
  language_detection=True,
  speaker_labels=True,
)

transcript = aai.Transcriber().transcribe(audio_file, config)

for utterance in transcript.utterances:
  print(f"Speaker {utterance.speaker}: {utterance.text}")

To enable Speaker Diarization, set speaker_labels to true in the POST request body:

import fs from "fs-extra";

const baseUrl = "";

const headers = {
  authorization: "<YOUR_API_KEY>",
};

const path = "./audio/audio.mp3";
const audioData = await fs.readFile(path);

let res = await fetch(`${baseUrl}/v2/upload`, {
  method: "POST",
  headers,
  body: audioData,
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const uploadResponse = await res.json();

const uploadUrl = uploadResponse.upload_url;

const data = {
  audio_url: uploadUrl, // You can also use a URL to an audio or video file on the web
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
};

const url = `${baseUrl}/v2/transcript`;
res = await fetch(url, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    for (const utterance of transcriptionResult.utterances) {
      console.log(`Speaker ${utterance.speaker}: ${utterance.text}`);
    }
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

To enable Speaker Diarization, set speaker_labels to true in the transcription config.

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

// You can use a local filepath:
// const audioFile = "./example.mp3"

// Or use a publicly-accessible URL:
const audioFile = "";

const params = {
  audio: audioFile,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  for (const utterance of transcript.utterances ?? []) {
    console.log(`Speaker ${utterance.speaker}: ${utterance.text}`);
  }
};

run();

Example output

Speaker A: Smoke from hundreds of wildfires in Canada is triggering air quality alerts
throughout the US. Skylines from Maine to Maryland to Minnesota are gray and smoggy.
And in some places, the weights of the air qualitative index has exceeded 300.
Speaker B: Well, we have seen in the past when smoke from prior years and prior fires
has come up from prior years to the east coast. But this particular time around,
it's been notable in just how much of the of the country has been affected.

The default upper limit on the number of speakers depends on the audio duration:

• 0–2 minutes: no max speaker limit
• 2–10 minutes: max 10 speakers
• 10+ minutes: max 30 speakers

If you need a different limit, you can use speaker_options to set a range of possible speakers.

Set number of speakers expected

You can set the number of speakers expected in the audio file by setting the speakers_expected parameter.

Only use this parameter if you are certain about the number of speakers in the audio file.

Building on the Quickstart above, add speakers_expected to your transcription config:

data = {
    "audio_url": upload_url,
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": True,
    "speaker_labels": True,
    "speakers_expected": 5
}

config = aai.TranscriptionConfig(
  speech_models=["universal-3-pro", "universal-2"],
  language_detection=True,
  speaker_labels=True,
  speakers_expected=5,
)

const data = {
  audio_url: uploadUrl,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
  speakers_expected: 5,
};

const params = {
  audio: audioFile,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
  speakers_expected: 5,
};

Set a range of possible speakers

You can set a range of possible speakers in the audio file by setting the speaker_options parameter. By default, the maximum number of speakers depends on the audio duration (no limit for 0–2 minutes, 10 for 2–10 minutes, and 30 for 10+ minutes).

This parameter is suitable for use cases where there is a known minimum/maximum number of speakers in the audio file that is outside the bounds of the default limits.

Setting max_speakers_expected too high may reduce diarization accuracy, causing sentences from the same speaker to be split across multiple speaker labels.

When using multichannel with speaker_labels, the speaker_options parameters are applied per channel, not globally across the entire file. For example, setting min_speakers_expected: 5 and max_speakers_expected: 7 on a 5-channel file means the model will find 5–7 speakers on each channel, resulting in 25–35 total speakers. Adjust your speaker options accordingly when using multichannel transcription.

Building on the Quickstart above, add speaker_options to your transcription config:

data = {
    "audio_url": upload_url,
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": True,
    "speaker_labels": True,
    "speaker_options": {
      "min_speakers_expected": 3,
      "max_speakers_expected": 5
    }
}

config = aai.TranscriptionConfig(
  speech_models=["universal-3-pro", "universal-2"],
  language_detection=True,
  speaker_labels=True,
  speaker_options=aai.SpeakerOptions(
    min_speakers_expected=3,
    max_speakers_expected=5
  ),
)

const data = {
  audio_url: uploadUrl,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
  speaker_options: {
    min_speakers_expected: 3,
    max_speakers_expected: 5,
  },
};

const params = {
  audio: audioFile,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
  speaker_options: {
    min_speakers_expected: 3,
    max_speakers_expected: 5,
  },
};

API reference

Request

Speakers Expected

curl  \
--header "Authorization: <YOUR_API_KEY>" \
--header "Content-Type: application/json" \
--data '{
  "audio_url": "YOUR_AUDIO_URL",
  "speech_models": ["universal-3-pro", "universal-2"],
  "language_detection": true,
  "speaker_labels": true,
  "speakers_expected": 3
}'

Speaker Options

curl  \
--header "Authorization: <YOUR_API_KEY>" \
--header "Content-Type: application/json" \
--data '{
  "audio_url": "YOUR_AUDIO_URL",
  "speech_models": ["universal-3-pro", "universal-2"],
  "language_detection": true,
  "speaker_labels": true,
  "speaker_options": {
    "min_speakers_expected": 3,
    "max_speakers_expected": 5
  }
}'

Key	Type	Description
speaker_labels	boolean	Enable Speaker Diarization.
speakers_expected	number	Set number of speakers.
speaker_options	object	Set range of possible speakers.
speaker_options.min_speakers_expected	number	The minimum number of speakers expected in the audio file.
speaker_options.max_speakers_expected	number	The maximum number of speakers expected in the audio file.

Response

Key	Type	Description
utterances	array	A turn-by-turn temporal sequence of the transcript, where the i-th element is an object containing information about the i-th utterance in the audio file.
utterances[i].confidence	number	A score between 0 and 1 indicating the model's confidence in the accuracy of the transcribed text for this utterance.
utterances[i].end	number	The ending time, in milliseconds, of the utterance in the audio file.
utterances[i].speaker	string	The speaker of this utterance, where each speaker is assigned a sequential capital letter. For example, "A" for Speaker A, "B" for Speaker B, and so on.
utterances[i].start	number	The starting time, in milliseconds, of the utterance in the audio file.
utterances[i].text	string	The transcript for this utterance.
utterances[i].words	array	A sequential array for the words in the transcript, where the j-th element is an object containing information about the j-th word in the utterance.
utterances[i].words[j].text	string	The text of the j-th word in the i-th utterance.
utterances[i].words[j].start	number	The starting time for when the j-th word is spoken in the i-th utterance, in milliseconds.
utterances[i].words[j].end	number	The ending time for when the j-th word is spoken in the i-th utterance, in milliseconds.
utterances[i].words[j].confidence	number	The confidence score for the transcript of the j-th word in the i-th utterance.
utterances[i].words[j].speaker	string	The speaker who uttered the j-th word in the i-th utterance.

The response also includes the request parameters used to generate the transcript.

Identify speakers by name

Speaker Diarization assigns generic labels like "Speaker A" and "Speaker B" to each speaker. If you want to replace these labels with actual names or roles, you can use Speaker Identification to transform your transcript.

Before Speaker Identification:

Speaker A: Good morning, and welcome to the show.
Speaker B: Thanks for having me.

After Speaker Identification:

Michel Martin: Good morning, and welcome to the show.
Peter DeCarlo: Thanks for having me.

The following example shows how to transcribe audio with Speaker Diarization and then apply Speaker Identification to replace the generic speaker labels with actual names.

import requests
import time

base_url = ""

headers = {
    "authorization": "<YOUR_API_KEY>"
}

audio_url = ""

# Configure transcript with speaker diarization and speaker identification
data = {
    "audio_url": audio_url,
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": True,
    "speaker_labels": True,
    "speech_understanding": {
        "request": {
            "speaker_identification": {
                "speaker_type": "name",
                "known_values": ["Michel Martin", "Peter DeCarlo"]
            }
        }
    }
}

# Submit the transcription request
response = requests.post(base_url + "/v2/transcript", headers=headers, json=data)
transcript_id = response.json()["id"]
polling_endpoint = base_url + f"/v2/transcript/{transcript_id}"

# Poll for transcription results
while True:
    transcript = requests.get(polling_endpoint, headers=headers).json()

    if transcript["status"] == "completed":
        break
    elif transcript["status"] == "error":
        raise RuntimeError(f"Transcription failed: {transcript['error']}")
    else:
        time.sleep(3)

# Print utterances with identified speaker names
for utterance in transcript["utterances"]:
    print(f"{utterance['speaker']}: {utterance['text']}")

const baseUrl = "";

const headers = {
  authorization: "<YOUR_API_KEY>",
  "content-type": "application/json",
};

const audioUrl = "";

// Configure transcript with speaker diarization and speaker identification
const data = {
  audio_url: audioUrl,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  speaker_labels: true,
  speech_understanding: {
    request: {
      speaker_identification: {
        speaker_type: "name",
        known_values: ["Michel Martin", "Peter DeCarlo"],
      },
    },
  },
};

async function main() {
  // Submit the transcription request
  const response = await fetch(`${baseUrl}/v2/transcript`, {
    method: "POST",
    headers: headers,
    body: JSON.stringify(data),
  });
  if (!response.ok) throw new Error(`Error: ${response.status}`);

  const { id: transcriptId } = await response.json();
  const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

  // Poll for transcription results
  while (true) {
    const pollingResponse = await fetch(pollingEndpoint, { headers });
    if (!pollingResponse.ok) throw new Error(`Error: ${pollingResponse.status}`);
    const transcript = await pollingResponse.json();

    if (transcript.status === "completed") {
      // Print utterances with identified speaker names
      for (const utterance of transcript.utterances) {
        console.log(`${utterance.speaker}: ${utterance.text}`);
      }
      break;
    } else if (transcript.status === "error") {
      throw new Error(`Transcription failed: ${transcript.error}`);
    } else {
      await new Promise((resolve) => setTimeout(resolve, 3000));
    }
  }
}

main().catch(console.error);

For more details on Speaker Identification, including how to identify speakers by role and how to apply it to existing transcripts, see the Speaker Identification guide.

Best practices for accurate diarization

Follow these tips to get the best results from Speaker Diarization:

• Ensure sufficient speech per speaker. Each speaker should speak for at least 30 seconds uninterrupted. The model may struggle to create separate clusters for speakers who only contribute short phrases like "Yeah", "Right", or "Sounds good".
• Minimize cross-talk. Overlapping speech between speakers can reduce diarization accuracy. Where possible, ensure speakers take turns.
• Reduce background noise. Background noise, echoes, or playback of recorded audio during a conversation can interfere with speaker separation.
• Use speaker_options instead of speakers_expected when uncertain. Only use speakers_expected when you are confident about the exact number of speakers. If this number is incorrect, the model may produce random splits of single-speaker segments or merge multiple speakers into one. It's generally recommended to use min_speakers_expected and set max_speakers_expected slightly higher (e.g., min_speakers_expected + 2) to allow flexibility.
• Avoid setting max_speakers_expected too high. Setting the maximum too high may reduce accuracy, causing sentences from the same speaker to be split across multiple speaker labels.
• Be aware of speaker similarity. If speakers sound similar, the model may have difficulty distinguishing between them.

Frequently asked questions

The default upper limit on the number of speakers depends on the audio duration:

• 0–2 minutes: no max speaker limit
• 2–10 minutes: max 10 speakers
• 10+ minutes: max 30 speakers

If you need a different limit, you can use speaker_options to set a range of possible speakers.

The accuracy of the Speaker Diarization model depends on several factors, including the quality of the audio, the number of speakers, and the length of the audio file. See the best practices section above for tips on improving accuracy.