For clean Markdown of any page, append .md to the page URL. For a complete documentation index, see For full documentation content, see For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at

Entity Detection

Extract named entities from your audio file

For the complete documentation index, see llms.txt

cURL quickstart — Entity Detection

bash

# Step 1: Submit transcription with entity_detection enabled
curl  \
  --header "Authorization: YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "audio_url": "",
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": true,
    "entity_detection": true
  }'

# Step 2: Poll for result (replace TRANSCRIPT_ID)
curl  \
  --header "Authorization: YOUR_API_KEY"

The response includes an entities array with each entity's text, entity_type, start, and end timestamps.

US & EU

The Entity Detection model lets you automatically identify and categorize key information in transcribed audio content.

Here are a few examples of what you can detect:

Names of people
Organizations
Addresses
Phone numbers
Medical data
Social security numbers

For the full list of entities that you can detect, see Supported entities.

Entity Detection is available in multiple languages. See Supported languages.

Quickstart

Enable Entity Detection by setting entity_detection to True in the JSON payload.

python

import requests
import time

base_url = ""

headers = {
    "authorization": "<YOUR_API_KEY>"
}

with open("./local_file.mp3", "rb") as f:
    response = requests.post(base_url + "/v2/upload",
                            headers=headers,
                            data=f)

upload_url = response.json()["upload_url"]

data = {
    "audio_url": upload_url, # You can also use a URL to an audio or video file on the web
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": True,
    "entity_detection": True
}

url = base_url + "/v2/transcript"
response = requests.post(url, json=data, headers=headers)

transcript_id = response.json()['id']
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

print(f"Transcript ID: {transcript_id}")

while True:
    transcription_result = requests.get(polling_endpoint, headers=headers).json()

    if transcription_result['status'] == 'completed':
      for entity in transcription_result['entities']:
        print(entity['text'])
        print(entity['entity_type'])
        print(f"Timestamp: {entity['start']} - {entity['end']}\n")
      break
    elif transcription_result['status'] == 'error':
        raise RuntimeError(f"Transcription failed: {transcription_result['error']}")
    else:
        time.sleep(3)

Enable Entity Detection by setting entity_detection to True in the transcription config.

python

import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

# audio_file = "./local_file.mp3"
audio_file = ""

config = aai.TranscriptionConfig(
    speech_models=["universal-3-pro", "universal-2"],
    language_detection=True,
    entity_detection=True
)

transcript = aai.Transcriber().transcribe(audio_file, config)
print(f"Transcript ID: {transcript.id}")

for entity in transcript.entities:
    print(entity.text)
    print(entity.entity_type)
    print(f"Timestamp: {entity.start} - {entity.end}\n")

Enable Entity Detection by setting entity_detection to true in the JSON payload.

javascript

import fs from "fs-extra";

const baseUrl = "";

const headers = {
  authorization: "<YOUR_API_KEY>",
};

const path = "./my-audio.mp3";
const audioData = await fs.readFile(path);
let res = await fetch(`${baseUrl}/v2/upload`, {
  method: "POST",
  headers,
  body: audioData,
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const uploadResponse = await res.json();
const uploadUrl = uploadResponse.upload_url;

const data = {
  audio_url: uploadUrl, // You can also use a URL to an audio or video file on the web
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  entity_detection: true,
};

const url = `${baseUrl}/v2/transcript`;
res = await fetch(url, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
console.log("Transcript ID: ", transcriptId);

const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    for (const entity of transcriptionResult.entities) {
      console.log(entity.text);
      console.log(entity.entity_type);
      console.log(`Timestamp: ${entity.start} - ${entity.end}\n`);
    }
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Enable Entity Detection by setting entity_detection to true in the transcription config.

javascript

import { AssemblyAI } from "assemblyai";

const client = new AssemblyAI({
  apiKey: "<YOUR_API_KEY>",
});

// const audioFile = './local_file.mp3'
const audioFile = "";

const params = {
  audio: audioFile,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
  entity_detection: true,
};

const run = async () => {
  const transcript = await client.transcripts.transcribe(params);

  for (const entity of transcript.entities) {
    console.log(entity.text);
    console.log(entity.entity_type);
    console.log(`Timestamp: ${entity.start} - ${entity.end}\n`);
  }
};

run();

Example output

plain

Canada
location
Timestamp: 2548 - 3130

the US
location
Timestamp: 5498 - 6350

...

API reference

Request

bash

curl  \
--header "Authorization: <YOUR_API_KEY>" \
--header "Content-Type: application/json" \
--data '{
  "audio_url": "YOUR_AUDIO_URL",
  "entity_detection": true
}'

Key	Type	Description
`entity_detection`	boolean	Enable Entity Detection.

Response

Key	Type	Description
`entities`	array	An array of detected entities.
`entities[i].entity_type`	string	The type of entity for the i-th detected entity.
`entities[i].text`	string	The text for the i-th detected entity.
`entities[i].start`	number	The starting time, in milliseconds, at which the i-th detected entity appears in the audio file.
`entities[i].end`	number	The ending time, in milliseconds, for the i-th detected entity in the audio file.

The response also includes the request parameters used to generate the transcript.

Supported entities

The model is designed to automatically detect and classify various types of entities within the transcription text. The detected entities and their corresponding types is listed individually in the entities key of the response object, ordered by when they first appear in the transcript.

Entity name	Description	Example
`account_number`	Customer account or membership identification number	Policy No. 10042992; Member ID: HZ-5235-001
`banking_information`	Banking information, including account and routing numbers
`blood_type`	Blood type	O-, AB positive
`credit_card_cvv`	Credit card verification code	CVV: 080
`credit_card_expiration`	Expiration date of a credit card
`credit_card_number`	Credit card number
`date`	Specific calendar date	December 18
`date_interval`	Broader time periods, including date ranges, months, seasons, years, and decades	2020-2021; 5-9 May; January 1984
`date_of_birth`	Date of birth	Date of Birth: March 7, 1961
`drivers_license`	Driver's license number	DL# 356933-540
`drug`	Medications, vitamins, or supplements	Advil, Acetaminophen, Panadol
`duration`	Periods of time, specified as a number and a unit of time	8 months; 2 years
`email_address`	Email address	support@assemblyai.com
`event`	Name of an event or holiday	Olympics, Yom Kippur
`filename`	Names of computer files, including the extension or filepath	Taxes/2012/brad-tax-returns.pdf
`gender_sexuality`	Terms indicating gender identity or sexual orientation, including slang terms	female; bisexual; trans
`healthcare_number`	Healthcare numbers and health plan beneficiary numbers	Policy No.: 5584-486-674-YM
`injury`	Injuries or health issues	I broke my arm, I have a sprained wrist
`ip_address`	Internet IP address, including IPv4 and IPv6 formats	192.168.0.1
`language`	Name of a natural language	Spanish, French
`location`	Any Location reference including mailing address, postal code, city, state, province, country, or coordinates.	Lake Victoria, 145 Windsor St., 90210
`marital_status`	Terms indicating marital status	Single; common-law; ex-wife; married
`medical_condition`	Name of a medical condition, disease, syndrome, deficit, or disorder	chronic fatigue syndrome, arrhythmia, depression
`medical_process`	Medical process, including treatments, procedures, and tests	heart surgery, CT scan
`money_amount`	Name and/or amount of currency	15 pesos, $94.50
`nationality`	Terms indicating nationality, ethnicity, or race	American, Asian, Caucasian
`number_sequence`	Numerical PII (including alphanumeric strings) that doesn't fall under other categories
`occupation`	Job title or profession	professor, actors, engineer, CPA
`organization`	Name of an organization	CNN, McDonalds, University of Alaska, Northwest General Hospital
`passport_number`	Passport numbers, issued by any country	PA4568332; NU3C6L86S12
`password`	Account passwords, PINs, access keys, or verification answers	27%alfalfa, temp1234, My mother's maiden name is Smith
`person_age`	Number associated with an age	27, 75
`person_name`	Name of a person	Bob, Doug Jones, Dr. Kay Martinez, MD
`phone_number`	Telephone or fax number
`physical_attribute`	Distinctive bodily attributes, including terms indicating race	I'm 190cm tall; He belongs to the Black students' association
`political_affiliation`	Terms referring to a political party, movement, or ideology	Republican, Liberal
`religion`	Terms indicating religious affiliation	Hindu, Catholic
`statistics`	Medical statistics	18%, 18 percent
`time`	Expressions indicating clock times	19:37:28; 10pm EST
`url`	Internet addresses
`us_social_security_number`	Social Security Number or equivalent
`username`	Usernames, login names, or handles	@AssemblyAI
`vehicle_id`	Vehicle identification numbers (VINs), vehicle serial numbers, and license plate numbers	5FNRL38918B111818; BIF7547
`zodiac_sign`	Names of Zodiac signs	Aries; Taurus

Frequently asked questions

The model is capable of identifying entities with variations in spelling or formatting. However, the accuracy of the detection may depend on the severity of the variation or misspelling.

No, the Entity Detection model doesn't support the detection of custom entity types. However, the model is capable of detecting a wide range of predefined entity types, including people, organizations, locations, dates, times, addresses, phone numbers, medical data, and banking information, among others.

To improve the accuracy of the Entity Detection model, it's recommended to provide high-quality audio files with clear and distinct speech. In addition, it's important to ensure that the audio content is relevant to the use case and that the entities being detected are relevant to the intended analysis. Finally, it may be helpful to review and adjust the model's configuration parameters, such as the confidence threshold for entity detection, to optimize the results.

Entity Detection ​

Quickstart ​

Example output ​

API reference ​

Request ​

Response ​

Supported entities ​

Frequently asked questions ​