AssemblyAI Enhances Speaker Diarization with New Languages and Improved Accuracy
Zach Anderson Jun 21, 2024 01:43
AssemblyAI announces major improvements to its Speaker Diarization service, enhancing accuracy by up to 13% and adding support for five new languages.
AssemblyAI has announced significant upgrades to its Speaker Diarization service, which is designed to identify individual speakers within a conversation. According to the company, these improvements have led to enhanced accuracy and expanded language support, making the service more effective and versatile for end-users.
Speaker Diarization Improvements
The updated Speaker Diarization model now offers up to 13% greater accuracy compared to its predecessor. The enhancements have been measured across various industry benchmarks, including a 10.1% improvement in Diarization Error Rate (DER) and a 13.2% improvement in concatenated minimum-permutation word error rate (cpWER). These metrics are critical in evaluating the performance of diarization models, with lower values indicating better accuracy.
DER measures the fraction of time an incorrect speaker is attributed to the audio, while cpWER accounts for the number of errors made by the speech recognition model, including those due to incorrect speaker assignments. AssemblyAI's improvements in both metrics highlight the model's enhanced capability in accurately identifying speakers.
Speaker Number Accuracy
Another significant upgrade is the 85.4% reduction in speaker count errors. This improvement ensures that the model can more accurately determine the number of unique speakers in an audio file. Accurate speaker count is essential for various applications, such as call center software that relies on identifying the correct number of participants in a conversation.
AssemblyAI's model now boasts the lowest rate of speaker count errors at just 2.9%, outperforming several other providers in the industry.
Increased Language Support
The service has also expanded its language support, now available in five additional languages: Chinese, Hindi, Japanese, Korean, and Vietnamese. This brings the total number of supported languages to 16, covering almost all languages supported by AssemblyAI's Best tier.
Technological Advancements
The improvements to Speaker Diarization stem from a series of technological upgrades:
- Universal-1 Model: The new Speech Recognition model, Universal-1, has enhanced transcription accuracy and timestamp prediction, which are critical for aligning speaker labels with automatic speech recognition (ASR) outputs.
- Improved Embedding Model: Upgrades to the speaker-embedding model have improved the model's ability to identify and differentiate between unique acoustical features of speakers.
- Increased Sampling Frequency: The input sampling frequency has been increased from 8 kHz to 16 kHz, providing higher-resolution input data and enabling the model to better distinguish between different speakers' voices.
Use Cases and Applications
Speaker Diarization is a critical feature for various applications across industries:
Transcript Readability
With the rise of remote work and recorded meetings, accurate and readable transcripts are more important than ever. Diarization improves the readability of these transcripts, making it easier for users to digest the content.
Search Experience
Many conversation intelligence products offer search features that allow users to find instances where specific people said particular things. Accurate diarization is essential for these features to function correctly.
Downstream Analytics and LLMs
Many analytical features and large language models (LLMs) rely on knowing who said what to extract meaningful information from recorded speech. This is crucial for applications like customer service software, which can use speaker information for coaching and improving agent performance.
Creator Tool Features
Accurate transcription and diarization are foundational for various AI-powered features in video processing and content creation, such as automated dubbing, auto speaker focus, and AI-recommended short clips from long-form content.
For more detailed information, you can visit the official AssemblyAI blog.
Image source: Shutterstock