How do I maximize my intelligence throughput?

Introduction

In the realm of audio analysis, Deepgram’s intelligence features prove useful for advanced tasks including sentiment analysis, intent recognition, topic detection, and summarization. However, these features require more computational power and processing time, which aren’t conducive to ultra-low latency and high-throughput transcription. By decoupling intelligence from the transcription itself, this becomes more scalable, reliable, and conforms to rate limits. Making subsequent intelligence API calls using transcribed text as input also enables streaming support.

Step 1: Transcribe Your Audio

Whether you have pre-recorded or streaming audio, you may transcribe the audio using a standard API request.

Step 2: Obtain Intelligence Results for the Audio

Given the text transcript as input, use the text intelligence API to obtain the intelligence results.

Depending on your application, you may want to initiate this request at different points.

For pre-recorded audio, you’ll likely want to make the intelligence request after transcribing an audio file. You can leverage callbacks for an event-driven workflow, where receiving a transcription callback response triggers a text intelligence request.

For streaming audio, you may want to make intelligence requests as a response to various events, such as a transcription final result, the end of a sentence (using smart formatting), a speaker change (using diarization), or when a stream closes.

Factors Affecting Intelligence Throughput

Rate Limits

Note that for Pay-As-You-Go and Growth users, rate limits for pre-recorded and streaming transcription are 100 concurrent requests, while rate limits for audio intelligence features are 10 concurrent requests. Given this discrepancy, adding intelligence parameters to your transcription requests will substantially limit your throughput.

Request Duration

Because our intelligence features use task-specific language models, adding intelligence features to transcription requests increases the request duration.

Imagine that you have 1-hour audio files that takes 30 seconds to transcribe, and an additional 30 seconds to summarize. If you add the summarization into the transcription requests, then not only do you reduce your concurrency from 100 to 10 requests, but each request takes twice as long, so your effective concurrency is halved again.

On the flip side, imagine that your 1-hour audio files are mainly comprised of callers waiting on hold, with little speech. Transcription duration is not impacted by proportion of speech in the audio, because it always needs to run through a transcription model to determine whether speech is indeed present. But if the transcript of a 1-hour audio file is 90% silence with only a few brief sentences spoken, then most of the request duration is in the transcription itself. Yet, if the request also includes summarization, then the time to transcribe also consumes one request in the summarization rate limit. Once the audio is transcribed, the summarization is relatively quick, because of the short transcript to be summarized.

The optimal solution in either case is to decouple transcription and intelligence, so that neither reduces the throughput of the other.

Diagram

The same results are obtained from performing transcription followed by text intelligence, versus a single request with transcription and audio intelligence. The former yields lower latency and higher throughput.

Demonstration

Check out our Live-Streaming Starter Kit, as well as our accompanying documentation, in order to transcribe streaming audio and then fetch the intelligence results once the stream has closed.

Conclusion

By separating audio transcription from intelligence, audio transcription can maintain the highest possible throughput, with intelligence features supplemented in standalone requests at the cadence of your choice.