Why do I sometimes get different results with the same audio?

You might have a situation where you find yourself submitting the same audio to Deepgram multiple times. This is especially common when starting out with the Deepgram API, or when implementing integration tests between your services and the Deepgram API.

You may notice in some of these tests that the Deepgram API may not always return the same transcription for a speech-to-text request, or the same exact summary for an audio intelligence request, or the same audio snippet for a text-to-speech request.

Different Inputs

Sometimes, this happens because the input data is not exactly the same. For example, you may have two different recordings of your own voice saying, "Hello, this is a test." While the words are the exact same, other aspects of the audio might be different, including variations in tone, background noise, and other factors. The exact bits of each audio file will vary, and you may get slightly different results.

In this case, the word timing results will likely be slightly different for each file. The transcription may even vary, especially if the submitted files have low audio quality or high background noise.

Identical Inputs

In other cases, your input data may be exactly the same, down to the final bit. For example, you may re-use a specific audio file as part of a test suite. When results vary even with the same audio file, this can be explained by the inherent non-determinism when doing inference on GPUs.

Deepgram's voice AI platform is powered by deep learning models, such as transformers. These models can be run on a variety of different hardware, with the most common today being Graphical Processing Units, or GPUs. GPUs pair nicely with deep learning models, such as neural networks, as described in the below excerpt:

“Embarrassingly parallel” is a term used in computing to describe problems that can be easily split into multiple parallel tasks with little or no need for communication between them. In the context of neural networks, this means that many operations, like matrix multiplications, can be performed simultaneously on different data points. GPUs, with their thousands of smaller cores, are naturally adept at handling such parallel tasks.

Source: https://deepgram.com/ai-glossary/ai-hardware#the-rise-of-gpus-in-deep-learning

One key aspect of parallelism is that there are pieces of the computations that have no defined order of operations. As an example, a certain inference request, like transcription, may be summarized as "take this audio file, split it into pieces A, B, and C, and process those three pieces in parallel to produce a transcript." When you split up the audio file and send the pieces to the GPU, the computations complete independently, and then have to be stitched back together. Depending on the order of completion, that stitching together may look slightly different, and therefore lead to slightly different results.

A technical explanation of this is that floating point operations in computer science are not associative. The parallelism of GPUs that enables incredibly efficient machine learning inference also means the ordering of operands in the billions of computations can vary. Variations in floating point operand order can introduce different rounding errors.

For a single operation, this is often insignificant. However, when compounded across billions of computations, these small individual rounding differences in aggregate can lead to different top-level results, such as different words predicted within a transcript.

In a speech-to-text context, for a clear, high-quality audio file, you may see an identical transcript every single time. The model may be so confident in its predictions that rounding errors do not affect the transcript results. However, for low-quality audio files, or for recordings with background noise and other confounding factors, it's more likely that the model will be less confident in its results, and the compounded effect of many rounding differences can lead to differences in the final transcript.