How Accurate is Caption.Ed?

Why is Accuracy Important?

For people who are deaf or have hearing and cognitive disabilities, captions can open the doors to greater accessibility because they allow individuals to see what cannot be heard.

This means that the accuracy of captions is fundamental. Accurate captions allow people to read the correct information and gain a solid understanding of what’s being said.

Caption.Ed provides highly accurate real-time captions for both live and pre-recorded media but just how accurate is Caption.Ed? And how does the accuracy of Caption.Ed stack up against other captioning software? To find out the answer, our team conducted a series of tests – let’s explore the results.

Test results

Based on a test we ran against two pieces of media with complex terminology, Caption.Ed had an average word error rate of 3%, compared to 4% and 12% from other captioning providers.

Here is a diagram illustrating the number of word errors between Caption.Ed and two other providers based on an extract of the transcript from our test media.

Test Methodology

The two pieces of test media were selected based on the following criteria:

• Specialist subject matter: The media selected uses a high volume of complex medical terminology
• Accents: The media selected includes different accents
• Speed: The media selected includes speakers talking at different speeds

A stenographer manually transcribed both forms of media to provide a baseline of 100% accuracy.

Next, we ran the software for Caption.Ed, Beaver* and MarmeladeJob* against the two forms of media, and the generated captions were compared to the 100% baseline. For each captioning provider, errors were struck through with the correction highlighted in red. Any missing words were also highlighted in red. The number of errors and missing words were added to calculate the word error rate percentage for all three captioning providers.

*Captioning provider names have been changed

How Can You Increase The Accuracy Of Your Captions?

Automatically generated captions are never 100% accurate 100% of the time. This is because the accuracy of captions can vary according to audio quality, topic and speaker.

However, there are some simple steps you can take to improve the accuracy of your captions:

• Ensure you have a strong internet connection. Slow broadband can interrupt your live stream captioning and affect accuracy.
• Use an external USB microphone or wear a good-quality headset with a dedicated microphone arm rather than using the built-in microphone on your device. This can help reduce background noise, often leading to missing or incorrect words in your transcript.

If possible;

• Place your microphone in the middle of everyone who might be talking. For example, in the middle of a room rather than at one end.
• Minimise situations with echo and background noise. For example, choose to use an enclosed room rather than an open plan office and look for a smaller than bigger meeting room where possible.