2020 BEITC Proceedings

Towards Designing a Subjective Assessment System for the Quality of Closed Captioning Using Artificial Intelligence - $15

A novel quality assessment system design for Closed Captioning (CC) is proposed. CC is originally designed to serve Deaf and Hard of Hearing (D/HoH) audiences for enjoying audio/visual content, similar to hearing audiences. Traditional quality assessment models have focus on empirical methods only, measuring quantitative accuracy by counting the number of word errors in the captions of show. Errors are specifically defined to be quantitative (e.g., spelling errors) and/or assessed by trained experts. However, D/HoH audiences have been outspoken about their dissatisfaction with current CC quality. One solution to this could be inviting human evaluators who represent different groups to assess the quality of CC at the end of each show, however, in reality, this would be difficult to do and impractical. We have developed an artificial intelligence (AI) system to include human subjective assessment in the CC quality assurance procedure. The system is designed to replicate the human evaluation process and can predict the subjective score for a given caption file. Probabilistic models of human evaluators were developed based on actual data from D/HoH audiences. Deep Neural Networks-Multilayer Perceptron (DNN-MLP) were then trained with the probability models and data collected. To date, the major findings of this process are:

1. The human subjective ratings for given caption quality prediction performance of DNN-MLP was higher than that of using some of the basic statistical regression models (polynomial fitting),
2. The user probability models of Deaf viewer and Hard of Hearing viewer seemed to represent the different characteristics between two primary service consumer groups, and
3. The artificial intelligence prediction system created based solely on literature seemed to be improved after training with the data based on user probability models.

Somang Nam | University of Toronto | Toronto, ON, Canada
Deborah Fels | Ryerson University | Toronto, ON, Canada

Watson Captioning Live: Leveraging AI for Smarter, More Accessible Closed Captioning - $15

The requirements for closed captioning were established more than two decades ago , but many broadcasters still struggle to deliver accurate, timely, and contextually-relevant captions. Breaking news, weather, and entertainment programming often feature delayed or incorrect captions, further
demonstrating that there is great room for improvement. These shortcomings lead to a confusing viewing experience for the nearly 48 million Americans with hearing loss and any other viewers who need captioning to fully digest content.

Committed to transforming broadcasters? ability to provide all audiences with more impactful viewing experiences, IBM Watson Media launched Watson Captioning Live , a trainable, cloud-based solution producing accurate captions in real-time to ensure audiences have equal access to timely and vital
information. Combining breakthrough AI technology like machine learning models and speech recognition, Watson Captioning Live redefines industry captioning standards.

The solution uses IBM Watson Speech to Text API to automatically ingest and transcribe spoken words and audio within a video. Watson Captioning Live is trained to automatically recognize and learn from data updates to ensure timely delivery of factually accurate captions. The product is designed to learn over time to increase its long-term value proposition for broadcast producers.

This paper will explore how IBM Watson Captioning Live leverages AI and machine learning technology to deliver accurate closed captions at scale, in real-time? to make programming more accessible for all.

Brandon Sullivan | The Weather Company Solutions | Austin, TX, USA