Akamai closed captioning fix

Important video streaming issue

with closed captioning for major TV channels in USA was tacked by Redwerk

WorldNowWorldNowNew York, NY, United States

A renowned digital media company, which develops content management systems, advertising solutions, and streaming video platforms designed for local news broadcasting companies.

All Customers
Dedicated Team

We made sure we understood exactly what the problem was and assigned the right professional hardcore hackers to the team who we knew could get this job done.

Learn more
Media & Entertainment

Code we developed is working for over 80% US households delivering them local news on a daily basis.

Learn more

Challenge

Nearly all TV channels host videos on their websites, e.g. up-to-the-minute news broadcasts that give visitors to their sites quick and convenient access to what’s happening in the world as well as in their towns. In order to prepare these videos for the user, they are first uploaded as video files through the backend.

The real challenge though is a live stream that is often offered on TV channel websites. It’s being captured directly from the video source and re-streamed to the visitor’s web browser. In this particular case, videos also had to contain closed captions (CC).

Closed captioning is a process of displaying text as an overlay to the video, in order to provide additional or interpretive information for the viewer. A typical use of closed captioning in real life is the live transcription of the audio portion of a program, which may also include certain non-verbal elements.

This technology was not supported by the streaming solution used by Worldnow. It was therefore up to Redwerk to create an efficient and modern solution which would enable the display of closed captions together with the streamed video files.

Solution

The technology used by Worldnow for re-streaming and displaying live streams on their clients’ websites was provided by Akamai Technologies, a U.S. network which specializes in Internet content delivery. While this is one of the largest distributed computing platforms in the world and currently serves between 15 and 20 percent of all web traffic, the display of closed captions proved to be a problem as they just disappeared once a video signal containing CC data was uploaded to their servers. Our software experts quickly identified the source of the problem: Flash media live encoder (FMLE) actually deleted all CC data when encoding the video signal to AAC/H264.

In order to make this work in a way that would both satisfy our client and deliver a great end-user experience, two possible roads were identified for our project team to take:

1. Find a solution to the issues caused by the existing Workflow and FMLE

During the analysis of the existing solution, the following shortcomings of FMLE were detected:

  • Unable to read closed caption data from a device’s pin
  • Unable to copy streams without modifying it
  • CC data was deleted during h264 encoding
  • No configuration option available to remedy this behavior
  • No open source solution for modifying this information available
2. Find alternative encoder(s) capable of retaining CC data during encoding

We tried and tested various encoding software solutions, like Afritelly encoder, Wirecast, and Sorenson Squeeze. However, things proved difficult as all of these either had similar problems with CC data or were incompatible with streaming to Akamai. The reason for this was the secure connection used by Akamai, which is protected by usernames and passwords.

In the end and after much deliberation, the winner was… ffmpeg, which is an open source project with a highly flexible configuration and supports tons of formats for encoding and streaming. In order to adapt this solution to our needs, however, a lot of research was still needed:

  • Repeat the current workflow process of streaming video and audio to Akamai via ffmpeg. This proved successful as streaming ffmpeg to Akamai through basic auth worked faultlessly.
  • Encode CC data as a live stream and send this stream to Akamai via ffmpeg. Here, it was important to make sure that no CC data would be lost. In this stage, development was hampered due to some bugs in ffmpeg, which made it impossible to implement the functionality as required:
    • The program is unable to successfully read the pins of the device, so CC data cannot be read from it
    • FFmpeg deletes CC data when encoding to H264, just like FMLE; even if we can read closed captions and encode them to the video stream, ffmpeg will still delete CC data during encoding

This proved to be a complex project, due to the above-mentioned reasons, but the happy ending arrived when we found out that if the step of video encoding is skipped and ffmpeg is used only for transmitting the stream to Akamai, CC data is not removed.

After this piece of good news, the remaining task was much simplified: finding a way to encode video to H264 without losing CC data and to upload it to Akamai via ffmpeg. In order to overcome this final hurdle, we settled on MainConcept H264 encoder, which is able to use video and CC byte-streams as input data and produce a signal with the closed captions encoded to the video stream (a so-called elementary stream).

Need hackers to address toughest technical challenges?

Request Quote

2

developers worked on the project

1

QA engineer

3

months to tackle the issue

Results

Redwerk’s experts had to go through a lengthy testing process with a great number of different configurations for ffmpeg (also using two ffmpegs!) before finally achieving the project’s goal of sending CC data to Akamai and avoiding the loss of such data during encoding. All streamed content is now displayed correctly through Flash Player (RTMP, HDS streaming) and also on iDevices (HLS streaming). The customer’s problem could be solved in a satisfactory manner, and a great video experience is now guaranteed for all end-users!

Digital media company
We have worked with Redwerk for the past years. We are very happy with our value and payback equation. They provide great technical knowledge for what they bring to the table. Our efforts overseas/offshore has been very difficult, Redwerk's team have finally proven successful with our offshore strategy to complement our core development team state side.
Joe Sticca Worldnow parnter

Joe Sticca, Chief Product Officer at Worldnow

Want to complement your local team with hardcore hackers?

Contact Us
Streaming video platform development
Local news broadcasting company
Weather broadcast
Akamai development

Impressed? Hire us

Contact Now
Request Quote
×