The National Theatre is using augmented reality to enhance its shows

Now that it’s being effectively integrated into our mobile phones, augmented reality technology is really starting to take off. Where Google Glass managed to make AR experiences seem inaccessible with limited applications and maximum privacy concerns, simple AR apps like Pokemon Go and Snapchat have brought down consumer walls and made us see the fun side of the technology.

Now that ARKit has launched on iOS 11 and Apple has effectively created a huge platform for AR on the iPhone overnight we can only see it expanding in terms of depth and breadth.

Recently TechRadar got the chance to experience one of the really exciting directions augmented reality is taking right now. Fairly soon, augmented reality is going to be known for so much more than bringing Pokemon into the real world or changing our faces in real-time with animated lenses.

Live and legible

The National Theatre is using augmented reality technology to improve accessibility in its live performances as part of a collaboration with Epson and Accenture.

Using Epson’s latest Moverio BT-350 Smart Glasses, Accenture has created a new augmented reality system called Open Access Smart Capture which makes it possible for theater-goers to view real-time subtitling during live performances.

We tried the glasses out for ourselves during a short demo session. During the demo, actresses Olivia Coleman and Olivia Williams performed a scene from the play Mosquito on a makeshift stage. While they performed their words appeared on the lenses of the glasses, overlaying the real world without blocking it out.

There are already accessible National Theatre performances which have subtitles on screens at the side of the stage. However, these performances are at limited times – there are only four captioned performances and up to only three audio-described performances per production run.

Open Access Smart Capture will make it possible for the hearing-impaired to attend any National Theatre performance they like, sit in any seat they please and keep their focus more on the on-stage performance rather than the screen off to the side.

Front and center

Accenture’s software offers a surprising degree of customizability. Before the show started, we were able to choose the size, color, and placement of the subtitles as well as whether they remained stable or scrolled.

This could all be done using a small touchpad attached to the glasses and although we settled on what we wanted before the performance started, it’s completely possible to change any of your settings as the show is happening.

Though the glasses weren’t so lightweight we forgot they were on our face, they were far from uncomfortable and wearing them was no more annoying than wearing 3D glasses in a cinema.

Naturally, there are limitations but it appears they’re being actively worked on. Being live, theater is known for not always going according to plan and what’s written on the script isn’t always going to be exactly what appears on stage.

Fortunately, through a combination of preparation during the rehearsal process (making it possible to program sequences of events), the software’s ability to watch and listen to the performance in real time and adapt as necessary, and the fact that a human can monitor and change things during the show in real-time, everything is likely to go smoothly. The aim is to achieve 97% accuracy when it comes to timing of captions and descriptions.

Sticking to the script

For example, if the stage background is due to change color to white during the performance, the software can be programmed to make sure the subtitles that are appearing aren’t also in white so they remain legible. However, it can also recognize if this change happens earlier or later than planned and adapt as necessary.

It’s also possible for the viewer to interfere themselves, though, which is especially convenient given color blindness has the potential to make certain color combinations problematic for individual viewers.

Another potential problem is, of course, battery life. When we asked an Accenture representative how long the glasses could be used on a full charge he told us around two and a half hours. Though this could get through many National Theatre productions, in some cases it could be cutting it fine.

This relatively short battery life would be especially problematic if there was a rush to move the glasses between theatres for matinee and evening performances.

A solution could be to tether the headsets to the seats or have longer intermissions to allow for slightly more charge time but we’d hope the battery technology in the headsets will eventually be improved to make wearing them as free and seamless an experience as possible.

After starting the partnership at the beginning of 2017, the equipment and software is now entering its pilot phase. Over the next year it will be tested during real National Theatre performances in three of its theaters.

The hope is that after extensive testing the glasses will be rolled out to all National Theatre performances by October 2018. After this, it’ll then be opened up as a possibility for other theaters of all sizes up and down the country.

When we asked representatives at the event whether the glasses and software could be rolled out for a wider number of applications such as cinemas or museums they agreed that in the future this is certainly a possibility.

Future plans

Applications like this make it clear why many state that augmented reality has a much wider appeal than virtual reality. A theater filled with people watching something on closed off virtual reality headsets just wouldn’t have the same atmosphere or sense of being connected with what was happening that these augmented reality headsets made possible.

With Epson’s glasses we still felt connected to the real world and the rest of the audience around us while we were enjoying a completely personalized viewing experience. It truly does augment the real world and it proves that ancient arts such as theater can actually be opened up to more people and given a wider appeal by new technology rather than destroyed by it.