New immersive captioning tech to revolutionise theatres’ provision for deaf audiences

Thursday, 10 October 2019

New and easy to use immersive captioning technology is to be made freely available to UK theatre companies to boost their provision for deaf audiences. 

The package of training and software has been developed by experts in drama and performance at the University of Nottingham, in collaboration with Red Earth Theatre, a pioneering company based in the East Midlands.

The experts have worked with digital researchers in the University’s Mixed Reality Lab to develop the new projection mapping software. It can be used with traditional theatre projection systems to make captioning part of the artistic design of the show using any part of the set, props and even actors themselves.

Soonchild by Red Earth Theatre © Robert Day

The new tech also allows much faster generation of captions via an easy import of text from a Word document – where scripts often reside - rather than someone having to hand type sections of script in multiple files.

The first production to showcase this creative way to display the words and sounds of a show is the epic Arctic adventure story, Soonchild by Russell Hoban. The book has been adapted for the stage by Red Earth Theatre and is currently on tour in the UK, including Lakeside Arts on University Park in Nottingham. The technology will be made available open source to other theatre companies with training and the loan of hardware included once the tour is finished.

Soonchild by Red Earth Theatre © Robert Day

A special performance of Soonchild will be staged at Nottingham Trent University’s Waverley Studio theatre on Thursday 17th October at 6pm to celebrate 21 years of Red Earth Theatre and showcase the new technology. 

Head of Drama and Creative Writing at the University of Nottingham, Professor Jo Robinson, said: “We were really conscious that for deaf audiences performances are not generally immersive, as you need to keep switching focus between the action that’s happening on stage and the captions at the side or bottom of the stage. So our question was, can we put the captions into the action and make them an integral part of the production on stage?”

“Our answer was to enlist the skills of researchers in our Mixed Reality Lab to develop new software to allow theatre designers to do just that. It was essential to involve deaf people in the project as well and we are thrilled that we have been able to bring everyone’s skills and ideas to life in this ground-breaking new package of captioning technology.”

Joanna Robinson
This is a truly ground-breaking package of technology and training, freely available to theatre companies.
Professor Joanna Robinson, School of English

Dr Paul Tennent from the University’s Mixed Reality Lab explained the technical process: “A key innovation we have made is to simplify the process of projection mapping. Almost every set designer makes a scale model (either digital or physical) of the set before building the real thing. We make use of these models to create a 3D digital model of the set which we project right onto the real set. We then align the physical space with its digital twin. The process of alignment is simply a matter of clicking on specific locations on the real physical set, then the system can use this knowledge to align the digital twin in 3D space, so the projection is perfectly mapped regardless of the position of the projector or set pieces in a fraction of the time taken with traditional projection mapping software.

“We are able to manipulate the digital version, for example by adding captions, graphics, or video to it, and those objects appear on the projection, and subsequently on the real set, in the correct place and accounting for occlusion and projector position. Then we simply hide the digital twin, leaving only the captions/graphics displayed on the set."

Dr Pierre-Alexis Mével from the University's Department of Modern Languages has drawn on expertise in screen translation to develop guidance for legibility and positioning of creative captioning in live performances. He has also produced a short video featuring creative captions to be used by the company and the venues to which they tour in order to build awareness of what captions can do. 

Red Earth Theatre

Red Earth Theatre has a pioneering track record in making integrated and accessible theatre using sign language and projected captioning as part of the action. Co-Artistic Director, Amanda Wilde, said: “This research project with the University of Nottingham has encouraged us to evaluate our current access provision and to be more creative with how access is integrated into our future touring shows. As a small-scale touring company committed to providing access for D/deaf audiences, it has afforded us the space with our research colleagues, to be more ambitious and inventive in developing our access offer and to push the boundaries of captioning in theatre. Because the work is research-based we are allowing ourselves to challenge current captioning conventions and test whether our ideas work with audiences. The research element of the project gives us the opportunity to both excel and fail in equal measure, a factor most important to testing new ideas and new approaches to access.”

The project has been funded by the AHRC and EPSRC as part of their ‘next-generation’ immersive experiences funding stream. This provides opportunities to multi-disciplinary research teams to promote and develop immersive experiences in the performance industries.  

A review for Soonchild from is available here

Story credits

For more information please contact Professor Jo Robinson, School of English, via email or Emma Rayner, Media Relations Manager for the Faculties of Arts on +44 (0)115 748 4413, 

Emma Rayner - Media Relations Manager, Faculty of Arts
Phone: 0115 748 4413

Notes to editors:

The University of Nottingham

Our academics can now be interviewed for broadcast via our Media Hub, which offers a Quicklink fixed camera and ISDN line facilities at Jubilee campus. For further information please contact a member of the Press Office on +44 (0)115 951 5798, email

For up to the minute media alerts, follow us on Twitter

The University of Nottingham is a research-intensive university with a proud heritage. Studying at the University of Nottingham is a life-changing experience, and we pride ourselves on unlocking the potential of our students. We have a pioneering spirit, expressed in the vision of our founder Sir Jesse Boot, which has seen us lead the way in establishing campuses in China and Malaysia - part of a globally connected network of education, research and industrial engagement. Ranked 18th in the UK by the QS World University Rankings 2023, the University’s state-of-the-art facilities and inclusive and disability sport provision is reflected in its crowning as The Times and Sunday Times Good University Guide Sports University of the Year twice in three years, most recently in 2021. We are ranked seventh for research power in the UK according to REF 2021. We have six beacons of research excellence helping to transform lives and change the world; we are also a major employer and industry partner - locally and globally. Alongside Nottingham Trent University, we lead the Universities for Nottingham initiative, a pioneering collaboration which brings together the combined strength and civic missions of Nottingham’s two world-class universities and is working with local communities and partners to aid recovery and renewal following the COVID-19 pandemic.

More news…

Media Relations - External Relations

The University of Nottingham
C Floor, Pope Building (Room C4)
University Park
Nottingham, NG7 2RD

telephone: +44 (0) 115 951 5798