Blog Archives

*Holographic, Interactive Nun

Posted on by

Maker Monday Commission: Magdalene Laundries

I’ve recently finished an artistic collaboration with multi-discipline Birmingham-based Live Artist Kate Spence.  Kate is embarking on a new long-term direction of work connected with the Magdalene Asylums of the Catholic Church.  Kate was able to begin this work thanks to being chosen as one of four recipients of the Maker Monday Commissioning Project (a fantastic achievement as it was highly competitive).  As part of this commission we were able to focus on building one aspect of Kate’s big vision.

Staring Vicki MacWinyers as the nun

The Magdalene Asylums is a delicate subject matter which has to be approached with care.  I was impressed with Kate’s bravery in tackling something so important.  The Magdalene Asylums were homes or more accurately work-houses for the ‘fallen women’ of the Catholic Church.  It was common that these woman would spend decades locked away and have their children taken from them.  The documentary ‘The Forgotten Maggies’ & the film ‘The Madgalene Sisters’ cover this in more detail.

After an extensive R&D phase looking at a wide range of technology, we decided to create a piece to simulate the way that the nuns treated the ‘fallen women’ in these work houses.

The ‘one-on-one’ installation

What was produced was a single-person installation piece which takes around 10 minutes in a large darkened space.  A full size holographic nun appears to float in one corner of the room. The nun orders the player to do tasks of increasing uneasiness and her reaction changes based on participant’s responses.

Just some of the bewildering and terrifying CYOA snippets

Kate essentially wrote a branching script (similar to a personal favourite of mine – the Choose Your Own Adventure Books) where in this case the choices are made by the player’s location or by what they say.  The nun’s reactions change based on whether you follow her instructions.  The software chooses from 50+ video clips of the nun to create the overall experience.

Technology breakdown

The technology behind this is a holographic projection ‘Pepper’s Ghost’, a ‘Kinect for Windows V2’ sensor and a custom software application.

Pepper’s Ghost illusion dates from Victorian-era magic – where a brightly lit offstage actor appears on stage by the reflection of their image in a sheet of glass.  Our slightly more modern version uses a projector and large sheet of Perspex.

The ‘Kinect for Windows V2’ is an incredible piece of technology produced by Microsoft – originally intended as a game controller for Xbox One (turning your body into the controller) it has been widely used outside of gaming.  The Kinect is a depth sensing camera allowing it to see distance as well as both the colour and Infra-red image.  This depth information can be used for a variety of purposes – most interesting is the ability to track the body and limbs of humans within view.  In ideal circumstances the Kinect can track up to 6 people at any one time.

The ‘Pepper’s Ghost’ hologram showing overlaid Kinect sensor information.

The software has been developed to handle the information from the Kinect and hence it knows your location in the room as well as your vocal responses to the nun’s questioning.

Next Steps

I was particularly interested in this project as I’m fascinated by the ideas of surveillance, control and manipulation.  I’m hoping to explore this further using similar technology but away from the difficult subject matter of the Magdalene Asylum nun.

I’m thinking a more abstract experience that really allows the participants to be more playful and push the boundaries of the interaction.  How can participants be manipulated in a one-on-one experience?  Do they play-along particularly when they are aware that this is an artificial experience?  Do players suspend their disbelief or play with the ‘fourth-wall’?  And can we use their behaviour to profile them further?

I was particularly interested in the idea that while an outsider would consider the actions of the nuns as ‘evil’ the nuns themselves would have been convinced that they were doing ‘good’.

* while this is often marketed as Holographic projection it has nothing to do with holography / holograms.  Thought I’d mention that before someone else does 😉

Category: My Projects

Make & Take – Create your own Audio Tour Experiences

I’m a big fan of predominantly audio experiences in real spaces.  Staying connected with the real environment while augmenting it with audio can be powerful.  Definitely preferable to being distracted by a mobile screen!

You can see this in some of my recent projects:


Take & Make is a current project (on behalf of West Midland Museum Development) and running until December 2016 working with 4 museums across the West Midlands.  The museums are involved are Hereford, Leamington Spa, Nuneaton and Whitchurch.  Fortunately for me these museums are at the far flung reaches of the West Midlands so I’m getting to explore our beautiful county some more 🙂

Make & Take Workshop at Birmingham Open Media

‘Make & Take’ Workshop at Birmingham Open Media

Each museum involved receives some equipment and training to get them started:

  • a small cuddly toy
  • a collection of NFC tags
  • NFC capable phones


This combination of hardware allows a museum to build their own magical audio tour guide.  If you’d like to try this on your own there are some materials here that help you to build your own:

Tutorial – Cuddly Audio Tour Guide

The project began with a workshop hosted at Birmingham Open Media where we talked about a variety of audio experiences.  I presented for discussion some of my favourite existing audio experiences and some innovative work from BBC Radio:

(I’ll spare you my analysis & deconstruction of these!)

During the day we also covered the technology in more detail – building a simple prototype and discussing ways in which it could be extended (e.g. using Bluetooth to share the audio, or ‘flipping’ the hardware – so having the audience carry the NFC tags instead).  In addition – each museum gets two half-day visits from me (those lucky people!) where we deal with issues specific to their individual project.  In between the mentoring visits the project teams continues their work by developing scripts, creating voice recordings, editing the audio and making simple graphical displays.


The projects are incredibly varied with some developed for adults, others for families while some are more linear and some more game-like.

Progress has been excellent so far and am really pleased with how confident the museums are with the technology.  I’m just about to begin my 2nd round of mentoring visits and then we’ll be having a wrap-up event in December where we share our results.  The experiences should go live for public visitors in the New Year.

Category: My Projects

Farmer George’s Flock Game at The Great Pagoda

[While I’m on a blog posting roll – here’s some details of another project that I’ve been collaborating on – again with interactive theatre makers ‘The Other Way Works’]

The idea: Cuddly Sheep as the Game Controller

We were fortunate to win a competition earlier in the year to develop a ‘playful experience’ to bring ‘The Great Pagoda’ to life.  The competition website is still up if you want to take a look at what we had to work with.

Our idea was an advanced audio tour guide in the shape of a cuddly sheep.  It would be more playful and interactive than the usual audio tour guides as it would be the sheep telling the story.  As a family you collect a sheep from the ground floor and it tells you to climb the Pagoda while it talks about educational & entertaining stories.  There are fun activities to do on the way – such as making sure the sheep is well fed and taking them to particular windows so that it can talk about the view.


One of our favourite features was that sheep would know the whereabouts of other sheep – so when you went close to another group the sheep would start talking to each other.

I should add that we hadn’t come up with the sheep idea just because of our natural love of sheep – there is a great history of sheep connected with both King George III & the pagoda.

“George III earned the nickname ‘Farmer George’ through his keen interest in farming and its improvement. One of his agricultural experiments was centred on improving the quality of his sheep’s wool by cross-breeding them with Spanish Merino sheep. Export of these animals was, at the time, banned by the Spanish authorities. However, Kew’s botanist, Joseph Banks, managed to acquire several Merinos for the King that were duly integrated into His Majesty’s flock. Subsequently sheep from the King’s flock were sold at public auctions that were held annually at the base of the Pagoda every August between 1804 and 1810 before the royal flock was dispersed in 1813.”


Katie Day, Artistic Director of The Other Way Works and me (in the background) playing with sheep & scales

Extending the location to outside The Pagoda

After a number of discussions with Historic Royal Palaces the project changed slightly to instead focus on the area immediately surrounding the Pagoda.  This was a positive decision that very much helped our experience.  It makes more sense for the sheep to be spending their time on the grassy area outside instead of climbing the stairs of the Pagoda.  This enabled us to include many more fun sheep activities and extend the experience to be much more game like.

Technology Tour

When building these types of experiences I often use the word magical –as while they are very high tech, that is all kept behind the scenes.  Players shouldn’t even notice the technology and instead just focus on the fact you are helping a talking sheep on their adventures.  This ‘smart-sheep’ essentially becomes our game controller and all interaction with the game is through it.


The exciting shot of ‘me working’

For those that are interested – the intelligent technology is contained within the sheep – so an Android phone (with Bluetooth 4.0 & Near Field Communication capabilities).  The dumb technology is hidden in the environment (Beacons & NFC tags).  This allows the sheep to know when it is within a certain area or sitting on a certain location.  The players make game decisions purely by positioning the sheep in the world.


Our ‘game controller’ sitting on our prototype marker (secretly covered in NFC tags)

The game system is the clever part and is a custom software application.  It is written in C# using the Xamarin cross-platform development tools and runs on the Android phone – so a mobile app on a phone.  By using Xamarin it means that we’re not limited by hardware – it would be possible to have this running on iOS or Windows if required (although currently Apple don’t allow developer access to the NFC hardware!)

First ever Artists in Residence @ HRPStudio 🙂

As part of this commission we were invited to be the first ever Artists in Residence at the Historical Royal Palaces Studio.  The main goal of this residency is to work on the project with HRP input, engage with the public and test with internal groups.


We built a prototype before we arrived in residence and then with access to the vast knowledge of HRP staff were able to customise aspects of it to ensure we were historically accurate (so it would educational as well as fun – although mainly fun!).

This ‘vertical slice’ (note there is some ambiguity & debate over this term) – so a complete playable experience from beginning to end with work from graphic artists and real voice actors.  This allowed a version of the game to be played as well as testing some of the game mechanics e.g. cheekily eating forbidden vegetables from the palace gardens before being caught, or figuring out a password to get into another area.  This prototype game takes a small group just under 30 minutes to play through in its current form.   It contains over 80 individual audio segments spoken by 5 different game characters.


Play-testing in the ‘Clock Court’

One of the main benefits was to run two afternoon play-test sessions:  the first with staff and then the second with their families.  We were delighted with the feedback from both sessions.  The most useful was seeing families engage with it.  No matter how many times you’ve done this before there is always something you can learn by standing back and watching your target audience playing with the game.  The weather wasn’t idea for the second test as it was absolutely pouring down – but it was heartening to see smiling families enjoying themselves and playing it through it to the end.


Feedback from the youngest play-testers!


New rule – all my games to be made with such stunning backdrops!

The feedback from the staff was also very positive and they were impressed at just how far we had got with this prototype at this stage.  The fact that it was a complete experience was clearly above their expectations at this point 😉

What happens next?

It has been a fantastic opportunity and surely one the most beautiful and historically important places that I have made games!  It was hugely exciting to walk into ‘work’ each day when work is the magnificent Hampton Court Palace.  We made sure that on our lunch breaks to take in different parts of the palace so got to see the fabulous location and amazing interpretation (both static & live!).

The technology and game mechanics proved so successful that I’m sure we’ll be using a variation of this game for another location in the near future.  Watch this space!

Our beautiful cardboard prototype destroyed by the great British weather

Our beautiful cardboard prototype destroyed by the great British weather

Category: My Projects