Transcript
Amber Gibson: Welcome to Inside ACMI X: A series where we discuss TV, film, videogames, virtual, augmented, and mixed reality with the residents who work at ACMI's screen focus coworking space; ACMI X. I'm Amber Gibson, the Community Coordinator. Before we begin, I would like to acknowledge the traditional owners, the Wurundjeri and Boonwurrung people of the Kulin nation, on whose land we meet, share, and record this podcast, here in Melbourne. It always was, and always will be Aboriginal land.
Here with me are Emma Parker and Chris Leaver from experiential training company, Isonomic, to explain how holograms are being used to upskill the Australian workforce. Emma is Founder and CEO, and has worked as a Transformation and Change Expert for over 20 years, consulting in major blue-chip environments. Chris is Partner of Isonomic, handling production and delivery. He's a visual effects artist, producer, and sculptor. Together they lead Isonomic; a cutting-edge, immersive, augmented, virtual and mixed reality company, made up of talented developers, game designers and 3D artists, who transform complex training into state-of-the-art immersive training in the HoloLens. Welcome, Emma and Chris.
Emma Parker & Chris Leaver: Hi!
AG: Emma. It would be great to start with you and hear about when you first started to work with augmented, virtual and mixed-reality technologies.
EP: I had an unusual move into the field. I started about 15 years ago in Change Management and moved into that straight from Human Resources. It sounds like a strange move. Actually, both are really human behaviour focused, in terms of what you're looking at. You're looking at why people make certain decisions, what behaviour changes you are looking at, at an organizational level, and how transformation can actually be effective. I was very lucky to work at some really big blue-chip companies early on in my career. I had a lot of experience and input into operating model designs and delivery. So as part of that, I started to actually get into training. That started to tick an idea in the back of my mind that there was a better way to do some of these things and there was a way to start to use technology to make a change in people, in a way that's more interesting than how we deliver it now in an online learning environment.
At the time, the only exposure I had to that, was sitting on the board of Orbitvfx - which is Chris - our partners of the company and as part of that, I started to get to know designers, artists, 3D specialists, (I) had my first go in (a) virtual reality headset. I then just started thinking; "okay. I can either stay on the pathway I'm going at, which is very much on the banking side, or I can take a risk now when the market is quite new and we have a lot of expertise across our skillset, and just go." And we did.
AG: And how about you, Chris? How did you use your skills as a visual effects artist, producer, and sculptor to create those holograms that people are using for training?
CL: Well, really my background really informed our pipeline and the processes of which we came to build all the models, and realised those in augmented reality and mixed reality. My experience has been involved with the making of models for film and television, and we use that same pipeline and the same quality of artists and outcomes to put them in the pipeline for the AR stuff. From the 25 years I've had experience in the film industry, I've got a large network of designers or 3D guys or coders. There's a big pool of people out there who we can draw on.
So we use the same sort of scenario that you would in 2D, as you would, going into AR. So we built the storyboards, we did the scripting, model making, and the same approval process. So the same pipeline that you would, with a film, we use the same scenario for the AR stuff as well. So, we tend to get a better quality of outcome than you would, for example, if you'd only been making games or you'd only been making industrial applications because we bring that knowledge of how things look in the space as well.
EP: You understand how complex a scene can be because if you've not come from that background, you could think a scene is very easy to do and simple. But actually having some knowledge of the film industry, like you do, we can understand that one scene that a client wants is actually really complicated in a virtual world or in the Hololens and so we can work with them on a better way to do things. It's more economical.
AG: Can you describe the service and products that Isonomic provides?
EP: All our work involves new technology, ranging from machine learning to AR, down to mixed reality with the HoloLens, which is one of the more expensive pieces of equipment that we program into. The nature of the work that we do, we do very large infrastructure immersive experiences, and the type of machinery that we build is built for a purpose. So it's actually built for that specific job. Whether it's South West Metro or whether it's the rail extension in Sydney, they spend a lot on designing specific pieces of machinery and plant, which we actually build holograms of, before that machine even exists, and train users on it, before it's been put together and actually delivered. One of the benefits of that is they can actually look at it from a risk assessment perspective and actually then recommend any changes to that machinery before it's even used in service.
They can then actually have users ready to go on-site so they've been inducted and they understand the safety parameters of the machinery. Also; the machines are massive. They're immovable beasts. You can't be trained on one of the machines. We do a lot around tunnel boring machines and they're three times the size of an airplane. So you are not going to be in a training room with one of them anytime soon, whereas we can build a large but scaled-down version of that so you still get the sense of scale. In particular, in the project we've just done - which is with the Victorian Tunnelling Centre - we built part sections of that machine but life-size, so that you could be in it and then it would change to another part of the machine, and then another part of the machine, but in holographic form. They enabled you to move through it, but not have to have a training room that was the size of the actual machine. They're the major projects that we work on.
AG: Can you describe for listeners, the difference between virtual, augmented and mixed reality?
CL: In virtual reality, you're closed off from the environment. You're within a VR headset. You cannot see anything else except the world that you're in. With mixed and augmented reality, you can still see the world around you. So if you've got the headset on, you can still see your loungeroom or wherever you might be; the boardroom or the office or the warehouse. You can still see that environment, so you're not going to trip over and you can inter-react in actual pieces of machinery.
EP: Virtual reality is very much capped at what it can do. It's very much gamification of learning and you can't really move beyond that in terms of the tool. AR and its implicit term is obviously holding up the iPad and the Pokemon experience, certainly not hands-free because it can't be. Whereas mixed reality is very much seen that you can actually work in a blended augmented environment, at the same time on the task, with your hands-free.
AG: You spoke about holograms. Can you describe the Microsoft HoloLens for listeners who actually haven't experienced using mixed reality technology?
CL: So you've got some glasses - a headset - which has got the computer and all the sensors in it and you can see out of it, so you can see the room around you, and you're presented with the hologram in front of your eyes which is fixed at the space. The HoloLens, in particular, works by gathering data about the room and the space that you're in and relaying that back to the computer, and then projecting the Hololens within that environment, very stably, and then projecting that into your eyeballs.
AG: Does it use gaze and head-tracking capabilities? Is that what you mean; that the data's being filtered back in from the device?
CL: Yes. Well, there are two things going on: it's firing a laser or photons out into the room and collecting that data to build a model of the room, a spatial model of the room. So it knows where it is, at all times. And also, it's seeing where your eyes are looking at the same time. So it's doing two things, to build that holographic experience in the situation that you're in. It's basically a laser system that fires into a series of mirrors that bounces back through a lens and then fires into your eyes so you can actually see it.
AG: So if someone is learning how to use a different type of equipment, they could also be taking notes while wearing this headset.
EP: It's actually even more than that. They can do the job that they're doing. For example, how it's used in the aeronautical industry is in repairing plans at the same time as wearing the headset.
CL: So you're hands-free, you're basically hands-free. You can call up data, there's voice recognition as well. You can remote into someone else like a technology expert if you need any other information, you can zoom them in if you wanted to and they can see what you're (doing) (and say) you know; "don't cut the green wire, cut the red wire." So it's greater than the sum of its parts in terms of what it can do.
AG: How do the employees react when their boss gives them a headset to put on and learn some training around a machine?
EP: It's probably the most enjoyable part, as you get to see their excitement just as soon as you hand them the headset really, because they've never seen this thing before. So, as soon as you hand them the headset, they know that they're going to do something that's really different and they're not quite sure what it is. So there's a bit of nerves and anticipation that comes in.
We also take them through a video orientation before we even take them into a training room, because the training rooms can be quite specific and a little bit dark, so you want to get people ready for the experience. That adds to it as well, because they've got this waiting time before they go in, so it feels like they're going into a paintballing extreme version or something.
Then the rooms themselves - like the particular one we've got the training going into now in Chadstone - I mean, it's a life-size mind tunnel. So it's got real flooring on it, which is rock. So it's an uneven surface or concrete, it's dark, it's dusty, it really has got the whole thing, and so when they open that door and walk in, they have that sense of just the enormity of what they're about to go through.
Then you get the surprise element because as soon as they put a headset on, a lot of people expect virtual reality. They kind of expect that you're just going to replace what they're going to see. When they walk into that room and realise that actually being in that room is part of the experience and what they're going to get is augmented into that space in front of them. When suddenly something appears and starts animating in front of them, the first thing they say is, they're just like; "wow, that's incredible. That's blown my mind. How does that even work?" That's always (within) the first few minutes, which is always lovely to see, but then you see them getting engrossed.
AG: What are some of the training outcomes that you've seen after implementing the roll-outs?
EP: We have noticed that recall is much stronger. It's definitely much stronger in people that have gone through the immersive experience in comparison to being presented with a PowerPoint. One of the other things that we've noticed is that particularly in construction, it gives people an understanding of the whole pipeline. So why A relies on B and why B then has to deliver to D on time. What happens if a dump truck ends in a ditch and can't get the delivery to W person? And what does that really do to the whole chain? Because you can build mini-worlds that people can see operating, they walk away for the first time - some of them we've met had been working in the environment for over 10 years - and they were like; "oh. So, that's what happens. And that's why that's important." It just changed their whole perspective on the whole work environment, which you can't really do in another tool. You can't show that micro-level in such an immersive way in any other tool; it just doesn't exist. That changes people's behaviour because they suddenly buy into their own role within a whole chain of events and that's what you want to do to reduce risk and improve safety.
AG: What is in store for you for the next 12 months? What projects are you working on?
EP: We're looking at a much more innovative use of the HoloLens and... I can't talk too much about it. But what we're looking at is a much more technology-driven use of the Hololens in its future intended states. So actually working on machinery, wearing a headset, but combining that with machine learning. What we will be doing is looking at bringing a couple of mentors over from New Zealand and from the States, so we can get some mentoring onto the project of local talent in this area and really build some game-changing tech. It's exciting. So, that'll be the next 18 months.
AG: Thanks so much, Emma and Chris, for chatting to us today.
EP & CL: Thanks for having us!
AG: Thanks for joining us on Inside ACMI X. To learn more about ACMI X: our website and Twitter are listed in the show notes, alongside information about Isonomic and each of our guests.