Virtual Reality game with a Robotic Arm
Work for: Polidea (acquired by Snowflake)
Gaming, Employee training
5 team members
Sort-it VR is a game designed exclusively for the MCE 2018 Redefining Tech international conference. The aim of the game is to gain as many points as possible under time pressure by sorting objects on the assembly line.
Learn more how we hot there
When I was working at a software company Polidea (acquired by Snowflake), the aim of our interdisciplinary team (1 UX Designer, 1 VR Designer, 1 VR Engineer, 1 Robotic Arm Engineer, 1 Product Manager, 1 Product Owner) was to build a Virtual Reality game that can inspire clients. At the same time, it was supposed to be an entertainment experience to be launched at the MCE Redefining Tech 2018 international conference. Moreover, the VR experience was supposed to by synchronised with the Robotic Arm in real-time.
My role: UX/UI design for VR experience (starting from the concept for the game, 2D Interfaces Design in Sketch and 3D Interfaces Prototypes in Unity), a Robotic Arm choreography design, interaction design, testing with users
At the beginning, we wanted to understand Virtual Reality better and also get some inspirations for our idea. We were researching different kind of Experiences in VR, business cases and games. Also, how it applied in various industries. We have also checked what are the possibilities and examples of a Robotic Arm applications.
With the aim of providing business and entertainment value, and referencing the blocks concept from the conference visual materials in mind, we settled on Sorting objects on the assembly line idea.
In order to maintain consistency with MCE conference colourful blocks, from its Visual Identity, we kept the similar colour palette for our visual language. Furthermore, we created a Moodboard inspired by Environment design.
We made a User Flow diagram of the whole VR Experience to see the logic and all interactions that we need to design. We iterated and developed it further as the project progressed.
User Journey with Interactions
While working on designing User Interactions in, new to me in 3D, I came up with the below tool to help me visualise them on the paper. Once generalised to a template, I made VR storyboard with all necessary interactions representing the User Journey in our game.
The VR environment – a spacious factory has been imported to Unity3D and redesigned to get the visual style from the moodboard.
Low-fidelity Interface Design
In the VR environment, we considered interfaces attached to different points in the space, such as the controllers, in-game objects or above them, in front of the user, or hints somewhere in the space. Also, there might appear and disappear in various moments, it depends on the individual user flow.
This is why, using a flowchart to visualise the flow of some basic interfaces, often grouping them and adding comments related to interactions, turned out to be the best solution. I have also experimented with 3D mockups but it was not as effective.
High-fidelity Interface Design
Originally, I designed them as 2D interfaces in Sketch, and later learned how to build them as 3D objects in Unity, so that a VR Engineer could easily add a code to objects in the VR environment.
I worked on separate branches which might be integrated in Unity by an engineer later. He taught me how to add some simple interactions necessary for our game and would follow with building more complicated ones himself.
In the Sort-it VR, there are different kind of interactions. Firstly, there is a Tutorial to quickly teach the user how to play the game. Secondly, there are dynamic interfaces with explanations and practical exercises.
VR interaction design poses many challenges, some of which I encountered while trying to answer these questions: 1.How users should click buttons on the flat interface (laser beam interaction or more like a physical touch pad interaction), 2. What distance from the user and hight the flat interface should have, 3. How to use controllers to catch objects in the game, 4. How an on-hand tutorial should look like and where it should be, 5. How score points should be presented and where should they appear.
Prototyping and Testing
Interactions have been prototyped and tested with users to find out the most intuitive and suitable UX solutions for the game.
I was supposed to design a Robotic Arm choreography that was to be an attraction for the observers. The choreography was synchronised with the user’s movements in VR.
A Robotic Arm has a choreography while counting down until the game begins and ends. Also every time when a user throws a special MCE block into the MCE board in front of them, a Robot takes a physical MCE block and puts it into the same place on the physical MCE board/shelf.
In case of any errors with the Robotic Arm during the gameplay, I have designed info graphics that will appear on the screen, so that the monitoring staff fixes the problem.
Stand at the MCA 2018 conference
The conference took place in the Palace of Culture and Science in Warsaw. A building wing housed the VR Space that included a Stand with our project: a user in VR and a Robotic Arm with other objects that it interacts with its choreography.