Now that Curious Blocks has been submitted to the App Store, I want to step back and take a minute to talk about what we’ve learned about the control aspect of AR apps and games.
We decided last year to make a game that utilised AR, and we started experimenting with ARKit by making a number of prototypes. They ranged from local multiplayer ping pong, fighting games, to MOBA experiences. We wanted to know how AR can improve games and interactive experiences (alongside it’s limitations).
In one of the prototypes, we decided to explore one particular aspect of AR – control. With AR, controlling the position and angle of the (virtual) camera in a 3D space all of a sudden becomes incredible easy and intuitive. The way we control the camera in 3D is standardised on many platforms; on the PC, we usually use the mouse + command/alt/control to look around, arrow/WASD keys to pan, and mouse wheel to zoom. On consoles with controllers, we use the left thumbstick to look around and the right thumbstick to pan.
(image: navigating the 3D space in Unity)
Then on mobile, we normally use one or two fingers to swipe across the screen to rotate, pan, and zoom. The touch gesture is fairly intuitive, however it also comes with a few limitations:
(photo: by Lina Yatsen)
With augmented reality, we can tackle both of these issues. AR allows the player to control the camera in 3D by moving and rotating the device naturally. There is no learning curve – this is how people take photos on their phones, and this is how we move the camera in real life. As we began prototyping an AR creative app, we quickly realised the experience of creating a model in Curious Blocks actually feels like making a sculpture! When you start to move the device to find the right spots to put down your blocks, you move your arms and tilt your head to create something in-between the virtual and the real. Sometimes it even feels meditative!
For developers and 3D artists, manoeuvring the 3D space with keyboard+mouse is second nature, and it’s the same for gamers with joysticks. However, I still remember hours spent getting my hands used to the these types of controls. There’s a certain complexity whenever we map a virtual 3D environment onto a 2D screen.
(image: MagicaVoxel on PC)
With augmented reality, however, we no longer try to navigate a 3D environment via a 2D screen. The 3D environment is brought back to the physical world – which is naturally 3D. Curious Blocks may look like it’s all about stacking blocks to create 3D models, but at the core of it, it is the first aspect of AR we decided to explore and create an experience for – a _native_ 3D navigation / control.
Further more, with augmented reality, the line between what we perceive in the real-world starts to blur with what we experience on screen in the virtual space. I’ve surprised myself many times when I have reached behind the phone to try to move the blocks! Or I’ve been puzzled for a brief second wondering where the model went after I put down the phone.
Augmented reality has a lot to offer the entertainment world and we intend to continue to explore its various properties – physical, technical, and also physiological. It’s a new and exciting field and I can’t wait to share with you what we are cooking next!