Goals and Motivation

The motivation that all members share is the creation of a game where creativity can be enhanced through physics simulation in an AR environment. Moreover, the common interest in learning about AR technology and exploitation of the possibilities. This motivation brings us down to the main goal of the project which is the development of a visually pleasing, augmented reality based experience application, to life. The result of this is a physics simulated based building experience in the AR context, namely Kapl'AR.

Technologies

The game’s core gameplay functionality is the manipulation of gravity so that the placement of the building blocks will interact as realistically as possible. For that reason a game engine that supports both physics simulation and libraries for AR was the best choice. Hence, Unity engine was chosen as the main tool. MS Visual Studio was used for IDE as C# is the main programming language for writing the scripts in Unity. The project consisted of four members, therefore the Trello project management tool was more than sufficient in order to keep track of the user cases and task delegation. Finally, different branches and versions of the code was handled by the git version control system.

Challenges and obstacles

KaplAR was built during the autumn semester of 2020 when Covid-19 was active and all KTH education was held at a distance. Group work at a distance proved to be a big challenge, especially for maintaining communication channels and motivation during a long project. Another challenge during the development has been interactively testing and debugging AR applications using Unity. A simple parameter change can take up to 6 minutes to test.

Related work

Our project took inspiration and was built on several other previous projects. Here are the three main articles that helped us for the development of Kapl'AR:

• A Physics-Based Augmented Reality Jenga Stacking Game
• Real-Time 3D Gesture Sensing For Interaction With Head-Mounted DisplayFor Augmented Reality Jenga Game
• A Vision-based 3D finger interactions for mixed reality games with physics simulation

Of course, our main inspiration has been the original Kapla game.

Lessons learned

There are several things we learned and wish we knew before we started the project:
When working with AR efficient debug tools need to be developed first to ease testing.
Especially when working remotely, meetings should be scheduled regularly and uniformly to keep the project work active and progressing, and keep the members motivated.
Continuous testing and asking users to play the game is important and many times vital.

Fails and successes

We succeeded in creating a fully functional single player building/construction game in the context of an AR application. A full game-loop from “start a game” state to a winning or losing state is present. Two stages are currently available with special features and setting in each of them (you can blow on your phone to create wind in the game in the first one and random earthquakes appear in the second). An inventory system has been created to allow custom blocks with special charateristics in each level. In addition, audio/music control is present. Other advanced physics, graphics and interaction features are present that we will let you discover in the game!

Unfortunately, we had to put aside some ideas, prototypes and non-functional features:

• Edvin: Mesh cut (demo)

  The mesh cut loads an Unity mesh into a half-edge data structure. Given a plane, the user can then cut, or clip, this mesh into two parts along the plane. This feature didn’t make it into the application due to the following reasons: it does not support UV-coordinates, the triangulation algorithm used introduces numerical instability due to skinny triangles, and it is not clear how it would be implemented in the app in a way that makes sense. See the video for a demo of the feature in the Unity editor.

• Ted: Hand tracking and interaction (demo)

  The current version allows the user to place blocks by moving their phone and tapping the screen, but the initial idea was to allow the user to use their hand inside of the augmented reality to move and place blocks. We tried this using ManoMotion, which is a library for Unity which tracks and recognizes hand gestures. Unfortunately, the tracking had several issues of unknown reasons. After many attempts to integrate it into the application, we finally decided to scrap it, as it was taking more time than we thought would be better spent implementing more solid things for the core aspect of the game.