Starlight / Coca-Cola / Tool of NA

To integrate the Starlight product,  we asked people to scan the Starlight can to begin the experience. A challenge was that during early testing we found that we were getting poor accuracy due to the reflectiveness and shininess of the cans. Through trial and error and using a combination of spherical and rectangular image targets we achieved acceptable scanning results using the built- in 8Th Wall target recognition.

We also created a custom trained AI model - ssd_mobilenet_v2_fpnlite_320x320 -  specifically for the Starlight cans running in the browser. This model is very robust and has a very good tracking rate even in different and more difficult lighting conditions.

‍

All the experiences were built with our own custom WebGL pipeline that we then connected with 8thWall. To make sure we would have a consistent look and feel and to easily integrate all the shaders and effects across the 3 performances we created our own node based timeline system in Javascript/json. This enabled us to easily control, add/remove and time all shaders/effects, particles and geometry within each scene for all the different songs.

To have high fidelity, we spent a lot of effort in creating custom shaders for the hologram effect, the particles, the portals and the shading of the giant base rock.

‍