拾光 - Collection of Time
Skills: Unity, OpenCV, Maya, Shader Graph, VR/MR
拾光(Collection of Time) is an interactive VR album. Instead of simply displaying the photos, 拾光 converts the posts of a given Instagram user into folded letters hidden in the game scene. By finding the letters, the player can then read the content of the post and see the corresponding photo. With this application, the player can use their Instagram account as a diary or a blog, and over the course of time, they will find they have created an interactive album full of memories.
Motivation of Creating the Application
The original idea of 拾光 is triggered by the intention to comfort my friend Vivi, who was informed that her dear cat OneOne had got cancer and her days were numbered. As a person who knows how hard it is when life hits that deep, I empathized with her and decided to create an interactive VR album. I aspire the app to comfort people and help them remember the time and stories they shared with their beloved.
Gameplay
In the game, the player first creates and decorates their own scene with scene objects I made. After setting up the scene, the player enters the scene and starts to find the letters, which are converted in the run-time from a given Instagram user's posts, hidden in the scene. Once the player finds a letter, they can open their left hand, and the letter will fly to their left hand and open for them to read the content of the post. When finishing reading, the player can then hold the left hand to grab a slingshot and use the right hand to aim and then shoot the slingshot into the sky. After blasts of fireworks, the photo of the Instagram post will appear in the form of a constellation. The player can find as many letters as possible before the BGM ends.
Design Concept
The first thing the player sees on the main menu is a scene of a sky lantern festival.
In Chinese culture, by writing the wishes on the sky lanterns(天燈) and then releasing them, people believe that their wishes can be sent to and heard by people in heaven. I borrow this notion into my game and hope the player knows no matter where the people we miss are, they will always be with us and hear us.
In the game, instead of plainly showing the photos and texts using slides, I hope the player can enjoy the game when trying to find the letter, using the slingshot to shoot, seeing the constellation, and guessing what photo the stars on the sky represents.
Programming and Technique
1. Setting the game scene
The program uses the LittleWorm MR Builder, a Unity package I made to help VR developers solve the design problem of not being able to know and customize the game level based on the player's playing area. By using the MR Builder, the player can use the premade scene objects I designed to build and decorate the game scene themselves.
2. Converting the Instagram posts into letters in the game
In the program, I first use a web crawler to fetch the Instagram posts and convert it into photos and texts. Then, the photos will be processed by an image process pipeline I designed using OpenCV. The photo will first be converted into a grayscale texture. After that, by eroding and dilating the texture, two textures are created. Finally, the outline texture is produced by subtracting the eroded one with the dilated one.
User Experience
Since the game was intended for my friend Vivi, I invited her as the first person to experience the game.
I first created an Instagram account for Vivi's cat OneOne, who got cancer. Then, I posted some photos of OneOne and Vivi with that account. By giving that account to the program, the game had automatically become a personalized interactive VR album for Vivi.
Vivi the human & OneOne the cat
While I had done many user trials, I still found that my UX design is not good enough. For example, I found that the UIs, especially the tool icons, of my MR Builder were not as intuitive as I thought, so I ended up setting the game scene for Vivi. Another failed UX design is insufficient instructions to help the player to use pure hand gesture/hand tracking throughout the game. It is much harder than I expected to give the player instructions on where and how they should place their hands and fingers in order to play the game.
However, all my time and efforts paid off when Vivi started playing and then came out with an "Ohh~" after she saw OneOne's image appeared in the sky. We couldn’t even finish the whole game because Vivi was so overwhelmed with emotions that she started crying. Although my intention was to bring the player joy and happiness, I was still touched and glad to see that my game was able to help Vivi release her feeling. I really hope my game has lifted some of the depression clouded on her mind.
Conclusion
Overall, I am proud of this project. Surely, there are some weaknesses, such as the imperfect UX design mentioned above, the design of the image process pipeline does not come out with great result when processing certain kind of photos, or the fact that the MR builder does not solve the level design problem that well. However, I love how this game can touch and comfort the player's heart, especially after I saw Vivi played the game. I never know that I am capable of creating a work that can bring out such strong emotions from the player.
As the world and international situation become more and more uncertain and treacherous, I hope I can continue to tell the truth of my heart and deliver my thoughts through my games.