Projection Mapping Multimedia Art Project with the use of Projector, Foam core, Paper Origami

Software: HeavyM, Adobe AfterEffects, Audition, Photoshop, Illustrator

Gifs

Artist Statement

This project and the images/songs used are an ode to images of the past, present, and future. I used some of my own images, popular culture gifs from Giphy, and glitch effect videos from YouTube. I also really like city pop and 80s music, and unironically think “Never Gonna Give You Up” is a good song.

The 5 pyramids in the middle represents the 5 stages of grief (denial, bargaining, anger, depression, acceptance), through their loosely circular shape. I know my mental health had some ups and downs especially during Covid-19, and I still feel it ebbs and flows. As humans, we’re always growing and changing, and there will always be grief and loss. This projection mapping project is where I tried to bring together some feelings I’ve had towards my past, present, and the future.

The music compliment was arranged to give a spooky, horror ambiance with strong use of sound effects to highlight the changes in the projection mapping.

The story of the projection mapping is a longing for the past, to time when I was younger and enjoyed more forms of popular media. The images are mostly things that a Millennial or older Gen Z would recognize. I sampled some parts of Rick Astley’s “Never Gonna Give You Up” and “Together Forever” for the recognizable catchiness of the tracks, but split it up/pitched it/distorted it in a way that aided the strange unsettling feeling I was creating. The other 2 songs used are: The Mii Theme Song aka “The Wii Song” and    MACINTOSH PLUS – FLORAL SHOPPE – 02 リサフランク420 – 現代のコンピュー , a  popular vaporwave meme song. These sounds are also highly nostalgic for people of my age group and  provide a dreamy nostalgic quality to the piece.

The VHS screen and glitch video effects are also used to hark back to older forms of media, before everything was digital.

The cutting in and out of Rick Astley’s “Together Forever” music video is to represent past memories, how they fade in and out, and you eventually forget the details.  

Sketches

The idea for the project came from my desire to experiment with Projection Mapping. As an Illustrator, I’m always interested in bringing new life into my images and pushing the boundaries of my craft. Projection Mapping felt like a harmonious choice of blending live art, music, and physical ephemeral experiences.

Process

Part 1 – Brainstorming Ideas, Compositing Ideas

Originally, my ideas were much more grandiose and combined Projection Mapping with SparkAR, web-based AR. But upon further timeline, it would have been a very heavy workload to fit all the non-integral parts.

Part 2 – Projector Use

Plugging in the projector and figuring out the best cast length and which possible surfaces to cast onto. Testing the projector’s color accuracy and the quality of casting based on the types of surfaces (white paper/plastic/foam core)

Part 3 – HeavyM Software Exploration

While using the free Demo version of HeavyM, I created shapes using their basic shape tool(square, circle ,triangle) and organic shapes with the pen tool. The shapes were either filled-in with the software’s inherit line, color, shading, animation effects.

Part 4 – HeavyM Export/Rendering Issues + Paper Pyramid Origami Creation/Attachment

Since the video and shader rendering was unnecessarily long, I scavenged for more gifs that would suite background/shader use. The origami paper pyramids were created from a YouTube tutorial, as a substitute from the official triangular surface, Olga, that is sold by HeavyM. The origami were reinforced with double sided tape and pressed by heavy books to make sure the edges were crisp and clear.

Part 5 – HeavyM Geometric Shapes Mapping, Gif/Video Shader Mapping

Creation of a new HeavyM project file to map the specific triangular shapes with the projector turned on. This is also where taping down the spots of all physical components were necessary, as something off by a few degrees or centimeters would ruin the effect. The HeavyM file had to also keep the layers labelled and organized, as each pyramid needed its own layer, and each varying overlay and background effect needs its own layer as well. Many test renders were exported, as different blending modes created highly specific effects. Sounds were also not exported in HeavyM, though I was thinking of using AfterEffects to add sound afterwards, so this was not a big issue.

Part 6 – Audition Sound Effects/Sounds Mix Creation

I consolidated many .mp3 and .wav files in Adobe Audition to create the audio mix. I was going for strong sounds that would aid in creating nice shifting transitions for the various HeavyM .mov exports.

Part 7 – Timing HeavyM Rendered Clips to Audition sound mix in AfterEffects

The Audition Audio was exported out and brought into AfterEffects, where the various effects were timed to the audio. So video effects were also applied to aid in the transitions between scenes, where necessary.