LUTUM! An Exploration of Clay as a Digital Medium
Abstract
LUTUM! is an interactive computer-based experience made entirely out of clay. Inspired by point-and-click media throughout history LUTUM! Is an experiment in using analog methods in a digital space. In the space, users are greeted by the main character, a clay figure representative of myself, in her apartment. Users are prompted to accomplish various tasks such as cleaning her room. Through these seemingly menial tasks, the character speaks and interacts with the items in her room, alluding to a conversation about coming of age. LUTUM! pushes the limitations of clay as a digital medium, re-imagines early stop-motion media, and contributes to the contemporary conversation surrounding claymation as an art form and avenue for interactivity. Since its earliest origin in the 1926 film, Long Live the Bull, claymation has charmed the world as plasticine manipulates our perception of reality. While Gumby, Wallace, and Gromit have remained some of the world’s most beloved cartoon icons, stop motion has lost popularity due to the tedious nature of its production as media technologies especially as new VFX and CGI technologies improve. In an effort to revive claymation as a relevant medium, all of the game assets are made out of plasticine clay, programmed in Unity, and stop-motion animated in DragonFrame.
Introduction
LUTUM! is a point-and-click adventure game following the story of the protagonist and her childhood cat Zachary. In the game, the protagonist is woken up in her Brooklyn apartment at night by the sound of an alien screech. Upon going outside to investigate, the protagonist is greeted by her deceased childhood pet cat, Zachary, who has become a human-alien-cat hybrid.
There are 3 main components to the introduction of this project: Story, Medium/Aesthetics, and Game.
Story
The story takes place in a replica of my apartment in Williamsburg, Brooklyn. When exploring my apartment, players will learn about her feelings, anxieties, and questions about the world as she makes comments and interacts. She feels particularly overwhelmed and unhopeful about her future, can’t finish any artwork, and hates her job.
As the player comes to realize her struggles through the exploration of her items, the player is then introduced to the spirit of her dead childhood cat, Zachary. The loss of a childhood pet marks the end of adolescence for many, creating a new awareness of time and age. Zachary serves as a physical symbol of this uncomfortable transition between adolescence and adulthood.
Zachary traveled through space and time to come back to Earth to ask Julia for her help. He explains that the planet he’s living on is no longer sustainable for life and he is responsible for leading their escape. Julia must find all of the pieces Zachary is looking for in order to help him build a ship for escape.
Medium & Aesthetics
The focal point of the game, however, is not the story. It is instead the medium. All of the LUTUM! assets are made entirely out of plasticine clay, a putty-like modeling material most commonly used for claymation. All of the sprites were animated by hand using DragonFrame, the industry standard for stop-motion animation.
Why clay?
I have always had a fascination with the magic of stop-motion animation. As someone who loves to draw, animate, and sculpt, claymation has been a way for me to practice all of my artistic interests. I am also interested in the history of claymation as it relates to modern methods of animation. In a world where VFX and CGI tools have evolved so rapidly, stop-motion animation has become increasingly less favorable. My goal is to make claymation a more relevant medium for art and interactivity. I also want to show the possibilities of clay when used in a digital space. I want to reimagine how clay has been used in the past for a new audience: An audience that has witnessed the landscape of digital technology rapidly evolve in the past two decades into the highly polished Internet that is today. (For example, Mac OS.) I want to show that the future of technology does not have to exist without the qualities and methods of analog media.
I hope to reintroduce love as an essential component to production. With the tedious nature of stop motion animation, I believe there must be a degree of love and passion to complete a project. LAIKA Studio, the home of stop-motion masterpieces Coraline and Frankenweenie, returns very little profit in its films. But Travis Knight, the company’s president said in an interview with the blogsite Den of Geek, “What we want to do is have this medium live up to its potential, which means always pushing, trying to advance it, trying to get the next level of performance, nuance and subtlety, and to make it as dynamic as it can possibly be…We’re trying to tell stories in this medium that we love in the best possible way, not settling but always pushing for the next innovation.” Unfortunately, love alone is not enough to fund films. Luckily Knight is also the co-founder of Nike.
Internet Aesthetics
Another goal of mine is tapping into the ever-evolving web of internet aesthetics. Webcore, dreamcore, and weirdcore have taken over the niches of the internet. Derived from the word hardcore, the Internet has taken -core and to categorize and name new genres of visual and audio aesthetics. It is difficult to pinpoint the exact origin of these aesthetics, but it definitely can be attributed to tumblr and TikTok.
Webcore can be characterized by a celebration of the lost days of the unrefined self-expression that way the early internet.
Artwork that is created with webcore intentions will often include elements from early computer graphics, use filters that compress imagery or incorporate early fonts. A key quality that all of these choices combine to make is a nostalgic chaos of the wild west of the early internet. I think millennials and older gen-z individuals feel especially nostalgic when consuming webcore media because it is the interface we grew up on.
Weirdcore is characterized by empty spaces, blurred faces, and highly digitized imagery meant to make you feel weird.
It could be described as the surrealist movement of the new digital art and meme age. Images often depict broken realities and liminal spaces which can trigger the sensation of depersonalization or derealization. The ominous undertones of weirdcore are nods to the dark side of the internet and technology.
An artist that specifically created mixed media stop motion videos that are considered weirdcore is Jack Stauber. Jack Stauber is one of my main points of inspiration for this project because of his use of clay and other materials as art that brings up unsettling topics.
Dreamcore is similar to weirdcore through its use of surrealist imagery except it focuses on ideas associated with dreams, daydreams, or nightmares.
The intention of dreamcore is not to illicit feelings of dread or impending doom, but instead, its goal is to just create the feeling of a dream state through media. Dreamcore always uses bright colors but other common imagery includes bubbles, sparkles, rainbows, and glitter. Still, though, it feels just kind of off.
You could argue that all of these aesthetics are interchangeable. The elements do overlap in many cases. For example, this image has elements of all three.
There are many, many other niche internet aesthetics. Bloomcore, subjectivecore, campcore, and the list goes on. For now, I’ll stick with webcore, dreamcore, and weirdcore.
An artist that specifically created mixed media stop motion videos that are a mix of all of these aesthetics is Jack Stauber. Jack Stauber is one of my main points of inspiration for this project because of his use of clay and other materials as art that brings up unsettling topics.
What does this all have to do with claymation?
There are two ways internet aesthetics are connected to LUTUM!
- I think claymation can naturally lend itself to the webcore aesthetic. Claymation is nostalgic for many people because it was the medium of several classic films and TV Shows such as Wallace and Gromit, Gumby, and Shaun the Sheep. My claymation assets is in some ways reminiscent of these 2000s clay icons, but with a fresh take. LUTUM! also uses a font similar to monospaced fonts of the early 200s.
- My aesthetic goals for the color palette, character design, and visual hierarchy are inspired by webcore, dreamcore, and weirdcore. I chose to make references to all three of these internet aesthetics because my goal was to make LUTUM! relevant to new media; to separate it almost entirely from the preconceived notions of stop-motion animation. I’m not necessarily interested in capturing one aesthetic perfectly, but rather creating something that lives online somewhere between. I don’t want it to feel like The Nightmare Before Christmas or Coraline. I want this to feel like LUTUM!
By designing LUTUM! with internet aesthetics in mind, I am reimagining claymation with a modern internet-age perspective.
The Game
When I was planning what kind of interactivity I wanted LUTUM! To have, I knew I wanted it to have a point-and-click feel, but not entirely. I explored the basics of point-and-click adventure games and RPG games to get a better sense of where I would go with my game.
Point-and-click adventure games
A point-and-click adventure game is a game that focuses on navigating an environment to assemble clues, uncover a mystery, and complete a quest. Computer interactions are usually limited to clicking, drag-and-drop, and double click. Interactions with the game are usually limited to talking, looking, taking, using, and combining items. All of these actions are used as tools to solve a problem. A popular contemporary point-and-click style game are Escape Room games.
These games date all the way back to the earliest days of PC gaming in 1980 when On-Line Systems combined graphics and text for one of the first point-and-click games, Mystery House.
Related Works
Early Claymation Games
In terms of stop-motion animation point-and-click games, I am not the first to make one. One of the first claymation games was The Neverhood, created by Doug TenNapel and released in 1996 by Dreamworks Interactive, the creative lovechild of Bill Gates and Steven Spielberg. It was the first game to have all assets made entirely out of clay — 3 1/2 tons of plasticine.
The main character, Klaymen, solves puzzles and interacts with other characters. Throughout gameplay, there are several cutscenes that help to develop the plot.
The game received overwhelmingly positive reviews and several awards and nominations for Best Computer game. Real reviews of the game on IMBd praise its “childlike claymation interface but adult-level difficulties in game-play, all interwoven with quirky elements.”
Other notable reviews include:
- “Although it may appear to be, I don’t think that this game is for children. I say this because it genuinely freaked the hell out of me at several points and there is just an eerie, almost horror-like feeling shadowing the mood of the game-play.”
- “When Klaymen’s ultimate journey begins, there is no plot. There is no story. There is no big bad guy. There is exploration and there are puzzles and plenty of them.”
These quotes stuck out to me as characteristics that I knew I wanted to emulate in LUTUM! which are to challenge players’ ideas about claymation being for children by creating an uneasy mood, and simply encouraging the player to explore the clay space.
In the documentary The Making of Neverhood, the creators discuss building 1:2 scale models for the sets out of wood, drawing out every scene, and programming it all from scratch.
However, it was commercially unsuccessful only selling 37,000 copies in the US in its first year, an embarrassing figure in the industry. This was a result of point-and-click games falling out of popularity by the time it was released in the late 90s.
Despite its low figures, the same creators went on to create Armikrog, released in 2015, another claymation point-and-click game that was enjoyed by the small group of fanatic Neverhood followers from decades before.
Armikrog received mixed and average reviews with some describing it as “critically underrated” and stunning” and others describing it as “a lackluster adventure” and “boring and bland, it’s a dismal failure as an adventure game.”
In between The Neverhood and Armikrog, more claymation games were developed with some well-known titles such as Skullmonkeys and other lesser-known titles like Tanita: Plasticine Dream. Many of these games were award-winning, but I don’t feel particularly drawn to any of the visuals. They all feel like your typical Gumby claymation: a characteristic I want to challenge.
Though fascinating to research, I realized that I wasn’t able to get that much relevant insight about the actual development in the game because they all used pretty outdated technologies.
Contemporary
As stop motion animation and claymation have become less favorable over time due to an abundance of new SFX technologies, I didn’t expect to discover so many recent developments in stop motion game making. Two outstanding projects I researched were Hylics and Harold Halibut.
Harold Halibut
Harold Halibut is a claymation “narrative game,” as described on its homepage, “about friendship and life on a city-sized spaceship submerged in an alien ocean.” The game developer is Slow Bros. based in Germany and has not been released yet.
The images and trailer for the game are truly spectacular. It feels like a crossover between Wes Anderson and The Matrix with incredible attention to detail and futuristic set design.
The process for making the game hasn’t been explicitly stated, but in some behind the scene photos, I see some 3D modeling and scanning in addition to intricate fabrication of physical puppets and sets. The Unity logo can be seen on the game’s website.
The game hasn’t been released yet, but it seems to be a crossover between a game and a film with cinematic cutscenes and lighting.
Hylics
Hylics (2015) and Hylics 2 (2020) are surreal stop-motion JRPG games created by Mason Lindroth known for their claymation graphics, abstract art style, and wild storytelling. As opposed to an RPG game, a JRPG is made in the style of a Japanese Roleplaying Game or influenced by the style of early JRPGs such as Dragon Quest, Final Fantasy, or Pokemon.
At first glance, it’s easy to assume all of the moving parts of Hylics are 3D models. In fact, everything in Hylics is stop-motion animated.
I purchased this game on Steam and was completely blown away by the visuals, the music, and the overall experience. I haven’t gotten around to finishing it yet, but I really loved the structure of the game. The player begins in the map pictured above. The control instructions are floating in space around the map. The player has to explore and interact with NPCs to figure out the story themselves. It never explicitly says, “You need to find the key before you leave.” Everything is kind of cryptic but it encourages lots of thinking, problem-solving, and exploration.
From playing this, I decided to emphasize exploration as a focal point of the game. I loved how big the map was and how far I could make the main character walk around. The animation was seamless and fluid between jumping, walking and going up and down steps.
Though Hylics was first released the same year as Armikrog, Hylics is a fresh take on clay. Aesthetically, Lindroth has created this totally unique hybrid between the physical and digital worlds.
All of the assets have this hypnotic dream fuzz filter over them. It’s somewhere between bitmap art and highly compressed old webcore gifs, but whatever it is, it leaves you scratching your head trying to figure out if it’s clay or not.
It is never revealed entirely how Lindroth makes his assets, but he alludes to 3D scanning, traditional stop-motion animation, and the use of Blender to achieve his signature look.
After all of my research and findings, Hylics has become my main point of inspiration in regard to the success of re-imagining clay as a new medium for interactivity.
Methodology
When building out LUTUM!, I was building in the digital and physical worlds. It was definitely a challenge to switch back and forth between physical, digital, 2D, and 3D.
Planning the Game
Before I could start anything, I needed to have a solid visual plan. I started by sketching out a map with a rough idea of where interactions would take place.
- I made a list of all of the clickable items, their location, significance, animation, and what events they trigger.
- I made a list of all of the character animations and when they would be triggered.
- I made a rough script including all of these actions and items with cutscenes I wanted to include.
After I organized all of these assets, I sketched out rough stills in Procreate to use as placeholders as I develop the game.
It was helpful to visualize and interact with the scene. It made me think about details such as wall posters I might want to include with clay.
Programming the Game
The game is built using a plug-in for Unity, Adventure Creator. Adventure Creator, according to the website, is a fully-featured toolkit that has everything needed to make an adventure game. Its interface allows for powerful visual scripting with tools to create NavMeshes for walkable areas, Hotspots for interactivity, and a vast selection of different action types. I chose this plugin for Unity because I wanted to avoid programming the game from scratch considering my two-month time limitation. The visual scripting would allow me to focus more on the idea of clay in a digital space. Because it was designed to create point-and-click games, I was able to use some of the characteristics of point-and-click methodology in my game, but I still had to freedom to make it my own.
Using all of the temporary 2D assets, I began creating a 2D point-and-click game.
When I finished making the game with temporary assets and determined which part of the game was actually feasible, I began making the real clay assets.
Production
Building the Characters
From what I’ve learned during my time studying stop motion, watching behind the scenes videos, and appreciating the craft, I decided to take a unique approach to puppet building. Traditionally, stop motion puppets are made of a non-malleable material like silicone, wood, or resin castings. The material is assembled on top of a posable ball-and-socket armature skeleton.
I wanted to retain the organic look of soft clay, so I covered it in tinfoil and put clay right on top. I used this process for the puppet modeled after myself. When designing myself as a clay character, I decided that my most defining facial features were my bangs and soft nose and eyes. I gave myself an outfit I might actually wear in real life including a pair of Converse Chuck 70s.
For Zachary, I did not use a posable armature skeleton because I wouldn’t be doing any complex motion. I shaped tinfoil into an approximate cat shape and then covered it in white clay. I fixed a bendable wire in the rear of Zachary’s body which I then covered in clay. This would allow me to pose the tail for animation frames.
Building the Set
Just like the set designers of The Neverhood, I built the set out of wood then covered it in clay. I referenced my real-life bedroom for this, trying to stay true to the colors, window AC unit, and the artwork on my walls. I also made sure to pay attention to details such as wall trim and wood panel flooring. I created these effects using my fingers and various polymer clay tools. The size of the model is approximately 12” x 12” x 12”. The bed, vanity, and wall shelf were also constructed out of wood then covered in clay. All of the bedroom furniture pieces were moveable in order to allow for separate positioning in Unity.
Photography and Set Animations
I took photographs of the bedroom set with bright, clean lighting as far above overhead that I could manage. The camera I used was the Canon Mark IIII was positioned from an angle above to make the game isometric. I first photographed without the furniture that I wanted to be interactive or animated. Then, I inserted the furniture into the room and took more photographs. My idea was that I would remove these furniture items from the room using Photoshop. This way, I would ensure that the furniture angles would match the angles of the entire set.
The two items in the room I animated were the drawers and the Kit Kat clock. I used Dragonframe to capture several frames of the drawers opening and closing as well as the iconic Kit Kat clock swinging its tail and moving its eyes. I planned to insert the animations as a result of interaction with the item.
I also took photographs with and without dirty clothes scattered around my room. This would allow me to extract them in photoshop and place them in the room in Unity.
Animating Sprites
All of the moving parts of my game would be turned into sprites, two-dimensional objects used in computer graphics and video games. Combining frames creates sprite animations. To be able to play these animations in Unity, I had to create sprite sheets. Sprite sheets are documents that have animation frames for a particular sprite. I removed the background and rigging armature, leaving behind the characters with a transparent background and lined up all of the frames for each animation on a transparent background. I exported this as a PNG, brought into Unity where I created a new animation controller for each sprite. Once the animation was created, I was able to control when animation is played with the animation control panel.
Animating the Julia puppet was much more challenging. The shot list included eight walk cycle sequences from different directions: walk cycle left, walk cycle right, walk cycle up, walk cycle down, walkcycle diagonal-up-left, walkcycle diagonal-up-right, walkcycle diagonal-down-left, walkcycle diagonal-down-right. It also included a talk sequence and a blink sequence.
I used a reference layer in DragonFrame to rotoscope the animation of my puppet. By using a reference of another walk cycle, I knew how to position the legs and arms.
Because I used non-hardening polymer clay, I had to constantly repair and resculpt the body parts during the animation process. When I would move the arm, a crack would appear in the joints. I think this actually added a layer of noise I wouldn’t have been able to achieve otherwise.
The table I animated on is made specifically animating stop motion puppets. The table is made from a perforated sheet metal where the armature can be screwed in place. The metal armature has of the puppet has holes in the feet so that every time a foot needs to make contact with the ground, I can screw it in place.
Here is the final sprite sheet for Julia including walk and talk cycles for each direction as well as an unused animation of the face smooshing away.
Fonts and UI
To really tie everything together, I wanted to make my own fonts and graphics for UI features. These features included the inventory, the menu, the cursor, and speech bubbles. For the custom font, I printed out the alphabet in the font. I sculpted clay over the letters and then scanned them. Using the Photoshop plug-in Fontsself, I was able to turn my clay letters into an .otf file.
I sculpted them onto a piece of paper which I then scanned. For legibility, the font for the text in the game was SyneMono. I chose this monospaced source code inspired font because it reminded me of the early point and click games like Mystery House. The edges were rounded and organic, feeling like it could have been made of clay.
The other UI elements included an inventory box, volume controls, and menus. I sculpted these in the same way I sculpted the font letters on a piece of paper and scanned them in with my printer.
The largest UI element I made was the windows screensaver border. I did this because the edges of the scene weren’t interactive and kind of dull. I thought making it look like it was on a desktop would not only play into the webcore aesthetic, but also make the composition brighter.
And lastly, I made clay pointers.
Music and Sound Effects
Music and sound effects were the last things I worked on for the project. I wanted an ambient, noisy, Aphex Twin inspired soundtrack which I got from Noah Cote. Thanks Noah!
In the opening scene when the user clicks the “Lutum!” banner to enter the space, the sound effect that is triggered is a layering of my voice and Zachary’s meow with an 8-bit texture added to it created by Ethan O’Donovan.
A couple other thinsg make osund in the room. The Kit Kat Clock will make a clock sound when you press it and the air conditioner will also beep and make a fan noise. These sounds are from freesound.org.
Dialogue
When clicking on the interactive objects around the room, Julia will make various comments hinting towards the task to clean the room. For example, she will say, “No time to sit, I have to clean my room!” when the chair is clicked on. The conversation with Zachary alludes to his death and his ressurection.
The Final Product
The final gameplay goes as following:
There are no initial instructions so the player begins by exploring the space. They soon realize that the clothes are clickable. As the continue to explore the space, they discover the inventory. The clothes can be put into the drawers.
Conversation with Zachary can begin at any time. There are three different dialogue options with different responses from Zachary.
You can try out the game yourself at: https://jaf765.itch.io/lutum
I created a video about the initial planning and production stages of this project:
As well as a followup video which covers more of the UI stuff:
Evaluation
Finally, after all of these steps, I felt my game was ready to be playtested. I sat alongside my playtesters and took note of any confusion they encountered. Some of the adjectives my playtesters used to describe the game were surreal, dreamlike, silly, dark, and uncanny. There were very few issues navigating the space and understanding the UI, but they noted some player control instructions could’ve been helpful. I noticed that people who had experience with point-and-click games had less trouble understanding the player controls.
Conclusion
At a basic level, I achieved my goal of creating an interactive stop motion. I successfully integrated clay into a digital workspace by ocreated everything from fonts and UI elements to intricate walking characters that ould be controlled by your mouse.
Over the past three months, this project almost completely consumed my life. With so many moving parts of the project including stop motion, game design, and narrative, it was a challenge to find a balance between all three. While I did decide to put most of my time into the aesthetics of the stop motion animation, I still constantly thought about how the assets were contributing to my narrative, what the story meant to me, and what I hoped I would gain from it. In the beginning, LUTUM! was an exercise and experiment in interactive stop motion animation. Now, it is a time capsule of my life in this very moment. It’s an archive of my apartment on South 2nd St. It’s the memory of my dead cat. It’s the way I express myself and want to be perceived at this time in my life.
Acknowledgements
I couldn’t have done it without the help and support of the IMA faculty, friends and family.
IMA Professor: Christina Dacanay Graduate Assistants: Tinrey Wang, Alexandra 3D Scanning: Webb Hunt Unity Knowledge: Sarah Rothberg Music: Noah Cote SFX: Ethan O’Donovan Photography Resources: Noah Pivnik, ITP Documentation Lab, Carter Beardmore Playtesters: Isabelle Rieken, Webb Hunt, Ethan O’Donovan, George Gildersleeve, Bailey Foltz, XY Zhou, Max Chu, Elliot Wright, Aidan Massie, Bailey Foltz, Shiva Viswanathan
Bibliography
“Adventure Creator.” Unity Asset Store, 2021, assetstore.unity.com/packages/tools/game-toolkits/adventure-creator-11896.
“A Plasticine Dream.” Tanita Games, 2016, store.steampowered.com/app/372560/Tanita_A_Plasticine_Dream/.
“Dreamcore.” Aesthetics Wiki, Fandom, 2021, aesthetics.fandom.com/wiki/Dreamcore.
“DreamWorks Interactive.” Avid Wiki, Miraheze, 2021, avid.miraheze.org/wiki/DreamWorks_Interactive.
“Harold Halibut.” Slow Bros., 2021, store.steampowered.com/app/924750/Harold_Halibut/.
“Inside The Collaborations Between Stop-Motion Animation Studio Laika And Nike.” Fast Company, Mansueto Ventures, LLC, 2016, www.fastcompany.com/3062998/inside-the-collaborations-between-stop-motion-animation-studio-laika-and-ni.
“Jack Stauber.” YouTube, uploaded by Jack Stauber, 2021, www.youtube.com/c/JackStauber.
Lindroth, Mason. “Material World.” Kill Screen, Kill Screen Media, 2016, killscreen.com/previously/articles/material-world-mason-lindroth/.
“Stop Motion Animation and Early Special Effects.” The Cambridge Companion to the Silent Era, edited by Charlie Keil and Ben Singer, Cambridge University Press, 2014, pp. 193–214. JSTOR, www.jstor.org/stable/j.ctt2005zgm.11.
“Stop Motion Animation: A Brief History.” Animation World Network, AWN, Inc., 2013, www.awn.com/animationworld/stop-motion-animation-brief-history.
“Stardew Valley Creator Eric Barone on Self-Publishing, the “Everything” Update, and What’s Next.” GQ, Condé Nast, 2017, www.gq.com/story/stardew-valley-eric-barone-profile.
“The Neverhood.” Retro Games Online, 2018, www.retrogames.onl/2018/01/the-neverhood-ps1.html.
“The Rise and Fall of Adventure Games.” Digital Game Museum, 2021, www.digitalgamemuseum.org/the-rise-and-fall-of-adventure-games/.
“Wallace and Gromit: The Complete Collection.” Wallace and Gromit, Aardman Animations, 2017, www.wallaceandgromit.com/films/.
“Why I Love Chicken Run.” IMDB, Amazon, 2000, www.imdb.com/title/tt0213117/reviews.
“Hylics.” Mason Lindroth, 2015, store.steampowered.com/app/397740/Hylics/.
“Dujanah.” Jack King-Spooner, 2018, store.steampowered.com/app/681240/Dujanah/.
