My final project at ITP Camp 2018 was “Murder Ballad: The Game”, an interactive modular storytelling card game in which the players are tasked with avenging the ghosts of victims from traditional murder ballads. Rewriting new story songs through recombining phrases from varying folk idioms, the players serve justice to the murderer and let these ghosts rest in peace at last.
From the ITP Camp Show description:
Come hither to go yonder in MURDER BALLAD: The Game!
In this multi-player storytelling card game, the high lonesome sounds of murder ballads still echo far and wide, through the hills and hollers of Appalachia, across the dusty western plain where the cactus blossoms bloom, downriver deep in the Mississippi Delta, the Rio Grande, and even far across the ocean through the British Isles...
But as long as guitars will ring and voices regale these grizzly deeds, the victims' ghosts of these woeful age-old songs cannot be laid to rest until their infamous killers' stories are retold and revenge served cold, either by the law or "hill justice" that catches up to them on judgement day. Through a recombinant phraseology of ballad and song lyrics from an array of historical folk traditions, players will steer our old-timey fugitives into new-timey fates, building new songs and casting the villains six feet beneath the clay with updated tombstones befitting their crimes.
Regenerative Songwriting: Toward a Lyrical Grammar of Folksong
Murder Ballad: The Game is an interactive generative songwriting experience in augmented reality that explores and re-evaluates a storytelling tradition within songs that recount tragic tales known lovingly as “murder ballads.” Typically, these songs feature female victims who reject the advances of a man to their own peril. Ranging from hundreds of years old and passed down within cultures all around the world, what is it about this narrative song style and content that persists? As purveyors of tradition, there is a prerogative to keep an age-old practice alive. However, as we consider how to preserve and pass on this cultural inheritance, it is time we take a closer look at what exactly we are trying to preserve and the possible dangers therein. There is violence in this music, and it’s time we start singing another side of these stories. In recent years, folk music and folksong have seen an enormous revival, and there have been a few artists to challenge this narrative paradigm by writing new songs where victims defend themselves and manage to escape an otherwise expected fatal outcome. This project aims to do just that, rewrite the narrative in which victim can become hero, justice can be restored, and villains can reap what they sow.
As a teacher, I like to encourage songwriters at all levels to explore folksong as a safe space to experiment finding expression. Songcraft within traditional idioms has a fairly modular phraseology, and it is very common that many of these songs share some of the same lyrics, sometimes in light variation or slightly different syntax if not verbatim, but all borrowing from the same lexical framework and poetic grammar. It is this quality that puts the “folk” in folk music, that total strangers from vastly different parts of the continent or globe can exchange and interchange the poetics of traditional vernacular lyrics and play together with no rehearsal. If you forget a line, you can borrow one from another song, and that is part of what makes this style of songwriting infinitely renewable and recombinant. As my songwriting practice began to adopt design-thinking and computational techniques, I developed decks of cards of stock lyrics and turns of phrase, and using principles of tarot card spreads, narrative songs began to write themselves very quickly. This caught on with my students as a playful creative iteration exercise, but the narrative affordances of the form and its content leaped from the cards as if they were trying to tell me something. And so, I listened— and the vision of this songwriting tool with the sorcery of technological application emerged from my desire to animate the inanimate and give voice to the voiceless. As this concept is partly drawn from tarot, I also wanted to build a ritual dimension to songwriting, as if the spirits of these songs’ victims beseech the player and enlist their help so they can find eternal peace at last.
Immersive Design, Tabletop Gaming, & the Future of the Album
The original mechanic of Murder Ballad was inspired by an award-winning game called Gloom by Keith Baker, a macabre collaborative storytelling card game where the objective is to put your poor unfortunate characters out of their misery through ridiculous and tragic twists of fate while your opponents attempt to thwart you by bringing them good fortune. Months after I developed my first prototype of Murder Ballad, I discovered that the Decemberists had enlisted the services of Keith Baker to collaborate on a game together called Illimat, as an asset toward their next LP release. The first time I played Illimat, my friend and I put on the new Decemberists’ album, setting the environmental tone and sonic backdrop to the world-building that Illimat effectively carried out, a highly aesthetic and tangible ritual experience that enhanced our listening of the music beyond our expectation. Just as no two play-throughs of a game will ever be identical, this dimension is similarly built into the listening experience in a fashion that rewards the players with each new listen.
Glimpses of Yonder: Augmented Reality & Machine Learning Implementation
Murder Ballad: The Game is an immersive mixed-reality generative songwriting experience that bridges centuries of traditional narrative songcraft and composition to the bleeding edge of technology through augmented reality and machine learning capabilities. Remaining true to the aesthetics, dynamics, and mechanics of cleromancy (divination through cards), the card deck element serves not only as ritual object, but most importantly as a tangible user interface that leverages the spatial configuration of the card layout as specific musical data. Once detected by a smartphone camera through the game’s companion app, this spatial information along with the images printed on the cards are taken as input toward the generative song, and then processed through simple machine learning algorithms into lyrical, melodic, and harmonic output that evolves throughout gameplay, modifying song templates that result in a genuine original song every play-through. Not only that, but the characters and images on cards are brought to life through the player’s phone screen! Each card, named for a genuine lyric or song from the corpus of traditional folk music repertoire, features its own animated sequences and musical output. With the tools included in ARKit SDK, this companion app also has the capability to reanimate these characters on the faces of the cards. Utilizing your front-facing camera for live facial motion capture blend-shapes, this AI maps the mesh of the user’s face movements and lip synchronization to those of the printed character! The spirit of this game is unbroken by intrusion of technology: this is not just an experience that showcases cool new tech features as a gimmick accessory, but rather, the player’s phone will mount onto an old-timey wooden frame with wheeled casters, resembling Victorian-era planchettes used in fortune telling practices like Ouija, enchanting your phone into a haunted lens and crystal ball in one. This songwriting experience is designed for solo, collaborative, or competitive multiplayer use. While no musical experience is required, songwriters and musicians of all levels can look to Murder Ballad: The Game for revelation.
Development Timeline
Dust on the Trail…
Here’s a look at some projects this year that advanced the vision
FACIAL MOTION CAPTURE w/ ARKit
Computer Vision and PoseNet: Spatial Composition, the Camera as Instrument
Lyrical Computation & Regenerative Song Code
Reference
“The Mystery of Murder Ballads and the Women who Flipped the Script”, Karen Hogg, https://sheshredsmag.com/the-history-of-murder-ballads/
"This Murder Done": Misogyny, Femicide, and Modernity in 19th-Century Appalachian Murder Ballads, Christina Ruth Hastie, University of Tennessee, Knoxville