Take Lev Manovich’s The Language of New Media, in one hand, your iPhone in another, and try these exercises in transcoding.
Bloom. Creates paradigmatic montage, with an applied algorithm creating syntagmatic variations. Takes input to initiate the montage, but also sets up syntagmatic expectations in response. The music is looped, with variations creating a syntagmatic sequencing. But the loops are placed paradigmatically in layers.
Koi Pond: Montage, paradigmatic, to create a syntagmatic scene for narrative, if not a narrative itself. Composites image, motion, sound, and reaction: ripples can be created by touch, and come audio; fish will feed if given food or touch a finger held to the surface. Lilypads can be moved on the surface. Motion and sound will continue without interaction. The app doesn\’t engage narrative, but it encourages experiment with setting for narratives.
Shopper and other grocery list apps: Menu selection from a larger to create a subset, which is then accessed and modified in situ. Might be location based, in which case the app selects the subset. Might be mapped to the store, in which case the app draws on prior use and movement through the store to re-create the narrative movement. Items in these lists do not need to be real objects. The Shopper database can store elements the user sees fit to enter, using paradigmatic alternatives to Groceries. A user can create a fully motivated (if linear) narrative of shopping: Turning into the Tunnel of Love, you purchase a vinyl copy of McGough’s Summer with Monika. Crossing Shaftsbury Avenue, you buy a pickup for your steel guitar.
StarMap and others. A database of stellar objects, presented by selectable criteria: place; time of day, day, and year; direction. Data on each object is accessed in typical ways of touching, zooming in. Access points are mapped to cosmological traditions (constellations) and measures of astronomy (celestial equator, horizon).
Enigmo, Crayon Physics. The object in these is to create a device that achieves a simple goal (get the fluid in the container, the ball to the other side of the screen) by selecting and placing surrogate objects on the screen. The objects interact with the agent (water, ball) and with each other in a simulation of physical properties. In Tetris, we don’t ask how we’re able to rotate the pieces as they fall: physical limits are set aside. In these games the object is to make explicit how agents on the screen can be acted upon. Can create a sense of picaresque narrative – a narrative of trial and error – in that the machine may have to be constructed and torn down more than once to complete the puzzle.
CameraBag. Selection of filters from a time-centered menu. Most of the filters are constructed to degrade the digital image towards material-based techniques: make it look like it was taken with a Holga, a Polaroid. In other cases, the filter vignettes and adjusts the image towards a cinematic frame. This is material nostalgia: nostalgia for lost tools. Even the title – CameraBag – replaces the idea of selection from a menu with a selection of cameras.
More to come. What’s on your iPhone?