The Changing Nature of Gaming Interfaces

The other day I was having coffee with friends who brought their 2 1/2 year old son over for a visit. He was bored, looking for anything to do in our (boring) house -- so I handed him an original Game Boy with Super Mario Land 2. I figured that a toddler would enjoy smashing the Game Boy’s bulletproof buttons, making Mario run and jump, and hearing the ear-piercing four-channel music. He took the Game Boy from my hand with interest, and held onto it in the familiar way that all of us hold portables. He looked at the cabbage-green screen and squealed, “MARRIOO!” I asked his mother if he had played games before, and she said, “Oh yeah. He loves playing kiddie games on our iPhone.”

I turned back to her son, and he was frowning intently at the Game Boy. He reached out tentatively and pushed on the plastic screen. Nothing happened. He pushed again, in a different spot. Nothing. I reached over and pushed a button -- Mario jumped. He looked at me with a puzzled expression, and turned back to the game. I eventually had to slide his fingers over to the D-Pad and buttons, pushed them down a few times to show him how it worked, and he started to “get it”.

I realized in that moment that we are now living in a time when the standard D-PAD + Buttons layout can no longer be assumed the “standard” way of playing a game. A new generation of players are growing up with motion-based interfaces from Sony (the upcoming Playstation Move), Nintendo (Wii MotionPlus, Balance Board), Harmonix (Rock Band), as well as touch based devices from Apple (iPod Touch/iPhone). Where the 1980s and 1990s almost always guaranteed a familiar mediating interface -- whether it be a keyboard, mouse, or D-Pad -- I wonder at how the recent explosion of alternative interfaces has changed the way gamers understand what a game is?

For instance, can we really say that Myst or Monkey Island 2 SE for PC are the “same games” as their iPhone variants? On what basis could we distinguish between our experience of playing the two (temporarily setting aside differences in sound quality, resolution, etc)? Is the “touch” aspect really that different from a point-n-click interface using the mouse?

I’m going to waffle here, because I just don’t know. And here’s why:

When I play any game, using a standard NES/PS2/PS3/Xbox/GameCube controller layout -- my fingers and thumbs find their places. If it’s a NES, my right thumb handles the A+B buttons while my left thumb takes care of the D-Pad. There are no moments of confusion, I never have to ask myself, “which button is it again?”. The same goes for the PS2 and PS3 games: my fingers know their business. As soon as I settle down to play the game, my fingers are no longer fingers to me. They are a part of the game -- my fingers become something like my mouth when I am speaking -- they spring into action when Mario needs to bound over a Chain Chomp or needs to go down a green pipe. My fingers never become a part of my foreground or focal experience -- in other words, my fingers become repressed parts of my bodily experience.¹ If I had to think about what I was going to do next before committing myself to the act, Super Mario 3 would become unplayable.² In other words, games like Mario 3 require us to forget that we have fingers for a few moments in order to bring a natural flow into the game. Without getting too artsy or mixing metaphors, many games demand that the player become a pianist of a kind.

Mouse-based interfaces that we typically see in adventure games require a different kind of skilfulness. My hand has to learn to map the horizontal two-dimensional space of the mouse to an on-screen virtual space. I have to learn that forwards-is-up, and backwards-is-down, and that I have to move the cursor to the right position in order to make my character do something. In this kind of interface, I still “repress” my hand -- at some point my hand disappears and the cursor becomes invisible to me. The cursor moves simultaneously with my hand. My hand knows where it needs to go on-screen in order to make Guybrush Threepwood pick up a wooden mallet. I don’t think to myself: there is a mallet, and I need to click ‘pick up’, then click on the mallet. Exploring the world of Monkey Island 2 becomes a natural gesture for me.

(this review demonstrates how focal one’s finger can become when playing Myst on a touch device)

But can the same be said for touch-based devices that require us to make physical contact with the display in order to play the game? For instance, while the Myst interface is more or less the same between the PC/Mac and iPhone versions, the fact that I have to occlude some of the screen with my fingertip in order to “do” something changes the game subtly. Every time I reach forward and click on the screen with my finger I feel the cool glass push back at me, and I leave a fingerprint. There is something very focal in interacting with touch-based devices, because my finger does not fall into the background as easily. Compare that to the PC version: my hand is always on the mouse, my fingers always in their familiar positions on the mouse buttons. They never leave that surface, and the mouse becomes an extension of my body. On the iPod, my finger is constantly leaving the surface, popping in and out of my visual field.

But it’s the same game, right? Not for me. While the iPhone version of Myst is a wonderful port of the original game, I cannot quite dwell in the world simply because I cannot repress my awareness of my fingertips. I feel like I am playing a game. It is not quite bad enough to totally remove me from the world, but it is enough to remind me that yes -- I am playing a game on my iPod Touch and this is a virtual/fictional world that I am interacting with. The PC version of Myst is nothing like that -- when I click something I am reaching into the world and flipping a toggle switch.

Returning to my anecdote: does my friend’s 2 1/2 year old son experience his favourite iPod Touch game as a ‘real’ world? Or is his experience like mine — somewhat disembodied and self-conscious? Is this an inherent problem with touch-based interfaces, or do some of us already experience bodily repression that allows us to ignore our fingertips when we touch the display? How much have designers appreciated the qualitative change in gameplay experience as a result of the massive turn towards touch-based gaming, and have they done anything to respond to it? What are your experiences with touch-based (or even motion-based) interfaces; how do they change your experience of the game?