Atypical Instruments: Musings on Gaming, Musicianship, and Neuroscience

Guitar Hero and similar games started popping up in American video arcades and electronics stores in around 2005, following on the heels of earlier (and decidedly more aerobic) rhythm-based games like Dance Dance Revolution (which have actually been around, at least in Japan, for nearly a decade). I really only started noticing the proliferation of "guitar controller"-based games in early 2007, and honestly, I didn't really see what the appeal was. Wikipedia accurately describes the game dynamic as follows:

An extended guitar neck is shown vertically on the screen (the frets horizontal), and as the song progresses, colored markers indicating notes travel down the screen in time with the music; the note colors and positions match those of the five fret keys on the guitar controller. Once the note(s) reach the buttom, the player must play the indicated note(s) by holding down the correct fret button(s) and hitting the strumming bar in order to score points. Success or failure will cause the on-screen Rock Meter to change, showing how well the player is playing (denoted by red, yellow, and green sections). Should the Rock Meter drop below the red section, the song will automatically end, with the player booed off the stage by the audience.

So, basically, the player uses a guitar-shaped game controller to activate "notes" (which are represented by colored buttons and, as far as I can tell, don't actually correspond to real notes) as they glide toward her on the television screen. Hit the right note, and you get to keep playing (and the guitar track in the background continues). Hit the wrong note, or fail to hit the note on time, and you hear an unpleasant, tinny noise, followed by an interruption of the guitar track, followed by simulated "booing" noises as you are none-too-subtly "escorted" off stage. When I saw kids playing this in stores, I couldn't help but wonder, Why?

While My Guitar Gently...Beeps?

You see, some of my very earliest memories involve guitars -- specifically, my parents playing songs like the Beatles' Blackbird, or Cat Stevens' Wild World. We always had acoustic guitars in the house while I was growing up; I started getting curious about learning to play myself at around age fourteen, and received an electric guitar (along with a small amplifier) for my fifteenth birthday. I never took any formal lessons; my self-teaching technique consisted mainly of looking at and copying chord diagrams in various books my father had lying around, and listening to songs I liked and trying to match the notes.

Eventually I got to the point where I could identify and play most basic major and minor chords, and consequently pick out things that sounded at least vaguely like a number of rock, folk, and classical melodies. Due to a combination of changing interests and priorities, college, and time/space constraints I never really got very far beyond this point, but I'm still glad I went through the process of becoming at least a mediocre guitar player. There's an odd sort of thrill in being able to hear something pleasant and output something yourself that sounds at least vaguely similar, and playing familiar melodies can be incredibly relaxing once you know how to do it.

Additionally, learning to play an instrument (even if one never reaches virtuoso status) falls squarely into the category of "cool opportunities for personal brainhacking", at least in my book. I was definitely not someone to whom learning to play the guitar came easily, so the process of getting to the point of even mediocre musicianship was bizarre, difficult, mystifying, and incrementally rewarding all at once.

I remember just sitting for what felt like hours in the beginning with my fingers stuck in chord formations, pressing on the strings until it became unbearably painful to do so. I remember crying at school after a teacher told me (probably somewhat hyperbolically, but I was a very literal kid) that in order to ever become a good guitarist I'd need to practice "until my fingers bled". I remember being really annoyed at the fact that my little brother was so much better than me (he's definitely more of a natural musician than I am; I managed to partially make up through stubbornness what I lacked in talent, but he was always far quicker to learn new songs than me and a lot more competent of a player overall).

But perhaps most of all, I remember the incredible feeling of "Whoah, did I really do that?" that swept over me the first time I managed to output the first progression of Bouree in E Minor (thanks, in part, to some very patient help from the aforementioned musically-adept younger sibling).

One moment I was sitting there in a sea of fumbling discord with fingers that stubbornly refused to cooperate; the next moment, I was making the guitar do something that seemed to belie its nature as an inanimate lump of wood and metal. It was definitely one of those moments that made me very conscious of the fact that the brain and body comprise a dynamically interactive system, and that the "mind" is not some ethereal entity that hovers somewhere up above a person's shoulders. Plus, it was just plain neat!

So, I guess you could say guitars hold a special place in my mental storehouse of Significant Things. Video games do as well, actually, since they're (a) fun, (b) something I grew up with (being a card-carrying member of the Nintendo Generation and all), and (c) prone to prompting plenty of musings on interesting topics like learning, memory, and cognition.

But what to think of a guitar-inspired video game? Initially, I wasn't sure. I was mighty skeptical when my boyfriend Matt picked up the Guitar Hero III game for the Nintendo Wii about a month or so ago, but after several nights of glancing over my shoulder at the screen while he played, I found myself really wanting to give it a try.

So I did.

And of course I was awful at first -- my little purple-pigtailed avatar got "booed" off the virtual stage within 10 seconds or so since I basically failed to hit a single correct note. But after a few practice runs, I started hitting notes correctly at least occasionally, and eventually I managed to get through an entire song without getting booed out of the game (the first time that happened, I think my accuracy was something like 56%). Within a few days of spending 15 - 60 minutes playing in the evenings, I ramped up to the point where I could score 95% or above in "Easy" mode fairly consistently.

I've since tried "Medium" a few times and managed to get through a song or two, but mainly, I've been doing less playing lately and more thinking about the process of learning to play Guitar Hero and other games (and, for that matter, real musical instruments). What follows are some observations prompted by this thinking.

The Neuroscience of Gaming and Musicianship

Guitar Hero is, hands-down, far easier to learn than an actual guitar is -- it took me months to get to the point of being able to do anything decent with a real guitar, but only a few hours of GH to get to the point of being able to follow a song in the game from start to finish.

Nevertheless, GH is still more difficult in some very specific ways than the kinds of games I usually tend to favor and do well at. The first few times I played through a song I was quite literally exhausted afterward -- it was as if someone had attached a hose to my head and sucked the neurochemical energy out of my brain, leaving me with nothing more than a solid lump of inert clay in my skull. My eyes hurt from not blinking, my fingers were stiff and shaky, and I seriously considered never touching the game again after feeling its oddly pervasive somatic wrath.

However, the experience of eventually, occasionally coming to hit the right buttons at the right time was powerful enough to urge me to continue practicing. The more I practiced, the longer the time and distance between each note and the next started to seem -- as if my brain was somehow speeding up. That's a really interesting feeling. I really have to wonder how this kind of learning works -- I figure that as I practice more, certain neural connections form to the point where actions can occur without my having to consciously think about them, but it would be very interesting to know how exactly that happens.

Along those lines, The Frontal Cortex recently ran an article entitled Guitar Hero 2 and Plasticity, which featured a video clip of an eight-year-old zipping nonchalantly through one of GH2's songs in "Expert" mode. The article quotes a study of the somatosensory cortex in string players:

Magnetic source imaging revealed that the cortical representation of the digits of the left hand of string players was larger than that in controls. The effect was smallest for the left thumb, and no such differences were observed for the representations of the right hand digits. The amount of cortical reorganization in the representation of the fingering digits was correlated with the age at which the person had begun to play. These results suggest that the representation of different parts of the body in the primary somatosensory cortex of humans depends on use and changes to conform to the current needs and experiences of the individual.

Frontal Cortex suggests that it might be interesting to look at the young Guitar Hero expert's somatosensory cortex -- however, I'd extend this interest to the somatosensory cortices of people who play all sorts of games. The comment in the article lamenting the 8-year-old's time spent playing a game (when he presumably could have been learning to play a "real" guitar) doesn't make a whole lot of sense to me.


Firstly, there's no way to tell from the video whether the kid does or does not already play an actual instrument. He might very well be a young violinist or pianist or classical guitarist. Secondly, I don't see too many people suggesting that kids should be learning instruments instead of playing non-GH games -- sure, plenty of parents would rather their offspring be doing homework or folding laundry or picking up after the dog than staring at Super Mario Bros. for entire weekends, but it seems a bit odd to even try and classify instrument-inspired games as "substitutes for playing real music".

Guitar Hero is no more a "substitute" for real guitar-playing than Street Fighter is a substitute for learning martial arts. While video games may help teach or enhance particular skills, and while certain games and game elements might, in fact, actually translate to isometrically-similar offscreen abilities, I would wager that gaming probably doesn't tend to correlate with the meatspace analogues of what it represents in polygon-land. I imagine that my ability to pick up Guitar Hero despite a very rough start has more to do with my experience playing other video games than with my experience playing the guitar -- you just can't compare the two on the basis that the GH controller looks vaguely guitar-like.

As far as the mapping of the somatosensory cortex, it does seem possible that both instrument-playing and electronic gaming might increase the size of digit representation -- however, given the drastically different skill sets needed to make music with a guitar versus to play Guitar Hero, I'd guess that most people's brains probably don't provide much in the way of instantaneous skill crossover in this regard (not that anyone was suggesting this exactly, I'm just conjecturing here).

For one thing, when you're playing the guitar "for real", you are responsible for producing all the sounds you want to hear (assuming you're playing without a band or accompaniment). As a commenter to the aforementioned Frontal Cortex article noted, when you play a real instrument, you are performing more of a creative than a reactive process (and generally speaking, video games tend to demand reaction moreso than creation on the part of the player).

For another thing, sometimes the skills video games actually grant to a person are very individual (and often unexpected). Back in 2005 I spent a fair bit of time engrosed in the (very visually prettty) online roleplaying game, Guild Wars. Much to my surprise, I found at one point that some of the cognitive tasks in Guild Wars were actually helping me learn extremely useful, transferrable skills that benefitted me both at work and in the process of going about my daily life.

Guild Wars basically forces you to "load" certain skill modules prior to stepping out into the wilderness to fight or quest -- you have a limited number of these module slots, and until you come upon another safe town area, you can only use the modules you loaded last time you were in a town location. As I got better and better at "thinking ahead" in terms of what skills might serve me best for particular quests (and loading those skills accordingly), I found myself starting to realize that I could do the same thing in 'real life', to great benefit.

One thing I've always struggled with is context-switching -- I have a lot of stored knowledge (of facts, of how to do things, etc.), but I can't often access the exact piece of required knowledge on command -- I need time to bring that knowledge "on-line", and I also sometimes need particular triggers to bring it out. Prior to playing Guild Wars it hadn't even occurred to me that I could essentially try and look ahead to what situations I'd be likely to face at work or while running errands and consequently "pre-load" the modules that would give me the best chance of dealing well with the anticipated situations. So while pre-loading doesn't work as smoothly or reliably in meatspace as it does in the context of Guild Wars, it most certainly has given me another cognitive tool to work with. And the discovery of that unanticipated benefit has had me on the lookout ever since for similar opportunities.

As far as Guitar Hero goes in this regard, I'm not sure whether (or when) it will proffer any external, unexpected "enhancements" for me personally. I'm hoping that it at least helps my 29-year-old brain exercise its plasticity circuits to some extent. Mainly, I'm interested in what this game might help me learn about how I process information and execute certain tasks, because that in itself can be both useful and intriguing.

Notes on Speech and Multitasking

This might be an autistic thing (at least partially), but so far it seems that no matter how good/accurate I get at playing Guitar Hero, I can't play the game and simultaneously speak. Moreover, if I even think about forming speech, I almost immediately start missing notes. Even if playing seems "effortless", and even though I can perform other seemingly complex cognitive tasks while playing (like visualizing going into the kitchen and making tea), talking or planning to talk will kill my score every time.

This is really interesting because I've known for a while that spontaneous speech formation is incredibly demanding for me -- to a degree that it doesn't seem to be for others. For a long time I was confused as to why everyone didn't communicate via e-mail and instant-message practically all the time, given how much "easier" writing was than talking. Eventually, though, I started noticing that some people actually tended to complain about how slow and annoying the process of writing was. This came as a bit of a shock!

But once I got past that initial shock, I came to understand that despite sometimes getting "stuck" in an apparently chatty, monologue-laden mode, I actually do have a fair bit of difficulty with speech-based communication. It was a revelation to learn this, and one that I've been able to make plenty of use of since -- e.g., I've learned that pre-loading "speech modules" is essential to effective communication at work when I'm expected to talk, and that sometimes, I need to make use of non-speech-based communication options to the greatest extent possible in order to free up "CPU space" for other complex cognitive tasks. I'd be very curious to know whether other people (autistic or not) have attempted to play games like Guitar Hero either while speaking or while planning out speech in their minds.

The Future?

In thinking about what games like Guitar Hero, video games in general, and modern trends in instrument-playing and instruction might imply for the future, several possibilities come to mind. The Guitar Hero interface might very well be adaptable to a kind of "guitar tutorial" that would actually help teach people to read music (or at least tablature) and play a real guitar. While I stand by my earlier assertion that video games don't have to teach someone how to play an instrument or perform some other physical or mental skill outside the gaming environment, video games have most certainly proven themselves as far as promoting motivated behavior in humans.

It would almost seem silly for those interested in helping others learn to play music, etc., not to consider the vast possibilities in this area offered by electronic entertainment. Even though as noted before, video games tend to require "reactive" responses from those who play them, there's no reason that this has to be the case -- e.g., a fully-functional GH-style instrument tutorial game would basically provide players with feedback very similar to that they would receive from a real instrument -- which is to say that if the player didn't do the right thing, they wouldn't get to hear any music at all. The game interface would also help people keep track of their progress automatically (appealing to the "stats geek" in many of us).

I've also been wondering lately what we "first generation gamers" are going to be like, neurologically speaking, once we're elderly. While video games have been around for as long as I can remember (my dad had an Atari 2600 back in around 1980, and I recall trying to sneak out of bed as a two-year-old so I could watch my babysitters play "Space Invaders"), it is interesting indeed to acnowledge that there aren't presently any living humans old enough to be the subjects of a study of the effects of early video-game-playing in the elderly.

Are those of us who spent our Saturday mornings engaged with what amounted to interactive cartoons going to display different brain-aging patterns than those who only had access to more passive forms of screen-based entertainment?

If we keep playing games, will we be able to maintain neural plasticity longer, or perhaps even ward off certain forms of dementia?

Does gaming constitute a feedback mechanism powerful enough to be classed as a "neural modification"?

The questions are endless, and while they might not all ever be answered, I have to admit that I'm going to be eagerly watching for research news in this area. Considering the deep ties gaming has to brain states and the opportunity it provides for monitoring one's own skill development, I anticipate that people will eventually see it as anything but frivolous!