Welcome

A while back I was playing Rock Band, and decided to see if i could use a webcam to read the screen and call out the notes. I got it to work, and put a video of it up on YouTube.
http://www.youtube.com/watch?v=9jz_2tCt_qg&feature=channel_page
and
http://www.youtube.com/watch?v=GGFbVxJlue4&feature=channel_page

FIGURING OUT WHAT NOTES TO PLAY:

Using an OLD(circa 1999) webcam that I had in a drawer, and some custom C# code, i was able to analyze each picture, and watch for the notes that need to be played. It's not that complicated, it simply looks at a pre-defined area of the screen and watches it to get 'bright'. When it sees the certain area of the screen get bright, it plays a wav file of me saying the color. This was useful when I wanted to watch the screen and hear the note it thought it should play.


Since I put together the note-reader part of the system, I've seen a bunch of people put together similar systems. I haven't seen any use a webcam though. Many use direct video input, which gives you a perfect signal, but I don't have a video input card. I've also seen people attach light sensors to the screen, but I don't want to do anything like that. I'm using a D-Link DSB-C300 webcam that I got in the fall of 1999. It's Old. Focus is pretty much impossible, and the frame rate isn't all that great. It's good enough to get up to about the HARD level at 320x240 resolution. Once I complete the system, and get all the bugs worked out, I'll look into replacing it if it becomes the limiting factor.


PLAYING THE NOTES BACK TO THE GAME

The next step was to get the notes to play back into the game. The easiest way is probably to use a Parallel port to drive some relays. I found a few parallel port driven relay's on ebay, but who's got a computer with a parallel port anymore? I decided to go with an Arduino. It's something that's programmable, and has a LOT more power than a simple USB or parallel port driven relay board. Once this project is done, I can use it to do something else.
This Christmas season, I saw a good deal on an XBOX 360 GHIII controller, so I decided to resurrect this project. I also purchased an Arduino Duemilanove, the latest version of the Arduino board. So far, very cool.

The Arduino connects to your PC with a USB cable and appears as a regular serial COM Port. Using System.IO.Ports in C#, I'm able to send commands to the Arduino. The Arduino reads the serial commands and then 'strums' a note.

The second part of this of course, is actually sending a command to the game. I opened up the GHIII controller and soldered a wire to each of the inputs of the buttons. I'm and idiot and forgot to take pictures, but I may have to open it up again, so expect pictures later. For now, check out:
http://egyokeo.com/gh2mod/
.
I used a GHIII controller, which, for our purposes is the same. I didn't cut the wire though, I soldered into where the button board connects to the wire. It's small, and I've never really done much soldering before(another fun part of this project :) ), but I made it work.

It's a little strange how the wires work. There's 8 wires for 5 buttons. It seems like you could use 6 wires to get the work done, but that's not how it works. It looks like there is either .3V across the button, or .6, depending on the button. I've come from a Computer Science background, not an electrical engineering one so I don't really understand why it is done this way, but since I plan to just use a relay to connect the wires, it should be OK.
I'm currently waiting for a 8 channel relay board from ebay to show up at my door. Right now, I can read the screen and light up an LED for each button to play and one more LED for the Strum bar. Once I get the relay, I'll wire it up, and trip the relay instead of an LED. I am using relay's that are powerful enough to drive a 15amp 120V signal. They are TOTALLY overkill, but I want to be able to use them for something else in the future, like driving Christmas lights or something like that. Their response time is around 10ms max, which may become a problem on the faster notes. I've started looking into reed relay's, which are around 1ms, or using some sort of MOSFET or something else, that are around 1ns response time. I'll cross that bridge when I come to it.

Hopefully it'll all work, but I foresee a lot of debugging and getting the timing right. It should be interesting, if at least to boost up my XBOX Live scorecard :P.