Christian Gingras- The Man Who Fixed Robotron
- “Firing in the corner” glitch- The original program included in the Rom set on ALL Robotron machines in the 80s had a defect in the code. A bug situation could happen where the player shoots an enforcer traveling on the side of the playfield causing the program to try reading from a bogus location in the code. Thus the code didn’t know what to do and the watchdog for the program would reset the game to the carpet pattern.
- per Eugene Jarvis- Around 1987 a kid in Canada named Christian Gingras completely analyzed the machine code in the ROMS and synthesized an entire Robotron source listing from the Roms. He then sent a list of 7 bugs he found in the game just by reading the code, and we fixed them. One of them was the infamous diagonal shot crash bug!
- per Sean Riddle- Williams Arcade Classics actually came out in 1995. I assumed they got the bug-fixed ROM file from Eugene. I beta-tested the program and emailed the author quite a bit back then. (http://www.vavasour.ca/jeff/general.html) He had made a few ROM changes to make emulation easier (and he redid Stargate as Defender II, which must have been a lot of work)
- Sean Riddle wrote: …I did a little googling, and it looks like people think the blue ROMs fix the shot-in-the-corner bug, but that’s not true. The patch came later- the text in the patch actually reads “1987 CHR.G”. I think I originally got the patch from the Digital Eclipse Williams Arcade Classics for Windows 95 CD
- NOTE- Although Christian is renowned for his work with Robotron, his true passion was Joust, whereupon he is in the world record books with a 98 million marathon in 1983! He also wrote a script for MAME to allow Joust to play on its’ own, without human interaction.
- June 2014 UPDATE: Scott Tunstall project to reverse engineer Robotron HERE (SeanRiddle site) “On Sean Riddle’s website there is an ongoing project to reverse engineer Robotron 2084. Thanks to Sean’s info on the hardware I have made great inroads into the way the game works. I aim to reverse engineer the full game, purely for educational purposes. I will try to produce an updated disassembly every week.”
- Christian is French-Canadian, raised in Canada and currently residing in California
- If you wish to contact him he can be found on Facebook
Christian’s story in his own words (Nov 2012)-
I was fairly advanced in Electronic around 1982 and could design analog and digital systems. It became clear with the new video games that were appearing in these years, Williams games for example, that something inside the 100 chips on the Joust/Robotron board allowed the system to perform more complex functions than what was possible with standard TTL logic gates.
Studying the schematic in detail, there was one obvious choice: the Motorola 6809 microprocessor was the “brain” and the 12 EPROM were storing whichever data needed to create the quasi intelligent behavior of the computer controlled birds in Joust.
I realized that I needed to understand what this mysterious 6809 chip really do if I wanted to design myself any complex electronics systems. If I would have been one lucky kid with access to large “mainframe”, I would have picked that as reference to study the microprocessors. In my world, the most powerful computer which I could have access and study were the video games in the arcade. It was clear that any of these games was orders of magnitude faster than any “home computer” such as the TRS-80 sold at radio shack or the TI99a sold in other stores.
I didn’t need too much time to determine which video game was the fastest. Robotron 2084, with about 120 objects moving at full speed in wave 10 was the clear winner. Contrast that with 10 birds, 3 pterodactyl and 2 players in Joust, Robotron is much more challenging on performance point of view. Then, if we downgrade to a computer using a Z80 processor such as Miss Pac-man, with 4 ghosts, one fruit and one player, the 6809 processor is obviously much faster.
It took me a few years to find a technician in an arcade who would remove the 12 EPROM of a Robotron board and allow me a few days to read these EPROM and than give back the EPROM. This was 1984 or 1985.
At that time, I already bought a “COCO2”, the radio shack “color computer” with the cartridge “edtasm”, a 6809 debugger written by Microsoft. I already reverse engineered “edtasm”, so I understood how a 6809 assembler and disassembler are created. However, I was assuming that Robotron:2084 is so much more complex that I didn’t dare study it yet. In 1985 and 1986, I reverse engineered the 8K ROM that create the basic language of the COCO2 and studied dozens of other applications such as cassette / diskette copy application, to save my work, some game to understand the method to access the graphic mode of the COCO2, etc.
Finally, around august 1986, I was ready to start the study of Robotron:2084. I made a custom version of “edtasm” which could read each EPROM, 4K of binary, and disassemble that code with a “virtual” address set to match the address generated in the actual hardware. Instead on printing to the screen, that disassembler was dumping the result to the printer port. It took more than 3 days to print the entire 512 pages. From time to time, I was adding a few drop of ink in the cartridge, starting with black, than blue, than red. To save space and money, I reversed the pile of paper after finishing the first 256 pages, after 6 EPROM were finished to disassemble.
I was pleasantly surprised to realize that I was able to follow that code in a similar way as the other 6809 applications that I studied in the previous years. There was, however, some visible differences:
– Robotron:2084 was using a lot of indexed instructions such as “LDA A,Y++, STB B,X+$0A” (read one byte at memory location referenced par the register X, increment that register, save that byte to the 10th bytes of the memory pointed by the register Y.
It turned out that Robotron:2084, despite being written in assembler, was a very clean object oriented code.
Robotron relied on a set of common shared functions to perform operations such as “malloc”, reserve an array of RAM, “sleep(x millisecond), where the processor was free to execute other operation and resumed the execution of the code that follow the call to “sleep” when the delay expired.
It turned out that Robotron was a fully mufti-threaded application where each moving object (enforcer, thank, etc) are controlled by a distinct thread of execution
The IRQ interrupt, which is fired 300 times per second, if my memory is right, is actually a fully pre-emptive thread, performing high priority task, such as drawing to a portions of the screen when the CRT is scanning far enough from that location to avoid creating a flickering. A state machine read the vertical position of the CRT scan, and select which object in the current list of objects to refresh can be safely redrawn. For example, if the player that we control with the joystick is near the top of the screen, the erase/redraw operation will be performed only when the CRT scan between the middle and the bottom of the screen.
It took me 8 months to study these 512 pages of code. Every night where I was working as security guard, between two building inspection time, I would unroll the stack of paper and draw lines with color ink to represent the branch instructions with arrow, or to find every places that call a specific function or read/write a specific port, etc.
After a few months, I got the answer to my main question: what makes Robotron:2084 so much faster than any other video game. The “blitter” (an X/Y DMA chip) was the main element to explain that performance. However, Defender appeared to draw on the screen about as fast as Robotron and didn’t have a blitter (this chip was designed for Robotron… then Stargate got the extra boost of the blitter).
I decided to continue the study of Robotron, while looking also at Joust and Stargate to compare the common features and difference. I was constantly impressed by the way the code solved some difficult algorithm. For example, how it uses the “MUL” instruction of the 6809 to perform a division, using “fixed point” binary mathematic.
When studying the code that managed the things, I could guess that this code created some problems in the past. There were more tests to double check possible problems, such as comparing the position of the bullet with the 4 walls to make sure that they stay confined inside the visible screen. On the total of 48K of RAM, the first 40K is used to display every pixel on the CRT, and the last 8K is the memory used by the code for the stack pointer, the “objects” (similar to the C++ notion objects, any small array of RAM used to memorize the position of each enforcers, thanks, bullets, etc and all other parameters associated with each type of objects), the local variables (on the stack, pointed by the register S) and the global variables (just a few, as any good object oriented application is mandated to do). If an object would, by mistake, by drawn too far on the right side of the screen, that would actually overwrite and corrupt the 8K of RAM needed by the program.
Later, at about the 4th month of the study, I reached the part of the code that managed the “enforcers”. I was curious to understand how the bullets could make the spiral motion, assuming a very efficient code without sin/cos, log or exponent. I wanted to understand how the code was making this strange “scared puppy” dance where an enforcer approach slowly the player on the left side, and then seems to be “scared” and fly to the right side of the screen, then slow down and make an elegant curve to come back toward the player. It turned out to be a mistake, a 8 bit binary overflow which was finally left in the code voluntarily. The screen is 320 pixel width, but an 8 bit register can only represent the first 256 pixels on the left. The enforcer would “perceive” the player as if it was located way far on the right side of the screen when it actually was very close on the left. So, the enforcer would compute a new X/Y speed trying to catch-up with this imaginary player. Then, in the middle of the mad race, the enforcer would perceive the player position correctly, on the left, so would decelerate smoothly then reverse direction, creating the gentle curve motion.
The study of this bug helped also to clarify another property of this multi-threaded/multi-tasked high performance software: each enforcer carrying it’s own private bloc of data, keep it own personality independently of every other enforcer. One of the 8 bit variable inside the enforcer data structure is a counter that specify how long (how many refresh cycle) the enforcer keep it’s current “mood” before changing it’s mind and do something different. When the enforcer erroneously flies in the direction of the right, it will keep chasing the “ghost” of the player for a fixed number of cycle. When that counter expire, decrementing to zero, the enforcer then “wake up”, re-evaluate where is the player, then start a new “mood”, which is to try to reach the player position in a smooth spiral motion.
After about 6 months of study, I visited every location of the code. I made a list of every call to functions (a cross reference describing who is calling who, how many time each function is called, etc). Similarly, I also listed every global variable that can be accessed with the register “DP” using efficient 8 bit pointer style of instructions. I noted every location which was reading or writing to each of these variables. I could see that some variables used only by objects shooting bullets (tanks, enforcers), some are used only by the mysterious code that perform “malloc” and similar operation (the operating system, the real-time multi-tasker), etc
I also found every encrypted text and every protection, these secret instructions that prevented any pirate to ever copy Robotron, Joust, etc If a pirate would dare changing “Williams” to another fake name, the game would not work. However, I needed to disable these protections if I wanted to modify the game to test if I really fixed the random reset bug.
I understood that the apparently useless “fancy mode, yes/no” was actually a way to prevent the random reset problem. When turned off, there was no drawing of explosion when shooting objects. It was more and more clear that one of these explosion drawing routines was the actual cause of the random reset.
In autumn 1986, I decided to write a letter to Eugene P. Jarvis and Larry E. DeMar, telling that I studied Robotron:2084 in detail and though I may have fixed the problem. I knew their name from a video game magazine. When I decrypted the secret text encoded in one part of the code, I saw these two names again, so I was convinced to have identified the real designer of the game.
While waiting for an answer, I got sick with pneumonia… I realized that I may not be able to fight this disease (I never had antibiotic before, didn’t know that the doctor was able to help. I waited to visit the doctor as I felt from all previous contacts that he didn’t care and was not there to help). I felt the urge to conclude that most important work in my life before it would be too late, fearing that all this work would be wasted if I am not there, nobody around me being aware of the work that I did.
I used all my energy to revise the code and write a detailed letter to Eugene and Larry. I documented every mistake that I could identify, explaining the behavior, the reason why such mistake may have been done and a solution to fix these. I documented the six bytes to change in order to carry my own tests, saying “only Eugene and Larry will know why”. I didn’t want them to think that I am a hacker paid by some illegal sponsor to defeat the protections. I discretely indicated that I understood absolutely every part of the code, and that I didn’t commit a stupid mistake such as triggering a protection while attempting to fix the random reset bug. The letter was 6 pages, with technical, condensed, precise description.
I concluded the letter saying “I did all this study for my love of the hardware (6809 based graphic display), it’s software (incredibly efficient multitask code) and it’s designers. That last part was because all these 48K bytes are equivalent to a novel, or any other very well written story which somebody took time to put on paper, with all the great care of a talented writer. The fact that such work have a life of it’s own, that thousands of video game players feel challenged and with all their strength and sweat, push and pull on these joystick to “beat” the machine, maybe it is no more spectacular than feeling emotion for some characters invented in a good novel story.
Using a 555 oscillator to generate IRQ interrupt (disconnected the 300 Hz from IRQ on the 6809 chip, connecting to the 555 instead) and adjusting the frequency to be so high that the CPU is swamped by the interrupt and allow little execution time to the main code, a few kilo-hertz, I could make the player draw the initial implosion for many minutes, while the game was starting at very low speed. I spent hours playing in slow motion, enjoying a much easier and less stressful game, even though much less challenging. I marveled at this code to never crash despite this crude violation of timing. I knew that I could crash the machine right away by moving my player to the upper left corner. I found that with fancy mode off, without the initial implosion, there was no was to ever crash this machine. Brief, this was one more clue that the bug was related to the drawing of implosion/explosion. The excellent players, such as the twin brothers Ian and Yvan Girouard were swearing that each time the machine crashed, an enforcer was standing in the upper left corner of the screen. These incredible players learned to never shoot an enforcer in diagonal, so that they could play for a few hours with a single credit.
A few weeks later, the phone rang early on a Saturday morning and somebody was speaking in English on the line. I told Michel-Guy Huot, my partner, wake up and speak to that man, this is the designer of Robotron calling me. Michel acted as translator, announcing that they want to see me in Chicago. They will pay for the train, the hotel, etc
The day where we where going to Chicago, I printed the routine that draw the diagonal explosion and started the patient work of drawing arrow on each branch instruction to illustrate the flow of the code. Then, while we almost arrived in Chicago, I noticed one more time the 6809 instruction to perform a jump thru an array of pointers. This code allowed to skip the drawing of so many lines when an object was too close to a wall and could not allow drawing the full shape of explosion/implosion. So, if the enforcer is close to the wall, less lines are drawn. What if the enforcer is so close that not even a single line would be allowed ? Looking one more time at the jump table, it is clear that the case for zero line have been forgotten as this location in the table contain 6809 op-code for the software instead of a valid address. Furthermore, there is no explicit test such as “BEQ…”, avoid the jump table if no line can be drawn. So, the processor would jump at an address that is determined by the 6809 op-code. This address turned out to make the processor fail, and the watchdog, detecting the “dead code” would finally apply a hardware reset on the CPU.
I was so happy. I would be able to tell Eugene and Larry: today, I found the real bug, the one that make the machine crash.
When discussing about the enforcer bug that make them behave like scared animals, approaching carefully, then running away, then coming back with some hesitation, I asked them: isn’t it an overflow, an error in the 320 horizontal pixels evaluated like if there was only 256 pixels ? Am I right to think that you found that bug but decided to leave it as is because of this exquisite complex motion it induce in the enforcers.
Larry smiled and confirmed my intuition. Then Larry paused and said: in the last 5 years, I never discussed the design of Robotron with anybody. None of my friends would be able to follow such technical issue. But today, with you, a total stranger who barely speaks our language, we discussed like if you had worked with us, like if you were one of Robotron original designer…
Later, when I asked how would they rate my understanding of Robotron. Larry said that it is quite good, but certainly not exceed the level that he and Eugene has. Eugene was saying, however, that I have a clear understanding of absolutely every part of the code, ranging from the boot test and other “operating system” type of operation as well as the game itself, so he was willing to imagine that I may understand the entire system more than either himself or Larry.
Later that day, Larry had to leave for work and me and Michel-Guy were staying with Eugene. He presented me the designer of Joust. Eugene was annoyed that I appear to prefer Joust to Robotron. I told the designer that I didn’t find a single mistake. I didn’t like a test made in the IRQ routine which could trigger a voluntary reset, but understood that this code would never be triggered, unless a hardware problem would corrupt the RAM. Mister Newcommer explained to me the way he could detect a collision between a bird and one of the floating platform. He evaluated a crude but efficient collision using an array built in RAM. When the wave changed, the array is modified to reflect adding or removing some platform.
Larry insisted to come back and drive us to the airport. When he aggressively honked at a city bus driver for trying to cut him, this felt like a life lesson for me
I could write as a follow up after meeting Eugene Jarvis my experiments, 2 years later. I bought a sample of the micro-processor that Williams selected to replace the 6809 (they were developing the game Narc with it). The TMS34010 was fascinating in many aspects. I created my own processor/display board and wrote all the code needed to initialize the graphic mode, download code thru parallel port, paint 40 millions pixel per second with circle, lines, etc. At that time, only expensive (about $2000 dollars) video card could match such speed.
<Christian chose to end the story here, pending further curiosity or questions from the audience. If you have questions/comments/feedback for Mr. Gingras feel free to email me at firstname.lastname@example.org and I’ll pass it along>
CLICK to read about Christian’s 98 million marathon on Joust
Click HERE for Part 2 of his story- Who wrote the patch?
Click HERE for Part 3 of his story- The Results of Learning Robotron Development
More Technical information about the Patched ROMS is available HERE