Pac-Man, the classic 1980s arcade game developed by Namco where the believed yellow headed protagonist makes his way through a maze, chomping Pac-Pellets while being chased by four multi-colored ghosts: Blinky, Inky, Pinky, and Clyde. Later the game was given a spin-off where players are able to play as Pac-Man’s wife Ms. Pac-Man, through generally the same concept as the original. Since the games released, players have made history accumulating incredibly high scores of over 1 million, Billy Mitchell was the first person to achieve the maximum score of Pac-Man of 3,333,360 on July 3rd, 1999 a record which he still holds today along with 6 other players, As for the highest possible score on Ms.Pac-Man AI created by Microsoft’s Maluuba team in Canada has scored the highest possible score of 999,990 beating the original human score of 266,330 by a Brazilian player by four times!
Achieving the maximum score on Ms. Pac-Man is no small feat, as the score has eluded gamers due to the unpredictability of the game, the score has only been achieved by humans through cheat codes. The AIs were able to achieve the high score through a process Microsoft’s Maluuba team called Hybrid Reward Architecture in which they basically taught the AI to work together. 150 AI agents were taught to work in parallel on the game, one group of agents were rewarded for eating Pac-Pellets while other Agents were rewarded for avoiding ghosts, the top AI agent then collected the data from the others and calculated a weighted average to make decisions on what to do in the game.
It was the top AI agents job to decide what was the best course of action for the team as a whole as well as weighing the options given by the other AI in the team, if one AI want to turn left to avoid left to avoid a ghost and another AI wanted to go up to avoid the ghost the top AI agent would weigh the consequences and importance of each decision and decide which course of action would help them achieve their goal. Harm Van Seijen a researcher at Maluuba describes the cooperation between the AIs, “There’s this nice interplay, between how they have to, on the one hand, cooperate based on the preferences of all the agents, but at the same time each agent cares only about one particular problem. It benefits the whole.” Maluuba states the AIs have much more potential and could be used for predicting sales, stocks, and advancing technology and with further study could be a great help to businesses.
Featured Image via Wikimedia
