Building an AI-based Automated Bot for Pokemon Revolution Online

It’s christmas, and like every year, I like to play some online games. But if you know me, you know that more than playing, I like to build bots that play for me. This year, I decided to build an automation bot for the online game Pokémon Revolution Online. It’s a free-to-play, fan-made MMORPG based on the Pokémon series. And like any MMORPG, it involves a lot of farming, which can be a bit redundant, that’s why a bot can definitely help. Nevertheless, a strong anti-cheat software blocks usual automation strategies like packet sniffing and injection, requiring a different approach using an external CV-based bot.

Here’s how I built it.

def main():
    bot = MeyzhBOT()

Overview

The core logic of the bot resides in handling Pokémon encounters and determining the action to take based on user-defined settings. The bot has the following core functionalities:

1. Walking in the Wild: Automatically navigates through the game world.
2. Detecting game state: Uses a trained Computer Vision model to identify what are the in-game pop-ups.
3. Detecting Encounters: Uses another model to identify which Pokémon we encounter during battles.
4. Action Decisions: Based on the detected Pokémon, it decides whether to kill, capture, or run.

Computer Vision Models

Sauron Model

The bot leverages a computer vision model for detecting battles, pop-ups and other in-game windows.

To know what to perform (on which button to click), and what is the current status in the game (am I walking in the wild or am I in combat?) the bot needs to understand the game’s state from the screen image. To do so, I trained a Computer Vision model called Sauron. I named it this way because it “sees” the game world and detects the presence of Pokémon at any time, always performing inference on screenshots. It’s a classification model that detects the possible game windows like ‘YES’ , ‘NO’ choice buttons, the “Fight” window that displays when a battle starts, the “Evolve” window when a Pokémon is ready to evolve etc.

I had to take a lot of screenshots of the game and label them manually. I used Roboflow, an awesome tool, to label the images, augment the dataset and split them into test/train/validation. The dataset is available here.

The Sauron model is built on YOLOv5. I trained this model using Google Colab on NVIDIA Tesla T4 GPU. What’s awesome is that you can pull your dataset from Roboflow directly into Google Colab with only four lines of code:

from roboflow import Roboflow
rf = Roboflow(api_key="YOUR_API_KEY")
project = rf.workspace("ryzen-gaming-m4c6x").project("pokemonrevoonline")
dataset = project.version(5).download("yolov5")

The model was trained during 208 epochs with a batch size of 16. The training took around 38 minutes to complete and reached a mAP50 of at least 0.95 on all the classes.

Once trained, I defined a Sauron class that loads the model and performs inference on the screen image to detect the game windows, screenshots are taken using pyautogui. Here’s a snippet of the class:

class Sauron:
    def __init__(self, TKWindow):
        self.model = hub.load(  'ultralytics/yolov5',
                                'custom',
                                path='sauron_model/best.pt', force_reload=True) 

        self.TKWindow = TKWindow
        self.inference = self.model(pyautogui.screenshot())
        self.xyxy  = self.inference.pandas().xyxy[0] 
        self.xywh  = self.inference.pandas().xywh[0] 
        self.xywhn = self.inference.pandas().xywhn[0] 

    def inference_func(self, _):
        self.inference = self.model(pyautogui.screenshot())
        self.xyxy  = self.inference.pandas().xyxy[0] 
        self.xywh  = self.inference.pandas().xywh[0] 
        self.xywhn = self.inference.pandas().xywhn[0] 

    def main(self):
        while self.TKWindow.window_variables.is_active:                
            while self.TKWindow.window_variables.is_paused and self.TKWindow.window_variables.is_active:
                sleep(1)                
            # Continuously perform inference every .5 seconds
            self.inference_func(self.model)

Pokémon Classifier

A secondary model is used for identifying the specific Pokémon encountered during battles. This model is based on the ViT (Vision Transformer) architecture. I did not train this model myself, but rather used a pre-trained model available on the HuggingFace model hub. 🤗

Setting Up the Bot

User Interface

The user interface is built using tkinter, which provides a simple and intuitive way for users to interact with the bot. It allows users to:

• Add Pokémon to a list for specific actions (kill, capture, run).
• Set global actions for all other Pokémon.
• Start, stop, and pause the bot.

Here’s a brief look at the main interface components:

Running the Bot

The bot is wrapped in the MeyzhBOT class, which initializes the two CV models, and takes the TKinter window as argument. The sauron’s main method is attached to a different thread to run in parallel with the main bot loop. The bot continuously captures the screen, performs inference using the Sauron model, and makes decisions based on the detected game state.

class MeyzhBOT:
    def __init__(self, TKWindow):
        self.device = "cuda" if cuda.is_available() else "cpu"
        self.model_pokemon_classifier = ViTForImageClassification.from_pretrained("imjeffhi/pokemon_classifier").to(self.device)
        self.feature_extractor = ViTFeatureExtractor.from_pretrained('imjeffhi/pokemon_classifier')
        self.TKWindow = TKWindow
        self.sauron = Sauron(TKWindow=self.TKWindow)
        t = Thread(target = self.sauron.main)
        t.start()
        self.screen_shape = pyautogui.size()

While the Sauron model continuously performs inference on the screen, the MeyzhBOT checks for the classes detected using asynchronous methods:

async def am_i_in_combat(self):
    """
    returns True if in combat
    """
    return 'combatbox' in self.sauron.xywhn['name'].values

async def is_there_an_evo(self):
    """
    returns True if there is an evolution
    """
    xyxy, xywh, xywhn = await self.process_img(self.global_screenshot())
    return 'evolve' in xywhn['name'].values

And based on these game states, the main script takes actions, according to the user-defined settings:

while not(TKWindow.window_variables.is_paused) and \
    await bot.am_i_in_combat():

    routine_counts = 0
    mine, other = await bot.poke_in_combat()
    
    if other is not None:
        other = bot.find_poke_in_img(other).lower()

        if other in RUN_POKE:
            await run_away(TKWindow)
            await asyncio.sleep(2)
        
        elif other in CATCH_POKE:
            while await bot.am_i_in_combat(): 
                await capture(TKWindow)
                await asyncio.sleep(6)

        elif other in KILL_POKE:
            k+=1
            while await bot.am_i_in_combat(): await attack(TKWindow)

        else:
            if TKWindow.SCRIPT_PARAMS['others'] == 'kill':
                k+=1
            while await bot.am_i_in_combat(): await others_func(TKWindow)
            
        await asyncio.sleep(3)

You can find the whole code including the sauron model’s weights on the github repo.

Pierre.

Resources

• The repo
• Roboflow
• Google Colab
• Pokémon Revolution Online