Autonomous agent with visual sensing for game environments

Abstract

This paper presents the design and validation of a DRL reinforcement learning pipeline that operates exclusively using logical visual information. Unlike conventional approaches based on convolutional neural networks (CNNs) trained on raw pixels, the proposed system prioritizes the use of explicit Feature Engineering based on computer vision (CV) techniques, with the aim of favoring both almost direct sim-to-real transfer and complete adaptability to any version of the game, both with only slight modifications to the feature engineering without losing the DRLtraining performed, thus reducing computational cost and improving interpretability. The 1978 video game Space Invaders is used as a controlled experimental environment and is deliberately treated as a black box, being observed onlythroughscreenshotsandwithoutaccess to internal source code states. Visual observations are processed in real time to construct a structured state representation that encodesthemainelementsofthegame. Thisrepresentation is encapsulated in a customized Gym environment that standardizes interaction with a Deep Q-Network (DQN)agent. Training is performed under computational constraints, prioritizing functional validation of the pipeline over optimal performance. Beyondtheperformanceachieved,themaincontributionofthisworkistodemonstratethatFeature Engineering (in this case based on computer vision CV), integrated in a modular way with reinforcement learning, constitutes an optimal and efficient alternative to variations for highly complex DRL models, leaving a more viable option to take advantage of simulation training in real environments (sim-to-real).