This paper describes the design and implementation of robotic agents for the RoboCup Simulation 2D category that learns using a re- cently proposed Heuristic Reinforcement Learning algorithm, the Heuris- tically Accelerated Q–Learning (HAQL). This algorithm allows the use of heuristics to speed up the well-known Reinforcement Learning algo- rithm Q–Learning. A heuristic function that influences the choice of the actions characterizes the HAQL algorithm. A set of empirical evalua- tions was conducted in the RoboCup 2D Simulator, and experimental results show that even very simple heuristics enhances significantly the performance of the agents. Keywords: Reinforcement Learning, Cognitive Robotics, RoboCup Sim- ulation 2D.