Research Topics (Last update: November 2024)

Multi-agent reinforcement learning. Application to pedestrians simulation. Keywords: Reinforcement Learning, pedestrians simulation. Emergent collective behaviors. My  Doctoral Thesis applied Multi-Agent reinforcement learning to simulate pedestrians trajectories in 3D graphics environments. We have also developed a consultoring task about this subject with the company Toolchef in  UK. Multi-agent reinforcement learning applied to games.  Recently, we have assessing the use of  state-of-the-art Multi-agent reinforcement learning algorithms such as PPO, VDN and QMIX for learning behaviors in social networks-oriented role playing games. We have developed behaviors that surpass human skills in a multiplayer online roleplaying game settled on  Discord. Paper being reviewed. Multi-agent simulation framework based on evolutionary techniques for the analysis of the Electrical Vehicle (EV) market share. In colaboration with the University Carlos III of Madrid I have participated in a project founded by REPSOL S.A. company. The developed simulator uses Genetic Algorithms to optimize the configurations and placement of electric charging stations for EVs. Paper being reviewed. Hybrid techniques for behavior learning in 3D graphics characters. We are studying the application of state-of-the-art Quality Diversity (QD) techniques (Map-Elites) with deep learning and deep reinforcement learning to get diverse high quality behaviors in 3D graphics standard locomotion problems (HalfCheetah, Ant, Humanoid). Machine Learning techniques for the ADN analysis in biological remains to identify dog owners. Dog droppings on public roads is a sanitary problem in many cities. Using samples of these biological remains, we can obtain the ADN signature of the dog. I participate in a project for improving the detection accuracy of these ADN tests to identify  dog owners who did not comply with current legislation to pick up their dogs' excrement from public roads.  Deep learning to locate different flow pressure areas in the aorta inner walls in human heart. I participate in a study for predicting blood preassure in the inner walls of aorta based on  specific topological and geometrical information. We use deep convolutional networks to learn to predict flow and pressure in internal walls of aorta using images coding geometrical and topological characteristics.

Recent publications

A Reinforcement Learning Approach for Multiagent Navigation Francisco Martinez-Gil, Fernando Barber, Miguel Lozano, Francisco Grimaldo, Fernando Fernández. ICAART 2010 - Proceedings of the International Conference on Agents and Artificial Intelligence, Volume 1 - Artificial Intelligence, Valencia, Spain, January 22-24, 2010. INSTICC Press 2010, ISBN 978-989-674-021-4 Multi-Agent Reinforcement Learning for Simulating Pedestrian Navigation Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. Adaptive and Learning Agents Workshop at AAMAS (ALA'2011). LNAI 7113 (P. Vrancx, M. Knudson, M. Grzes Eds.) . Pags.54-69. Springer. 2012 Calibrating a motion model based on reinforcement learning for pedestrian simulation Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. ACM SIGGRAPH Conference on Motion in Games (MIG 2012) Rennes (France). LNCS 7660 (M. Kallmann, K. Bekris eds.) Pages 302-313. Springer. 2012. Emergent collective behaviors in a multi-agent reinforcement learning based pedestrian simulation Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. Extended abstracts booklet of the First Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM2013). Princeton University. New Jersey. 25-27 October 2013. Accepted as a full paper in the AAMAS 2014 Workshop: Multi-Agent-Based Simulation (MABS 2014) Paris, France. 2014 Strategies for simulating pedestrian navigation with multiple reinforcement learning agents Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. Autonomous Agents and Multi-Agent Systems. 2014. Springer. DOI: 10.1007/s10458-014-9252-6 (In Press) MARL-Ped: a Multi-Agent Reinforcement Learning Based Framework to Simulate Pedestrian Groups Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. Simulation Modelling Practice and Theory. 47: 259-275 (2014). Elsevier. Strategies for simulating pedestrian navigation with multiple reinforcement learning agents Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. Autonomous Agents and Multi-Agent Systems. 29 (1): 98-130 (2015). Springer. DOI: 10.1007/s10458-014-9252-6  Emergent behaviors and scalability for multi-agent reinforcement learning-based pedestrian models  Francisco Martinez-Gil, Miguel Lozano, Fernando Fernández. Simulation Modelling Practice and Theory 74: 117-133 (2017). Elsevier. Modeling, Evaluation and Scale on Artificial Pedestrians: A literature review  Francisco Martinez-Gil, Miguel Lozano, Ignacio García, Fernando Fernández. ACM Computing Surveys (CSUR) 50 (5): Article 72 (2017). ACM. Using Inverse Reinforcement Learning with Real Trajectories to Get More Trustworthy Pedestrian Simulations. F Martinez-Gil, M Lozano, I García-Fernández, P Romero, D Serra. Mathematics 8 (9), (2020)