This repository contains the code for simulating embodied neural agents controlled by the Haken-Kelso-Bunz (HKB) equations. The code uses Pytorch to enable parallelized solving of differential equations.
- basicHKB.py Illustration of how to solve the HKB equations with PyTorch
- two_oscillators.py Animation of two interacting oscillators with HKB equations
- agent_RL.py Contains classes of agents
- environment.py Contains classes of environments in which the agents move
- simulations.py Contains functions to perform agent runs with agent and environment classes
- agent_evaluation.py Executes single run simulations of single agents and plots trajectories
- social_agent_evaluation.py Executes single run simulations of multiple agents and plots trajectories
- grid_search.py Executes and saves single_agent runs for many different parameter values
- grid_search_social.py Executes and saves single_agent runs for many different parameter values
- grid_search_evaluation.py Evaluates the saved runs of the single agent grid search
- grid_search_social_evaluation.py Evaluates the saved runs of the multi-agent grid search
- visualizations.py Helper functions for plotting simulated runs
- ternary_plot.py Example code for generating ternary plots
- utils.py Helper functions
Our agent design (named 'Guido' for 'guided oscillator' in agent_RL.py) consists of four oscillators, corresponding to the sensory system (1 and 2) and motor system (3 and 4). The grey boxes represent the two eyes (or sensors). The left eye feeds the change in stimulus intensity to oscillator 1; the right eye to oscillator 2. The agent's orientation in space changes according to the phase difference between the two motor oscillators. The agent travels at a uniform speed. Both the simulations with individual agents and the multi-agent simulations in the main paper are run with four-oscillator agents. The agent_RL.py also contains a 'SocialGuido' class with an additional 5th oscillator that represents a 'socially sensitive' oscillator that is directly sensitive to the phases of other agents. This class has not been used in the main paper.
The environment classes in environment.py and the agent classes in agent_RL.py are made with a similar structure as standard reinforcement learning approaches. The different layers of the agent classes in agent_RL.py are all differentiable using Autograd to ensure that they can possibly be trained using policy gradient methods in Pytorch.
Coucke, N., Heinrich, M. K., Cleeremans, A., Dorigo, M. & Dumas, G. (2023). Collective decision making with embodied neural agents (in prep).
Aguilera, M., Bedia, M. G., Santos, B. A., & Barandiaran, X. E. (2013). The situated HKB model: how sensorimotor spatial coupling can alter oscillatory brain dynamics. Frontiers in computational neuroscience, 7, 117. doi:10.3389/fncom.2013.00117
Frank, T. D., Daffertshofer, A., Peper, C. E., Beek, P. J., & Haken, H. (2000). Towards a comprehensive theory of brain activity:: Coupled oscillator systems under external forces. Physica D: Nonlinear Phenomena, 144(1-2), 62-86. doi:10.1016/S0167-2789(00)00071-3
Haken, H., Kelso, J. S., & Bunz, H. (1985). A theoretical model of phase transitions in human hand movements. Biological cybernetics, 51(5), 347-356. doi:10.1007/BF00336922
Ramsauer, H., Schäfl, B., Lehner, J., Seidl, P., Widrich, M., Adler, T., ... & Hochreiter, S. (2020). Hopfield networks is all you need. arXiv preprint arXiv:2008.02217. doi:10.48550/arXiv.2008.02217
Zhang, M., Beetle, C., Kelso, J. S., & Tognoli, E. (2019). Connecting empirical phenomena and theoretical models of biological coordination across scales. Journal of the Royal Society Interface, 16(157), 20190360. doi:10.1098/rsif.2019.0360