MELGYM is a Gymnasium-based interface designed to facilitate interactive control over MELCOR 1.8.6 simulations.

Every control functionality in MELCOR is determined by Control Functions (CFs). However, the batch execution mode of MELCOR makes it difficult to interactively control and modify functions under certain user-defined conditions. Control conditions are defined a priori and sometimes requires the concatenation of several CFs that must be done in an unfriendly way.

MELGYM allows the definition of external user-defined controllers, allowing the use of reinforcement learning agents or any other custom control algorithm.

⚙️ How it works?

MELGYM leverages MELCOR's restart capabilities to modify CFs dynamically during simulation cycles. Here's how it works:

1. On-restart control: MELGYM discretizes simulations by modifying the MELCOR input file just before warm starts are performed.
2. Dynamic control: the specified CFs are updated based on the latest simulation state, allowing real-time control.
3. Integration with RL: MELGYM can integrate reinforcement learning agents to optimize control strategies.

This approach enables interactive control over MELCOR simulations, overcoming the limitations of its batch execution mode.

Check the MELGYM documentation for more detailed information.

🖥️ Setting up experiments

MELGYM environments adhere to the Gymnasium interface, and can be combined with DRL libraries such as Stable-Baselines3.

import melgym
import gymnasium as gym

from stable_baselines3 import PPO

env = gym.make('pressure-v0')

# Training
agent = PPO('MlpPolicy', env)
agent.learn(total_timesteps=10_000)

# Evaluation
obs, _ = env.reset()
done = trunc = False

while not (done or trunc):
    env.render()
    act, _ = agent.predict(obs)
    obs, rew, done, trunc, info = env.step(act)      

env.close()

🚀 Contributing

We welcome contributions to MELGYM! Here's how you can contribute:

1. Fork this repository.
2. Create a new branch for your feature or bug fix:

   git checkout -b feature/new-feature

Remember to check our contributing guidelines.

🧰 Side projects

MELGYM rely on the auxiliar toolbox MELKIT. Feel free to help us improving both projects!