Features
BenchMARL has several features:
A test CI with integration and training test routines that are run for all simulators and algorithms
Integration in the official TorchRL ecosystem for dedicated support
Logging
BenchMARL is compatible with the TorchRL loggers.
A list of logger names can be provided in the experiment config.
Example of available options are: wandb, csv, mflow, tensorboard or any other option available in TorchRL.
You can specify the loggers
in the yaml config files or in the script arguments like so:
python benchmarl/run.py algorithm=mappo task=vmas/balance "experiment.loggers=[wandb]"
The wandb logger is fully compatible with experiment restoring and will automatically resume the run of the loaded experiment.
Checkpointing
Experiments can be checkpointed every experiment.checkpoint_interval collected frames.
Experiments will use an output folder for logging and checkpointing which can be specified in experiment.save_folder.
If this is left unspecified,
the default will be the hydra output folder (if using hydra) or (otherwise) the current directory
where the script is launched.
The output folder will contain a folder for each experiment with the corresponding experiment name.
Their checkpoints will be stored in a "checkpoints" folder within the experiment folder.
python benchmarl/run.py task=vmas/balance algorithm=mappo experiment.max_n_iters=3 experiment.on_policy_collected_frames_per_batch=100 experiment.checkpoint_interval=100
Reloading
To load from a checkpoint, you can do it in multiple ways:
You can pass the absolute checkpoint file name to experiment.restore_file.
python benchmarl/run.py task=vmas/balance algorithm=mappo experiment.max_n_iters=6 experiment.on_policy_collected_frames_per_batch=100 experiment.restore_file="/hydra/experiment/folder/checkpoint/checkpoint_300.pt"
If you do not need to change the config, you can also just resume from the checkpoint file with:
python benchmarl/resume.py ../outputs/2024-09-09/20-39-31/mappo_balance_mlp__cd977b69_24_09_09-20_39_31/checkpoints/checkpoint_100.pt
In Python, this is equivalent to:
from benchmarl.hydra_config import reload_experiment_from_file
experiment = reload_experiment_from_file(checkpoint_file)
experiment.run()
Evaluating
To evaluate an experiment, you can:
from benchmarl.hydra_config import reload_experiment_from_file
experiment = reload_experiment_from_file(checkpoint_file)
experiment.evaluate()
This will run an iteration of evaluation, logging it to the experiment loggers (and to json if create_json==True.
There is a command line script which automates this:
python benchmarl/evaluate.py ../outputs/2024-09-09/20-39-31/mappo_balance_mlp__cd977b69_24_09_09-20_39_31/checkpoints/checkpoint_100.pt
Callbacks
Experiments optionally take a list of Callback which have several methods
that you can implement to see what’s going on during training, such
as: