ns3-gym: OpenAI Gym integration

The Playground for Reinforcement Learning in Networking Research
4.11/5 (3375 reviews)
Categories: Machine Learning

OpenAI Gym is a toolkit for reinforcement learning (RL) widely used in research. The network simulator ns-3 is the de-facto standard for academic and industry studies in the areas of networking protocols and communication technologies. ns3-gym is a framework that integrates both OpenAI Gym and ns-3 in order to encourage usage of RL in networking research.

Examples

A brief description of available examples can be found here

How to reference ns3-gym?

If you use this module in your research, please cite our paper:

P. Gawlowicz and A. Zubow, "ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research", MSWiM, 2019. Available here.

Contact

  • Piotr Gawlowicz, TU-Berlin, gawlowicz@tkn
  • Anatolij Zubow, TU-Berlin, zubow@tkn
  • Mikolaj Chwalisz, TU-Berlin, chwalisz@tkn
  • tkn = tkn.tu-berlin.de

1.0.2

Works with ns-3.36

Latest Build :

Build History : ns3-gym: OpenAI Gym integration 1.0.2

Release Notes

<p>The third release of ns3-gym App.</p>
Bakefile : Link


1.0.1

Works with ns-3.35

Latest Build :

Build History : ns3-gym: OpenAI Gym integration 1.0.1

Release Notes

<p>The second release of the ns3-gym App. Updated to support the new version of the protobuff and zmq libs.</p>
Bakefile : Link


1.0.0

Works with ns-3.29

Build History : ns3-gym: OpenAI Gym integration 1.0.0

Release Notes

<p>The first release of ns3-gym app.</p>
Bakefile : Link


Dependencies

Compulsory

  • libzmq5
  • libzmq5-dev
  • libprotobuf-dev
  • protobuf-compiler
  • python3
  • pip3
  • ns3gym

ns-3.36+ (with cmake) ======== ### Installation from the download button * Download the source archive and unpack it: ``` tar -xzf ns3-gym-1.0.2.tar.gz ``` * Move (and rename) the ns3-gym-1.0.2 directory to: ``` ns-3.36/contrib/opengym ``` * Install ZMQ and Protocol Buffers libs: ``` sudo apt-get update ``` ``` apt-get install libzmq5 libzmq5-dev ``` ``` apt-get install libprotobuf-dev ``` ``` apt-get install protobuf-compiler ``` * Configure and build ns-3 project: Note: if you use Python virtual environment, please execute these commands inside it. Note: Opengym Protocol Buffer messages (C++ and Python) are build during configure. ``` ./ns3 configure --enable-examples ``` ``` ./ns3 build ``` * Install ns3gym Python module (Python3 required) ``` cd ./contrib/opengym/ ``` ``` pip3 install -U ./model/ns3gym ``` ### Installation from bake * Download the `ns3-gym.xml` configuration file, and place it in the contrib folder of bake (you may need to mkdir to create this contrib folder). * Configure bake such as: ``` ./bake.py configure -e ns-3.36 -e ns3-gym -m ``` * Run bake installation as usual: ``` ./bake.py check ``` ``` ./bake.py show ``` ``` ./bake.py download ``` ``` ./bake.py build ``` ### Test your installation - simple example * Terminal 1: start simulation script (i.e. environment): ``` ./ns3 run "opengym" ``` * Terminal 2: start an agent that will connect to simulation: ``` cd ./contrib/opengym/examples/opengym/ ``` ``` python ./test.py --start=0 ``` ns-3-35 (with waf) ======= ### Installation from the download button * Download the source archive and unpack it: ``` tar -xzf ns3-gym-1.0.1.tar.gz ``` * Move (and rename) the ns3-gym-1.0.1 directory to: ``` ns-3.35/contrib/opengym ``` * Install ZMQ and Protocol Buffers libs: ``` sudo apt-get update ``` ``` apt-get install libzmq5 libzmq5-dev ``` ``` apt-get install libprotobuf-dev ``` ``` apt-get install protobuf-compiler ``` * Configure and build ns-3 project: Note: if you use Python virtual environment, please execute these commands inside it. Note: Opengym Protocol Buffer messages (C++ and Python) are build during configure. ``` ./waf configure --enable-examples ``` ``` ./waf build ``` * Install ns3gym Python module. Note: Python3 is recommended. Compile Protobuf messages manually (not required if `./waf configure` was executed): ``` cd ./ns-3.35/contrib/opengym/ ``` ``` protoc -I=./model/ --python_out=./model/ns3gym/ns3gym ./model/messages.proto ``` Install ns3gym Python module: ``` pip install ./contrib/opengym/model/ns3gym ``` ### Installation from bake * Download the `ns3-gym.xml` configuration file, and place it in the contrib folder of bake (you may need to mkdir to create this contrib folder). * Configure bake such as: ``` ./bake.py configure -e ns-3.35 -e ns3-gym -m ``` * Run bake installation as usual: ``` ./bake.py check ``` ``` ./bake.py show ``` ``` ./bake.py download ``` ``` ./bake.py build ``` ### Test your installation - simple example * Terminal 1: start simulation script (i.e. environment): ``` ./waf --run "contrib/opengym/examples/opengym" ``` * Terminal 2: start an agent that will connect to simulation: ``` cd ./contrib/opengym/examples/opengym/ ``` ``` python ./test.py --start=0 ```

To report bugs, please open an issue in the [module GitHub repository](https://github.com/tkn-tub/ns3-gym/issues) or write an email to the maintainers.

Contributions to the module are welcome. Please open a Pull Request on the main repository against the [app branch](https://github.com/tkn-tub/ns3-gym/tree/app). You can also get in touch with the module maintainers, to improve the coordination of the development and avoid duplicated efforts.

This ns-3 extension is one or more contributed modules.

Version 1.0.2

Released July 6, 2022

Works with ns-3.36

Bake bakeconf.xml