ns3-gym: OpenAI Gym integration

The Playground for Reinforcement Learning in Networking Research
(26)

OpenAI Gym is a toolkit for reinforcement learning (RL) widely used in research. The network simulator ns-3 is the de-facto standard for academic and industry studies in the areas of networking protocols and communication technologies. ns3-gym is a framework that integrates both OpenAI Gym and ns-3 in order to encourage usage of RL in networking research.

Examples

A brief description of available examples can be found here

How to reference ns3-gym?

If you use this module in your research, please cite our paper:

P. Gawlowicz and A. Zubow, "ns3-gym: Extending OpenAI Gym for Networking Research", in CoRR, 2018. Available here.

Contact

  • Piotr Gawlowicz, TU-Berlin, gawlowicz@tkn
  • Anatolij Zubow, TU-Berlin, zubow@tkn
  • Mikolaj Chwalisz, TU-Berlin, chwalisz@tkn
  • tkn = tkn.tu-berlin.de

1.0.0

Works with ns-3.29

Release Notes

<p>The first release of ns3-gym app.</p>
Bakefile : Link


### Installation from the download button * Download the source archive and unpack it: ``` tar -xzf ns3-gym-1.0.0.tar.gz ``` * Move (and rename) the ns3-gym-1.0.0 directory to: ``` ns-3.29/contrib/opengym ``` * Install ZMQ and Protocol Buffers libs: Note: to install protobuf-3.6 on ubuntu 16.04, run: ``` sudo add-apt-repository ppa:maarten-fonville/protobuf && sudo apt-get update ``` Then: ``` apt-get install libzmq5 libzmq5-dev ``` ``` apt-get install libprotobuf-dev ``` ``` apt-get install protobuf-compiler ``` * Configure and build ns-3 project: Note: if you use Python virtual environment, please execute these commands inside it. Note: Opengym Protocol Buffer messages (C++ and Python) are build during configure. ``` ./waf configure --enable-examples ``` ``` ./waf build ``` * Install ns3gym Python module. Note: Python3 is recommended. Compile Protobuf messages manually (not required if `./waf configure` was executed): ``` cd ./ns-3.29/contrib/opengym/ ``` ``` protoc -I=./model/ --python_out=./model/ns3gym/ns3gym ./model/messages.proto ``` Install ns3gym Python module: ``` pip install ./ns-3.29/contrib/opengym/model/ns3gym ``` ### Installation from bake * Download the `ns3-gym.xml` configuration file, and place it in the contrib folder of bake (you may need to mkdir to create this contrib folder). * Configure bake such as: ``` ./bake.py configure -e ns-3.29 -e ns3-gym -m ``` * Run bake installation as usual: ``` ./bake.py check ``` ``` ./bake.py show ``` ``` ./bake.py download ``` ``` ./bake.py build ``` ### Test your installation - simple example * Terminal 1: start simulation script (i.e. environment): ``` ./waf --run "contrib/opengym/examples/opengym" ``` * Terminal 2: start an agent that will connect to simulation: ``` cd ./contrib/opengym/examples/opengym/ ``` ``` python ./test.py --start=0 ```

To report bugs, please open an issue in the [module GitHub repository](https://github.com/tkn-tub/ns3-gym/issues) or write an email to the maintainers.

Contributions to the module are welcome. Please open a Pull Request on the main repository against the [app branch](https://github.com/tkn-tub/ns3-gym/tree/app). You can also get in touch with the module maintainers, to improve the coordination of the development and avoid duplicated efforts.

This ns-3 extension is one or more contributed modules.

Version 1.0.0

Released Nov. 26, 2018

Works with ns-3.29

Bake bakeconf.xml