DEFIANCE

A framework for simulating multi-agent reinforcement learning (rl) deployments in realistic scenarios
4.56/5 (9 reviews)
Categories: Machine Learning

We provide a framework for Reinforcement Learning (RL) research using ns3 and ns3-ai. You can find our design documentation and user documentation here.

For a practical example of how our framework is used, see this Medium article. The blog is divided into two parts and demonstrates building a balancing inverted pendulum in a network scenario using our framework.

This module supports the integration of RL components into network scenarios to simulate their deployment and the communication between them. This allows the simulation of distributed RL in networks. Both single-agent and multi-agent RL scenarios are supported.

v1.40.0

Works with ns-3.40

Latest Build :

Build History : DEFIANCE v1.40.0

Release Notes

<p>This is the first release of the defiance module for ns-3-40.</p> <p>Head to <a href="https://github.com/DEFIANCE-project/ns3-DEFIANCE?#setup-the-development-environment">https://github.com/DEFIANCE-project/ns3-DEFIANCE?#setup-the-development-environment</a> for detailed setup instructions. This module also depends on our <a href="https://github.com/DEFIANCE-project/ns3-ai">fork of ns3-ai</a>.</p> <p>If you have any questions or encounter any problems, feel free to reach out.</p>
Bakefile : Link


Dependencies

Compulsory

  • ns-3-dev
  • dce-meta-dev
  • dce-meta-dev
  • net-next-nuse-4.4.0
  • iproute2-4.4.0
  • dce-meta-dev
  • net-next-sim-2.6.36
  • iproute-2.6.38-fix-01
  • dce-quagga-dev
  • dce-umip-dev
  • dce-meta-dev
  • iproute-2.6.38-fix-01
  • libdb-dev
  • bison
  • flex
  • libssl-dev
  • libpcap-dev
  • libexpat-dev
  • glibc-2.31
  • gawk
  • libdb-dev
  • bison
  • flex
  • libssl-dev
  • libdb-dev
  • bison
  • flex
  • libssl-dev
  • libdb-dev
  • bison
  • flex
  • libssl-dev
  • bc
  • bc
  • dce-meta-dev
  • freebsd-sim-10.0.0
  • libsysfs-dev
  • dce-quagga-dev
  • dce-meta-dev
  • umip
  • dce-meta-dev
  • quagga
  • iputils
  • indent
  • gawk
  • libxml2-dev
  • mercurial
  • libxml2-dev
  • mercurial
  • libxml2-dev
  • mercurial
  • libxml2-dev
  • mercurial
  • clang-dev
  • llvm-dev
  • cmake
  • clang-dev
  • llvm-dev
  • cmake
  • clang-dev
  • llvm-dev
  • cmake
  • clang-dev
  • llvm-dev
  • cmake
  • clang-dev
  • llvm-dev
  • cmake
  • castxml
  • setuptools
  • cxxfilt
  • gccxml-ns3
  • setuptools
  • castxml-0.1+git98a626ec
  • setuptools
  • cxxfilt
  • castxml-0.2
  • setuptools
  • cxxfilt
  • castxml-0.3.6
  • setuptools
  • cxxfilt
  • castxml-0.4.3
  • setuptools
  • cxxfilt
  • python3-setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • setuptools
  • libc
  • libc-debug
  • mercurial
  • mercurial
  • qt
  • g++
  • qt4
  • g++
  • qt4
  • g++
  • qt
  • g++
  • qt
  • g++
  • ns-3-dev
  • automake
  • cmake
  • mercurial
  • mercurial
  • mercurial
  • mercurial
  • mercurial
  • mercurial
  • mercurial
  • cmake
  • cmake
  • cmake
  • cmake
  • cmake
  • ns-3.24
  • ns-3.25
  • ns-3.26
  • ns-3.27
  • ns-3.28
  • ns-3.28.1
  • ns-3.29
  • automake
  • ns-3.30
  • automake
  • ns-3.30.1
  • automake
  • ns-3.31
  • automake
  • ns-3.32
  • automake
  • ns-3.33
  • automake
  • ns-3.34
  • automake
  • ns-3.35
  • automake
  • ns-3.36
  • automake
  • ns-3.36.1
  • automake
  • ns-3.37
  • automake
  • ns-3.38
  • automake
  • ns-3.39
  • automake
  • ns-3.40
  • automake
  • ns-3.41
  • automake
  • ns-3.24
  • mercurial
  • dce-meta-1.7
  • mercurial
  • dce-meta-1.7
  • net-next-sim-2.6.36
  • iproute-2.6.38-fix-01
  • mercurial
  • dce-meta-1.7
  • freebsd-sim-10.0.0
  • mercurial
  • quagga
  • dce-meta-1.7
  • iputils
  • mercurial
  • dce-linux-1.7
  • dce-quagga-1.7
  • dce-meta-1.7
  • umip
  • mercurial
  • ns-3.25
  • mercurial
  • dce-meta-1.8
  • mercurial
  • dce-meta-1.8
  • net-next-sim-2.6.36
  • iproute-2.6.38-fix-01
  • mercurial
  • dce-meta-1.8
  • freebsd-sim-10.0.0
  • mercurial
  • quagga
  • dce-meta-1.8
  • iputils
  • dce-linux-1.8
  • dce-quagga-1.8
  • dce-meta-1.8
  • umip
  • mercurial
  • ns-3.26
  • mercurial
  • dce-meta-1.9
  • mercurial
  • dce-meta-1.9
  • net-next-sim-2.6.36
  • iproute-2.6.38-fix-01
  • mercurial
  • dce-meta-1.9
  • freebsd-sim-10.0.0
  • mercurial
  • quagga
  • dce-meta-1.9
  • iputils
  • mercurial
  • dce-linux-1.9
  • dce-quagga-1.9
  • dce-meta-1.9
  • umip
  • mercurial
  • ns-3.28
  • dce-meta-1.10
  • dce-meta-1.10
  • net-next-nuse-4.4.0
  • iproute2-4.4.0
  • dce-meta-1.10
  • freebsd-sim-10.0.0
  • dce-meta-1.10
  • quagga
  • iputils
  • dce-quagga-1.10
  • dce-meta-1.10
  • umip
  • dce-quagga-1.11
  • dce-meta-1.11
  • umip
  • ns-3.34
  • dce-meta-1.11
  • dce-meta-1.11
  • net-next-nuse-4.4.0
  • iproute2-4.4.0
  • dce-meta-1.11
  • quagga
  • iputils
  • ns-3.35
  • glibc-2.31
  • linux-dev
  • dce-meta-1.12
  • dce-meta-1.12
  • net-next-nuse-4.4.0-fix1
  • iproute2-4.4.0
  • dce-meta-1.12
  • quagga
  • iputils

Optional

  • elf-loader
  • libaspect
  • iperf
  • ccnx
  • wget
  • thttpd
  • bash
  • lksctp-dev
  • python-dev
  • pygraphviz
  • pygoocanvas
  • python3-dev
  • pygraphviz
  • pygobject
  • gi-cairo
  • gir-bindings
  • python3-dev
  • pygccxml-1.9.1
  • python-dev
  • python3-dev
  • python3-dev
  • python3-dev
  • python3-dev
  • python3-dev
  • python-dev
  • python-dev
  • pygccxml
  • python-dev
  • pygccxml
  • python-dev
  • pygccxml
  • python-dev
  • netanim
  • pyviz-gtk3-prerequisites
  • click-dev
  • openflow-dev
  • BRITE
  • pyviz-gtk3-prerequisites
  • cppyy
  • netanim-3.106
  • pybindgen-0.17.0.post41+ngd10fa60
  • pyviz-prerequisites
  • netanim-3.107
  • pybindgen-0.17.0.post49+ng0e4e3bc
  • pyviz-prerequisites
  • netanim-3.107
  • pybindgen-0.17.0.post57+nga6376f2
  • pyviz-prerequisites
  • netanim-3.108
  • pybindgen-ns3.27-castxml
  • pyviz-prerequisites
  • netanim-3.108
  • pybindgen-ns3.27-castxml
  • pyviz-prerequisites
  • netanim-3.108
  • pybindgen-ns3.27-castxml
  • pyviz-prerequisites
  • netanim-3.108
  • pybindgen-0.19.0.post4+ng823d8b2
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.20.0
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.20.0
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.21.0
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.21.0
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.21.0.post15+nga587377
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.22.0
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.22.0
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.22.1
  • pyviz-gtk3-prerequisites
  • netanim-3.108
  • pybindgen-0.22.1
  • pyviz-gtk3-prerequisites
  • pyviz-gtk3-prerequisites
  • cppyy
  • pyviz-gtk3-prerequisites
  • cppyy
  • pyviz-gtk3-prerequisites
  • cppyy
  • pyviz-gtk3-prerequisites
  • cppyy
  • pyviz-gtk3-prerequisites
  • cppyy
  • netanim-3.106
  • nsc-0.5.3
  • pybindgen-0.17.0.post41+ngd10fa60
  • pyviz-prerequisites
  • click-ns-3.20
  • openflow-ns-3.17
  • pygccxml-1.0.0
  • gccxml-ns3
  • BRITE
  • netanim-3.107
  • nsc-0.5.3
  • pybindgen-0.17.0.post49+ng0e4e3bc
  • pyviz-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • pygccxml-1.0.0
  • gccxml-ns3
  • BRITE
  • netanim-3.107
  • nsc-0.5.3
  • pybindgen-0.17.0.post57+nga6376f2
  • pyviz-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • pygccxml-1.0.0
  • gccxml-ns3
  • BRITE
  • netanim-3.108
  • nsc-0.5.3
  • pybindgen-ns3.27-castxml
  • pyviz-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • pygccxml-1.9.1
  • BRITE
  • netanim-3.108
  • nsc-0.5.3
  • pybindgen-ns3.27-castxml
  • pyviz-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • pygccxml-1.9.1
  • BRITE
  • netanim-3.108
  • nsc-0.5.3
  • pybindgen-ns3.27-castxml
  • pyviz-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • pygccxml-1.9.1
  • BRITE
  • netanim-3.108
  • castxml-0.1+git98a626ec
  • pygccxml-1.9.1
  • pyviz-gtk3-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • BRITE
  • netanim-3.108
  • castxml-0.2
  • pygccxml-1.9.1+git2b1efbb9+patch
  • pyviz-gtk3-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • BRITE
  • netanim-3.108
  • castxml-0.2
  • pygccxml-1.9.1+git2b1efbb9+patch
  • pyviz-gtk3-prerequisites
  • click-ns-3.25
  • openflow-ns-3.25
  • BRITE
  • netanim-3.108
  • castxml-0.2
  • pygccxml-1.9.1+git2b1efbb9+patch
  • pyviz-gtk3-prerequisites
  • click-ns-3.25
  • openflow-dev
  • BRITE
  • netanim-3.108
  • castxml-0.2
  • pygccxml-1.9.1+git2b1efbb9+patch
  • pyviz-gtk3-prerequisites
  • click-ns-3.25
  • openflow-dev
  • BRITE
  • netanim-3.108
  • castxml-0.3.6
  • pygccxml-2.0.1
  • pyviz-gtk3-prerequisites
  • click-ns-3.25
  • openflow-dev
  • BRITE
  • netanim-3.108
  • pygccxml-2.2.1
  • pyviz-gtk3-prerequisites
  • click-ns-3.34
  • openflow-dev
  • BRITE
  • netanim-3.108
  • pygccxml-2.2.1
  • pyviz-gtk3-prerequisites
  • click-ns-3.34
  • openflow-dev
  • BRITE
  • netanim-3.108
  • pygccxml-2.2.1
  • pyviz-gtk3-prerequisites
  • click-ns-3.34
  • openflow-dev
  • BRITE
  • netanim-3.108
  • pygccxml-2.2.1
  • pyviz-gtk3-prerequisites
  • click-ns-3.34
  • openflow-dev
  • BRITE
  • netanim-3.108
  • pyviz-gtk3-prerequisites
  • click-ns-3.37
  • openflow-dev
  • BRITE
  • netanim-3.109
  • pyviz-gtk3-prerequisites
  • click-ns-3.37
  • openflow-dev
  • BRITE
  • netanim-3.109
  • pyviz-gtk3-prerequisites
  • click-ns-3.37
  • openflow-dev
  • BRITE
  • netanim-3.109
  • pyviz-gtk3-prerequisites
  • click-ns-3.37
  • openflow-dev
  • BRITE
  • netanim-3.109
  • pyviz-gtk3-prerequisites
  • click-ns-3.37
  • openflow-dev
  • BRITE
  • elf-loader
  • iperf
  • ccnx
  • wget
  • thttpd
  • bash
  • elf-loader
  • iperf
  • ccnx
  • wget
  • thttpd
  • bash
  • elf-loader
  • iperf
  • ccnx
  • wget
  • thttpd
  • bash
  • elf-loader
  • libaspect
  • iperf
  • ccnx
  • wget
  • thttpd
  • bash
  • lksctp-dev
  • elf-loader
  • libaspect
  • ccnx
  • iperf
  • wget
  • thttpd
  • bash
  • lksctp-dev
  • libaspect
  • iperf
  • wget
  • thttpd
  • bash
  • lksctp-dev

## Setup the development environment ### Installation with bake See [bake-defiance](https://github.com/DEFIANCE-project/bake-defiance) for easy instructions. ### Docker If you just want a docker container with a built ns3-defiance, head to [bake-defiance](https://github.com/DEFIANCE-project/bake-defiance) and follow the steps to build the full image `defiance-full`. For development, we supply a `Dockerfile` here, in which you can build your custom changes to ns3-defiance. It builds upon `defiance-full`, so make sure you have built or downloaded it beforehand. Then, you can build a docker image containing your local changes with a simple `docker build .`. The ns3 root directory is at `$NS3_HOME`; the default working directory. By default, ns3 and ns3-defiance are built directly. To skip the build, add `--build-arg BUILD_NS3=False`. ### Installation without bake The manual way: Requirements: Depending on your use-case, different dependencies are needed. For a complete list of all possible development dependencies, refer to [our devcontainer Dockerfile](https://github.com/DEFIANCE-project/bake-defiance/blob/main/.devcontainer/Dockerfile#L9) 1. Clone ns3 `git clone https://gitlab.com/nsnam/ns-3-dev.git -b ns-3.40` 2. Some of our code needs that the environment variable `NS3_HOME` is set. Set it with `export NS3_HOME=$(pwd)/ns-3.40` 3. Clone ns3-ai and ns3-defiance into `ns3/contrib`: ```shell-c cd ns-3-dev/ git clone https://github.com/DEFIANCE-project/ns3-ai contrib/ai git clone https://github.com/DEFIANCE-project/ns3-defiance contrib/defiance ``` 4. Make sure, you have all dependencies. Running `./ns3 configure --enable-python --enable-examples --enable-tests` should succeed. 5. Then, compile ns3-ai to generate the message types with protobuf: `./ns3 build ai` 6. Install the python requirements with `poetry install -C contrib/defiance` 7. Manually install the ns3-ai message types: `pip install -e ./contrib/ai/python_utils -e ./contrib/ai/model/gym-interface/py` 8. Compile everything with `./ns3 build` 9. You are now able to start the training of our example scenario, such as `defiance-balance2` with `run-agent train -n defiance-balance2`. See `run-agent --help` for more info. ### Development tools This repo comes with additional developer tools, which may be installed with `poetry install --with dev`. - We format and lint our python code with `ruff`. Simply run `ruff check` to lint and `ruff format` to format. - We enforce type-checking on our code with `mypy`. Simply run it! - Optional jupyter notebook support can be installed with `poetry`. Simply run `poetry install --with ipynb`! In order to test ns3, it needs to be configured correctly. Refer to [bake-defiance](https://github.com/DEFIANCE-project/bake-defiance) for a complete command suggestion. The ns3 testsuites in the `test` directory can be run with `./test.py -s `, e.g. `./test.py -s defiance-agent-application` for the `defiance-agent-application` testsuite in `/test/agent-application-test.cc`. For further information refer to . The special ns3-ai tests need to be executed with `./contrib/defiance/tests/run-python-tests.sh`

To report bugs, please open an issue in the module GitHub repository.

Contributions to the module are welcome. To do so, please make a pull request against the main branch on the corresponding GitHub repositories.

This ns-3 extension is one or more contributed modules.

Version v1.40.0

Released Aug. 23, 2024

Works with ns-3.40

Bake bakeconf.xml