Skip to content

Latest commit

 

History

History
252 lines (190 loc) · 7.88 KB

1.1-Shadow.md

File metadata and controls

252 lines (190 loc) · 7.88 KB

This page describes how to setup and install Shadow from a bare-bones Linux installation and is the recommended setup and installation method.

Note: It is recommended to install on Ubuntu 18.04 as it is currently the only platform tested in our CI environment.

Dependencies

Requirements

If you are installing Shadow within a Docker container, you must increase the size of the container's /dev/shm mount and make it executable by passing --tmpfs /dev/shm:rw,nosuid,nodev,exec,size=1g (with a suitable size for your system and experiments) to docker run.

Required:

  • gcc, gcc-c++
  • python 3, pyelftools (python module)
  • glib (version >= 2.32.0)
  • igraph (version >= 0.5.4)
  • cmake (version >= 2.8.8)
  • make
  • xz-utils
  • glibc debuginfo

Recommended Python Modules (for helper/analysis scripts):

  • numpy, scipy, matplotlib, networkx, lxml

Recommended System Tools:

  • git, dstat, screen, htop

Installation

Most of the libraries/tools used by Shadow are available through the system package managers of the distributions we support. An exception is the Python 3 libraries, which will need to be installed by alternative means (for example with pip or conda) if they are not available in your distribution.

YUM (Fedora/CentOS):

In more recent versions of Fedora (and maybe at some point CentOS), yum can be exchanged for dnf in these commands. Before running these commands, please check any platform-specific requirements below.

Warning: YUM and DNF often install 32-bit (i686) versions of libraries. You may want to use the --best option to make sure you're installing the 64-bit (x86_64) versions, which are required by Shadow.

sudo yum install -y \
    cmake \
    gcc \
    gcc-c++ \
    glib2 \
    glib2-devel \
    igraph \
    igraph-devel \
    make \
    python3 \
    python3-pyelftools \
    xz \
    xz-devel \
    yum-utils
sudo yum debuginfo-install -y glibc
sudo yum install -y \
    python3-numpy \
    python3-lxml \
    python3-matplotlib \
    python3-networkx \
    python3-scipy
sudo yum install -y \
    dstat \
    git \
    htop \
    screen
CentOS 7

You must enable the EPEL repository using:

yum install -y https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
CentOS 8

As procps-ng-devel, igraph, and igraph-devel are not available on CentOS 8, you must install them manually.

dnf remove -y procps-ng procps-ng-devel
dnf install -y http://vault.centos.org/centos/7.7.1908/os/x86_64/Packages/procps-ng-3.3.10-26.el7.x86_64.rpm
dnf install -y http://vault.centos.org/centos/7.7.1908/os/x86_64/Packages/procps-ng-devel-3.3.10-26.el7.x86_64.rpm
dnf install -y https://dl.fedoraproject.org/pub/archive/epel/7.7/x86_64/Packages/i/igraph-0.7.1-12.el7.x86_64.rpm
dnf install -y https://dl.fedoraproject.org/pub/archive/epel/7.7/x86_64/Packages/i/igraph-devel-0.7.1-12.el7.x86_64.rpm

APT (Debian/Ubuntu):

sudo apt-get install -y \
    cmake \
    gcc \
    g++ \
    libc-dbg \
    libglib2.0-dev \
    libigraph-dev \
    make \
    python3 \
    python3-pyelftools \
    xz-utils
sudo apt-get install -y \
    python3-numpy \
    python3-lxml \
    python3-matplotlib \
    python3-networkx \
    python3-scipy
sudo apt-get install -y \
    dstat \
    git \
    htop \
    screen

Shadow Setup

git clone https://github.com/shadow/shadow.git
cd shadow
./setup build --clean --test
./setup install
./setup test

You should add /home/${USER}/.shadow/bin to your shell setup for the PATH environment variable (e.g., in ~/.bashrc or ~/.bash_profile).

echo 'export PATH="${PATH}:/home/${USER}/.shadow/bin"' >> ~/.bashrc && source ~/.bashrc

Check that Shadow is installed and runs:

shadow --version
shadow --help

Setup Notes

  • All build output is generated out-of-source, by default to the ./build directory.
  • Use ./setup build --help to see all build options; the most useful build options are:
    • -g or --debug to build Shadow with debugging symbols
    • --include and --library if you installed any dependencies in non-standard locations or somewhere other than ~/.shadow.
    • --prefix if you want to install Shadow somewhere besides ~/.shadow
  • The setup script is a wrapper to cmake and make. Using cmake and make directly is also possible, but strongly discouraged.

TGen Setup

Installing Shadow gives you the simulation environment, but you'll almost certainly want to run some processes inside of Shadow. The TGen traffic generator is useful for generating and transferring traffic through Shadow.

TGen was moved to its own repo on April 3, 2019 as of this commit for organizational reasons, but installing it is easy (TGen's dependencies are a subset of Shadow's):

git clone git@github.com:shadow/tgen.git
cd tgen && mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/home/$USER/.shadow
make && make install

Now you can run ~/.shadow/bin/tgen either inside of Shadow, or outside of Shadow. See the TGen repo for more info.

System Configs and Limits

Some Linux system configuration changes are needed to run large-scale Shadow simulations (more than about 1000 nodes).

Number of Open Files

There is a default linux system limit on the number of open files. If each node in your Shadow plug-in opens many file or socket descriptors (if you have many nodes, this is very likely to happen), you'll likely want to increase the limit so you application doesn't start getting errors when calling open() or socket().

System-wide Limits

Check the system-wide limits with:

sysctl fs.nr_open # per-process open file limit
sysctl fs.file-max # system-wide open file limit

Use cat /proc/sys/fs/file-nr to find:

  1. the current, system-wide number of used file handles
  2. the current, system-wide number of free file handles
  3. and the system-wide limit on the maximum number of open files for all processes

Change the limits, persistent across reboots, and apply now:

sysctl -w fs.nr_open=10485760
echo "fs.nr_open = 10485760" >> /etc/sysctl.conf
sysctl -w fs.file-max=10485760
echo "fs.file-max = 10485760" >> /etc/sysctl.conf
sysctl -p
User Limits

Check the maximum number of open file descriptors currently allowed in your session:

ulimit -n

Check the number of files currently used in a process with pid=PID:

/bin/ls -l /proc/PID/fd/ | wc -l

You will want to almost certainly want to raise the user file limit by modifying /etc/security/limits.conf. For example:

rjansen soft nofile 10485760
rjansen hard nofile 10485760

The max you can use is your fs.nr_open system-wide limit setting from above. You need to either log out and back in or reboot for the changes to take affect. You can watch /proc/sys/fs/file-nr and reduce the limit according to your usage, if you'd like.

Number of Maps

There is a system limit on the number of mmap() mappings per process. Most users will not have to modify these settings. However, if an application running in Shadow makes extensive use of mmap(), you may need to increase the limit.

Process Limit

The process limit can be queried in these ways:

sysctl vm.max_map_count
cat /proc/sys/vm/max_map_count

You can check the number of maps currently used in a process with pid=PID like this:

wc -l /proc/PID/maps

Set a new limit, make it persistent, apply it now:

sudo sysctl -w vm.max_map_count=1073741824
sudo echo "vm.max_map_count = 1073741824" >> /etc/sysctl.conf
sudo sysctl -p

For more information

https://www.kernel.org/doc/Documentation/sysctl/fs.txt
https://www.kernel.org/doc/Documentation/sysctl/vm.txt

man proc
man ulimit
cat /proc/sys/fs/file-max
cat /proc/sys/fs/inode-max