Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix that on ubuntu 20.04, the current version of the code does not compile as a python package #224

Merged
merged 1 commit into from
Apr 25, 2022

Conversation

molysgaard
Copy link
Contributor

Steps to reproduce:

  # install required packages
  apt-get install build-essential libncurses5-dev libpcap-dev pybind11-dev

  # create working directory
  mkdir nethogs-py
  cd nethogs-py

  # create virtualenv
  virtualenv -p python3 venv

  # activate virtualenv
  source venv/bin/activate

  # create requirements.txt file for pip
  echo "git+https://github.com/raboof/nethogs.git" > requirements.txt

  # build nethogs using pip
  pip install -r requirements.txt

The last step gives the following compile error on my Ubuntu 20.04
machine:

Processing ./nethogs
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: nethogs
  Building wheel for nethogs (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/lysgaard/work/nethogs-py/venv/bin/python /tmp/tmpbzq5d0ej build_wheel /tmp/tmpxbnq_o9a
       cwd: /tmp/pip-req-build-uxzyh5t6
  Complete output (50 lines):
  running bdist_wheel
  running build
  running build_ext
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/lysgaard/work/nethogs-py/venv/include -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
  building 'nethogs' extension
  creating build
  creating build/temp.linux-x86_64-cpython-38
  creating build/temp.linux-x86_64-cpython-38/python
  creating build/temp.linux-x86_64-cpython-38/src
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Isrc/ -I/tmp/pip-build-env-kvg6xf6h/overlay/lib/python3.8/site-packages/pybind11/include -I/home/lysgaard/work/nethogs-py/venv/include -I/usr/include/python3.8 -c python/bindings.cpp -o build/temp.linux-x86_64-cpython-38/python/bindings.o -std=c++17 -fvisibility=hidden -g0 -Wall -Wextra -Wno-missing-field-initializers --std=c++0x -O3 -fPIC -DVERSION=\"0.8.7\"
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Isrc/ -I/tmp/pip-build-env-kvg6xf6h/overlay/lib/python3.8/site-packages/pybind11/include -I/home/lysgaard/work/nethogs-py/venv/include -I/usr/include/python3.8 -c src/connection.cpp -o build/temp.linux-x86_64-cpython-38/src/connection.o -std=c++17 -fvisibility=hidden -g0 -Wall -Wextra -Wno-missing-field-initializers --std=c++0x -O3 -fPIC -DVERSION=\"0.8.7\"
  src/connection.cpp: In function ‘Connection* findConnectionWithMatchingSource(Packet*, short int)’:
  src/connection.cpp:162:37: error: no matching function for call to ‘std::multiset<Connection*, ConnectionComparator>::lower_bound(Packet*)’
    162 |   auto it = connList->lower_bound(&p);
        |                                     ^
  In file included from /usr/include/c++/9/set:62,
                   from src/process.h:29,
                   from src/connection.cpp:34:
  /usr/include/c++/9/bits/stl_multiset.h:810:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    810 |       lower_bound(const key_type& __x)
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:810:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    810 |       lower_bound(const key_type& __x)
        |                   ~~~~~~~~~~~~~~~~^~~
  /usr/include/c++/9/bits/stl_multiset.h:814:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::const_iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) const [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::const_iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    814 |       lower_bound(const key_type& __x) const
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:814:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    814 |       lower_bound(const key_type& __x) const
        |                   ~~~~~~~~~~~~~~~~^~~
  src/connection.cpp: In function ‘Connection* findConnectionWithMatchingRefpacketOrSource(Packet*, short int)’:
  src/connection.cpp:187:41: error: no matching function for call to ‘std::multiset<Connection*, ConnectionComparator>::lower_bound(Packet*&)’
    187 |   auto it = connList->lower_bound(packet);
        |                                         ^
  In file included from /usr/include/c++/9/set:62,
                   from src/process.h:29,
                   from src/connection.cpp:34:
  /usr/include/c++/9/bits/stl_multiset.h:810:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    810 |       lower_bound(const key_type& __x)
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:810:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    810 |       lower_bound(const key_type& __x)
        |                   ~~~~~~~~~~~~~~~~^~~
  /usr/include/c++/9/bits/stl_multiset.h:814:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::const_iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) const [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::const_iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    814 |       lower_bound(const key_type& __x) const
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:814:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    814 |       lower_bound(const key_type& __x) const
        |                   ~~~~~~~~~~~~~~~~^~~
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for nethogs
Failed to build nethogs
ERROR: Could not build wheels for nethogs which use PEP 517 and cannot be installed directly

I have tracked this down to that the setup.py file uses the wrong
C++-standard-flag.

This is corrected in this commit.

I have also reduced the optimization level from 3 to 2.
The rationale is that optimization level 3 contains
optimazitions that potentially change the behaviour of code, eg.
floating point behaviour. General convention is that optimization level
2 is more than sufficient unless you are working with extremely tight
budget, eg linear algebra kernels or similar. If optimization level 3
is still wanted, I can revert this.

…mpile as a python package

Steps to reproduce:

```
  # install required packages
  apt-get install build-essential libncurses5-dev libpcap-dev pybind11-dev

  # create working directory
  mkdir nethogs-py
  cd nethogs-py

  # create virtualenv
  virtualenv -p python3 venv

  # activate virtualenv
  source venv/bin/activate

  # create requirements.txt file for pip
  echo "git+https://github.com/raboof/nethogs.git" > requirements.txt

  # build nethogs using pip
  pip install -r requirements.txt
```

The last step gives the following compile error on my Ubuntu 20.04
machine:

```
Processing ./nethogs
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: nethogs
  Building wheel for nethogs (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/lysgaard/work/nethogs-py/venv/bin/python /tmp/tmpbzq5d0ej build_wheel /tmp/tmpxbnq_o9a
       cwd: /tmp/pip-req-build-uxzyh5t6
  Complete output (50 lines):
  running bdist_wheel
  running build
  running build_ext
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/home/lysgaard/work/nethogs-py/venv/include -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
  building 'nethogs' extension
  creating build
  creating build/temp.linux-x86_64-cpython-38
  creating build/temp.linux-x86_64-cpython-38/python
  creating build/temp.linux-x86_64-cpython-38/src
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Isrc/ -I/tmp/pip-build-env-kvg6xf6h/overlay/lib/python3.8/site-packages/pybind11/include -I/home/lysgaard/work/nethogs-py/venv/include -I/usr/include/python3.8 -c python/bindings.cpp -o build/temp.linux-x86_64-cpython-38/python/bindings.o -std=c++17 -fvisibility=hidden -g0 -Wall -Wextra -Wno-missing-field-initializers --std=c++0x -O3 -fPIC -DVERSION=\"0.8.7\"
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -Isrc/ -I/tmp/pip-build-env-kvg6xf6h/overlay/lib/python3.8/site-packages/pybind11/include -I/home/lysgaard/work/nethogs-py/venv/include -I/usr/include/python3.8 -c src/connection.cpp -o build/temp.linux-x86_64-cpython-38/src/connection.o -std=c++17 -fvisibility=hidden -g0 -Wall -Wextra -Wno-missing-field-initializers --std=c++0x -O3 -fPIC -DVERSION=\"0.8.7\"
  src/connection.cpp: In function ‘Connection* findConnectionWithMatchingSource(Packet*, short int)’:
  src/connection.cpp:162:37: error: no matching function for call to ‘std::multiset<Connection*, ConnectionComparator>::lower_bound(Packet*)’
    162 |   auto it = connList->lower_bound(&p);
        |                                     ^
  In file included from /usr/include/c++/9/set:62,
                   from src/process.h:29,
                   from src/connection.cpp:34:
  /usr/include/c++/9/bits/stl_multiset.h:810:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    810 |       lower_bound(const key_type& __x)
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:810:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    810 |       lower_bound(const key_type& __x)
        |                   ~~~~~~~~~~~~~~~~^~~
  /usr/include/c++/9/bits/stl_multiset.h:814:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::const_iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) const [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::const_iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    814 |       lower_bound(const key_type& __x) const
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:814:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    814 |       lower_bound(const key_type& __x) const
        |                   ~~~~~~~~~~~~~~~~^~~
  src/connection.cpp: In function ‘Connection* findConnectionWithMatchingRefpacketOrSource(Packet*, short int)’:
  src/connection.cpp:187:41: error: no matching function for call to ‘std::multiset<Connection*, ConnectionComparator>::lower_bound(Packet*&)’
    187 |   auto it = connList->lower_bound(packet);
        |                                         ^
  In file included from /usr/include/c++/9/set:62,
                   from src/process.h:29,
                   from src/connection.cpp:34:
  /usr/include/c++/9/bits/stl_multiset.h:810:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    810 |       lower_bound(const key_type& __x)
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:810:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    810 |       lower_bound(const key_type& __x)
        |                   ~~~~~~~~~~~~~~~~^~~
  /usr/include/c++/9/bits/stl_multiset.h:814:7: note: candidate: ‘std::multiset<_Key, _Compare, _Alloc>::const_iterator std::multiset<_Key, _Compare, _Alloc>::lower_bound(const key_type&) const [with _Key = Connection*; _Compare = ConnectionComparator; _Alloc = std::allocator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::const_iterator = std::_Rb_tree_const_iterator<Connection*>; std::multiset<_Key, _Compare, _Alloc>::key_type = Connection*]’
    814 |       lower_bound(const key_type& __x) const
        |       ^~~~~~~~~~~
  /usr/include/c++/9/bits/stl_multiset.h:814:35: note:   no known conversion for argument 1 from ‘Packet*’ to ‘Connection* const&’
    814 |       lower_bound(const key_type& __x) const
        |                   ~~~~~~~~~~~~~~~~^~~
  error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for nethogs
Failed to build nethogs
ERROR: Could not build wheels for nethogs which use PEP 517 and cannot be installed directly

```

I have tracked this down to that the `setup.py` file uses the wrong
C++-standard-flag.

This is corrected in this commit.

I have also reduced the optimization level from 3 to 2.
The rationale is that optimization level 3 contains
optimazitions that potentially change the behaviour of code, eg.
floating point behaviour. General convention is that optimization level
2 is more than sufficient unless you are working with extremely tight
budget, eg linear algebra kernels or similar. If optimization level 3
is still wanted, I can revert this.
@raboof raboof merged commit 918642f into raboof:main Apr 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants