quicksetup fails without sudo #1382

giovannipizzi · 2018-04-06T08:59:01Z

On dev-jupyter.materialscloud.org, when running the import-export demo from the aiida_demos repository (AiiDA tutorials app), I get this error:

Traceback (most recent call last):
  File "/usr/local/bin/verdi", line 11, in <module>
    sys.exit(run())
  File "/usr/local/lib/python2.7/dist-packages/aiida/cmdline/verdilib.py", line 1049, in run
    aiida.cmdline.verdilib.exec_from_cmdline(sys.argv)
  File "/usr/local/lib/python2.7/dist-packages/aiida/cmdline/verdilib.py", line 1034, in exec_from_cmdline
    CommandClass.run(*argv[command_position + 1:])
  File "/usr/local/lib/python2.7/dist-packages/aiida/cmdline/verdilib.py", line 621, in run
    quicksetup.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 27, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/aiida/cmdline/verdilib.py", line 654, in quicksetup
    success = postgres.determine_setup()
  File "/usr/local/lib/python2.7/dist-packages/aiida/control/postgres.py", line 113, in determine_setup
    non_interactive=bool(not self.interactive), **dbinfo):
  File "/usr/local/lib/python2.7/dist-packages/aiida/control/postgres.py", line 281, in _try_subcmd
    _pg_execute_sh(r'\q', **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/aiida/control/postgres.py", line 345, in _pg_execute_sh
    result = subprocess.check_output(sudo_su_psql, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 629, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 825, in __init__
    restore_signals, start_new_session)
  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 1574, in _execute_child
    raise child_exception_type(errno_num, err_msg)
OSError: [Errno 2] No such file or directory: 'sudo'

I think that there should be some fallback and not a "crash".
This is also needed for the demos. @DropD do you think it's something easy/quick to fix?

The text was updated successfully, but these errors were encountered:

giovannipizzi · 2018-04-06T09:07:14Z

Also, @yakutovicha : what would be the correct approach to create a new DB inside the jupyter software stack? I guess there one does not need sudo as the DB is run as the user?

DropD · 2018-04-06T11:10:09Z

You are sure that sudo is not required in this case? By fallback you mean try to use psql without sudo first? Or do you mean trying to use `su root -c` instead? Or just exit gracefully if su rights are required but sudo is not available?

…

On Fri, Apr 6, 2018, 10:59 Giovanni Pizzi ***@***.***> wrote: Assigned #1382 <#1382> to @DropD <https://github.com/DropD>. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#1382 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABMqBPFjFB2xzJDwLYi5UlK5Pv3dWEjtks5tly5bgaJpZM4TJuM5> .

giovannipizzi · 2018-04-07T07:10:46Z

So, I think that in the "worst" case, one would try to exit gracefully with a message.
Before that, probably yes, trying psql without sudo could be an option, and also su root -c (maybe first su root, and then falling back to psql? Or if one can try psql without sudo as the first thing and fall back to the rest it would be even better, so if it works, the user is never prompted for a password.

I think sudo should not be required on jupyter because the DB is installed and runs with the privileges of the current user, and the DB files live in the home. @yakutovicha might confirm, but probably it's easier if you try it directly in your jupyter.materialscloud.org account.

Ideally, it would be great if quicksetup works smoothly (and without user interactiveness) also in dev-jupyter; I imagine is possible.

giovannipizzi · 2018-04-13T09:12:28Z

Also, as a note, running the tests in a ubuntu environment without sudo fails with:

[test-seekpath] Running shell script

+ .travis-data/test_script.sh


case "$TEST_TYPE" in

    docs)

        # Compile the docs (HTML format); 

        # -C change to 'docs' directory before doing anything

        # -n to warn about all missing references

        # -W to convert warnings in errors

        SPHINXOPTS="-nW" make -C docs

        ;;

    tests)

        # Add the .travis-data folder to the python path such that defined workchains can be found by the daemon

        export PYTHONPATH=${PYTHONPATH}:${TRAVIS_BUILD_DIR}/.travis-data


        # Run the AiiDA tests

        python ${TRAVIS_BUILD_DIR}/.travis-data/test_setup.py

        python ${TRAVIS_BUILD_DIR}/.travis-data/test_fixtures.py

        python ${TRAVIS_BUILD_DIR}/.travis-data/test_plugin_testcase.py


        verdi -p test_$TEST_AIIDA_BACKEND devel tests


        # Run the daemon tests using docker

        verdi -p $TEST_AIIDA_BACKEND run ${TRAVIS_BUILD_DIR}/.travis-data/test_daemon.py

        ;;

    pre-commit)

        pre-commit run --all-files || ( git status --short ; git diff ; exit 1 )

        ;;

esac

EFEE

======================================================================

ERROR: test_postgres_faillure (__main__.QuicksetupTestCase)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "./.travis-data/test_setup.py", line 72, in test_postgres_faillure

    catch_exceptions=False

  File "/usr/local/lib/python2.7/dist-packages/click/testing.py", line 279, in invoke

    prog_name=self.get_default_prog_name(cli), **extra)

  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 697, in main

    rv = self.invoke(ctx)

  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 895, in invoke

    return ctx.invoke(self.callback, **ctx.params)

  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 535, in invoke

    return callback(*args, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/click/decorators.py", line 27, in new_func

    return f(get_current_context().obj, *args, **kwargs)

  File "/var/jenkins_home/workspace/test-seekpath/aiida/cmdline/verdilib.py", line 655, in quicksetup

    success = postgres.determine_setup()

  File "/var/jenkins_home/workspace/test-seekpath/aiida/control/postgres.py", line 113, in determine_setup

    non_interactive=bool(not self.interactive), **dbinfo):

  File "/var/jenkins_home/workspace/test-seekpath/aiida/control/postgres.py", line 281, in _try_subcmd

    _pg_execute_sh(r'\q', **kwargs)

  File "/var/jenkins_home/workspace/test-seekpath/aiida/control/postgres.py", line 345, in _pg_execute_sh

    result = subprocess.check_output(sudo_su_psql, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 629, in check_output

    process = Popen(stdout=PIPE, *popenargs, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 825, in __init__

    restore_signals, start_new_session)

  File "/usr/local/lib/python2.7/dist-packages/subprocess32.py", line 1574, in _execute_child

    raise child_exception_type(errno_num, err_msg)

OSError: [Errno 2] No such file or directory: 'sudo'


======================================================================

ERROR: test_user_configure (__main__.SetupTestCase)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "./.travis-data/test_setup.py", line 84, in setUp

    self.pg_test = PGTest()

  File "/usr/local/lib/python2.7/dist-packages/pgtest/pgtest.py", line 268, in __init__

    self._pg_ctl_exe = which('pg_ctl')

  File "/usr/local/lib/python2.7/dist-packages/pgtest/pgtest.py", line 89, in which

    results = subprocess.check_output(locate_cmd)

  File "/usr/lib/python2.7/subprocess.py", line 567, in check_output

    process = Popen(stdout=PIPE, *popenargs, **kwargs)

  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__

    errread, errwrite)

  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child

    raise child_exception

OSError: [Errno 2] No such file or directory


======================================================================

ERROR: test_user_setup (__main__.SetupTestCase)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "./.travis-data/test_setup.py", line 84, in setUp

    self.pg_test = PGTest()

  File "/usr/local/lib/python2.7/dist-packages/pgtest/pgtest.py", line 268, in __init__

    self._pg_ctl_exe = which('pg_ctl')

  File "/usr/local/lib/python2.7/dist-packages/pgtest/pgtest.py", line 89, in which

    results = subprocess.check_output(locate_cmd)

  File "/usr/lib/python2.7/subprocess.py", line 567, in check_output

    process = Popen(stdout=PIPE, *popenargs, **kwargs)

  File "/usr/lib/python2.7/subprocess.py", line 711, in __init__

    errread, errwrite)

  File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child

    raise child_exception

OSError: [Errno 2] No such file or directory


======================================================================

FAIL: test_user_setup (__main__.QuicksetupTestCase)

----------------------------------------------------------------------

Traceback (most recent call last):

  File "./.travis-data/test_setup.py", line 53, in test_user_setup

    self.assertFalse(result.exception, msg=get_debug_msg(result))

AssertionError: [Errno 2] No such file or directory: 'sudo'

---

Output:



----------------------------------------------------------------------

Ran 4 tests in 0.369s


FAILED (failures=1, errors=3)

script returned exit code 1

giovannipizzi · 2018-04-13T09:29:52Z

Moreover, note that some of the errors are due to the fact that pg_test requires locate as a dependency from apt-get or similar

ltalirz · 2018-12-11T10:17:26Z

Looked at this issue on aiidalab together with @sphuber

On aiidalab (formerly jupyter):

we run aiida 0.12.2
the database can be accessed by the user without sudo, e.g. this works:
psql -h localhost -d template1

I.e. quicksetup should determine that it can connect via psycopg.

The reason it failed to connect via psycopg is, very likely, that the host was not set to localhost by default:

In [2]: from aiida.control.postgres import _try_connect
In [4]: _try_connect(**{'user':None,'database':'template1'})
Out[4]: False
In [5]: _try_connect(**{'user':None,'database':'template1','host':'localho
   ...: st'})
Out[5]: True

This is already fixed in develop
https://github.com/aiidateam/aiida_core/blob/6e9711046753332933f982971db1d7ac7e7ade58/aiida/control/postgres.py#L72

But not in 0.12
https://github.com/aiidateam/aiida_core/blob/a045c451166eca0ab221a309c39c81edee76177b/aiida/control/postgres.py#L67

I.e. one should simply backport the postgres improvements to release_0.12.3 in order to close this issue.

sphuber · 2019-02-01T08:03:30Z

Fixed for v0.12.3 in PR #2433

giovannipizzi added type/bug topic/verdi priority/important labels Apr 6, 2018

giovannipizzi assigned DropD Apr 6, 2018

DropD assigned ltalirz and unassigned DropD Aug 28, 2018

ltalirz mentioned this issue Oct 24, 2018

verdi quicksetup assumes presence of sudo #2004

Closed

2 tasks

giovannipizzi added this to the v1.0.0 milestone Dec 3, 2018

sphuber modified the milestones: v1.0.0, v1.0.0b1 Dec 4, 2018

ConradJohnston mentioned this issue Dec 10, 2018

Add test for availability of sudo in quicksetup #2333

Merged

ltalirz modified the milestones: v1.0.0b1, v0.12.3 Dec 11, 2018

ConradJohnston mentioned this issue Dec 11, 2018

Document how to set up local Postgres instance #2335

Closed

ltalirz mentioned this issue Jan 31, 2019

backport postgres improvements #2433

Merged

sphuber closed this as completed Feb 1, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

quicksetup fails without sudo #1382

quicksetup fails without sudo #1382

giovannipizzi commented Apr 6, 2018

giovannipizzi commented Apr 6, 2018

DropD commented Apr 6, 2018 via email

giovannipizzi commented Apr 7, 2018

giovannipizzi commented Apr 13, 2018

giovannipizzi commented Apr 13, 2018

ltalirz commented Dec 11, 2018 •

edited

Loading

sphuber commented Feb 1, 2019

quicksetup fails without sudo #1382

quicksetup fails without sudo #1382

Comments

giovannipizzi commented Apr 6, 2018

giovannipizzi commented Apr 6, 2018

DropD commented Apr 6, 2018 via email

giovannipizzi commented Apr 7, 2018

giovannipizzi commented Apr 13, 2018

giovannipizzi commented Apr 13, 2018

ltalirz commented Dec 11, 2018 • edited Loading

sphuber commented Feb 1, 2019

ltalirz commented Dec 11, 2018 •

edited

Loading