pre_build in labspec.json fails!

[apurv111]

I am running a shell script to install packages required for a lab with in pre_build of the labspec.json.

I have tested the script on a vanila ubuntu container. The script runs fine without throwing any error.

The same script when passed to ovpl via labspec.json, installation fails. I have set the execute permissions too.
"pre_build": ["sh pre-build-script.sh "],

My script "pre-build-script.sh":
(sudo apt-get update; sudo apt-get install -y make m4 build-essential gcc gfortran libssl-dev python-software-properties; sudo -E apt-add-repository -y ppa:aims/sagemath; sudo apt-get update; sudo apt-get install -y dvipng; sudo dpkg --configure -a; sudo apt-get install -f; sudo apt-get install -y sagemath-upstream-binary --force-yes)

I have tested the same on a new container multiple times and it is successfull. but not via ovpl.

[ecthiender]

Can you attach the logs of OVPL when it fails?

[anhsirksai]

sure!! These are from yesterday evening run.

02/25/2015 05:54:31 PM: DEBUG: /root/ovpl/src/adapters/CentOSVZAdapter.py:195 -> vm_id = 58136, src_dir=/root/labs/databases-iiith, dest_dir=/vz/root/58136/root/labs
02/25/2015 05:54:31 PM: DEBUG: /root/ovpl/src/adapters/CentOSVZAdapter.py:168 -> Command = rsync -arz --progress /root/labs/databases-iiith /vz/root/58136/root/labs
02/25/2015 05:54:31 PM: DEBUG: /root/ovpl/src/utils/execute_commands.py:26 -> command: rsync -arz --progress /root/labs/databases-iiith /vz/root/58136/root/labs
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/adapters/CentOSVZAdapter.py:171 -> Copy successful
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/adapters/CentOSVZAdapter.py:128 -> CentOSVZAdapter: start_vm_manager(): command = /usr/sbin/vzctl exec 58136 "su - root -c 'python /root/ovpl/src/VMManager/VMManagerServer.py VMManager &'"
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/utils/execute_commands.py:26 -> command: /usr/sbin/vzctl exec 58136 "su - root -c 'python /root/ovpl/src/VMManager/VMManagerServer.py VMManager &'"
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/utils/execute_commands.py:26 -> command: /usr/sbin/vzlist -a | grep 58136
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/adapters/CentOSVZAdapter.py:100 -> CentOSVZAdapter: init_vm(): success = True, response = {'vm_ip': '10.2.58.136', 'vmm_port': '9089', 'vm_id': '58136'}
02/25/2015 05:54:32 PM: DEBUG: adapters/AdapterServer.py:49 -> success status returned True from init_vm
02/25/2015 05:54:32 PM: DEBUG: adapters/AdapterServer.py:51 -> init vm result = {'vm_ip': '10.2.58.136', 'vmm_port': '9089', 'vm_id': '58136'}
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/VMPool.py:87 -> VMPool: create_vm(): Response text from adapter: {"vm_ip": "10.2.58.136", "vmm_port": "9089", "vm_id": "58136"}
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/Controller.py:34 -> test_lab(): Returned from VMPool = {'lab_history': {'released_by': None, 'destroyed_on': None, 'released_on': None, 'destroyed_by': None}, 'lab_spec': {'lab_id': 'cse06', 'revision_tag': None, u'lab': {u'build_requirements': {u'platform': {u'os': u'ubuntu', u'service_pack': u'', u'build_steps': {u'status': [], u'pre_build': [], u'post_build': [], u'build': [u'sh change-shell.sh', u'./create-sage.exp', u'make all -C ../src', u'sh start_sage'], u'configure': []}, u'installer': [], u'osVersion': u'12', u'arch': u'i386'}}, u'runtime_requirements': {u'platform': {u'servicepack': u'', u'arch': u'i386', u'lab_actions': {u'restore': [], u'pause': [], u'stats': [], u'shutdown': [], u'resume': [], u'stop': [], u'publish': [], u'start': [], u'init': [u'sh pre-build-script.sh'], u'clean': [], u'backup': []}, u'storage': {u'min_required': u'15gb'}, u'hosting': u'dedicated', u'memory': {u'min_required': u'256mb', u'max_required': u'2gb'}, u'installer': [], u'osVersion': u'12', u'os': u'ubuntu'}, 'hosting': 'dedicated'}, u'description': {u'discipline': [u'Computer Science & Engineering'], u'status': u'Released', u'name': u'Data Mining', u'type': u'', u'server-side': True, u'integration_level': 5, u'id': 'cse06', u'developer': [{u'web': u'', u'name': u'Vikram Pudi', u'title': u'', u'institute': u'IIIT Hyderabad', u'contact': {u'alternate_email': u'', u'email': u'vikram@iiit.ac.in', u'office_number': u'', u'mobile_number': u''}, u'role': u'Lab Developer', u'department': u'', u'organization': u''}, {u'web': u'', u'name': u'Harshit Dubey', u'title': u'', u'institute': u'IIIT Hyderabad', u'contact': {u'alternate_email': u'', u'email': u'harshit.dubey@students.iiit.ac.in', u'office_number': u'', u'mobile_number': u''}, u'role': u'Lab RA', u'department': u'', u'organization': u''}]}}, 'lab_repo_name': 'databases-iiith', u'template': u'1.0', 'lab_src_url': 'https://github.com/Virtual-Labs/databases-iiith.git'}, 'vmpool_info': {'adapter_ip': u'http://localhost', 'vmpool_id': 1, 'adapter_port': u'8000', 'vm_description': u'LINUX'}, 'vm_info': {'vm_ip': u'10.2.58.136', 'vmm_port': u'9089', 'vm_id': u'58136'}, 'vm_status': {'last_known_status': None, 'disk_usage': None, 'reconnect_attempts': None, 'mem_usage': None, 'last_successful_connection': None}}
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/Controller.py:38 -> test_lab(): vmmgrurl = http://10.2.58.136
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/LabManager.py:46 -> vmmgr_ip = http://10.2.58.136, port = 9089, lab_src_url = https://github.com/Virtual-Labs/databases-iiith.git
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/LabManager.py:51 -> url = http://10.2.58.136:9089/api/1.0/test-lab, payload = {'version': None, 'lab_src_url': 'https://github.com/Virtual-Labs/databases-iiith.git'}
02/25/2015 05:54:34 PM: ERROR: /root/ovpl/src/LabManager.py:64 -> Error installing lab on VM for the first attempt with error: ('Connection aborted.', error(111, 'Connection refused'))
02/25/2015 05:54:36 PM: DEBUG: /root/ovpl/src/LabManager.py:58 -> response = <Response [200]>
02/25/2015 05:54:36 PM: ERROR: /root/ovpl/src/Controller.py:46 -> test_lab(); Test failed with error:Test lab failed

[ecthiender]

Hey, not the logs printed on the screen. Can you paste the logs from the
logs/ovpl.log file?

On Thu, Feb 26, 2015 at 3:31 PM, sai krishna notifications@github.com
wrote:

sure!! These are from yesterday evening run.

02/25/2015 05:54:31 PM: DEBUG:
/root/ovpl/src/adapters/CentOSVZAdapter.py:195 -> vm_id = 58136,
src_dir=/root/labs/databases-iiith, dest_dir=/vz/root/58136/root/labs
02/25/2015 05:54:31 PM: DEBUG:
/root/ovpl/src/adapters/CentOSVZAdapter.py:168 -> Command = rsync -arz
--progress /root/labs/databases-iiith /vz/root/58136/root/labs
02/25/2015 05:54:31 PM: DEBUG: /root/ovpl/src/utils/execute_commands.py:26
-> command: rsync -arz --progress /root/labs/databases-iiith
/vz/root/58136/root/labs
02/25/2015 05:54:32 PM: DEBUG:
/root/ovpl/src/adapters/CentOSVZAdapter.py:171 -> Copy successful
02/25/2015 05:54:32 PM: DEBUG:
/root/ovpl/src/adapters/CentOSVZAdapter.py:128 -> CentOSVZAdapter:
start_vm_manager(): command = /usr/sbin/vzctl exec 58136 "su - root -c
'python /root/ovpl/src/VMManager/VMManagerServer.py VMManager &'"
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/utils/execute_commands.py:26
-> command: /usr/sbin/vzctl exec 58136 "su - root -c 'python
/root/ovpl/src/VMManager/VMManagerServer.py VMManager &'"
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/utils/execute_commands.py:26
-> command: /usr/sbin/vzlist -a | grep 58136
02/25/2015 05:54:32 PM: DEBUG:
/root/ovpl/src/adapters/CentOSVZAdapter.py:100 -> CentOSVZAdapter:
init_vm(): success = True, response = {'vm_ip': '10.2.58.136', 'vmm_port':
'9089', 'vm_id': '58136'}
02/25/2015 05:54:32 PM: DEBUG: adapters/AdapterServer.py:49 -> success
status returned True from init_vm
02/25/2015 05:54:32 PM: DEBUG: adapters/AdapterServer.py:51 -> init vm
result = {'vm_ip': '10.2.58.136', 'vmm_port': '9089', 'vm_id': '58136'}
02/25/2015 05:54:32 PM: DEBUG: /root/ovpl/src/VMPool.py:87 -> VMPool:
create_vm(): Response text from adapter: {"vm_ip": "10.2.58.136",
"vmm_port": "9089", "vm_id": "58136"}
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/Controller.py:34 ->
test_lab(): Returned from VMPool = {'lab_history': {'released_by': None,
'destroyed_on': None, 'released_on': None, 'destroyed_by': None},
'lab_spec': {'lab_id': 'cse06', 'revision_tag': None, u'lab':
{u'build_requirements': {u'platform': {u'os': u'ubuntu', u'service_pack':
u'', u'build_steps': {u'status': [], u'pre_build': [], u'post_build': [],
u'build': [u'sh change-shell.sh', u'./create-sage.exp', u'make all -C
../src', u'sh start_sage'], u'configure': []}, u'installer': [],
u'osVersion': u'12', u'arch': u'i386'}}, u'runtime_requirements':
{u'platform': {u'servicepack': u'', u'arch': u'i386', u'lab_actions':
{u'restore': [], u'pause': [], u'stats': [], u'shutdown': [], u'resume':
[], u'stop': [], u'publish': [], u'start': [], u'init': [u'sh
pre-build-script.sh'], u'clean': [], u'backup': []}, u'storage':
{u'min_required': u'15gb'}, u'hosting': u'dedicated', u'memory':
{u'min_required': u'256mb', u'max_required' : u'2gb'}, u'installer': [],
u'osVersion': u'12', u'os': u'ubuntu'}, 'hosting': 'dedicated'},
u'description': {u'discipline': [u'Computer Science & Engineering'],
u'status': u'Released', u'name': u'Data Mining', u'type': u'',
u'server-side': True, u'integration_level': 5, u'id': 'cse06',
u'developer': [{u'web': u'', u'name': u'Vikram Pudi', u'title': u'',
u'institute': u'IIIT Hyderabad', u'contact': {u'alternate_email': u'',
u'email': u'vikram@iiit.ac.in', u'office_number': u'', u'mobile_number':
u''}, u'role': u'Lab Developer', u'department': u'', u'organization': u''},
{u'web': u'', u'name': u'Harshit Dubey', u'title': u'', u'institute':
u'IIIT Hyderabad', u'contact': {u'alternate_email': u'', u'email': u'
harshit.dubey@students.iiit.ac.in', u'office_number': u'',
u'mobile_number': u''}, u'role': u'Lab RA', u'department': u'',
u'organization': u''}]}}, 'lab_repo_name': 'databases-iiith ', u'template':
u'1.0', 'lab_src_url': '
https://github.com/Virtual-Labs/databases-iiith.git'}, 'vmpool_info':
{'adapter_ip': u'http://localhost', 'vmpool_id': 1, 'adapter_port':
u'8000', 'vm_description': u'LINUX'}, 'vm_info': {'vm_ip': u'10.2.58.136',
'vmm_port': u'9089', 'vm_id': u'58136'}, 'vm_status': {'last_known_status':
None, 'disk_usage': None, 'reconnect_attempts': None, 'mem_usage': None,
'last_successful_connection': None}}
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/Controller.py:38 ->
test_lab(): vmmgrurl = http://10.2.58.136
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/LabManager.py:46 -> vmmgr_ip
= http://10.2.58.136, port = 9089, lab_src_url =
https://github.com/Virtual-Labs/databases-iiith.git
02/25/2015 05:54:33 PM: DEBUG: /root/ovpl/src/LabManager.py:51 -> url =
http://10.2.58.136:9089/api/1.0/test-lab, payload = {'version': None,
'lab_src_url': 'https://github.com/Virtual-Labs/databases-iiith.git'}
02/25/2015 05:54:34 PM: ERROR: /root/ovpl/src/LabManager.py:64 -> Error
installing lab on VM for the first attempt with error: ('Connection
aborted.', error(111, 'Connection refused'))

02/25/2015 05:54:36 PM: DEBUG: /root/ovpl/src/LabManager.py:58 -> response

02/25/2015 05:54:36 PM: ERROR: /root/ovpl/src/Controller.py:46 ->
test_lab(); Test failed with error:Test lab failed

—
Reply to this email directly or view it on GitHub
#29 (comment).

[anhsirksai]

I got these from the log file!! Did I miss some thing?? /root/logs/<log file on 25th feb> ​10.2.56.212 : user/pwd: vlead/1

[ecthiender]

Okay. Thanks! I'll look into it. But looks like 'Connection refused', so a
probably some network/internet issue.

On Thu, Feb 26, 2015 at 4:31 PM, sai krishna notifications@github.com
wrote:

I got these from the log file!! Did I miss some thing??
/root/logs/<log file on 25th feb>

​10.2.56.212 : user/pwd: vlead/1

—
Reply to this email directly or view it on GitHub
#29 (comment).

[anhsirksai]

observed this. Not an actual issue. but still ovpl does this wired
configuration by default.

When a container is created with ovpl :
/etc/apt/apt.conf looks like this :
#Acquire::http::Proxy "http://proxy.iiit.ac.in:8080";
Acquire::http::Proxy "http://proxy.iiit.ac.in:8080";
Acquire::https::Proxy "https://proxy.iiit.ac.in:8080";

On Thu, Feb 26, 2015 at 5:35 PM, Anon Ray notifications@github.com wrote:

Okay. Thanks! I'll look into it. But looks like 'Connection refused', so a
probably some network/internet issue.

On Thu, Feb 26, 2015 at 4:31 PM, sai krishna notifications@github.com
wrote:

I got these from the log file!! Did I miss some thing??
/root/logs/<log file on 25th feb>

​10.2.56.212 : user/pwd: vlead/1

—
Reply to this email directly or view it on GitHub
#29 (comment).

—
Reply to this email directly or view it on GitHub
#29 (comment).

[travula]

ADS is unable to contact the VMManager service to install the lab. This is
a network issue. Can you please give the github url of the lab you are
deploying.

On Thu, Feb 26, 2015 at 5:49 PM, sai krishna notifications@github.com
wrote:

observed this. Not an actual issue. but still ovpl does this wired
configuration by default.

When a container is created with ovpl :
/etc/apt/apt.conf looks like this :
#Acquire::http::Proxy "http://proxy.iiit.ac.in:8080";
Acquire::http::Proxy "http://proxy.iiit.ac.in:8080";
Acquire::https::Proxy "https://proxy.iiit.ac.in:8080";

On Thu, Feb 26, 2015 at 5:35 PM, Anon Ray notifications@github.com
wrote:

Okay. Thanks! I'll look into it. But looks like 'Connection refused', so
a
probably some network/internet issue.

On Thu, Feb 26, 2015 at 4:31 PM, sai krishna notifications@github.com
wrote:

I got these from the log file!! Did I miss some thing??
/root/logs/<log file on 25th feb>

​10.2.56.212 : user/pwd: vlead/1

—
Reply to this email directly or view it on GitHub
#29 (comment).

—
Reply to this email directly or view it on GitHub
#29 (comment).

—
Reply to this email directly or view it on GitHub
#29 (comment).

[anhsirksai]

Now ADS is able to contact the VMManager. The pre-build script is being called.

Although the script itself is fine, With further enhancements(capturing the exit code and handling it in pre-build-script.sh) also I could reproduce the same issue.

#!/bin/sh
if [ "$?" -eq "0" ]; then
(sudo apt-get update; sudo apt-get install -y make m4 build-essential gcc gfortran libssl-dev python-software-properties expect expect-dev; sudo -E apt-add-repository -y ppa:aims/sagemath; sudo apt-get update; sudo apt-get install -y dvipng)
if [ "$?" -eq "100" ]; then
(sudo dpkg --configure -a; sudo apt-get install -f; sudo apt-get install -y sagemath-upstream-binary --force-yes)
fi
fi

When "sudo apt-get install -y dvipng" is run, exit code 100 is thrown. So I capture this in my script and run it. Still ovpl fails.

Thanks,

[anhsirksai]

To manually install the lab: the order of scripts to be executed:

pre-build-script.sh
change-shell.sh
make all -C ../src
sh start_sage

I must say, the lab is deployed after this step.

[travula]

Can you guys create an ansible script to deploy this lab?

[travula]

ADS can be enhaced to deploy a lab using ansible.