Set up vagrant and a get it working

[ryanlerch]

Resolves: #185

[abompard]

Using the /srv/venv/bin/poetry binary should be enough, no need to source activate.

[ryanlerch]

i get permissions errors when doing it this way (had the same issue witht he datanommer vagrant setup)

doing it this way works, is there an issue with doing it in this manner?

[abompard]

No, it's fine. Maybe add a comment to explain it? I suppose the permission errors come from SELinux, a service file may not be allowed to run an executable in /srv. Anyway.

[ryanlerch]

ok, a few more tweaks added to get tests passing in CI

This one is ready to go.

[abompard]

That's apparently still not enough, because datagrepper installs a version of datanommer.models that takes precedence over the development install that poetry does.

On my fresh Vagrant machine, running the consumer fails on the first message because it's calling the old version of datanommer.models.add() (which does not know how to handle FM messages). And when I look at /srv/venv/lib/python3.9/site-packages/datanommer/models/__init__.py, it's the old version. Maybe running pip uninstall datanommer.models before running poetry install would help ?

[abompard]

Another sign of the same issue: the database created with datanommer-create-db is in the old format.

[ryanlerch]

okay, i think the least doddy solution here is to do some pre-release versions of datanommer, and require them here in datagrepper for development purposes.

I have resorted back to the relative develop dep for the moment to just get it working

also have added a fix to set a randomish UUID for the fedoramessaging queue in the config.toml we use in the vagrant setip

[abompard]

Nice use of the securerandom ruby lib! :-) But Ansible can do UUIDs by itself, as long as it's in a template. I think it's a better idea to template out the entire fedora-messaging config file anyway because then we can set other things like the client app name (it'll help broker sysadmins to know where the connection is coming from).

Can you look at this commit from another PR and tell me what you think of it?

[abompard]

I would suggest to do the following for the datanommer dependency:

• install datagrepper after datanommer, so it can pick up the previous datanommer install
• set the datanommer dependency to "^0.9.1" because that's what's current, otherwise it'll refuse the previous install
• run poetry update to update the lock file
• use the developper's workdir for datanommer:

--- a/Vagrantfile
+++ b/Vagrantfile
@@ -13,6 +13,7 @@ Vagrant.configure(2) do |config|
 
     datagrepper.vm.synced_folder '.', '/vagrant', disabled: true
     datagrepper.vm.synced_folder ".", "/home/vagrant/datagrepper", type: "sshfs"
+    datagrepper.vm.synced_folder "../datanommer", "/home/vagrant/datanommer", type: "sshfs"
 
     datagrepper.vm.provider :libvirt do |libvirt|
       libvirt.cpus = 2
diff --git a/devel/ansible/roles/datagrepper/tasks/main.yml b/devel/ansible/roles/datagrepper/tasks/main.yml
index 0cec830..d62b7f4 100644
--- a/devel/ansible/roles/datagrepper/tasks/main.yml
+++ b/devel/ansible/roles/datagrepper/tasks/main.yml
@@ -23,13 +23,6 @@
     owner: vagrant
     group: vagrant
 
-- name: check out datanommer from git
-  git:
-    repo: https://github.com/fedora-infra/datanommer.git
-    dest: /home/vagrant/datanommer
-  become: yes
-  become_user: vagrant
-
 - name: Create a directory for the virtualenv
   file:
     name: /srv/venv

• use a loop in ansible to avoid repeating ourselves:

- name: Install datanommer with poetry
  shell: /srv/venv/bin/poetry install
  args:
    chdir: /home/vagrant/datanommer/datanommer.{{item}}/
  loop:
    - models
    - commands
    - consumer
  become: yes
  become_user: vagrant

[ryanlerch]

I would suggest to do the following for the datanommer dependency:

• install datagrepper after datanommer, so it can pick up the previous datanommer install
• set the datanommer dependency to "^0.9.1" because that's what's current, otherwise it'll refuse the previous install
• run poetry update to update the lock file
• use the developper's workdir for datanommer:

--- a/Vagrantfile
+++ b/Vagrantfile
@@ -13,6 +13,7 @@ Vagrant.configure(2) do |config|
 
     datagrepper.vm.synced_folder '.', '/vagrant', disabled: true
     datagrepper.vm.synced_folder ".", "/home/vagrant/datagrepper", type: "sshfs"
+    datagrepper.vm.synced_folder "../datanommer", "/home/vagrant/datanommer", type: "sshfs"
 
     datagrepper.vm.provider :libvirt do |libvirt|
       libvirt.cpus = 2
diff --git a/devel/ansible/roles/datagrepper/tasks/main.yml b/devel/ansible/roles/datagrepper/tasks/main.yml
index 0cec830..d62b7f4 100644
--- a/devel/ansible/roles/datagrepper/tasks/main.yml
+++ b/devel/ansible/roles/datagrepper/tasks/main.yml
@@ -23,13 +23,6 @@
     owner: vagrant
     group: vagrant
 
-- name: check out datanommer from git
-  git:
-    repo: https://github.com/fedora-infra/datanommer.git
-    dest: /home/vagrant/datanommer
-  become: yes
-  become_user: vagrant
-
 - name: Create a directory for the virtualenv
   file:
     name: /srv/venv

• use a loop in ansible to avoid repeating ourselves:

- name: Install datanommer with poetry
  shell: /srv/venv/bin/poetry install
  args:
    chdir: /home/vagrant/datanommer/datanommer.{{item}}/
  loop:
    - models
    - commands
    - consumer
  become: yes
  become_user: vagrant

I have tried this order in the past, (other than syncing the workdir with sshfs) and when installing datagrepper, it still overwrites the installed 0.9.1 installed by datanommer with the 0.9.1 from pip.

will try with the directory synced with sshfs to see if that makes a difference, but not a fan of this approach, as it requires the user to checkout datanommer in a specifric location for the vagrant box to work.

[ryanlerch]

yeah, even when installing datanommer first, when installing datagrepper, this happens:

Updating datanommer.models (0.9.1 /home/vagrant/datanommer/datanommer.models -> 0.9.1)

and after updating to the newly alpha release, this happens too:

Updating datanommer.models (1.0.0a1 /home/vagrant/datanommer/datanommer.models -> 0.9.1)

[abompard]

Looks good, and tested, thanks!