Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade vagrant environment #492

Merged
merged 14 commits into from
Mar 7, 2023
Merged

Upgrade vagrant environment #492

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
7afa977
Vagrant environment changes
evmiguel Feb 13, 2023
45672d8
adding memory increase and cron job
evmiguel Feb 13, 2023
2897a22
Automatically upgrade to Node 18 and fix cron job on Vagrant environment
evmiguel Feb 14, 2023
f63a664
Adding cores to Vagrant machine
evmiguel Feb 15, 2023
43d4019
Adding environment variable to differentiate between deployment envir…
evmiguel Feb 15, 2023
4f20861
Changing permissions to /vagrant/server folder
evmiguel Feb 15, 2023
5be4dd6
separating out upgrade tasks
evmiguel Feb 21, 2023
ec00840
explicitly adding patch-package to root
evmiguel Feb 22, 2023
2d88562
removing DEPLOY_ENVIRONMENT variable
evmiguel Feb 28, 2023
30cd4f1
remove DEPLOY_ENVIRONMENT from vagrant config
evmiguel Feb 28, 2023
4f9dfb9
adding ALLOW_FAKE_ROLE env variable to vagrant env
evmiguel Mar 1, 2023
ca13a13
World write rsync permission on server folder on vagrant machine
evmiguel Mar 7, 2023
315030e
Document new Vagrant configuration
jugglinmike Mar 7, 2023
2c1c215
Remove obviated Ansible Task
jugglinmike Mar 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion Vagrantfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,18 @@ VAGRANTFILE_API_VERSION = '2'
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = 'debian/buster64'

config.vm.provider "virtualbox" do |v|
v.memory = 4096
v.cpus = 2
end

config.vm.synced_folder '.', '/vagrant', type: 'rsync',
rsync__exclude: ['node_modules/', 'client/dist/']
rsync__exclude: ['node_modules/', 'client/dist/'],
# Because Vagrant sets the owner of the synced folders to be the `vagrant`
# user, the `aria-bot` user (which runs the server process) will not be
# able to write to the `server` directory by default. The following
# configuration makes the directory world-writable to enable this.
rsync__args: ['-r', '--perms', '--chmod=Du=rwx,Dg=rwx,Do=rwx,Fu=rwx,Fg=rwx,Fo=rwx', './server']
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jugglinmike This is a naive world writable solution for the rsync options on the server folder. I know it's not setting the ownership on aria-bot, as suggested, but it does circumvent a Vagrant error on rsync when aria-bot isn't created on first provision. Given the time constraints, could we push this or something like this forward so that we can get the upgrade in place before the sprint ends?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jugglinmike I can also confirm that with this change, I am no longer running into permissions issues.


# Ideally, this IP will be unique, so the entry added to /etc/hosts won't
# conflict with that of another project.
Expand Down
5 changes: 1 addition & 4 deletions client/components/App/useSigninUrl.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,7 @@ const useSigninUrl = () => {
if (matchedFakeRole) {
dataFromFrontend += `fakeRole-${matchedFakeRole[1]}`;
}
return (
`${process.env.API_SERVER}/api/auth/oauth?dataFromFrontend=` +
dataFromFrontend
);
return `/api/auth/oauth?dataFromFrontend=` + dataFromFrontend;
};

export default useSigninUrl;
5 changes: 4 additions & 1 deletion deploy/files/config-development.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,11 @@ PGUSER=atr
PGPASSWORD=atr
PGHOST=localhost
PGPORT=5432
ENVIRONMENT=dev
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@evmiguel I see dev being referenced in embed.js but I'm assuming development is being inferred throughout when this ENV var doesn't exist. Should those embed.js references to this var (or lack of) also be updated?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@howard-e Good catch! Since the vagrant environment mirrors production, it's technically not using the dev configuration. I think we should name the variable something else with a different value.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would be the consequence of leaving that variable unchanged?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jugglinmike The app would look for the server side rendering files for the embed feature in a different directory.

ENVIRONMENT=vagrant
ALLOW_FAKE_ROLE=true
IMPORT_CONFIG=/home/aria-bot/config.env
GITHUB_OAUTH_SERVER=https://github.com
GITHUB_GRAPHQL_SERVER=https://api.github.com
GITHUB_CLIENT_ID=7190d52e3c3ef8e48e70
GITHUB_CLIENT_SECRET=a5eebe76bdaaf126e34a9b5a5a9686cf1cdbb71f
GITHUB_TEAM_ORGANIZATION=w3c
Expand Down
8 changes: 8 additions & 0 deletions deploy/roles/application/tasks/cron.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,11 @@
name: "import latest aria-at tests"
minute: "15"
job: "curl -X POST https://{{fqdn}}/api/test/import"
when: deployment_mode != 'development'

- name: Set a cron job to build test results in development
cron:
name: "import latest aria-at tests"
minute: "15"
job: "curl -X POST http://localhost:5000/api/test/import"
when: deployment_mode == 'development'
11 changes: 2 additions & 9 deletions deploy/roles/application/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,13 +28,6 @@
when: deployment_mode != 'development'
notify: "restart server"

- name: Make vagrant folder writable
file:
path: /vagrant/server/scripts
mode: '0777'
recurse: yes
when: deployment_mode == 'development'

- name: Link application code
file:
dest: '{{source_dir}}'
Expand Down Expand Up @@ -66,12 +59,12 @@
ansible_ssh_pipelining: yes

- name: Migrate database
shell: DOTENV_CONFIG_PATH={{environment_config.dest}} node -r ./node_modules/dotenv/config $(npm bin)/sequelize-cli db:migrate --config ./config/config.js --migrations-path ./server/migrations/
shell: DOTENV_CONFIG_PATH={{environment_config.dest}} node -r ./node_modules/dotenv/config ./node_modules/.bin/sequelize-cli db:migrate --config ./config/config.js --migrations-path ./server/migrations/
args:
chdir: '{{source_dir}}'

- name: Seed database
shell: DOTENV_CONFIG_PATH={{environment_config.dest}} node -r ./node_modules/dotenv/config $(npm bin)/sequelize-cli db:seed:all --config ./config/config.js --seeders-path ./server/seeders/
shell: DOTENV_CONFIG_PATH={{environment_config.dest}} node -r ./node_modules/dotenv/config ./node_modules/.bin/sequelize-cli db:seed:all --config ./config/config.js --seeders-path ./server/seeders/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to other reviewers: npm recently removed this subcommand.

args:
chdir: '{{source_dir}}'

Expand Down
11 changes: 10 additions & 1 deletion deploy/roles/nodejs/tasks/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
- https://dl.yarnpkg.com/debian/pubkey.gpg
- https://deb.nodesource.com/gpgkey/nodesource.gpg.key

- include: upgrade.yml

- name: Add software repositories
apt_repository:
state: present
Expand All @@ -16,7 +18,7 @@
become: yes
with_items:
- deb https://dl.yarnpkg.com/debian/ stable main
- deb https://deb.nodesource.com/node_14.x buster main
- deb https://deb.nodesource.com/node_18.x buster main

- name: Install packages
apt:
Expand All @@ -25,3 +27,10 @@
- yarn
state: present
become: yes

- name: Check if Node exists
command: node --version
register: new_node_version

- name: Show status of nodejs installation
debug: msg="{{ new_node_version.stdout }}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's safe to trust apt to tell us if the installation failed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted this debug message for us to compare the version in the Check if Node exists task and the node version installed during the upgrade.

25 changes: 25 additions & 0 deletions deploy/roles/nodejs/tasks/upgrade.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
- name: Check if Node exists
shell: |
if ! node --version; then
echo "Node.js is not installed"
fi
register: node_version
ignore_errors: true

- name: Show status of nodejs installation
debug: msg="{{ node_version.stdout }}"

- name: Remove Node.js repo if not version 18
apt_repository:
state: absent
repo: deb https://deb.nodesource.com/node_14.x buster main
update_cache: True
become: yes
when: '"18" not in node_version.stdout'

- name: Remove Node.js if not version 18
apt:
name: nodejs
state: absent
become: yes
when: '"18" not in node_version.stdout'
5 changes: 4 additions & 1 deletion index.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
const express = require('express');
const history = require('connect-history-api-fallback');
const path = require('path');

const { listener } = require('./server/server');
listener.use(express.static('./client/dist'));
// Need to use a fallback api so that the server
// can redirect to the frontend routes
listener.use(history()).use(express.static('./client/dist'));

listener.get('*', (req, res) => {
res.sendFile(path.join(__dirname, './client/dist/index.html'));
Expand Down
7 changes: 5 additions & 2 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
"storybook": "yarn workspaces run storybook",
"sequelize": "dotenv -e config/dev.env npx sequelize-cli",
"sequelize:test": "dotenv -e config/test.env npx sequelize-cli",
"postinstall": "npx patch-package"
"postinstall": "patch-package"
},
"repository": {
"type": "git",
Expand All @@ -33,7 +33,10 @@
"url": "https://github.com/bocoup/aria-at-report/issues"
},
"homepage": "https://github.com/bocoup/aria-at-report#readme",
"dependencies": {},
"dependencies": {
"patch-package": "^6.5.1",
"postinstall-postinstall": "^2.1.0"
},
"workspaces": [
"client",
"server"
Expand Down
4 changes: 3 additions & 1 deletion server/apps/embed.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,9 @@ app.get('/reports/:pattern', async (req, res) => {
// Usage: https://aria-at.w3.org/embed/reports/command-button?title=Link+Example+(span+element+with+text+content)
const queryTitle = req.query.title;
const pattern = req.params.pattern;
const protocol = process.env.ENVIRONMENT === 'dev' ? 'http://' : 'https://';
const protocol = /dev|vagrant/.test(process.env.ENVIRONMENT)
? 'http://'
: 'https://';
const {
title,
allBrowsers,
Expand Down
1 change: 1 addition & 0 deletions server/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
"apollo-server-express": "^3.4.0",
"axios": "^0.27.2",
"body-parser": "^1.20.0",
"connect-history-api-fallback": "^2.0.0",
"connect-pg-simple": "^8.0.0",
"cross-fetch": "^3.1.5",
"cross-spawn": "^7.0.3",
Expand Down
71 changes: 68 additions & 3 deletions yarn.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4387,7 +4387,13 @@
resolved "https://registry.yarnpkg.com/@xtuc/long/-/long-4.2.2.tgz#d291c6a4e97989b5c61d9acf396ae4fe133a718d"
integrity sha512-NuHqBY1PB/D8xU6s/thBgOAiAP7HOYDQ32+BFZILJ8ivkUkAHQnWfn6WhL79Owj1qmUnoN/YPhktdIoucipkAQ==

"@yarnpkg/lockfile@^1.1.0":
version "1.1.0"
resolved "https://registry.yarnpkg.com/@yarnpkg/lockfile/-/lockfile-1.1.0.tgz#e77a97fbd345b76d83245edcd17d393b1b41fb31"
integrity sha512-GpSwvyXOcOOlV70vbnzjj4fW5xW/FdUF6nQEt1ENy7m4ZCczi1+/buVUPAqmGfqznsORNFzUMjctTIp8a9tuCQ==

abab@^2.0.0, abab@^2.0.6:
abab@^2.0.6:
version "2.0.6"
resolved "https://registry.yarnpkg.com/abab/-/abab-2.0.6.tgz#41b80f2c871d19686216b82309231cfd3cb3d291"
integrity sha512-j2afSsaIENvHZN2B8GOpF566vZ5WVk5opAiMTvWgaQT8DkbOqsTfvNAvHoRGU2zzP8cPoqys+xHTRDWW8L+/BA==
Expand Down Expand Up @@ -5846,7 +5852,7 @@ chalk@^3.0.0:
ansi-styles "^4.1.0"
supports-color "^7.1.0"

chalk@^4.0.0, chalk@^4.1.0:
chalk@^4.0.0, chalk@^4.1.0, chalk@^4.1.2:
version "4.1.2"
resolved "https://registry.yarnpkg.com/chalk/-/chalk-4.1.2.tgz#aac4e2b7734a740867aeb16bf02aad556a1e7a01"
integrity sha512-oKnbhFyRIXpUuez8iBMmyEa4nbj4IOQyuhc/wy9kY7/WVPcwIO9VA668Pu8RkO7+0G76SLROeyw9CpQ061i4mA==
Expand Down Expand Up @@ -8384,6 +8390,13 @@ find-up@^5.0.0:
locate-path "^6.0.0"
path-exists "^4.0.0"

find-yarn-workspace-root@^2.0.0:
version "2.0.0"
resolved "https://registry.yarnpkg.com/find-yarn-workspace-root/-/find-yarn-workspace-root-2.0.0.tgz#f47fb8d239c900eb78179aa81b66673eac88f7bd"
integrity sha512-1IMnbjt4KzsQfnhnzNd8wUEgXZ44IzZaZmnLYx7D5FZlaHt2gW20Cri8Q+E/t5tIj4+epTBub+2Zxu/vNILzqQ==
dependencies:
micromatch "^4.0.2"

flat-cache@^3.0.4:
version "3.0.4"
resolved "https://registry.yarnpkg.com/flat-cache/-/flat-cache-3.0.4.tgz#61b0338302b2fe9f957dcc32fc2a87f1c3048b11"
Expand Down Expand Up @@ -11208,6 +11221,13 @@ kind-of@^6.0.0, kind-of@^6.0.2:
resolved "https://registry.yarnpkg.com/kind-of/-/kind-of-6.0.3.tgz#07c05034a6c349fa06e24fa35aa76db4580ce4dd"
integrity sha512-dcS1ul+9tmeD95T+x28/ehLgd9mENa3LsvDTtzm3vyBEO7RPptvAD+t44WVXaUjTBRcrpFeFlC8WCruUR456hw==

klaw-sync@^6.0.0:
version "6.0.0"
resolved "https://registry.yarnpkg.com/klaw-sync/-/klaw-sync-6.0.0.tgz#1fd2cfd56ebb6250181114f0a581167099c2b28c"
integrity sha512-nIeuVSzdCCs6TDPTqI8w1Yre34sSq7AkZ4B3sfOBbI2CgVSB4Du4aLQijFU2+lhAFCwt9+42Hel6lQNIv6AntQ==
dependencies:
graceful-fs "^4.1.11"

kleur@^3.0.3:
version "3.0.3"
resolved "https://registry.yarnpkg.com/kleur/-/kleur-3.0.3.tgz#a79c9ecc86ee1ce3fa6206d1216c501f147fc07e"
Expand Down Expand Up @@ -12517,7 +12537,7 @@ onetime@^5.1.2:
dependencies:
mimic-fn "^2.1.0"

open@^7.0.3, open@^7.1.0:
open@^7.0.3, open@^7.1.0, open@^7.4.2:
version "7.4.2"
resolved "https://registry.yarnpkg.com/open/-/open-7.4.2.tgz#b8147e26dcf3e426316c730089fd71edd29c2321"
integrity sha512-MVHddDVweXZF3awtlAS+6pgKLlm/JgxZ90+/NBurBoQctVOOB/zDdVjcyPzQ+0laDGbsWgrRkflI65sQeOgT9Q==
Expand Down Expand Up @@ -12816,6 +12836,26 @@ pascalcase@^0.1.1:
resolved "https://registry.yarnpkg.com/pascalcase/-/pascalcase-0.1.1.tgz#b363e55e8006ca6fe21784d2db22bd15d7917f14"
integrity sha512-XHXfu/yOQRy9vYOtUDVMN60OEJjW013GoObG1o+xwQTpB9eYJX/BjXMsdW13ZDPruFhYYn0AG22w0xgQMwl3Nw==

patch-package@^6.5.1:
version "6.5.1"
resolved "https://registry.yarnpkg.com/patch-package/-/patch-package-6.5.1.tgz#3e5d00c16997e6160291fee06a521c42ac99b621"
integrity sha512-I/4Zsalfhc6bphmJTlrLoOcAF87jcxko4q0qsv4bGcurbr8IskEOtdnt9iCmsQVGL1B+iUhSQqweyTLJfCF9rA==
dependencies:
"@yarnpkg/lockfile" "^1.1.0"
chalk "^4.1.2"
cross-spawn "^6.0.5"
find-yarn-workspace-root "^2.0.0"
fs-extra "^9.0.0"
is-ci "^2.0.0"
klaw-sync "^6.0.0"
minimist "^1.2.6"
open "^7.4.2"
rimraf "^2.6.3"
semver "^5.6.0"
slash "^2.0.0"
tmp "^0.0.33"
yaml "^1.10.2"

path-browserify@0.0.1:
version "0.0.1"
resolved "https://registry.yarnpkg.com/path-browserify/-/path-browserify-0.0.1.tgz#e6c4ddd7ed3aa27c68a20cc4e50e1a4ee83bbc4a"
Expand Down Expand Up @@ -13215,6 +13255,11 @@ postgres-interval@^1.1.0:
dependencies:
xtend "^4.0.0"

postinstall-postinstall@^2.1.0:
version "2.1.0"
resolved "https://registry.yarnpkg.com/postinstall-postinstall/-/postinstall-postinstall-2.1.0.tgz#4f7f77441ef539d1512c40bd04c71b06a4704ca3"
integrity sha512-7hQX6ZlZXIoRiWNrbMQaLzUUfH+sSx39u8EJ9HYuDc1kLo9IXKWjM5RSquZN1ad5GnH8CGFM78fsAAQi3OKEEQ==

prelude-ls@^1.2.1:
version "1.2.1"
resolved "https://registry.yarnpkg.com/prelude-ls/-/prelude-ls-1.2.1.tgz#debc6489d7a6e6b0e7611888cec880337d316396"
Expand Down Expand Up @@ -14403,6 +14448,26 @@ sax@^1.2.4:
resolved "https://registry.yarnpkg.com/sax/-/sax-1.2.4.tgz#2816234e2378bddc4e5354fab5caa895df7100d9"
integrity sha512-NqVDv9TpANUjFm0N8uM5GxL36UgKi9/atZw+x7YFnQ8ckwFGKrl4xX4yWtrey3UJm5nP1kUbnYgLopqWNSRhWw==

sane@^4.0.3:
version "4.1.0"
resolved "https://registry.yarnpkg.com/sane/-/sane-4.1.0.tgz#ed881fd922733a6c461bc189dc2b6c006f3ffded"
integrity sha512-hhbzAgTIX8O7SHfp2c8/kREfEn4qO/9q8C9beyY6+tvZ87EpoZ3i1RIEvp27YBswnNbY9mWd6paKVmKbAgLfZA==
dependencies:
"@cnakazawa/watch" "^1.0.3"
anymatch "^2.0.0"
capture-exit "^2.0.0"
exec-sh "^0.3.2"
execa "^1.0.0"
fb-watchman "^2.0.0"
micromatch "^3.1.4"
minimist "^1.1.1"
walker "~1.0.5"

sax@^1.2.4:
version "1.2.4"
resolved "https://registry.yarnpkg.com/sax/-/sax-1.2.4.tgz#2816234e2378bddc4e5354fab5caa895df7100d9"
integrity sha512-NqVDv9TpANUjFm0N8uM5GxL36UgKi9/atZw+x7YFnQ8ckwFGKrl4xX4yWtrey3UJm5nP1kUbnYgLopqWNSRhWw==

saxes@^6.0.0:
version "6.0.0"
resolved "https://registry.yarnpkg.com/saxes/-/saxes-6.0.0.tgz#fe5b4a4768df4f14a201b1ba6a65c1f3d9988cc5"
Expand Down Expand Up @@ -16958,7 +17023,7 @@ yallist@^4.0.0:
resolved "https://registry.yarnpkg.com/yallist/-/yallist-4.0.0.tgz#9bb92790d9c0effec63be73519e11a35019a3a72"
integrity sha512-3wdGidZyq5PB084XLES5TpOSRA3wjXAlIWMhum2kRcv/41Sn2emQ0dycQW4uZXLejwKvg6EsvbdlVL+FYEct7A==

yaml@^1.10.0, yaml@^1.7.2:
yaml@^1.10.0, yaml@^1.10.2, yaml@^1.7.2:
version "1.10.2"
resolved "https://registry.yarnpkg.com/yaml/-/yaml-1.10.2.tgz#2301c5ffbf12b467de8da2333a459e29e7920e4b"
integrity sha512-r3vXyErRCYJ7wg28yvBY5VSoAF8ZvlcW9/BwUzEtUsjvX/DKs24dIkuwjtuprwJJHsbyUbLApepYTR1BN4uHrg==
Expand Down