Skip to content

Setting up dev environment

Jump to bottom Edit New page
Yu Pei edited this page Sep 5, 2024 · 1 revision

Setting up dev environment

This is a guide for Windows and Mac users on how to set up coursemology on your local machine so that you can contribute as well.

Table of Contents

Windows

This is a complete guide for windows setup that can be used independently from the original README in the coursemology repository.

1. Install Ubuntu

Go to the Microsoft app store and search Ubuntu. Install the application Ubuntu 20.04. There are also other acceptable ways such as using a VM or dual booting, but it is recommended to use WSL2. WSL2 has support for headless Chrome which is needed for some tests, which WSL1 cannot support.

2. Cloning the Repository

Clone the coursemology2 repository.

cd
git clone https://github.com/Coursemology/coursemology2.git
cd coursemology2

This git repo uses submodules as well, so use this command to update them:

git submodule update --init --recursive

3. Setting up Ruby on Rails

This section is mostly taken from a guide made by the people at GoRails. To make sure we have everything necessary for Webpacker support in Rails, we're first going to start by adding the Node.js and Yarn repositories.

sudo apt install curl
curl -sL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -
echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list

sudo apt-get update
sudo apt-get install git-core zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev software-properties-common libffi-dev nodejs yarn

Next we install Ruby using a version managed called rbenv

cd
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL

git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc
exec $SHELL

Then we install Ruby version 3.1.4

rbenv install 3.1.4 --verbose

This commands will take 10-15 mins to complete. The --verbose flag will show what is going on so we can be sure it has not gotten stuck.

Once installed, we use the next command to tell rbenv which version to use. We can change the global flag to local instead if you have other Ruby projects on your machine that are on different versions.

rbenv global 3.1.4

4. Install Node.js

Similar to Ruby, here we are installing nodenv, which is a version manager for Node.js

git clone https://github.com/nodenv/nodenv.git ~/.nodenv
cd ~/.nodenv && src/configure && make -C src
echo 'export PATH="$HOME/.nodenv/bin:$PATH"' >> ~/.bashrc
~/.nodenv/bin/nodenv init

Installing a node-build

mkdir -p "$(nodenv root)"/plugins
git clone https://github.com/nodenv/node-build.git "$(nodenv root)"/plugins/node-build
curl -fsSL https://github.com/nodenv/nodenv-installer/raw/master/bin/nodenv-doctor | bash

We can use this command to verify that everything has been properly set up.

curl -fsSL https://github.com/nodenv/nodenv-installer/raw/master/bin/nodenv-doctor | bash

Install Node.js, similar to Ruby

nodenv install 18.12.1 --verbose
nodenv local 18.12.1

Appending the follow to your ~/.bashrc

echo 'eval "$(nodenv init -)"' >> ~/.bashrc
source ~/.bashrc

5. Install Yarn

npm install --global yarn

Installing more dependencies for client application

cd ~/coursemology2/client && yarn; cd -

6. For pg gem to work

sudo apt install libpq-dev

7. Install Ruby dependencies

cd ~/coursemology2
gem install bundler:2.5.9
bundle config set --local without 'ci:production'
bundle install

8. Install PostgreSQL

If you already have a version of PostgreSQL that is at least version 12, you can skip this section. This is because having two versions of PostgreSQL installed on your system can be problematic when it comes to which version is running on the default port 5432.

sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update
sudo apt-get -y install postgresql

Use this command to check the status of your PSQL clusters.

pg_lsclusters

Use this command to start the cluster with the corresponding version nummber and name.

sudo pg_ctlcluster 16 main start

9. Add role to PSQL

  1. Log into psql as postgres.
sudo -u postgres psql
  1. Create a PostgreSQL user. For example, if your user account is benleong:
CREATE ROLE benleong WITH CREATEDB LOGIN;
  1. Give the PostgreSQL user superuser status.
ALTER USER benleong WITH SUPERUSER;
  1. Create a database with the same name as your user account. By default, psql tries to connect to a database with the same name as your user.
CREATE DATABASE benleong WITH OWNER benleong;
  1. Exit the database by entering exit , then try to log into psql using just psql . If you set everything up correctly, you should now be logged in as a PostgreSQL user with the same name as your user account.

10. Create and seed the database

cd ~/coursemology2
bundle exec rake db:setup
bundle exec rake coursemology:seed
RAILS_ENV=test bundle exec rake db:setup

11. Access the Coursemology App

To access the App after having done setting up everything above, you need to run the following command

host lvh.me

on your terminal to bind 127.0.0.1 to lvh.me. Afterwards, you need to run 2 separate processes using 2 different terminals, each to run the Frontend and Backend, respectively. On the terminal that's supposed for the Frontend one, run the following command

yarn build:development

and on the terminal that's supposed for the Backend one, run the following command

bundle exec rails s -p 5000

Then, you can access the App by visiting http://lvh.me:8080

12. Sign in to Coursemology

Email: test@example.org
Password: Coursemology!

13. Installing and running Docker

  1. Update the apt package index and install packages to allow apt to use a repository over HTTPS
sudo apt-get update
sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release
  1. Add Docker’s official GPG key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
  1. Use the following command to set up the stable repository
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
  1. Update the apt package index, and install the latest version of Docker Engine, containerd, and Docker Compose
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin
  1. Verify that Docker Engine is installed correctly by running the hello-world image.
sudo docker run hello-world
  1. To start docker run
sudo service docker start

Note: If you are using WSL1, you may need to follow this when setting up Docker

14. Installing Imagemagick

Set up ImageMagick, which is used for image processing.

sudo apt-get install imagemagick

Mac

This is a guide for Mac users on how to set up coursemology on your local machine. Last updated: 5/8/24

Clone and update the repository

git clone https://github.com/Coursemology/coursemology2.git
cd coursemology2

Update the submodules:

git submodule update --init --recursive

System Requirements

Xcode

Install the latest Xcode from the app store. This process can take up to 15 to 20 minutes. Then, run the following command in your terminal:

xcode-select --install

You may need to click "Install" when prompted.

Homebrew

Homebrew is a package manager for macOS. To install it, run the following command in your terminal:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Ruby (= 3.1.4)

The easiest way to set up Ruby is using rbenv (or rvm will work as well, but this guide will focus on the former).

brew install rbenv

Then run this command:

rbenv init

You should see this message:

writing ~/.profile: now configured for rbenv.
💡 If the system has other instructions, follow that instead.

Now, we can finally install Ruby! Do the following:

rbenv install 3.1.4 --verbose

This command will take 10-15 minutes to complete. The --verbose flag will show you what’s going on so you can be sure it hasn’t gotten stuck.

Once installed, you can tell rbenv which version to use. You can either set this globally via:

rbenv global 3.1.4

Or locally (run this command at the project root):

rbenv local 3.1.4
rbenv rehash

Then update shims to recognize the new commands:

rbenv rehash

Check that everything has worked by typing ruby -v. The version should look something like this:

ruby 3.1.4p223 (2023-03-30 revision 957bb7cb81) [arm64-darwin23]

Gems

Install the bundler gem and use it to install the other gems (specified in the Gemfile):

gem install bundler:2.5.9
bundle config set --local without 'ci:production'
bundle install

Node.js (v18 LTS)

Similar to Ruby, the easiest way to set up and manage multiple versions of Node is using nodenv (or nvm will work as well, but this guide will focus on the former).

We can install it using Homebrew:

brew install nodenv
ℹ️ Note: If you need to add it to path please do so as well.

We will then configure the Node version at the project root. You can refer to the latest version at this website here. As of the time of writing, the latest version is v18.12.1

nodenv install 18.12.1 --verbose
nodenv local 18.12.1

Yarn

Yarn is a package manager for Node.js. You can install it using node package manager (npm):

npm install --global yarn

Then, install the dependencies for the client application:

cd client && yarn; cd -

ImageMagick or GraphicsMagick

Finally, we just need to set up ImageMagick or GraphicsMagick. It's generally easier to set up ImageMagick, so we will go ahead with that:

brew install imagemagick

As ImageMagick depends on Ghostscript fonts, we will also install that:

brew install ghostscript

Docker

Docker is a platform for developing, shipping, and running applications. You can install it from the official website.

Database and Authentication

This part is still quite janky, so please bear with us.

PostgreSQL (>= 16)

To get started with PostgreSQL, the easiest way is to download Postgres.app, which will help you set up everything you need.

Once installed, start the database.

Then run this command to create the coursemology and coursemology_test databases:

bundle exec rake db:setup

Don't worry about the other errors you see for now regarding keycloak.

Keycloak

Keycloack is an open-source Identity and Access Management (IAM) solution.

Build a Docker image named coursemology_auth using the Dockerfile located in the authentication directory

cd authentication
docker build -t coursemology_auth .

Copy the env file:

cp env .env

Create an empty coursemology_keycloak database in postgresql:

psql -c "CREATE DATABASE coursemology_keycloak;" -d postgres

Spin up the Keycloak server:

docker compose up

The authentication pages can be accessed via http://localhost:8443/admin

  1. Sign-in to authentication pages by inputting the following credentials:

Username: admin (whatever defined in KEYCLOAK_ADMIN inside ./.env)

Password: password (whatever defined in KEYCLOAK_ADMIN_PASSWORD inside ./.env)

  1. Navigate to coursemology realm by choosing Coursemology in the top-left dropdown box, or simply access Coursemology realm

  2. Navigate to Client, then click on the Client ID in which name is coursemology-backend

  3. Over there, navigate to Credentials and you will see the Client Secret. If whatever is defined there does not match with the Client Secret defined in your environment setup, simply copy-paste the client secret inside the page (you can possibly regenerate it if you want), then copy-paste it to KEYCLOAK_BE_CLIENT_SECRET inside ../.env

Seed the Database

Make sure the env files in the root directory and the client directory are set up correctly (copy the env files and add a dot in front of it).

cp env .env
ℹ️ Note: The client env requires and extra GOOGLE_RECAPTCHA_SITE_KEY as well. Look for a system admin to help with this.

Run the setup again, and we should not get any errors this time:

bundle exec rake db:setup
bundle exec rake coursemology:seed
RAILS_ENV=test bundle exec rake db:setup

Launch the App

To access the App after having done setting up everything above, you need to run the following command:

host lvh.me

on your terminal to bind 127.0.0.1 to lvh.me. Afterwards, you need to run 2 separate processes using 2 different terminals, each to run the Frontend and Backend, respectively. On the terminal that's supposed for the Frontend one, run the following command:

yarn build:development

and on the terminal that's supposed for the Backend one, run the following command

bundle exec rails s -p 3000

Then, you can access the App by visiting http://lvh.me:8080

Sign in to Coursemology

Email: test@example.org
Password: Coursemology!

Possible Issues and Fixes

Along the way, you might encounter certain issues. Here are some that we are aware of:

Failed to build gem native extension

If you get the following error with nokogiri/xcode-select when running bundle install --without ci:production:

failed to build gem native extension...

What you can do is to run:

sudo gem uninstall nokogiri
xcode-select –install
sudo gem install nokogiri

Check out this link for more information.

Could not find MIME type database in the following locations

If you get the following error with mimemagic when running bundle install --without ci:production:

could not find MIME type database in the following locations: ...

What you can do is to run:

brew install shared-mime-info
sudo gem install mimemagic

Check out this link for more information.

Failure to create the database

Sometimes while you tried to run either bundle exec rake db:create or bundle exec rake db:setup, the error occurs that basically mentions the database "postgres" does not exists. Referring to this link, basically there are 2 approaches that you might consider attempting. The first one is to create the postgres db manually (which is more recommended) by using the following command

createdb postgres

And the second one is to create the database coursemology and coursemology_test through psql, as follows

psql

Then, the interactive command will appear, and then you need to type the following

<username>=# CREATE DATABASE coursemology; CREATE DATABASE coursemology_test;

Regressions

Here are some known issues during development, all of which causes are still unknown

  1. The app needs to be restarted after updating any of the controllers under app/controllers
  2. The app needs to be restarted after updating any of the view files under app/views
  3. The CI test is a little bit flaky with the end to end tests during a PR, specifically ci/circleci: test. If any test related to "jobs" does not pass try running it again.

Local Testing

Compile assests (done in coursemology/client directory):

yarn build:test

Testing rails (Parallel -- much faster):

RAILS_ENV=test bundle exec rake parallel:spec

Running rails test on specific file:

RAILS_ENV=test bundle exec rspec `path to file`

Testing frontend (done in coursemology/client directory):

yarn test

Linting (done in coursemology/client directory):

yarn lint
yarn lint-fix

Useful Commands

Keeping Ruby gems updated:

bundle install

For debugging (insert anywhere in Ruby code):

byebug
Add a custom footer
Add a custom sidebar
Clone this wiki locally