The following components must be installed on the development computer:
✔️ Git
✔️ Docker
✔️ Node.js 20 LTS
Follow these steps to set up the development environment on your local machine:
Configure development/.env
according to the local service configuration.
Install node modules:
npm install
Generate prisma-client for database-access:
npm run prisma -- generate
- Start the development services.
- Open http://localhost:9001
- Sign in using the
STORAGE_USER
andSTORAGE_PASSWORD
of your development environment. - Navigate to Buckets and create a new bucket with the name
asset-sg
. - Navigate to the new bucket's browser and create an empty folder with the name
asset-sg
. - Navigate to Configuration and change the server region to
local
. - Navigate to Access Keys and create a new access key.
- Open the file
apps/server-asset-sg/.env.local
and modify the following variables:- Set
S3_ACCESS_KEY_ID
to your generated access key. - Set
S3_SECRET_ACCESS_KEY
to your generated access key's secret.
- Set
Start development services:
cd development
docker compose up
Start the application:
npm run start
# Or individually:
npm run start:server
npm run start:client
🔖App/Service | 🔗Link | 🧞User | 🔐Password |
---|---|---|---|
Assets (client) | localhost:4200 | admin |
admin |
Assets REST API (server) | localhost:3333/api/ | n/a | n/a |
postgreSQL (docker) | localhost:5432 | .env $DB_USER |
.env $DB_PASSWORD |
Elasticsearch (docker) | localhost:9200 | n/a | n/a |
Kibana (docker) | localhost:5601 | n/a | n/a |
pgAdmin (docker) | localhost:5051 | .env $PGADMIN_EMAIL |
.env $PGADMIN_PASSWORD |
MinIO (docker) | localhost:9001 | .env $STORAGE_USER |
.env $STORAGE_PASSWORD |
smtp4dev (docker) | localhost:5000 | n/a | n/a |
oidc-server (docker) | localhost:4011 | n/a | n/a |
You can dump data from a remote environment into a local file so you can initialize your development database with it.
To do so, use the following commands.
Be aware that you need to manually insert the {DB_*}
values beforehand.
cd development
docker compose exec db sh -c 'pg_dump --dbname=postgresql://{DB_USERNAME}:{DB_PASSWORD}@{DB_HOST}:5432/{DB_DATABASE} --data-only --exclude-table asset_user --exclude-table _prisma_migrations -n public > /dump.sql'
The export will output warnings related to circular foreign-key constraints. These can be safely ignored.
The export will only contain the database's data, not its structure. Data related to the authentication process is also excluded, so we don't run into conflicts when using a different eIAM provider.
To import the dumped data, run the following commands. Ensure to start your database service beforehand.
# Reset the database:
npm run prisma -- migrate reset -f
npm run prisma -- migrate deploy
# Import example data:
cd development
docker compose exec db sh -c 'psql --dbname=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5432/${POSTGRES_DB} -v ON_ERROR_STOP=1 -f /dump.sql'
You will need to manually sync the data to Elasticsearch via the admin panel in the web UI.
Tests execute automatically on every push to the Git repository.
The local tests require a running instance of both postgreSQL and Elasticsearch. Make sure that your local development environment is fully shutdown and then run the test services:
cd development
docker compose down
docker compose -f docker-compose.test.yml up
Then run all tests:
npm run test
It is also possible to run only specific tests:
# Run only the server tests:
nx run server-asset-sg:test
# Run only a specific test suite:
nx run server-asset-sg:test -t 'AssetRepo'
# Run only a specific, nested test suite:
nx run server-asset-sg:test -t 'AssetRepo create'
The file apps/server-asset-sg/.env.local
configures secrets for the SwissGeol Asset server.
An empty template for the file can be found in apps/server-asset-sg/.env.template
.
Variable | Example | Description |
---|---|---|
FRONTEND_URL | http://localhost:4200 | Public URL of the SwissGeol Asset web client. |
S3_REGION | local | Region of the S3 instance. |
S3_ENDPOINT | http://localhost:9000 | URL to the S3 instance. |
S3_ACCESS_KEY_ID | AP6wpeXraSc0IH4d42IN | Access Key for the S3 instance. |
S3_SECRET_ACCESS_KEY | fSx5Bfib0OeAyG1mwtslKA04Qj6oPStLcpnkACmF | Secret Key for the S3 instance. |
S3_BUCKET_NAME | asset-sg | S3 bucket name. |
S3_ASSET_FOLDER | asset-sg | Folder within the S3 bucket into which objects are stored. |
DATABASE_URL | postgres://postgres:postgres@localhost:5432/postgres?schema=public | PostgreSQL access URL. |
OAUTH_ISSUER | http://localhost:4011 | OAuth API URL. |
OAUTH_CLIENT_ID | assets | Name of the client within the OAuth issuer. |
OAUTH_SCOPE | openid profile email cognito | The scopes requested on each OAuth login. |
OAUTH_SHOW_DEBUG_INFO | true | Whether to show debug info about the OAuth process. |
OAUTH_TOKEN_ENDPOINT | http://localhost:4011/connect/token | The endpoint at which OAuth tokens can be fetched. |
OAUTH_AUTHORIZED_GROUPS | assets.swissgeol | The name of the groups (comma-separated) which should grant access. |
OCR_URL | Leave empty. | |
OCR_CALLBACK_URL | Leave empty. |
The local docker configuration contains an OIDC container supporting OAuth. Use the example values to use it instead of an external issuer.
The file development/.env
configures secrets for the services used in local development.
An empty template for the file can be found in development/.env.template
.
Make sure that your passwords have a minimal length of 8 and contain at combination of upper, lower and special characters. Some of the passwords will be checked for validity during startup.
Variable | Wert | Beschreibung |
---|---|---|
STORAGE_USER | custom | Username for the MinIO container. |
STORAGE_PASSWORD | custom | Password for the MinIO container. |
DB_USER | postgres | Username for the PostgreSQL container. |
DB_PASSWORD | custom | Password for the PostgreSQL container. |
PGADMIN_EMAIL | custom | Email for the PgAdmin container. |
PGADMIN_PASSWORD | custom | Password for the PgAdmin container. |
This project uses Prisma as its database ORM. The schema can be found at libs/persistence/prisma/schema.prisma.
To run prisma commands, you can use the following shortcut:
npm run prisma -- {command}
To apply all new migrations to your local database, run the following command:
npm run prisma -- migrate deploy
If your local database's state can't be migrated from, you might have to fully reset your database. This can happen when manually modifying the database, and will remove all local data.
npm run prisma -- migrate reset
To create a new migration, first modify the Prisma schema. Then, create a shadow database:
docker compose exec db sh -c 'psql --dbname=postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@localhost:5432/${POSTGRES_DB} -c "CREATE DATABASE postgres_shadow;"
Afterward, you can generate the new migration:
npm run prisma -- migrate dev --create-only
You can find and modify your new migration within the migrations/
directory.
The finalized migration can be applied like any other migration:
npm run prisma -- migrate deploy
Migrations are automatically applied to any environment running the swissgeol-assets-api Docker image.