diff --git a/README.md b/README.md index 7f4e5af..05139c2 100644 --- a/README.md +++ b/README.md @@ -1 +1,110 @@ -# iudx-rs-proxy \ No newline at end of file + + +![IUDX](./docs/iudx.png) + +# iudx-resource-proxy-server + +The resource server proxy is [IUDXs](https://iudx.org.in) data discovery portal. +The consumers can access data from the resource server proxy using HTTPs. + +

+ +

+ + +## Features + +- Provides data access from available resources using standard APIs. +- Search and count APIs for searching through available data: Support for Complex (Temporal + Attribute), Temporal (Before, during, After) and Attribute searches. +- Integration with authorization server (token introspection) to serve private data as per the access control policies set by the provider +- Secure data access over TLS +- Scalable, service mesh architecture based implementation using open source components: Vert.X API framework, Elasticsearch/Postgres for database. +- Hazelcast and Zookeeper based cluster management and service discovery + + +## Prerequisites +### External Dependencies Installation + +The resource server proxy connects with various external dependencies namely +- ELK stack +- PostgreSQL +- ImmuDB + + +Find the installations of the above along with the configurations to modify the database url, port and associated credentials in the appropriate sections + [here](SETUP.md) + +## Get Started + +### Docker based +1. Install docker and docker-compose +2. Clone this repo +3. Build the images + ` ./docker/build.sh` +4. Modify the `docker-compose.yml` file to map the config file you just created +5. Start the server in production (prod) or development (dev) mode using docker-compose + ` docker-compose up prod ` + + +### Maven based +1. Install java 13 and maven +2. Use the maven exec plugin based starter to start the server + `mvn clean compile exec:java@proxy-server` + +### JAR based +1. Install java 11 and maven +2. Set Environment variables +``` +export RS_URL=https:// **Doubt** +export LOG_LEVEL=INFO +``` +3. Use maven to package the application as a JAR + `mvn clean package -Dmaven.test.skip=true` +4. 2 JAR files would be generated in the `target/` directory + - `iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar` - clustered vert.x containing micrometer metrics + - `iudx.rs.proxy-dev-0.0.1-SNAPSHOT-fat.jar` - non-clustered vert.x and does not contain micrometer metrics + +#### Running the clustered JAR + +**Note**: The clustered JAR requires Zookeeper to be installed. Refer [here](https://zookeeper.apache.org/doc/r3.3.3/zookeeperStarted.html) to learn more about how to set up Zookeeper. Additionally, the `zookeepers` key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper. + +The JAR requires 3 runtime arguments when running: + +* --config/-c : path to the config file +* --hostname/-i : the hostname for clustering +* --modules/-m : comma separated list of module names to deploy + +e.g. `java -jar target/iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar --host $(hostname) -c configs/config.json -m iudx.rs.proxy.database.DatabaseVerticle,iudx.rs.proxy.authenticator.AuthenticationVerticle +,iudx.rs.proxy.metering.MeteringVerticle,iudx.rs.proxy.database.postgres.PostgresVerticle` + +Use the `--help/-h` argument for more information. You may additionally append an `RS_JAVA_OPTS` environment variable containing any Java options to pass to the application. + +e.g. +``` +$ export RS_JAVA_OPTS="-Xmx4096m" +$ java $RS_JAVA_OPTS -jar target/iudx.rs.proxy-cluster-0.0.1-SNAPSHOT-fat.jar ... +``` + +#### Running the non-clustered JAR +The JAR requires 1 runtime argument when running: + +* --config/-c : path to the config file + +e.g. `java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.rs.[proxy]-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json` + +Use the `--help/-h` argument for more information. You may additionally append an `RS_JAVA_OPTS` environment variable containing any Java options to pass to the application. + +e.g. +``` +$ export RS_JAVA_OPTS="-Xmx1024m" +$ java $RS_JAVA_OPTS -jar target/iudx.rs.proxy-dev-0.0.1-SNAPSHOT-fat.jar ... +``` + +## Contributing +We follow Git Merge based workflow +1. Fork this repo +2. Create a new feature branch in your fork. Multiple features must have a hyphen separated name, or refer to a milestone name as mentioned in Github -> Projects +3. Commit to your fork and raise a Pull Request with upstream + +## License +[MIT](./LICENSE.txt) diff --git a/SETUP.md b/SETUP.md new file mode 100644 index 0000000..1006730 --- /dev/null +++ b/SETUP.md @@ -0,0 +1,155 @@ +SETUP GUIDE +---- + +This document contains the installation and configuration processes +of the external modules of each Verticle in IUDX Resource Server Proxy. + +

+ +

+ +The Resource Server Proxy connects with various external dependencies namely +- `ELK` stack : used to capture and query temporal and spatial data. +- `PostgreSQL` : used to store and query data related to + - Token Invalidation + - Subscription Status +- `ImmuDB` : used to store metering information + + +The Resource Server Proxy also connects with various DX dependencies namely +- Authorization Server : used to download the certificate for token decoding +- Catalogue Server : used to download the list of resources, access policies and query types supported on a resource. + +## Setting up ELK for IUDX Resource Server Proxy +- Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/elk) to setup ELK stack + +**Note** : Access to HTTP APIs for search functionality should be configured with TLS and RBAC privileges + +## Setting up PostgreSQL for IUDX Resource Server Proxy + +- Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/postgres) to setup PostgreSQL + +**Note** : PostgresQL database should be configured with a RBAC user having CRUD privileges + +In order to connect to the appropriate Database, required information such as databaseIP, databasePort etc. should be updated in the DatabaseVerticle modules available in [config-example.json](example-config/config-dev.json). + +**DatabaseVerticle** +``` +{ + "id": "iudx.rs.proxy.database.DatabaseVerticle", + "verticleInstances": , + "databaseIP": "localhost", + "databasePort": , + "databaseName": , + "databaseUserName": + "databasePassword": , + "poolSize": "25" +} +``` +#### Schemas for PostgreSQL tables in IUDX Resource Server Proxy +1. Token Invalidation Table Schema +``` +CREATE TABLE IF NOT EXISTS revoked_tokens +( + _id uuid NOT NULL, + expiry timestamp with time zone NOT NULL, + created_at timestamp without time zone NOT NULL, + modified_at timestamp without time zone NOT NULL, + CONSTRAINT revoke_tokens_pk PRIMARY KEY (_id) +); +``` + + +---- + +## Setting up ImmuDB for IUDX Resource Server Proxy +- Refer to the docker files available [here](https://github.com/datakaveri/iudx-deployment/blob/master/Docker-Swarm-deployment/single-node/immudb) to setup ImmuDB. + +In order to connect to the appropriate ImmuDB database, required information such as meteringDatabaseIP,meteringDatabasePort etc. should be updated in the MeteringVerticle module available in [config-example.json](example-config/config-dev.json). + +**MeteringVerticle** + +``` +{ + "id": "iudx.rs.proxy.metering.MeteringVerticle", + "verticleInstances": , + "meteringDatabaseIP": "localhost", + "meteringDatabasePort": , + "meteringDatabaseName": , + "meteringDatabaseUserName": , + "meteringDatabasePassword": , + "meteringPoolSize": +} +``` + +**Metering Table Schema** +``` +CREATE TABLE IF NOT EXISTS rsproxyauditingtable +( + id uuid NOT NULL, + api varchar NOT NULL, + userid varchar NOT NULL, + epochtime integer NOT NULL, + resourceid varchar NOT NULL, + isotime timestamp with timezone NOT NULL, + providerid varchar NOT NULL, + CONSTRAINT metering_pk PRIMARY KEY (id) +); +``` + +## Connecting with DX Catalogue Server + +In order to connect to the DX catalogue server, required information such as catServerHost, catServerPort etc. should be updated in the AuthenticationVerticle and ApiServerVerticle modules available in [config-example.json](example-config/config-dev.json). + +**AuthenticationVerticle** +``` +{ + "id": "iudx.rs.proxy.authenticator.AuthenticationVerticle", + "verticleInstances": , + "keystore": , + "keystorePassword": , + "authServerHost": , + "catServerHost": , + "catServerPort": , + "serverMode": , + "jwtIgnoreExpiry": +} +``` + +**ApiServerVerticle** +``` +{ + "id": "iudx.rs.proxy.apiserver.ApiServerVerticle", + "ssl": true, + "production": false, + "keystore": , + "keystorePassword": , + "rsAdmin": , + "httpPort": , + "verticleInstances": , + "catServerHost": , + "catServerPort": +} +``` + +## Connecting with DX Authorization Server + +In order to connect to the DX authentication server, required information such as authServerHost should be updated in the AuthenticationVerticle module available in [config-example.json](example-config/config-dev.json). + +**AuthenticationVerticle** +``` +{ + "id": "iudx.resource.server.authenticator.AuthenticationVerticle", + "verticleInstances": , + "keystore": , + "keystorePassword": , + "authServerHost": , + "catServerHost": , + "catServerPort": , + "resourceServerId": [[]], + "serverMode": , + "jwtIgnoreExpiry": +} +``` diff --git a/docs/iudx.png b/docs/iudx.png new file mode 100644 index 0000000..e2d9eb3 Binary files /dev/null and b/docs/iudx.png differ diff --git a/docs/rs_overview.png b/docs/rs_overview.png new file mode 100644 index 0000000..115945f Binary files /dev/null and b/docs/rs_overview.png differ diff --git a/example-config/config-dev.json b/example-config/config-dev.json index 68171a3..66e6bda 100644 --- a/example-config/config-dev.json +++ b/example-config/config-dev.json @@ -1,58 +1,57 @@ { - "version": "1.0", - "zookeepers": [ - "zookeeper" - ], - "clusterId": "iudx-rs-proxy-cluster", - "host": "", - "modules": [ - { - "id": "iudx.rs.proxy.apiserver.ApiServerVerticle", - "ssl": true, - "production": false, - "keystore": "", - "keystorePassword": "", - "verticleInstances": 8, - "catServerHost": "", - "catServerPort": 1, - "port": 1 - }, - { - "id": "iudx.rs.proxy.authenticator.AuthenticationVerticle", - "verticleInstances": 1, - "keystore": "", - "keystorePassword": "", - "authServerHost": "", - "catServerHost": "", - "catServerPort": 1, - "jwtIgnoreExpiry": true - }, { - "id": "iudx.rs.proxy.cache.CacheVerticle", - "verticleInstances": 1, - "catServerHost": "", - "catServerPort": 1 - }, - { - "id":"iudx.rs.proxy.database.DatabaseVerticle", - "verticleInstances": 1, - "databaseIp": "", - "databasePort": 1, - "databaseName": "", - "databaseUserName": "", - "databasePassword": "", - "poolSize": 25 - - }, - - { - "id": "iudx.rs.proxy.metering.MeteringVerticle", - "verticleInstances": 1, - "meteringDatabaseIP": "", - "meteringDatabasePort": 1, - "meteringDatabaseName": "", - "meteringDatabaseUserName": "", - "meteringDatabasePassword": "", - "meteringPoolSize": 25 - } - ] + "version": "1.0", + "zookeepers": [ + "zookeeper" + ], + "clusterId": "iudx-rs-proxy-cluster", + "host": "", + "modules": [ + { + "id": "iudx.rs.proxy.apiserver.ApiServerVerticle", + "ssl": true, + "production": false, + "keystore": "", + "keystorePassword": "", + "verticleInstances": 8, + "catServerHost": "", + "catServerPort": 1, + "port": 1 + }, + { + "id": "iudx.rs.proxy.authenticator.AuthenticationVerticle", + "verticleInstances": 1, + "keystore": "", + "keystorePassword": "", + "authServerHost": "", + "catServerHost": "", + "catServerPort": 1, + "jwtIgnoreExpiry": true + }, + { + "id": "iudx.rs.proxy.cache.CacheVerticle", + "verticleInstances": 1, + "catServerHost": "", + "catServerPort": 1 + }, + { + "id": "iudx.rs.proxy.database.DatabaseVerticle", + "verticleInstances": 1, + "databaseIp": "", + "databasePort": 1, + "databaseName": "", + "databaseUserName": "", + "databasePassword": "", + "poolSize": 25 + }, + { + "id": "iudx.rs.proxy.metering.MeteringVerticle", + "verticleInstances": 1, + "meteringDatabaseIP": "", + "meteringDatabasePort": 1, + "meteringDatabaseName": "", + "meteringDatabaseUserName": "", + "meteringDatabasePassword": "", + "meteringPoolSize": 25 + } + ] }