From 9563ce5fac671e87a488dce0b6ba7c8ae5a7297e Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:00:06 -0500 Subject: [PATCH 1/8] Merge WebAPI.wiki --- CDM-Configuration.md | 6 +++--- PostgreSQL-Installation-Guide.md | 2 +- Roadmap.md | 13 +------------ Security-Configuration.md | 2 ++ WebAPI-Installation-Guide.md | 16 +++++++++++++--- 5 files changed, 20 insertions(+), 19 deletions(-) diff --git a/CDM-Configuration.md b/CDM-Configuration.md index 40bcf37..b2b4dab 100644 --- a/CDM-Configuration.md +++ b/CDM-Configuration.md @@ -8,7 +8,7 @@ This guide assumes that you have installed WebAPI which will create the necessar ## Overview -The WebAPI database created in the [WebAPI Installation Guide](WebAPI-Installation-Guide.md) allows you to configure one or more CDMs to use for performing various analyses through WebAPI. WebAPI's database contains two tables that hold this configuration: `source` and `source_daimon`. The JDBC connections to the CDM(s) are held in the `source` table and defines the connection to the CDM database. This source entry is then related to the `source_daimons` table using the `source_id` and is used to configure the schemas where the CDM, vocabulary, results and temp schemas reside. These schemas support different daimon-type functionality for WebAPI functionality. In this guide, we'll cover how to establish the additional schemas in your CDM required by WebAPI and how to configure the required `source` and `source_daimon` entries required for WebAPI to function properly. +The WebAPI database created in the [WebAPI Installation Guide](https://github.com/OHDSI/WebAPI/wiki/WebAPI-Installation-Guide) allows you to configure one or more CDMs to use for performing various analyses through WebAPI. WebAPI's database contains two tables that hold this configuration: `source` and `source_daimon`. The JDBC connections to the CDM(s) are held in the `source` table and defines the connection to the CDM database. This source entry is then related to the `source_daimons` table using the `source_id` and is used to configure the schemas where the CDM, vocabulary, results and temp schemas reside. These schemas support different daimon-type functionality for WebAPI functionality. In this guide, we'll cover how to establish the additional schemas in your CDM required by WebAPI and how to configure the required `source` and `source_daimon` entries required for WebAPI to function properly. ### Schemas @@ -59,7 +59,7 @@ The WebAPI `source` and `source_daimon` tables were created when you started the ### Example WebAPI SOURCE and SOURCE_DAIMON Inserts ```sql -INSERT INTO ohdsi.source (source_id, source_name, source_key, source_connection, source_dialect) VALUES (1, 'My Cdm', 'MY_CDM', ' jdbc:postgresql://server:5432/cdm?user={user}&password={password}', 'postgresql'); +INSERT INTO webapi.source (source_id, source_name, source_key, source_connection, source_dialect) VALUES (1, 'My Cdm', 'MY_CDM', ' jdbc:postgresql://server:5432/cdm?user={user}&password={password}', 'postgresql'); INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (1,1,0, 'cdm', 0); INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (2,1,1, 'vocab', 1); @@ -70,7 +70,7 @@ INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, tabl The above inserts creates a source with `source_id = 1` with 4 daimon entries, one for each daimon type (0 = CDM, 1 = Vocabulary, 2 = Results, 5 = TEMP). If you'd like to configure more than 1 source for use in WebAPI, repeat the steps above and increment the `source_id` used to distinguish the sources from one another. -👉 _Note: To see the new sources, open a browser and navigate to `:port/WebAPI/sources/refresh`_ +👉 _Note: To see the new sources, open a browser and navigate to `:port/WebAPI/source/refresh`_ ## Daimon Priority diff --git a/PostgreSQL-Installation-Guide.md b/PostgreSQL-Installation-Guide.md index 6063380..024e38d 100644 --- a/PostgreSQL-Installation-Guide.md +++ b/PostgreSQL-Installation-Guide.md @@ -143,7 +143,7 @@ Then follow these steps for each tabbed option: #### Tab: Properties #### -- **Name**: ohdsi +- **Name**: OHDSI - **Owner**: ohdsi_admin - **Comment**: OHDSI WebAPI database diff --git a/Roadmap.md b/Roadmap.md index bdd3cbc..f10cc5e 100644 --- a/Roadmap.md +++ b/Roadmap.md @@ -1,25 +1,14 @@ -# OHDSI WebAPI Roadmap 2019 +# OHDSI WebAPI Roadmap The [Atlas & WebAPI Working Group](http://www.ohdsi.org/web/wiki/doku.php?id=projects:workgroups:atlas-webapi) hold scheduled calls to review progress, technical roadblocks and timelines around new features or on-going issues. This roadmap is a reflection of the working group's commitments for this year. -## Atlas/WebAPI 2.7 - Bug fixes - -#### Release Date: March 2019 - -The 2.7 release will focus on addressing the following: -- Gaps from the v2.6.0 release. There were many items that the developer community didn't have an opportunity to finish for the v2.6.0 release and we're using the 2.7 release to close out the functionality gaps and focus on fixing outstanding issues. - ## Atlas/WebAPI 2.8 - Performance Optimizations -#### Planned Release Date: May 2019 - The 2.8 release will focus on optimizing features and functions in the platform. There will also be an effort to fix any outstanding bugs from the 2.7 release. 🙌 Important note: The 2.8 release will be the final release of Atlas/WebAPI that supports the CDM v5.x specification. ## Atlas/WebAPI 3.0 - CDM v6 Support -#### Planned Release Date: Q4 2019 - The 3.0 release of Atlas/WebAPI will provide support for CDM v6.x and will be a large overhaul of the code base with an eye towards maturing the platform's core components with unit/integration tests. diff --git a/Security-Configuration.md b/Security-Configuration.md index a36fe72..02a9369 100644 --- a/Security-Configuration.md +++ b/Security-Configuration.md @@ -64,6 +64,7 @@ The following settings are used to control Active Directory settings: true 30000 public + sAMAccountName ``` - **security.ad.url**: The LDAP endpoint for AD @@ -75,6 +76,7 @@ The following settings are used to control Active Directory settings: - **security.ad.ignore.partial.result.exception**: true/false - **security.ad.result.count.limit**: Limit the number of AD results to 0 means no limit - **security.ad.default.import.group**: The group to use for importing users from AD into WebAPI's configuration database. +- **security.ad.userMapping.usernameAttr**: The name of field in AD where user login is stored ### OpenID configuration diff --git a/WebAPI-Installation-Guide.md b/WebAPI-Installation-Guide.md index 21c2b92..a7a67d2 100644 --- a/WebAPI-Installation-Guide.md +++ b/WebAPI-Installation-Guide.md @@ -70,6 +70,16 @@ Create a folder (for example `C:\Git\OHDSI`) and navigate to it. From within you C:\Git\OHDSI> git clone https://github.com/OHDSI/WebAPI.git ``` +### Check out a released version of the software + +Once we've downloaded the code, we want to ensure that we are using a released version of WebAPI. This is done by performing a `git checkout` of the code at a specific tagged version. You can find the latest released version of WebAPI at https://github.com/OHDSI/WebAPI/releases and note the tag as shown in the left hand menu. For example, if I want to check out v2.7.4 I would change to the WebAPI directory and use the following command: + +``` +C:\Git\OHDSI\WebAPI> git checkout refs/tags/v2.7.4 +``` + +Where `v2.7.4` is the released version you would like to use. + ### Create settings.xml file In the root of the WebAPI project folder, there is a file named `sample_settings.xml` which 3 `` blocks for the 3 WebAPI database platforms. Copy this file into a new folder `WebAPIConfig` and rename it to `settings.xml`. **Note**: `WebAPIConfig` will be subfolder off of the root `WebAPI` folder. @@ -139,9 +149,9 @@ Update this to read: ``` - - 104857600 - 104857600 + + 154857600 + 154857600 0 ``` From ce7b6a4780657ad6f283a9175bcef245eb463716 Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:23:28 -0500 Subject: [PATCH 2/8] Update for source creation SQL --- CDM-Configuration.md | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/CDM-Configuration.md b/CDM-Configuration.md index b2b4dab..e4cb219 100644 --- a/CDM-Configuration.md +++ b/CDM-Configuration.md @@ -56,16 +56,37 @@ Once you have created the URL for your environment, open a browser and navigate The WebAPI `source` and `source_daimon` tables were created when you started the tomcat service with the WebAPI war deployed. These tables must be populated with a JDBC `source` connection and corresponding `source_daimon` that specify the location for the `cdm`, `vocabulary`, `results` and `temp` schemas associated to the source in order to use the OHDSI tools. For this example it is assumed that the CDM and Vocabulary exist as a separate schema in the same database instance. +Please note that the `source_id` must be > 0 and that the SQL below uses sequences to use the next available `source_id` and `source_daimon_id` respectively. + ### Example WebAPI SOURCE and SOURCE_DAIMON Inserts ```sql -INSERT INTO webapi.source (source_id, source_name, source_key, source_connection, source_dialect) VALUES (1, 'My Cdm', 'MY_CDM', ' jdbc:postgresql://server:5432/cdm?user={user}&password={password}', 'postgresql'); - -INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (1,1,0, 'cdm', 0); -INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (2,1,1, 'vocab', 1); -INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (3,1,2, 'results', 1); -INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) VALUES (4,1,5, 'temp', 0); - +INSERT INTO webapi.source (source_id, source_name, source_key, source_connection, source_dialect) +SELECT nextval('webapi.source_sequence'), 'My Cdm', 'MY_CDM', ' jdbc:postgresql://server:5432/cdm?user={user}&password={password}', 'postgresql'; + +INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) +SELECT nextval('webapi.source_sequence'), source_id, 0, 'cdm', 0 +FROM webapi.source +WHERE source_key = 'MY_CDM' +; + +INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) +SELECT nextval('webapi.source_sequence'), source_id, 1, 'vocab', 1 +FROM webapi.source +WHERE source_key = 'MY_CDM' +; + +INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) +SELECT nextval('webapi.source_sequence'), source_id, 2, 'results', 1 +FROM webapi.source +WHERE source_key = 'MY_CDM' +; + +INSERT INTO webapi.source_daimon (source_daimon_id, source_id, daimon_type, table_qualifier, priority) +SELECT nextval('webapi.source_sequence'), source_id, 5, 'temp', 0 +FROM webapi.source +WHERE source_key = 'MY_CDM' +; ``` The above inserts creates a source with `source_id = 1` with 4 daimon entries, one for each daimon type (0 = CDM, 1 = Vocabulary, 2 = Results, 5 = TEMP). If you'd like to configure more than 1 source for use in WebAPI, repeat the steps above and increment the `source_id` used to distinguish the sources from one another. From 07a4dcfc91f9316310631efc582e85d01c631438 Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:29:51 -0500 Subject: [PATCH 3/8] Add bearer token expiration --- Security-Configuration.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Security-Configuration.md b/Security-Configuration.md index 02a9369..6cacd9e 100644 --- a/Security-Configuration.md +++ b/Security-Configuration.md @@ -17,6 +17,7 @@ The `` settings are controlled via the settings.xml. The relevant sec ``` AtlasRegularSecurity +43200 * 8080 true @@ -24,6 +25,7 @@ The `` settings are controlled via the settings.xml. The relevant sec ``` - **security.provider**: The default is `DisabledSecurity`. To enable security in the application this value is set to `AtlasRegularSecurity`. +- **security.token.expiration**: The expiration for the security bearer token. - **security.origin**: Use `*` to allow any client to connect to the REST endpoints. This setting is used to narrow the scope of clients that are able to connect to the REST endpoints. You may set this to a specific client application (i.e. `http://ohdsi.org/web/atlas`) and WebAPI will only accept connections originating from this domain. - **server.port**: This setting specifies the port to use to listen for client connections. The default is `8080` and this value is generally switched to `443` when using a secured connection. - **security.ssl.enabled**: Set to true to enable SSL for encrypting connections to the REST endpoints. Check the [[SSL Configuration In Tomcat]] guide for more information on how to set up a server to use SSL. From 399849e8d19bace7d22896b055ccc83da4958dbb Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:33:01 -0500 Subject: [PATCH 4/8] Add link to issue with AD config details --- Security-Configuration.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Security-Configuration.md b/Security-Configuration.md index 6cacd9e..eb53772 100644 --- a/Security-Configuration.md +++ b/Security-Configuration.md @@ -80,6 +80,8 @@ The following settings are used to control Active Directory settings: - **security.ad.default.import.group**: The group to use for importing users from AD into WebAPI's configuration database. - **security.ad.userMapping.usernameAttr**: The name of field in AD where user login is stored +Some additional details around Active Directory configuration can be found in this [WebAPI GitHub Issue (#1373)](https://github.com/OHDSI/WebAPI/issues/1373). + ### OpenID configuration The following settings are used to control OpenID settings: From 0241cd1c29978f8f5b6e57b7233a82d3c23f71b9 Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:34:39 -0500 Subject: [PATCH 5/8] Add Kerberos --- Security-Configuration.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Security-Configuration.md b/Security-Configuration.md index eb53772..a461325 100644 --- a/Security-Configuration.md +++ b/Security-Configuration.md @@ -82,6 +82,10 @@ The following settings are used to control Active Directory settings: Some additional details around Active Directory configuration can be found in this [WebAPI GitHub Issue (#1373)](https://github.com/OHDSI/WebAPI/issues/1373). +### Kerberos + +Please see the setup notes included in the [WebAPI Repository](https://github.com/OHDSI/WebAPI/tree/master/src/main/java/org/ohdsi/webapi/shiro/realms). + ### OpenID configuration The following settings are used to control OpenID settings: From 0730af7244abbbec1fe561606343fa829a99f76e Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:48:39 -0500 Subject: [PATCH 6/8] Further updates to Basic Security Config --- Basic-Security-Configuration.md | 28 +++++++++++++++++----------- 1 file changed, 17 insertions(+), 11 deletions(-) diff --git a/Basic-Security-Configuration.md b/Basic-Security-Configuration.md index f8d598c..fee970d 100644 --- a/Basic-Security-Configuration.md +++ b/Basic-Security-Configuration.md @@ -12,6 +12,7 @@ The settings.xml file is used to configure your build of the OHDSI WebAPI in you ```xml AtlasRegularSecurity * +true 3 10 10 @@ -20,7 +21,7 @@ The settings.xml file is used to configure your build of the OHDSI WebAPI in you ohdsi ohdsi ohdsi -select password from ${security.db.datasource.schema}.demo_security where email = ? +select password,firstName,middleName,lastName from atlas_security.demo_security where email = ? ``` ## security.maxLoginAttempts @@ -32,25 +33,28 @@ This is the maximum number of login attempts allowed before the account is locke This represents the `initial` length of lockout and the `incremental` length of lockout in seconds. So, if there are more than `security.maxLoginAttempts`, the initial lockout time will start and for every subsequent failed login, the incremental value will be added to the total lockout time. ## database -Once you have completed the configuration of the profile for your OHDSI WebAPI you will need to create the table that will contain our sample login information. The script to create a minimal sample table in a postgresql environment is as follows: +Once you have completed the configuration of the profile for your OHDSI WebAPI you will need to create the database, schema and table that will contain our sample login information. To start, create database (eg. `Security`), and a schema within it as `atlas_security`. -```sql --- Table: ohdsi.demo_security - --- DROP TABLE ohdsi.demo_security; +The script to create a minimal sample table in a postgresql environment is as follows: -CREATE TABLE ohdsi.demo_security +```sql +CREATE TABLE atlas_security.demo_security ( - email character varying(255) COLLATE pg_catalog."default", - password character varying(255) COLLATE pg_catalog."default" + username character varying(255) COLLATE pg_catalog."default", + password character varying(255) COLLATE pg_catalog."default", + firstname character varying(255) COLLATE pg_catalog."default", + middlename character varying(255) COLLATE pg_catalog."default", + lastname character varying(255) COLLATE pg_catalog."default" ) WITH ( OIDS = FALSE ) TABLESPACE pg_default; -ALTER TABLE ohdsi.demo_security - OWNER to ohdsi; +ALTER TABLE atlas_security.demo_security +OWNER to ohdsi_app_user; + +GRANT ALL ON TABLE atlas_security.demo_security TO ohdsi_app_user WITH GRANT OPTION; ``` Next you will need to insert a sample record that will contain our demonstration username and password. The password is encrypted using BCrypt. You can create your own username and password or use the sample insert statement provided below where we have already encrypted the password 'ohdsi' for the user named 'ohdsi'. To create a different password hash using BCrypt you can use the following web site: @@ -64,6 +68,8 @@ insert into ohdsi.demo_security (email,password) values ('ohdsi', '$2a$04$Fg8TEiD2u/xnDzaUQFyiP.uoDu4Do/tsYkTUCWNV0zTCW3HgnbJjO') ``` +If you are still facing issues after following the steps in this guide, please refer to this [WebAPI Issue #1341](https://github.com/OHDSI/WebAPI/issues/1341), [WebAPI Issue #1099](https://github.com/OHDSI/WebAPI/issues/1099) or post an issue to the [WebAPI Issue Tracker](https://github.com/OHDSI/WebAPI/issues). + ## Configuring ATLAS Now that we have the OHDSI WebAPI configured, table created and populated we can now setup ATLAS to expect a secure OHDSI WebAPI. Please see the [[Atlas Security]] section for details. From c57df0addfe16124d20692c4d9d9f9a198d4e25a Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 11:56:56 -0500 Subject: [PATCH 7/8] Add hint for PG source config --- CDM-Configuration.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CDM-Configuration.md b/CDM-Configuration.md index e4cb219..4fe35c6 100644 --- a/CDM-Configuration.md +++ b/CDM-Configuration.md @@ -56,7 +56,9 @@ Once you have created the URL for your environment, open a browser and navigate The WebAPI `source` and `source_daimon` tables were created when you started the tomcat service with the WebAPI war deployed. These tables must be populated with a JDBC `source` connection and corresponding `source_daimon` that specify the location for the `cdm`, `vocabulary`, `results` and `temp` schemas associated to the source in order to use the OHDSI tools. For this example it is assumed that the CDM and Vocabulary exist as a separate schema in the same database instance. -Please note that the `source_id` must be > 0 and that the SQL below uses sequences to use the next available `source_id` and `source_daimon_id` respectively. +Please note the following: +- The `source_id` must be > 0 and that the SQL below uses sequences to use the next available `source_id` and `source_daimon_id` respectively. +- If you are configuring a PostgreSQL connection and are facing difficulities connecting, it may require the addition of the `OpenSourceSubProtocolOverride` to your `source_connection` connection string as described in [WebAPI Issue #592](https://github.com/OHDSI/WebAPI/issues/592). ### Example WebAPI SOURCE and SOURCE_DAIMON Inserts From ec60f4729f4830f721a6f655a118d04429adf99e Mon Sep 17 00:00:00 2001 From: Anthony Sena Date: Fri, 18 Dec 2020 12:03:06 -0500 Subject: [PATCH 8/8] Minor updates --- Roadmap.md | 2 -- WebAPI-Installation-Guide.md | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/Roadmap.md b/Roadmap.md index f10cc5e..d451c14 100644 --- a/Roadmap.md +++ b/Roadmap.md @@ -6,8 +6,6 @@ The [Atlas & WebAPI Working Group](http://www.ohdsi.org/web/wiki/doku.php?id=pro The 2.8 release will focus on optimizing features and functions in the platform. There will also be an effort to fix any outstanding bugs from the 2.7 release. -🙌 Important note: The 2.8 release will be the final release of Atlas/WebAPI that supports the CDM v5.x specification. - ## Atlas/WebAPI 3.0 - CDM v6 Support The 3.0 release of Atlas/WebAPI will provide support for CDM v6.x and will be a large overhaul of the code base with an eye towards maturing the platform's core components with unit/integration tests. diff --git a/WebAPI-Installation-Guide.md b/WebAPI-Installation-Guide.md index a7a67d2..9fb134e 100644 --- a/WebAPI-Installation-Guide.md +++ b/WebAPI-Installation-Guide.md @@ -75,10 +75,10 @@ C:\Git\OHDSI> git clone https://github.com/OHDSI/WebAPI.git Once we've downloaded the code, we want to ensure that we are using a released version of WebAPI. This is done by performing a `git checkout` of the code at a specific tagged version. You can find the latest released version of WebAPI at https://github.com/OHDSI/WebAPI/releases and note the tag as shown in the left hand menu. For example, if I want to check out v2.7.4 I would change to the WebAPI directory and use the following command: ``` -C:\Git\OHDSI\WebAPI> git checkout refs/tags/v2.7.4 +C:\Git\OHDSI\WebAPI> git checkout refs/tags/v2.8.0 ``` -Where `v2.7.4` is the released version you would like to use. +Where `v2.8.0` is the released version you would like to use. ### Create settings.xml file In the root of the WebAPI project folder, there is a file named `sample_settings.xml` which 3 `` blocks for the 3 WebAPI database platforms. Copy this file into a new folder `WebAPIConfig` and rename it to `settings.xml`. **Note**: `WebAPIConfig` will be subfolder off of the root `WebAPI` folder.