-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #12 from dittybijil/main
Approved the pull request.
- Loading branch information
Showing
17 changed files
with
1,704 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# when testing in this repo, we might have surveilr downloaded | ||
surveilr | ||
surveilr.exe | ||
|
||
# when testing in this repo, we might have created default RSSDs | ||
resource-surveillance.sqlite.db | ||
|
||
# when testing in this repo, we might have ingest samples / exemplars | ||
ingest/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,286 @@ | ||
# Direct Messaging Service | ||
|
||
This project is designed to facilitate the secure exchange of clinical data | ||
using the phiMail service. The project comprises four main modules, each serving | ||
a distinct purpose within the overall system. These modules work together to | ||
send, receive, process, and analyze clinical messages in a secure and efficient | ||
manner. | ||
|
||
# PhiMail Overview | ||
|
||
PhiMail is a secure messaging service that facilitates the exchange of protected | ||
health information (PHI) in compliance with healthcare regulations like HIPAA. | ||
It is designed to support healthcare providers, organizations, and other | ||
stakeholders in securely transmitting health-related data, ensuring both privacy | ||
and integrity. | ||
|
||
# Key Features | ||
|
||
Compliance with HIPAA: PhiMail ensures that all communications involving PHI | ||
meet the stringent requirements of the Health Insurance Portability and | ||
Accountability Act (HIPAA). This includes encryption of data in transit, secure | ||
storage, and proper handling of sensitive information. | ||
|
||
DIRECT Protocol Support: PhiMail is built on the DIRECT protocol, a standardized | ||
method for secure email communication in healthcare. This enables seamless and | ||
secure transmission of health information between different systems and | ||
organizations, such as hospitals, laboratories, and physician offices. | ||
|
||
Encryption and Security: PhiMail utilizes robust encryption mechanisms to | ||
protect data, ensuring that only authorized parties can access the information. | ||
This includes the use of SSL/TLS for secure connections and digital signatures | ||
to verify the authenticity of messages. | ||
|
||
Interoperability: PhiMail supports interoperability with various healthcare | ||
systems, enabling the exchange of data in different formats like CDA (Clinical | ||
Document Architecture). This allows for smooth integration with electronic | ||
health records (EHR) systems and other health information exchanges (HIEs). | ||
|
||
Message Tracking and Acknowledgments: PhiMail provides detailed tracking of | ||
message delivery, including delivery status and read receipts. This is crucial | ||
in healthcare settings, where confirmation of receipt and action on | ||
health-related messages is often required. | ||
|
||
Scalability and Customization: PhiMail can be tailored to the specific needs of | ||
healthcare organizations, whether they require a standalone messaging solution | ||
or integration with existing systems. It is scalable to accommodate the needs of | ||
small practices as well as large healthcare networks. | ||
|
||
Secure Attachments: PhiMail supports the secure exchange of attachments, such as | ||
medical records, lab results, and other documents. These attachments are | ||
encrypted and can be managed securely within the platform. | ||
|
||
# Use Cases | ||
|
||
Clinical Communication: Facilitates the exchange of medical records, referrals, | ||
and other critical health information between providers, ensuring timely and | ||
secure communication. | ||
|
||
Laboratory Results: Enables labs to send results directly to providers, | ||
integrating seamlessly with EHR systems for quick access. | ||
|
||
Patient Communication: Allows healthcare providers to securely send and receive | ||
information directly with patients, such as test results, care instructions, or | ||
appointment reminders. | ||
|
||
# Message Sender Module | ||
|
||
The Message Sender Module is responsible for sending direct messages containing | ||
clinical data. This module ensures that the messages are securely transmitted to | ||
the intended recipients using the phiMail service. | ||
|
||
![Sender](assets/phi-mail-sender.drawio.svg) | ||
|
||
# Message Receiving Module | ||
|
||
The Message Receiving Module handles the secure receipt of direct messages from | ||
various direct messaging service providers. It ensures that the incoming | ||
messages are appropriately processed and stored for further analysis. | ||
|
||
![Receiver](assets/phi-mail-receiver.drawio.svg) | ||
|
||
# `surveilr` Direct Messaging Service Patterns | ||
|
||
- `stateless-dms.surveilr.sql` script focuses on creating views that define how | ||
to extract and present specific direct messaging data from the | ||
`uniform_resource.content` JSONB column. It does not modify or store any | ||
persistent data; it only sets up views for querying. | ||
- `orchestrate-stateful-dms.surveilr.sql` script is responsible for creating | ||
tables that cache data extracted by views. These tables serve as "materialized | ||
views", allowing for faster access to the data but are static. When new data | ||
is ingested, the tables need to be dropped and recreated manually, and any | ||
changes in the source data will not be reflected until the tables are | ||
refreshed. | ||
|
||
## Try it out on any device without this repo (if you're just using the SQL scripts) | ||
|
||
Prepare the directory with sample files, download direct message inbox samples, | ||
download surveilr, and create resource-surveillance.sqlite.db RSSD file that | ||
will contain queryable direct message inbox content data and attachments | ||
including pdf and CCDA xml files. | ||
|
||
```bash | ||
# prepare a working directory | ||
$ mkdir -p /tmp/direct-messages | ||
$ cd /tmp/direct-messages | ||
``` | ||
|
||
Prepare the sample files for ingestion | ||
|
||
```bash | ||
# download the sample direct messages zip file using the below command. | ||
$ wget https://github.com/surveilr/www.surveilr.com/raw/main/pattern/direct-messaging-service/ingest.zip | ||
|
||
``` | ||
|
||
Extract the zip file | ||
|
||
```bash | ||
$ unzip ./ingest.zip | ||
``` | ||
|
||
Once unzipped, you should see the sample files in the ingest folder. The | ||
'direct-messages' directory structure should look like this: | ||
|
||
```bash | ||
direct-messages | ||
├── ingest | ||
├── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_sample.xml | ||
├── 20240808171534044_messageDeliveryStatus.json | ||
├── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_content.json | ||
└── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_sample.pdf | ||
``` | ||
|
||
Now | ||
[Download `surveilr` binary](https://docs.opsfolio.com/surveilr/how-to/installation-guide/) | ||
into 'direct-messages' directory, then ingest and query the data: | ||
|
||
```bash | ||
# ingest the files in the "ingest/" directory, creating resource-surveillance.sqlite.db | ||
$ ./surveilr ingest files -r ingest/ | ||
``` | ||
|
||
After ingestion your directory structure should look like this | ||
|
||
``` | ||
direct-messages | ||
├── ingest | ||
│ ├── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_sample.xml | ||
| ├── 20240808171534044_messageDeliveryStatus.json | ||
│ ├── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_content.json | ||
│ └── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_sample.pdf | ||
└── resource-surveillance.sqlite.db # SQLite database | ||
``` | ||
|
||
Post-ingestion, `surveilr` is no longer required, the `ingest` directory can be | ||
ignored, only `sqlite3` is required because all content is in the | ||
`resource-surveillance.sqlite.db` SQLite database which does not require any | ||
other dependencies. | ||
|
||
```bash | ||
# use SQLPage to preview content (be sure `deno` v1.40 or above is installed) | ||
$ surveilr shell https://surveilr.com/lib/pattern/direct-messaging-service/package.sql | ||
|
||
$ surveilr web-ui --port 9000 | ||
# launch a browser and go to http://localhost:9000/dms/index.sql | ||
``` | ||
|
||
Once you ingest all the files using `surveilr`, apply | ||
`orchestrate-stateful-dms.surveilr.sql` and `stateless-dms.surveilr.sql` all | ||
content will be accessed through views or `*.cached` tables in | ||
`resource-surveillance.sqlite.db`. | ||
|
||
At this point you can rename the SQLite database file, archive it, use in | ||
reporting tools, DBeaver, DataGrip, or any other SQLite data access tools. | ||
|
||
## Try it out in this repo (if you're developing SQL scripts) | ||
|
||
### Clone the Repository | ||
|
||
- Clone the repository containing the necessary scripts: | ||
```bash | ||
git clone https://github.com/surveilr/www.surveilr.com.git | ||
``` | ||
|
||
### Change to the Cloned Directory | ||
|
||
- After cloning, navigate to the repository folder: | ||
```bash | ||
cd lib/pattern//direct-messaging-service | ||
``` | ||
|
||
### Prepare the Sample Files for Ingestion | ||
|
||
wget https://github.com/surveilr/www.surveilr.com/raw/main/pattern/direct-messaging-service/ingest.zip | ||
|
||
#### Extract the Zip File | ||
|
||
Before beginning the ingestion process, extract the sample files from the zip | ||
archive named `ingest.zip`. | ||
|
||
### Verify the Directory Structure | ||
|
||
Once unzipped, you should see the sample files in the ingest folder. The | ||
'direct-messaging-service' directory structure should look like this: | ||
|
||
``` | ||
. | ||
├── ingest | ||
│ ├── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_sample.xml | ||
| ├── 20240808171534044_messageDeliveryStatus.json | ||
│ ├── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_content.json | ||
│ └── 00000191-31c8-a179-d9f7-ae67ed7c3b80_20240808171502355_sample.pdf | ||
├── orchestrate-stateful-dms.surveilr.sql | ||
├── package.sql.ts | ||
└── stateless-dms.surveilr.sql | ||
``` | ||
Now | ||
[Download `surveilr` binary](https://docs.opsfolio.com/surveilr/how-to/installation-guide/) | ||
into 'direct-messaging-service' directory, then ingest and query the data: | ||
```bash | ||
# ingest the files in the "ingest/" directory, creating resource-surveillance.sqlite.db | ||
$ ./surveilr ingest files -r ingest/ | ||
``` | ||
|
||
After ingestion, you will only work with these files: | ||
|
||
``` | ||
├── orchestrate-stateful-dms.surveilr.sql | ||
├── stateless-dms.surveilr.sql | ||
└── resource-surveillance.sqlite.db # SQLite database | ||
``` | ||
|
||
Post-ingestion, `surveilr` is no longer required, the `ingest` directory can be | ||
ignored, only `sqlite3` is required because all content is in the | ||
`resource-surveillance.sqlite.db` SQLite database which does not require any | ||
other dependencies. | ||
|
||
```bash | ||
# load the "Console" and other menu/routing utilities plus FHIR Web UI (both are same, just run one) | ||
$ deno run -A ./package.sql.ts | surveilr shell # option 1 (same as option 2) | ||
$ surveilr shell ./package.sql.ts # option 2 (same as option 1) | ||
|
||
# if you want to start surveilr embedded SQLPage in "watch" mode to re-load files automatically | ||
$ ../../universal/sqlpagectl.ts dev --watch . --watch ../../std | ||
# browse http://localhost:9000/ to see web UI | ||
|
||
# if you want to start a standalone SQLPage in "watch" mode to re-load files automatically | ||
$ ../../universal/sqlpagectl.ts dev --watch . --watch ../../std --standalone | ||
# browse http://localhost:9000/ to see web UI | ||
|
||
# browse http://localhost:9000/dms/info-schema.sql to see DMS-specific | ||
``` | ||
|
||
Once you apply `orchestrate-stateful-dms.surveilr.sql` and | ||
`stateless-dms.surveilr.sql` you can ignore those files and all content will be | ||
accessed through views or `*.cached` tables in | ||
`resource-surveillance.sqlite.db`. At this point you can rename the SQLite | ||
database file, archive it, use in reporting tools, DBeaver, DataGrip, or any | ||
other SQLite data access tools. | ||
|
||
## Automatically reloading SQL when it changes | ||
|
||
On sandboxes during development and editing of `.sql` or `.sql.ts` you may want | ||
to automatically re-load the contents into SQLite regularly. Since it can be | ||
time-consuming to re-run the same command in the CLI manually each time a file | ||
changes, you can use _watch mode_ instead. | ||
|
||
See: [`sqlpagectl.ts`](../../universal/sqlpagectl.ts). | ||
|
||
## How to Run the Tests | ||
|
||
To execute test and ensure that `surveilr` is functioning correctly: | ||
|
||
1. Run the tests using Deno: | ||
|
||
```bash | ||
deno test -A # Executes test | ||
``` | ||
|
||
This process will create an 'assurance' folder, where you can find the files | ||
related to the test, including the database and ingestion folder | ||
|
||
The `-A` flag provides all necessary permissions for the tests to run, including | ||
file system access and network permissions. |
1 change: 1 addition & 0 deletions
1
lib/pattern/direct-messaging-service/assets/phi-mail-receiver.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.