Split docs about Postgres hosting providers

docs/docs/changelog.md

@@ -80,7 +80,7 @@ keywords: ['Bemi Changelog', 'Bemi New Features', 'Postgres Audit Trails', 'Chan
   * Allow saving information about a PostgreSQL user who made data changes in the app context
 * Integrations
   * [Supabase](https://supabase.com/partners/integrations/bemi): new partnership integration
-  * [GCP Cloud SQL](https://docs.bemi.io/postgresql/source-database#gcp-cloud-sql): describe how to enable logical decoding and connect
+  * [GCP Cloud SQL](/hosting/gcp): describe how to enable logical decoding and connect
   * [PowerSync](https://www.powersync.com): make ingestion worker compatible with a separate PostgreSQL replication
 
 ## 2024-04
@@ -110,7 +110,7 @@ keywords: ['Bemi Changelog', 'Bemi New Features', 'Postgres Audit Trails', 'Chan
   * Add new helper methods for diffing and sorting changes
   * Filter out sensitive logs
 * Integrations
-  * [Render](https://docs.bemi.io/postgresql/source-database#render): create a dedicated integration runbook with their support
-  * [Neon](https://docs.bemi.io/postgresql/source-database#neon): describe how to integrate by using their new Logical Replication beta feature
-  * [Supabase](https://docs.bemi.io/postgresql/source-database#supabase): improve IPv6 error handling
+  * [Render](/hosting/render): create a dedicated integration runbook with their support
+  * [Neon](/hosting/neon): describe how to integrate by using their new Logical Replication beta feature
+  * [Supabase](/hosting/supabase): improve IPv6 error handling
   * [AWS DMS](https://aws.amazon.com/dms/): make ingestion worker compatible with DMS' logical replication decoding

docs/docs/home.md

@@ -50,11 +50,12 @@ This allows automatically enhancing low-level database changes with application-
 
 ## Supported ORMs
 
-#### Node.js
+#### JavaScript/TypeScript
 
 * **[Prisma](/orms/prisma)**
-* **[Supabase JS](/orms/supabase-js)**
 * **[TypeORM](/orms/typeorm)**
+* **[Supabase JS](/orms/supabase-js)**
+* **[MikroORM](/orms/mikro-orm)**
 
 #### Ruby
 

docs/docs/hosting/aws.md

@@ -0,0 +1,124 @@
+---
+title: Bemi and AWS RDS Integration - Audit Trail and Data Tracking
+sidebar_label: AWS RDS
+hide_title: true
+description: Learn how to configure your AWS RDS database with Bemi for real-time data tracking using Change Data Capture (CDC). Includes detailed setup instructions for connections and WAL levels.
+keywords: [Bemi, AWS RDS, PostgreSQL, Change Data Capture, real-time data tracking, audit trail, WAL, logical replication]
+image: 'img/social-card.png'
+---
+
+# AWS RDS
+
+## WAL level
+
+At a high level, these are the steps necessary to update the WAL level from `replica` to `logical`
+
+1. Create an RDS parameter group if it doesn’t exist
+2. Update `rds.logical_replication` parameter from 0 to 1
+3. Apply the parameter group to your RDS instance and restart it
+
+Now let's break down these steps.
+
+Create an RDS parameter group if it doesn’t exist by choose the group family depending on your PostgreSQL version and specifying any name and description:
+
+![](/img/wal_level-aws-1.png)
+
+Edit the created parameter group:
+
+![](/img/wal_level-aws-2.png)
+
+Find and change the `rds.logical_replication` parameter from 0 to 1:
+
+![](/img/wal_level-aws-3.png)
+
+Find and modify your RDS instance by using the parameter group:
+
+![](/img/wal_level-aws-4.png)
+
+Apply the modification by restarting your RDS instance:
+
+![](/img/wal_level-aws-5.png)
+
+If you have a Multi-AZ database cluster and you used a cluster parameter group, you will need to explicitly Reboot the Writer instance (it may take ~ 2 seconds if there is not a lot of activity).
+The Reader endpoint will continue to be available without downtime.
+
+![](/img/wal_level-writer-reboot.png)
+
+See the [AWS RDS user guides](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html) to learn more about parameter groups.
+
+## Connection
+
+You can specify the same regular database credentials you use to connect to PostgreSQL from your code.
+And that's it, everything should just work!
+
+## Read-only credentials
+
+Alternatively, you can manually create read-only PostgreSQL database credentials to connect to the primary instance's WAL.
+At a high level, you need to run these commands that are safe to execute without any downtime or performance issues:
+
+* `CREATE ROLE` creates a new read-only user for Bemi to read database changes.
+* `CREATE PUBLICATION` creates a "channel" that we'll subscribe to and track changes in real-time.
+* `REPLICA IDENTITY FULL` enhances records stored in WAL to record the previous state (“before”) in addition to the tracked by default new state (“after”).
+
+```sql
+-- Create read-only user
+CREATE ROLE [username] WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION PASSWORD '[password]';
+-- Grant RDS replication permission
+GRANT rds_replication TO [username];
+-- Grant SELECT access to existing tables for selective tracking
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO [username];
+-- Grant SELECT access to new tables created in the future for selective tracking
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO [username];
+
+-- Create "bemi" PUBLICATION to enable logical replication
+CREATE PUBLICATION bemi FOR ALL TABLES;
+
+-- Create a procedure to set REPLICA IDENTITY FULL for tables to track the "before" state on DB row changes
+CREATE OR REPLACE PROCEDURE _bemi_set_replica_identity() AS $$ DECLARE current_tablename TEXT;
+BEGIN
+  FOR current_tablename IN SELECT tablename FROM pg_tables LEFT JOIN pg_class ON relname = tablename WHERE schemaname = 'public' AND relreplident != 'f' LOOP
+    EXECUTE format('ALTER TABLE %I REPLICA IDENTITY FULL', current_tablename);
+  END LOOP;
+END $$ LANGUAGE plpgsql;
+-- Call the created procedure
+CALL _bemi_set_replica_identity();
+-- Create a trigger function that calls the created procedure
+CREATE OR REPLACE FUNCTION _bemi_set_replica_identity_func() RETURNS event_trigger AS $$
+BEGIN CALL _bemi_set_replica_identity(); END $$ LANGUAGE plpgsql;
+-- Create a trigger to set REPLICA IDENTITY FULL for all new created tables
+CREATE EVENT TRIGGER _bemi_set_replica_identity_trigger ON ddl_command_end WHEN TAG IN ('CREATE TABLE')
+EXECUTE FUNCTION _bemi_set_replica_identity_func();
+```
+
+## Read-only credentials with manually managed permissions for each table
+
+Run the following queries if you want to isolate read access only to logical replication for certain tables and manage permissions manually
+instead of relying on our robust built-in selective tracking manageable through our UI.
+
+```sql
+-- Create read-only user
+CREATE ROLE [username] WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE NOREPLICATION PASSWORD '[password]';
+-- Grant replication permission to allow using replication slots
+GRANT rds_replication TO [username];
+
+-- Create "bemi" PUBLICATION to enable logical replication for selected tables
+CREATE PUBLICATION bemi FOR TABLE [table1], [table2];
+
+-- Set REPLICA IDENTITY FULL for tables to track the "before" state on DB row changes
+ALTER TABLE [table1] REPLICA IDENTITY FULL;
+ALTER TABLE [table2] REPLICA IDENTITY FULL;
+```
+
+To enable data change tracking for a new table:
+
+```sql
+ALTER PUBLICATION bemi ADD TABLE [table3];
+ALTER TABLE [table3] REPLICA IDENTITY FULL;
+```
+
+To stop data change tracking for a table:
+
+```sql
+ALTER PUBLICATION bemi DROP TABLE [table3];
+ALTER TABLE [table3] REPLICA IDENTITY DEFAULT;
+```

docs/docs/hosting/digitalocean.md

@@ -0,0 +1,14 @@
+---
+title: Bemi and DigitalOcean Integration - Audit Trail and Data Tracking
+sidebar_label: DigitalOcean
+hide_title: true
+description: Learn how to configure your DigitalOcean database with Bemi for real-time data tracking using Change Data Capture (CDC). Includes detailed setup instructions for connections and WAL levels.
+keywords: [Bemi, DigitalOcean, PostgreSQL, Change Data Capture, real-time data tracking, audit trail, WAL, logical replication]
+image: 'img/social-card.png'
+---
+
+# DigitalOcean
+
+Navigate to the [DigitalOcean databases](https://cloud.digitalocean.com/databases) tab and specify your database credentials, which can be found in the Connection details:
+
+![](/img/perm-digitalocean.png)

docs/docs/hosting/gcp.md

@@ -0,0 +1,22 @@
+---
+title: Bemi and GCP Cloud SQL Integration - Audit Trail and Data Tracking
+sidebar_label: GCP Cloud SQL
+hide_title: true
+description: Learn how to configure your GCP Cloud SQL database with Bemi for real-time data tracking using Change Data Capture (CDC). Includes detailed setup instructions for connections and WAL levels.
+keywords: [Bemi, GCP Cloud SQL, PostgreSQL, Change Data Capture, real-time data tracking, audit trail, WAL, logical replication]
+image: 'img/social-card.png'
+---
+
+# GCP Cloud SQL
+
+## WAL level
+
+Logical replication is turned off by default. To turn it on, you have to update the [cloud flag](https://cloud.google.com/sql/docs/postgres/replication/configure-logical-replication#configure-your-postgresql-instance): `cloudsql.logical_decoding` = `on`. This will need a restart of your instance before `SHOW wal_level;` returns `logical`.
+
+## Connection
+
+Run the below command and then you can connect with the same credentials on the Bemi dashboard!
+```sql
+-- Grant replication permission to allow using replication slots
+ALTER USER [user] WITH REPLICATION;
+```

docs/docs/hosting/neon.md

@@ -0,0 +1,62 @@
+---
+title: Bemi and Neon Integration - Audit Trail and Data Tracking
+sidebar_label: Neon
+hide_title: true
+description: Learn how to configure your Neon database with Bemi for real-time data tracking using Change Data Capture (CDC). Includes detailed setup instructions for connections and WAL levels.
+keywords: [Bemi, Neon, PostgreSQL, Change Data Capture, real-time data tracking, audit trail, WAL, logical replication]
+image: 'img/social-card.png'
+---
+
+# Neon
+
+## WAL level
+
+To enable logical replication in [Neon](https://neon.tech/):
+
+1. Select your project in the Neon Console.
+2. On the Neon **Dashboard**, select **Settings**.
+3. Select **Beta**.
+4. Click **Enable** to enable logical replication. This will set the Postgres `wal_level` setting to `logical`.
+
+## Connection
+
+To connect a [Neon](https://neon.tech/docs/guides/bemi) Postgres database, specify your database credentials, which can be found on your Neon project's dashboard:
+
+**Note:** Please use the `Host` name without enabling the "Pooled connection" option.
+
+![](/img/perm-neon.png)
+
+And that's it, everything should just work!
+
+For a detailed setup guide, see [Create an automatic audit trail with Bemi and Neon](https://neon.tech/docs/guides/bemi), in the _Neon documentation_.
+
+## Read-only credentials
+
+Alternatively, you can manually create read-only PostgreSQL database credentials to connect to the primary instance's WAL.
+At a high level, you need to run these commands that are safe to execute without any downtime or performance issues:
+
+* `CREATE ROLE` creates a new read-only user for Bemi to read database changes.
+* `CREATE PUBLICATION` creates a "channel" that we'll subscribe to and track changes in real-time.
+* `REPLICA IDENTITY FULL` enhances records stored in WAL to record the previous state (“before”) in addition to the tracked by default new state (“after”).
+
+```sql
+-- Create read-only user with REPLICATION permission
+CREATE ROLE [username] WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE REPLICATION PASSWORD '[password]';
+-- Grant SELECT access to tables for selective tracking
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO [username];
+-- Grant SELECT access to new tables created in the future for selective tracking
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO [username];
+
+-- Create "bemi" PUBLICATION to enable logical replication
+CREATE PUBLICATION bemi FOR ALL TABLES;
+
+-- Create a procedure to set REPLICA IDENTITY FULL for tables to track the "before" state on DB row changes
+CREATE OR REPLACE PROCEDURE _bemi_set_replica_identity() AS $$ DECLARE current_tablename TEXT;
+BEGIN
+  FOR current_tablename IN SELECT tablename FROM pg_tables LEFT JOIN pg_class ON relname = tablename WHERE schemaname = 'public' AND relreplident != 'f' LOOP
+    EXECUTE format('ALTER TABLE %I REPLICA IDENTITY FULL', current_tablename);
+  END LOOP;
+END $$ LANGUAGE plpgsql;
+-- Call the created procedure
+CALL _bemi_set_replica_identity();
+```

docs/docs/hosting/render.md

@@ -0,0 +1,38 @@
+---
+title: Bemi and Render Integration - Audit Trail and Data Tracking
+sidebar_label: Render
+hide_title: true
+description: Learn how to configure your Render database with Bemi for real-time data tracking using Change Data Capture (CDC). Includes detailed setup instructions for connections and WAL levels.
+keywords: [Bemi, Render, PostgreSQL, Change Data Capture, real-time data tracking, audit trail, WAL, logical replication]
+image: 'img/social-card.png'
+---
+
+# Render
+
+## WAL level
+
+Please submit a Render support request, and they'll run a special runbook to set up Bemi:
+
+> In a few words, what can we help you with?
+
+```
+Configure database for Bemi
+```
+
+> Describe the issue in more detail.
+
+```
+- Set "wal_level" to "logical"
+- Add "REPLICATION" permission to the database user
+- Create "bemi" publication
+```
+
+## Connection
+
+To connect a [Render](https://render.com/) database, specify your database credentials that can be found on the Render dashboard:
+
+* Please use the full `Host` name that ends with `.render.com` from the External Database URL section
+
+![](/img/perm-render.png)
+
+*Note that you can't create new credentials with `REPLICATION` permissions in Render.*

docs/docs/hosting/self-managed.md

@@ -0,0 +1,93 @@
+---
+title: Bemi and Self-Managed Integration - Audit Trail and Data Tracking
+sidebar_label: Self-Managed
+hide_title: true
+description: Learn how to configure your self-managed database with Bemi for real-time data tracking using Change Data Capture (CDC). Includes detailed setup instructions for connections and WAL levels.
+keywords: [Bemi, Self-Managed, PostgreSQL, Change Data Capture, real-time data tracking, audit trail, WAL, logical replication]
+image: 'img/social-card.png'
+---
+
+# Self-managed PostgreSQL
+
+## WAL level
+
+Run the following SQL command to change the WAL level from `replica` to `logical` and restart your database:
+
+```sql
+ALTER SYSTEM SET wal_level = logical;
+```
+
+If you have issues in other PostgreSQL hosting environments, please [contact us](https://bemi.io/contact-us), and we will send you detailed instructions on how to set it up.
+
+## Connection
+
+You can specify the same regular database credentials you use to connect to PostgreSQL from your code.
+And that's it, everything should just work!
+
+## Read-only credentials
+
+Alternatively, you can manually create read-only PostgreSQL database credentials to connect to the primary instance's WAL.
+At a high level, you need to run these commands that are safe to execute without any downtime or performance issues:
+
+* `CREATE ROLE` creates a new read-only user for Bemi to read database changes.
+* `CREATE PUBLICATION` creates a "channel" that we'll subscribe to and track changes in real-time.
+* `REPLICA IDENTITY FULL` enhances records stored in WAL to record the previous state (“before”) in addition to the tracked by default new state (“after”).
+
+```sql
+-- Create read-only user with REPLICATION permission
+CREATE ROLE [username] WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE REPLICATION PASSWORD '[password]';
+-- Grant SELECT access to tables for selective tracking
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO [username];
+-- Grant SELECT access to new tables created in the future for selective tracking
+ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO [username];
+
+-- Create "bemi" PUBLICATION to enable logical replication
+CREATE PUBLICATION bemi FOR ALL TABLES;
+
+-- Create a procedure to set REPLICA IDENTITY FULL for tables to track the "before" state on DB row changes
+CREATE OR REPLACE PROCEDURE _bemi_set_replica_identity() AS $$ DECLARE current_tablename TEXT;
+BEGIN
+  FOR current_tablename IN SELECT tablename FROM pg_tables LEFT JOIN pg_class ON relname = tablename WHERE schemaname = 'public' AND relreplident != 'f' LOOP
+    EXECUTE format('ALTER TABLE %I REPLICA IDENTITY FULL', current_tablename);
+  END LOOP;
+END $$ LANGUAGE plpgsql;
+-- Call the created procedure
+CALL _bemi_set_replica_identity();
+-- Create a trigger function that calls the created procedure
+CREATE OR REPLACE FUNCTION _bemi_set_replica_identity_func() RETURNS event_trigger AS $$
+BEGIN CALL _bemi_set_replica_identity(); END $$ LANGUAGE plpgsql;
+-- Create a trigger to set REPLICA IDENTITY FULL for all new created tables
+CREATE EVENT TRIGGER _bemi_set_replica_identity_trigger ON ddl_command_end WHEN TAG IN ('CREATE TABLE')
+EXECUTE FUNCTION _bemi_set_replica_identity_func();
+```
+
+## Read-only credentials with manually managed permissions for each table
+
+Run the following queries if you want to isolate read access only to logical replication for certain tables and manage permissions manually
+instead of relying on our robust built-in selective tracking manageable through our UI.
+
+```sql
+-- Create read-only user with REPLICATION permission
+CREATE ROLE [username] WITH LOGIN NOSUPERUSER NOCREATEDB NOCREATEROLE REPLICATION PASSWORD '[password]';
+
+-- Create "bemi" PUBLICATION to enable logical replication for selected tables
+CREATE PUBLICATION bemi FOR TABLE [table1], [table2];
+
+-- Set REPLICA IDENTITY FULL for tables to track the "before" state on DB row changes
+ALTER TABLE [table1] REPLICA IDENTITY FULL;
+ALTER TABLE [table2] REPLICA IDENTITY FULL;
+```
+
+To enable data change tracking for a new table:
+
+```sql
+ALTER PUBLICATION bemi ADD TABLE [table3];
+ALTER TABLE [table3] REPLICA IDENTITY FULL;
+```
+
+To stop data change tracking for a table:
+
+```sql
+ALTER PUBLICATION bemi DROP TABLE [table3];
+ALTER TABLE [table3] REPLICA IDENTITY DEFAULT;
+```