docs: migrate releases from docs.opsfoli.com

astro.config.mjs

@@ -151,6 +151,11 @@ export default defineConfig({
           label: "Cookbook and Snippets",
           items: [],
         },
+        {
+          label: "Releases",
+          autogenerate: { directory: "docs/releases" },
+          collapsed: true,
+        },
       ],
       social: {
         github: "https://github.com/surveilr",

src/content/docs/docs/releases/v0_10_0.md

@@ -0,0 +1,27 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.10.0
+sidebar:
+  label: v0.10.0
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.10.0
+
+## What's New
+1. **Orchestration models**
+    - [x] `device` – `surveilr` already has a `device` table but check to see if we need to enhance it.
+    - [x] Add a table called `orchestration_nature` with a `orchestration_nature_id` TEXT ID primary key (not UUID), required `nature` TEXT, `elaboration` JSONB columns
+        - [x] Start with a single default row with ` orchestration_nature_id` called `V&V` and `nature` called `Verification and Validation`
+        - [x] We will start with one type of “orchestration” called “V&V” and then we will add more later after V&V is done
+    - [x] `orch_session` -> use it as it but call it `orchestration_session`
+         - [x] add a text column called `orchestration_nature_id` to this table and default it to `V&V` foreign key above.
+    - [x] `orch_session_entry` -> `orchestration_session_entry`
+    - [x] `orch_session_state` -> `orchestration_session_state`
+    - [x] `orch_session_exec` -> `orchestration_session_exec`
+    - [x] `orch_session_issue` -> `orchestration_session_issue`
+
+## Bug Fixes
+- **GitLab Batch Size**: The default used to be 20; however, it will now be determined by the `--batch-size` argument provided.
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.10.0).

src/content/docs/docs/releases/v0_10_1.md

@@ -0,0 +1,49 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.10.1
+sidebar:
+  label: v0.10.1
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.10.1
+
+## What's New
+
+1. **Orchestration Command Execution**
+    - Feature: `surveilr orchestrate -n "v&v" -s xyz.sql`
+        - Description: Execute an xyz.sql script stored in a VFS filesystem locally. This feature supports orchestrating SQL scripts for validation and verification (v&v) purposes.
+         - Usage: `surveilr orchestrate -n "v&v" -s xyz.sql`
+
+2. **Script Execution Order**
+   The orchestration service now accepts multiple SQL files and guarantees they are executed in the specified order, ensuring the correct sequence of operations.
+
+3. **SQLite Orchestration Helper Functions**
+    - `surveilr_orchestration_context_session_id`
+        - Description: Adds an SQLite function to return the current active session_id.
+        - Usage:` SELECT surveilr_orchestration_context_session_id();`
+    - `device_id`
+       - Description: Adds an SQLite function to return the device_id of the host device.
+        - Usage: SELECT device_id();
+    - `surveilr_version`
+       - Description: Retrieves the current surveilr version.
+       - Usage: SELECT surveilr_version();
+    - `surveilr_orchestration_context_active`: Returns a boolean indicating if the execution is in an active context.
+    - `surveilr_orchestration_context_session_id`: Gets the active session ID, which is the latest session still in progress and does not have a corresponding `surveilr_orch_completed` state.
+
+4. **Session Management**
+  When an orchestratio session is started through the command line, it inserts a record into `orchestration_session_state` with the `from_state` column set to `surveilr_orch_init` and the `to_state` filed to`surveilr_orch _progress`, indicating the session is in progress.
+  Upon completion of execution, a new record is inserted, rather than updating the previous record, with `to_state` colums set to `surveilr_orch_completed`, marking the session as complete.
+
+5. **Logging**
+      Feature: `orchestration_session_log` Table
+      Description: Adds a new table orch_session_log to log entries with the following columns:
+      ```sql
+      category: TEXT
+      content: TEXT NOT NULL
+      sibling_order: NUMERIC
+      parent_id: ULID
+      elaboration: JSON
+      ```
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.10.1).

src/content/docs/docs/releases/v0_10_2.md

@@ -0,0 +1,41 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.10.2
+sidebar:
+  label: v0.10.2
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.10.2
+
+## What's New
+### 1. Orchestrate with Script Storage
+ - **Command**: `surveilr orchestrate  -n "v&v" -s abc.sql --save-script`
+ - **Description**: This command now stores everything from the command line into the orchestration session. If the `--save-script` flag is set to true, which is the default, the script content is stored in the elaboration field for reference.
+
+### 2. Web UI Launch and Auto-Open
+  - **Command**: `surveilr web-ui -p 9000`
+  - **Description**: The `sqlpage` subcommand has been changed to `webui`. You can now launch the web interface using the command `surveilr web-ui`, which provides the address of the webpage. Additionally, using the `--open` flag will automatically open the webpage in the default browser. Read more [here](/surveilr/reference/cli/commands#surveilr-web-ui).
+
+### 3. Handling Streaming Content from STDIN
+  - **Functionality**: If streaming content is received through STDIN, it is treated as the first script to be executed, followed by any subsequent scripts provided via the `--sql` flag.
+  - Example: `curl https://abc.com/my.sql | surveilr orchestrate -n "v&v" -s abc.sql`
+
+### 4. Execute Scripts from Remote Filesystem via HTTPs
+  - **Command**: `surveilr orchestrate -n "v&v" --sql https://github.com/abc/xyz/xyz.sql`
+  - **Description**: This command will execute a script stored in a remote filesystem via HTTPs. This feature ensures that remote scripts are fetched and executed seamlessly.
+  - Example: `surveilr orchestrate -n "v&v" --sql https://github.com/abc/xyz/xyz.sql`
+
+**Combined Example of passing SQL scripts to `surveilr` orchestration**
+```bash
+curl https://raw.githubusercontent.com/opsfolio/resource-surveillance-commons/main/pattern/privacy/anonymize-sample/de-identification/deidentification.sql \
+ | surveilr orchestrate -n "v&v" \
+-s ./support/regression-tests/orchestrate.sql -s ./support/regression-tests/deidentify.sql \
+ -s "https://raw.githubusercontent.com/opsfolio/resource-surveillance-commons/main/pattern/privacy/anonymize-sample/stateless-privacy-surveilr.sql"
+```
+  - **Description**: This command pipes a script from a remote URL into the surveilr orchestrate command and specifies additional scripts to be executed from both local and remote sources. The initial script from STDIN is executed first, followed by the scripts provided via the --sql flag.
+
+## Fixes
+- `surveilr_device_id`: This function was previously named `device_id`, but now the word `surveilr_` has been added as a prefix to denote that it is a `surveilr` specific function.
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.10.2).

src/content/docs/docs/releases/v0_11_0.md

@@ -0,0 +1,32 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.11.0
+sidebar:
+  label: v0.11.0
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.11.0
+
+## What's New
+1. **Host Binding Customization:**
+    - The `web-ui` server now accepts a `--host` argument to specify the host to which it should be bound, providing more flexibility in deployment.
+    - By default, the server binds to localhost instead of 127.0.0.1.
+
+2. **Server Start Message:**
+    - Updated the server start message to read: "RSSD Web UI is now running on ...", enhancing clarity for users monitoring server status.
+
+3. **SQLPage**
+    - Updated SQLPage to the latest version, ensuring compatibility and access to the newest features and bug fixes.
+
+## Bug Fixes
+1. **Ctrl+C Shutdown Error**:
+     - Resolved the error encountered when stopping the web UI with Ctrl+C, which previously resulted in Error: No such file or directory (os error 2).
+
+2. **SQLPage Directory Creation:**
+    - Disabled the SQLPage feature that was creating an empty sqlpage directory with each execution, preventing unnecessary directory creation.
+
+3. **Multitenant Support**
+     - Fixed the error during ingestion related to multitenant support by seeding the `party_types` back into the models directly.
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.11.0).

src/content/docs/docs/releases/v0_13_0.md

@@ -0,0 +1,28 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.0
+sidebar:
+  label: v0.13.0
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.13.0
+
+## What's New
+### 1. Upgraded SQLPage
+- The SQLPage dependency has been upgraded to version `0.27.0` to ensure improved stability and performance.
+
+### 2. Enhanced Orchestration Session Listing
+- The `surveilr orchestrate sessions` command now produces a comprehensive list of orchestration sessions, regardless of their nature.
+
+### 3. Filtered Orchestration Session Listing
+- The `surveilr orchestrate sessions -n "v&v"/"csv"` command now allows users to filter and produce a list of orchestration sessions specific to a given nature.
+
+### 4. New Command: `surveilr doctor`
+- Added the `surveilr doctor` command, which displays the versions of `sqlite3`, `deno`, and `rust` (if installed) on the host machine.
+
+### 5. Version Tracking in Orchestration
+- When an orchestration is executed, the current versions of `sqlpage`, `rusqlite`, `pgwire`, and `udi_pgp` used by the current `surveilr` binary are now added to the `version` column in the `orchestration_session` table and the `device elaboration` field.
+
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.0).

src/content/docs/docs/releases/v0_13_1.md

@@ -0,0 +1,26 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.1
+sidebar:
+  label: v0.13.1
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.13.1
+
+### What's New
+
+1. **Notebook Orchestration**
+   - Introduced a powerful new feature that allows executing scripts stored within `code_notebook_cell`, similar to stored procedures. 
+   - You can use the `--notebook` (`-n`) and `--cell` (`-c`) arguments to select multiple notebooks and cells, orchestrating them in a single session.
+   - The `-arg` flag now creates a temporary table `surveilr_orchestration_session_arg` with columns `session_id`, `key`, and `value`. All provided arguments are inserted as rows in this table, allowing the `code_notebook_cell` SQL code to leverage these inputs dynamically.
+
+### Bug Fixes
+
+- **Cargo-Metadata Command Failure**
+  - Resolved an issue where all commands were failing due to `cargo-metadata`. This fix ensures stability and seamless execution of commands moving forward.
+- **`surveilr doctor` fixes** 
+   -  `surveilr doctor` only makes SQLite compulsory, all other dependencies are not required
+
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.1).

src/content/docs/docs/releases/v0_13_2.md

@@ -0,0 +1,30 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.2
+sidebar:
+  label: v0.13.2
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.13.2
+
+## What's New
+
+### `surveilr admin merge` Enhancements
+
+- **New Argument: `-table-name-patterns` or `-p`**
+  - Introduced a new argument `-table-name-patterns` (or shorthand `-p`) to the `surveilr admin merge` command.
+  - This argument allows users to specify a list of regular expressions to include specific table names in the merge process.
+  - Example usage:
+    ```bash
+    surveilr admin merge -p "audit_session%" -p "activity_log%"
+    ```
+## Bug Fixes
+
+### `surveilr orchestrate transform-csv` Command
+
+- **Resolved Hanging Issues**
+  - Fixed an issue where the `surveilr orchestrate transform-csv` command would hang during execution.
+
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.2).

src/content/docs/docs/releases/v0_13_3.md

@@ -0,0 +1,21 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.3
+sidebar:
+  label: v0.13.3
+---
+
+Release notes for Resource Surveillance (`surveilr`) 0.13.3
+
+## Bug Fixes
+
+### Orchestration Scripts
+- The execution and handling of orchestration scripts have been modified. Rather than running them as a batch, they now utilize prepared statements.
+
+### `admin merge` Command
+- The `--sql-only` flag is designed to output the SQL for the merge directly to the console. This functionality was not operational in previous versions, but has been corrected and is now functional.
+
+
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.3).

src/content/docs/docs/releases/v0_13_4.md

@@ -0,0 +1,15 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.4
+sidebar:
+  label: v0.13.4
+---
+Release notes for Resource Surveillance (`surveilr`) 0.13.4
+
+## Bug Fixes
+
+### Orchestration Scripts Execution
+- The execution and management of orchestration scripts have been restored to their initial configuration. They are now being executed in batches. Please note that only DML queries will be processed. More complex scripts should be automated using Deno. Additionally, support for executing Deno scripts with `surveilr` will be introduced in the upcoming release.
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.4).

src/content/docs/docs/releases/v0_13_5.md

@@ -0,0 +1,19 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.5
+sidebar:
+  label: v0.13.5
+---
+Release notes for Resource Surveillance (`surveilr`) 0.13.5
+
+## Bug Fixes
+
+### 1. Update to Party Table Insertion Logic
+- **Description**: Updated all logic or code inserting data into the `party` table to use the new `party_type_id` reference from the `partyType` table.
+- **Impact**: This change resolves the "Foreign Key Constraint Error in party Table" that occurred when executing the command:
+
+### 2. Removal of `DROP` Statements in RSSD Merging
+- Removed `DROP` statements when merging RSSDs together whenever the command `surveilr admin merge` is executed.
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.5).

src/content/docs/docs/releases/v0_13_6.md

@@ -0,0 +1,30 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.13.6
+sidebar:
+  label: v0.13.6
+---
+Release notes for Resource Surveillance (`surveilr`) 0.13.6
+
+## Bug Fixes
+
+#### 1. **Fix file content duplication during file ingestion**
+
+**Issue:**  
+We were facing file content duplication during the `surveilr ingest files` operation inside the `uniform_resource` table. This occurred when there were changes to file properties such as inode, even if the content inside the file remained the same.
+
+**Fix:**  
+The bug was resolved by removing the `last_modified` field from the unique index on the `uniform_resource` model. This prevents duplication when ingesting files with unchanged content.
+
+---
+
+#### 2. **Ensure idempotency during CSV auto-transformation ingestion**
+
+**Issue:**  
+When running the command `surveilr ingest files --csv-transform-auto`, tables are created based on the file names inside the folder (e.g., `code_quality_infrastructure.csv` creates a table `uniform_resource_code_quality_infrastructure`). The issue arose when the command was run multiple times—it inserted the same data repeatedly, leading to duplication.
+
+**Fix:**  
+The issue was fixed by dropping the CSV table first and then recreating it only if there are no changes to the resource. This ensures the command is idempotent and prevents duplicate entries on repeated runs.
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.13.6).

src/content/docs/docs/releases/v0_21_0.md

@@ -0,0 +1,45 @@
+---
+title: Releases
+description: Release notes for Resource Surveillance v0.21.0
+sidebar:
+  label: v0.21.0
+---
+Release notes for Resource Surveillance (`surveilr`) 0.21.0
+
+We are excited to announce the release of new features in `surveilr`, specifically focusing on the `shell` command. This release brings a lightweight, unopinionated alternative to `sqlite3` or `duckdb`, allowing users to execute SQL scripts easily, while offering minimal logging compared to the highly opinionated `orchestrate` command.
+
+The `shell` and `orchestrate` commands are now closely related, with `shell` providing a simpler execution flow without detailed tracking. In contrast, `orchestrate` offers robust logging and support functions. 
+
+## New Features
+
+### 1. Default Engine
+- **`surveilr shell --engine rusqlite`**: Rusqlite is the default engine for the `shell` command unless the `--engine` option is overridden.
+
+### 2. Multi-Script Execution
+- **`surveilr shell first.sql second.sql https://site.com/third.sql`**: 
+  - Supports executing multiple SQL scripts passed as arguments, whether from local or remote locations.
+  - **Globs**: You can now use globs such as `**/*.sql` and `**/*.sql.ts` to match and execute files in sorted depth-first order.
+  - **File Type Handling**: 
+    - Files ending in `.sql` are treated as direct SQL files.
+    - Files ending in other extensions (like `.sql.ts`) are treated as potentially runnable files:
+      - **Local Executable**: If the file is local and executable, it will be run, and the STDOUT output will be used for SQL execution.
+      - **Remote Executable**: For security reasons, remote executable files cannot be run and will produce an error message: `remote files cannot be executed due to safety reasons`.
+
+### 3. Command-Line SQL Execution
+- **`surveilr shell --cmd "select * from table"`**: Now supports executing SQL commands passed directly through the command line, similar to `sqlite3`'s `-cmd` option.
+
+### 4. STDIN Support
+- **`cat my.sql | surveilr shell`** and **`surveilr shell < my.sql`**: `surveilr` now supports executing SQL scripts via STDIN, allowing seamless integration with piping or file redirection.
+
+### 5. Observability and Logging
+- **Logging Execution**: Unless the `--no-observability` flag is used, `surveilr shell` will log the execution into the orchestration tables:
+  - **`orchestration_nature`**: Logs the nature of the execution as `shell-<engine>` (e.g., `shell-rusqlite`).
+  - **`orchestration_session`**: Logs a new session for each execution.
+  - **`orchestration_session_log`**: Logs any exceptions or errors that occur during the execution for better debugging and traceability.
+
+---
+
+
+
+## Assets
+Please find the release [here](https://github.com/opsfolio/releases.opsfolio.com/releases/tag/0.21.0).