Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/samples-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ jobs:
run: dotnet pack ./src/KeelMatrix.QueryWatch/KeelMatrix.QueryWatch.csproj -c Release --no-build --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages
- name: Pack QueryWatch.EfCore
run: dotnet pack ./src/KeelMatrix.QueryWatch.EfCore/KeelMatrix.QueryWatch.EfCore.csproj -c Release --no-build --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages
- name: Build QueryWatch.Redaction (Release)
run: dotnet build ./src/KeelMatrix.QueryWatch.Redaction/KeelMatrix.QueryWatch.Redaction.csproj -c Release --no-restore
- name: Pack QueryWatch.Redaction
run: dotnet pack ./src/KeelMatrix.QueryWatch.Redaction/KeelMatrix.QueryWatch.Redaction.csproj -c Release --no-build --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages

# 3) Restore samples using their NuGet.config (pins KeelMatrix.QueryWatch* to ../artifacts/packages)
- name: Restore samples
Expand Down
2 changes: 1 addition & 1 deletion Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
<!-- JSON serializer used by QueryWatch.Reporting -->
<!-- Pin a patched System.Text.Json so NuGet never resolves 8.0.0 -->
<PackageVersion Include="System.Text.Json" Version="8.0.5" />
<PackageVersion Include="Microsoft.EntityFrameworkCore" Version="8.0.8" />
<PackageVersion Include="Microsoft.EntityFrameworkCore" Version="9.0.9" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Relational" Version="8.0.8" />
<PackageVersion Include="Microsoft.Data.Sqlite" Version="8.0.8" />
<PackageVersion Include="Microsoft.EntityFrameworkCore.Sqlite" Version="8.0.8" /> <PackageVersion Include="Dapper" Version="2.1.35" />
Expand Down
170 changes: 117 additions & 53 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,51 +1,133 @@
> The project is currently under development. Keep an eye out for its release!
# QueryWatch

# KeelMatrix.QueryWatch
Guardrail your **database queries** in tests & CI. Capture executed SQL, enforce **budgets** (counts & timings), and fail builds on regressions. Works with **ADO.NET**, **Dapper**, and **EF Core**.

> Catch N+1 queries and slow SQL in tests. Fail builds when query budgets are exceeded.
**Quick links:**
- 👉 [Quick Start — Samples (local)](#quick-start--samples-local)
- 👉 [EF Core wiring](#ef-core-wiring) · [Dapper wiring](#dapper-wiring) · [ADO.NET wiring](#adonet-wiring)
- 👉 [CLI flags](#cli) · [Troubleshooting](#troubleshooting)

[![Build](https://github.com/OWNER/REPO/actions/workflows/ci.yml/badge.svg)](https://github.com/OWNER/REPO/actions/workflows/ci.yml) [![NuGet](https://img.shields.io/nuget/v/KeelMatrix.QueryWatch.svg)](https://www.nuget.org/packages/KeelMatrix.QueryWatch/)
---

## Install
## 5‑minute success (tests)

```bash
dotnet add package KeelMatrix.QueryWatch
# EF Core users:
dotnet add package KeelMatrix.QueryWatch.EfCore
```

## 5‑minute success (with JSON for CI)

**Per‑test scope → export JSON:**
Use the disposable scope to **export JSON** and enforce **budgets** in your test or smoke app.

```csharp
using KeelMatrix.QueryWatch.Testing;

// JSON is written even if assertions fail (helps CI).
using var q = QueryWatchScope.Start(
maxQueries: 5,
maxAverage: TimeSpan.FromMilliseconds(50),
using var scope = QueryWatchScope.Start(
maxQueries: 50,
maxAverage: TimeSpan.FromMilliseconds(25),
exportJsonPath: "artifacts/qwatch.report.json",
sampleTop: 50); // increase if you plan to use per‑pattern budgets in CLI
sampleTop: 200);

// ... run your code that hits the DB ...
```

**Gate in CI (already wired in ci.yml):**
The file `artifacts/qwatch.report.json` is now ready for the CLI gate.

```pwsh
dotnet run --project tools/KeelMatrix.QueryWatch.Cli -- --input artifacts/qwatch.report.json --max-queries 50
```
---

## Quick Start — Samples (local)

This repo ships three tiny sample apps (EF Core, ADO.NET, Dapper) that **consume local packages** you build from source.

1. **Pack the libraries** (run **at repo root**):
```bash
dotnet pack ./src/KeelMatrix.QueryWatch/KeelMatrix.QueryWatch.csproj -c Release --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages
dotnet pack ./src/KeelMatrix.QueryWatch.EfCore/KeelMatrix.QueryWatch.EfCore.csproj -c Release --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages
```
2. **Install local packages to samples** (pins to `./artifacts/packages` via `samples/NuGet.config`):
- Windows (PowerShell): `./samples/init.ps1`
- Linux/macOS (bash): `./samples/init.sh`
3. **Run a sample** (EF example shown):
```bash
dotnet run --project ./samples/EFCore.Sqlite/EFCore.Sqlite.csproj -c Release
```
4. **Gate with the CLI**:
```bash
dotnet run --project ./tools/KeelMatrix.QueryWatch.Cli -- --input ./samples/EFCore.Sqlite/bin/Release/net8.0/artifacts/qwatch.ef.json --max-queries 50
```

> CI uses the same flow and restores using `samples/NuGet.config` so **samples build after `dotnet pack` with no tweaks**.

---

## EF Core wiring

```csharp
using KeelMatrix.QueryWatch.EfCore; // extension lives in the EfCore adapter package
using Microsoft.EntityFrameworkCore;
using KeelMatrix.QueryWatch.EfCore;
using KeelMatrix.QueryWatch.Testing;

using var q = QueryWatchScope.Start(exportJsonPath: "artifacts/ef.json", sampleTop: 200);

var options = new DbContextOptionsBuilder<MyDbContext>()
.UseSqlite("Data Source=:memory:")
.UseQueryWatch(q.Session) // adds the interceptor
.Options;
```
> Interceptor only records **executed** commands. Use `QueryWatchOptions` on the session to tune capture (text, parameter shapes, etc.).

---

## Dapper wiring

```csharp
using Dapper;
using Microsoft.Data.Sqlite;
using KeelMatrix.QueryWatch.Dapper;
using KeelMatrix.QueryWatch.Testing;

using var q = QueryWatchScope.Start(exportJsonPath: "artifacts/dapper.json", sampleTop: 200);

await using var raw = new SqliteConnection("Data Source=:memory:");
await raw.OpenAsync();

// Wrap the provider connection so Dapper commands are recorded
using var conn = raw.WithQueryWatch(q.Session);

var rows = await conn.QueryAsync("SELECT 1");
```

> The extension returns the **ADO wrapper** when possible for high‑fidelity recording; otherwise it falls back to a Dapper‑specific wrapper.

---

## ADO.NET wiring

```csharp
using Microsoft.Data.Sqlite;
using KeelMatrix.QueryWatch.Ado;
using KeelMatrix.QueryWatch.Testing;

using var q = QueryWatchScope.Start(exportJsonPath: "artifacts/ado.json", sampleTop: 200);
await using var raw = new SqliteConnection("Data Source=:memory:");
await raw.OpenAsync();

using var conn = new QueryWatchConnection(raw, q.Session);
using var cmd = conn.CreateCommand();
cmd.CommandText = "SELECT 1";
await cmd.ExecuteNonQueryAsync();
```

---

## Budgets (counts & timing)

At **test time** (scope) you can enforce `maxQueries`, `maxAverage`, `maxTotal`. At **CI time** use the CLI for stronger rules including **per‑pattern budgets**. Patterns support wildcards (`*`, `?`) or `regex:` prefix.

Example per‑pattern budgets:

```bash
--budget "SELECT * FROM Users*=1"
--budget "regex:^UPDATE Orders SET=3"
```

> If your JSON is **top‑N sampled**, budgets evaluate only over those events. Increase `sampleTop` in your export to tighten guarantees.

---

## CLI

Expand All @@ -67,40 +149,22 @@ var options = new DbContextOptionsBuilder<MyDbContext>()
<!-- END:CLI_FLAGS -->

### Multi‑file support

Repeat `--input` to aggregate multiple JSONs (e.g., per‑project reports in a mono‑repo). Budgets evaluate on the aggregate.

### GitHub PR summary
When run inside GitHub Actions, the CLI writes a Markdown table to the **Step Summary** automatically.

When run inside GitHub Actions, the CLI writes a Markdown table to the **Step Summary** automatically, so reviewers see metrics and any violations at a glance.

### Note on per‑pattern budgets

Budgets match against the `events` captured in the JSON file(s). These are the top‑N slowest events by duration to keep files small. If you want strict coverage, export with a higher `sampleTop` in `QueryWatchJson.ExportToFile`, or pass a larger `sampleTop` to `QueryWatchScope.Start(...)`.

## Redactor ordering tips

If you use multiple redactors, **order matters**. A safe, effective default is:

1. **Whitespace normalizer** – make SQL text stable across environments/providers.
2. **High‑entropy token masks** – long hex tokens, JWTs, API keys.
3. **PII masks** – emails, phone numbers, IPs.
4. **Custom rules** – your app–specific patterns (use `AddRegexRedactor(...)`).

> Put *broad* rules (like whitespace) first, and *specific* rules (like PII) after. This lowers the chance one rule prevents another from matching.

## Typical budgets for Dapper‑heavy solutions

Dapper often issues *fewer, more targeted* commands than ORMs. Reasonable starting points (tune per project):
---

- **End‑to‑end web test:** `--max-queries 40`, `--max-average-ms 50`, `--max-total-ms 1500`.
- **Repository‑level test:** `--max-queries 10`, `--max-average-ms 25`, `--max-total-ms 250`.
- **Per‑pattern budgets:** cap hot spots explicitly, e.g.:
## Troubleshooting

```
--budget "SELECT * FROM Users*=5" --budget "regex:^UPDATE Orders SET=3"
```
- **“Budget violations:” but no pattern table** → you didn’t pass any `--budget`, or your JSON was **heavily sampled**. Re‑export with higher `sampleTop` (e.g., 200–500).
- **Baselines seem too strict** → tolerances are **percent of baseline**. Ensure your baseline is representative; use `--baseline-allow-percent` to allow small drift.
- **CLI help in README looks stale** → run `./build/Update-ReadmeFlags.ps1` (or `--print-flags-md`) to refresh the block between markers.
- **Hot path text capture** → disable per‑adapter: `QueryWatchOptions.Disable{Ado|Dapper|EfCore}TextCapture=true`.
- **Parameter metadata** → set `QueryWatchOptions.CaptureParameterShape=true` (emits `event.meta.parameters`), never values.

- Increase `sampleTop` in code (`QueryWatchScope.Start(..., sampleTop: 200)`) if you rely on many per‑pattern budgets.
---

Treat these as **guardrails**: keep design flexible but catch accidental N+1s or slow queries early.
## License
MIT
57 changes: 13 additions & 44 deletions samples/README.md
Original file line number Diff line number Diff line change
@@ -1,53 +1,22 @@
# QueryWatch Samples

This folder contains small **sample projects** that consume the `KeelMatrix.QueryWatch` NuGet package
from your local build. They are intentionally minimal so you can understand and test the core behaviors
quickly in different situations (EF Core + SQLite, and raw ADO).

> **Important:** These samples expect you to build the package(s) locally first and then add them to each sample.
> See the "Quick Start" below.
Tiny apps that consume the local `KeelMatrix.QueryWatch*` packages so you can see EF Core, ADO.NET, and Dapper wiring in action.

## Layout
- `EFCore.Sqlite/` – EF Core (SQLite provider) showing interceptor wiring and basic budgets.
- `Ado.Sqlite/` – plain ADO.NET over `Microsoft.Data.Sqlite` wrapped by `QueryWatchConnection`.
- `Dapper.Sqlite/` – tiny Dapper example over SQLite showing **async** and **transaction** usage with QueryWatch.
- `cli-examples.ps1` / `cli-examples.sh` – example commands for running the QueryWatch CLI gate.
- `NuGet.config` – forces the `KeelMatrix.QueryWatch*` packages to come from your local `../artifacts/packages`.
- `.gitignore` – ignores local build outputs and DB files for samples only.
- `EFCore.Sqlite/` – EF Core (SQLite) with interceptor wiring and basic budgets.
- `Ado.Sqlite/` – plain ADO.NET over `Microsoft.Data.Sqlite` via `QueryWatchConnection`.
- `Dapper.Sqlite/` – Dapper (async + transactions) via `WithQueryWatch(...)`.
- `cli-examples.ps1` / `cli-examples.sh` – quick commands to run the CLI gate.
- `NuGet.config` – pins `KeelMatrix.QueryWatch*` to `../artifacts/packages` (local build).

## Quick Start (local, step-by-step)
1. **Pack the libraries** at the repository root (one level *above* this folder):
```bash
dotnet pack ./src/KeelMatrix.QueryWatch/KeelMatrix.QueryWatch.csproj -c Release --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages
dotnet pack ./src/KeelMatrix.QueryWatch.EfCore/KeelMatrix.QueryWatch.EfCore.csproj -c Release --include-symbols --p:SymbolPackageFormat=snupkg --output ./artifacts/packages
```
2. **Install the packages into each sample** (runs the add+restore for you):
- Windows (PowerShell): `./init.ps1`
- Linux/macOS (bash): `./init.sh`
These scripts run:
```bash
dotnet add ./EFCore.Sqlite/EFCore.Sqlite.csproj package KeelMatrix.QueryWatch
dotnet add ./EFCore.Sqlite/EFCore.Sqlite.csproj package KeelMatrix.QueryWatch.EfCore
dotnet add ./Ado.Sqlite/Ado.Sqlite.csproj package KeelMatrix.QueryWatch
```
The included `NuGet.config` pins `KeelMatrix.QueryWatch*` to the local `../artifacts/packages` folder.
3. **Run a sample** (EF Core example shown):
```bash
dotnet run --project ./EFCore.Sqlite/EFCore.Sqlite.csproj
```
You should see console output and a file at `./EFCore.Sqlite/bin/Debug/net8.0/artifacts/qwatch.ef.json`.
4. **Gate with the CLI** (from repo root or here):
```bash
dotnet run --project ../tools/KeelMatrix.QueryWatch.Cli -- --input ./EFCore.Sqlite/bin/Debug/net8.0/artifacts/qwatch.ef.json --max-queries 50
```
## Start here
Follow the **[Quick Start — Samples (local)](../README.md#quick-start--samples-local)** in the root README.

### Notes
- These samples **compile only after** you add the `KeelMatrix.QueryWatch` and (for EF) `KeelMatrix.QueryWatch.EfCore` packages (Step 2).
- If you get restore errors, confirm that `../artifacts/packages` exists and contains your `*.nupkg` files.
- The EF Core sample uses a file-based SQLite DB under `./EFCore.Sqlite/app.db`. You can delete it safely.
> After you run `dotnet pack ...` at the repo root, use `./init.ps1` (or `./init.sh`) once to add local packages. No other tweaks are needed.

### Dapper sample
Run the Dapper sample:
### Run a sample
```bash
dotnet run --project ./Dapper.Sqlite/Dapper.Sqlite.csproj
dotnet run --project ./EFCore.Sqlite/EFCore.Sqlite.csproj -c Release
```

For CLI usage examples, see `cli-examples.ps1` / `cli-examples.sh`.
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
<PackageId>KeelMatrix.QueryWatch.EfCore</PackageId>
<Description>EF Core adapter for KeelMatrix.QueryWatch: wires a DbCommand interceptor and a UseQueryWatch(...) extension.</Description>
<PackageTags>queries;performance;testing;efcore;interceptor;ci;gate</PackageTags>
<RepositoryUrl>https://github.com/your-org/KeelMatrix.QueryWatch</RepositoryUrl>
<PackageProjectUrl>https://github.com/your-org/KeelMatrix.QueryWatch#readme</PackageProjectUrl>
<RepositoryUrl>https://github.com/KeelMatrix/QueryWatch</RepositoryUrl>
<PackageProjectUrl>https://github.com/KeelMatrix/QueryWatch#readme</PackageProjectUrl>
<PackageReadmeFile>README.md</PackageReadmeFile>
<PackageLicenseExpression>MIT</PackageLicenseExpression>
<PackageIcon>icon.png</PackageIcon>
Expand Down
Loading