Add top-level doc comment in Trustfall crate and improve main README. (…

…#270)

README.md

@@ -1,14 +1,36 @@
-# `trustfall` — How to Query (Almost) Everything
+# Trustfall — Engine for Querying (Almost) Everything
 
-This repository contains the `trustfall` query engine, which can be used to query any data source
-or combination of data sources: databases, APIs, raw files (JSON, CSV, etc.), git version control,
-etc. For a 10min video introduction to the project, see
-the ["How to Query (Almost) Everything" talk](https://www.hytradboi.com/2022/how-to-query-almost-everything)
-from the [HYTRADBOI 2022](https://www.hytradboi.com/) conference.
+Trustfall is a query engine for querying any kind of data source, from APIs and databases
+to any kind of files on disk — and even AI models.
+
+* [Try Trustfall in your browser](#try-trustfall-in-your-browser)
+* [10min tech talk + demo](#10min-tech-talk--demo)
+* [Examples of querying real-world data with Trustfall](#examples-of-querying-real-world-data-with-trustfall)
+
+## Try Trustfall in your browser
+
+The Trustfall Playground supports running queries against public data sources such as:
+- the HackerNews REST APIs: https://play.predr.ag/hackernews
+- the rustdoc JSON of top Rust crates: https://play.predr.ag/rustdoc
+
+For example,
+[this link](https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVlcnkgKEFsZ29saWEgKyBGaXJlYmFzZSk6CiMgRmluZCBjb21tZW50cyBvbiBzdG9yaWVzIGFib3V0ICJvcGVuYWkuY29tIiB3aGVyZQojIHRoZSBjb21tZW50ZXIncyBiaW8gaGFzIGF0IGxlYXN0IG9uZSBHaXRIdWIgb3IgVHdpdHRlciBsaW5rCnF1ZXJ5IHsKICAjIFRoaXMgaGl0cyB0aGUgQWxnb2xpYSBzZWFyY2ggQVBJIGZvciBIYWNrZXJOZXdzLgogICMgVGhlIHN0b3JpZXMvY29tbWVudHMvdXNlcnMgZGF0YSBpcyBmcm9tIHRoZSBGaXJlYmFzZSBITiBBUEkuCiAgIyBUaGUgdHJhbnNpdGlvbiBpcyBzZWFtbGVzcyAtLSBpdCBpc24ndCB2aXNpYmxlIGZyb20gdGhlIHF1ZXJ5LgogIFNlYXJjaEJ5RGF0ZShxdWVyeTogIm9wZW5haS5jb20iKSB7CiAgICAuLi4gb24gU3RvcnkgewogICAgICAjIEFsbCBkYXRhIGZyb20gaGVyZSBvbndhcmQgaXMgZnJvbSB0aGUgRmlyZWJhc2UgQVBJLgogICAgICBzdG9yeVRpdGxlOiB0aXRsZSBAb3V0cHV0CiAgICAgIHN0b3J5TGluazogdXJsIEBvdXRwdXQKICAgICAgc3Rvcnk6IHN1Ym1pdHRlZFVybCBAb3V0cHV0CiAgICAgICAgICAgICAgICAgICAgICAgICAgQGZpbHRlcihvcDogInJlZ2V4IiwgdmFsdWU6IFsiJHNpdGVQYXR0ZXJuIl0pCgogICAgICBjb21tZW50IHsKICAgICAgICByZXBseSBAcmVjdXJzZShkZXB0aDogNSkgewogICAgICAgICAgY29tbWVudDogdGV4dFBsYWluIEBvdXRwdXQKCiAgICAgICAgICBieVVzZXIgewogICAgICAgICAgICBjb21tZW50ZXI6IGlkIEBvdXRwdXQKICAgICAgICAgICAgY29tbWVudGVyQmlvOiBhYm91dFBsYWluIEBvdXRwdXQKCiAgICAgICAgICAgICMgVGhlIHByb2ZpbGUgbXVzdCBoYXZlIGF0IGxlYXN0IG9uZQogICAgICAgICAgICAjIGxpbmsgdGhhdCBwb2ludHMgdG8gZWl0aGVyIEdpdEh1YiBvciBUd2l0dGVyLgogICAgICAgICAgICBsaW5rCiAgICAgICAgICAgICAgQGZvbGQKICAgICAgICAgICAgICBAdHJhbnNmb3JtKG9wOiAiY291bnQiKQogICAgICAgICAgICAgIEBmaWx0ZXIob3A6ICI%2BPSIsIHZhbHVlOiBbIiRtaW5Qcm9maWxlcyJdKQogICAgICAgICAgICB7CiAgICAgICAgICAgICAgY29tbWVudGVySURzOiB1cmwgQGZpbHRlcihvcDogInJlZ2V4IiwgdmFsdWU6IFsiJHNvY2lhbFBhdHRlcm4iXSkKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBAb3V0cHV0CiAgICAgICAgICAgIH0KICAgICAgICAgIH0KICAgICAgICB9CiAgICAgIH0KICAgIH0KICB9Cn0%3D&v=ewogICJzaXRlUGF0dGVybiI6ICJodHRwW3NdOi8vKFteLl0qXFwuKSpvcGVuYWkuY29tLy4qIiwKICAibWluUHJvZmlsZXMiOiAxLAogICJzb2NpYWxQYXR0ZXJuIjogIihnaXRodWJ8dHdpdHRlcilcXC5jb20vIgp9)
+shows the results of the HackerNews query: "Which GitHub or Twitter
+users are commenting on stories about OpenAI?"
+
+In the Playground, Trustfall is configured to run client-side as WASM, performing
+all aspects of query processing (parsing, compilation, and execution) within the browser.
+While this demo highlights Trustfall's ability to be embedded within a target application,
+it is of course able to be used in a more traditional client-server context as well.
+
+## 10min tech talk + demo
+
+Trustfall was featured in the ["How to Query (Almost) Everything" talk](https://www.hytradboi.com/2022/how-to-query-almost-everything)
+talk at the [HYTRADBOI 2022](https://www.hytradboi.com/) conference.
 
 ![Terminal recording of running `cargo run --release -- query example_queries/actions_in_repos_with_min_10_hn_pts.ron` in the `demo-hytradboi` demo project. The system returns the first 20 results of the query in 6.36 seconds."](./demo-hytradboi/query-demo.gif)
 
-*Demo showing the execution of the cross-API query: "Which GitHub Actions are used in projects on the front page of HackerNews with >=10 points?"*
+*Demo from the talk showing the execution of the cross-API query: "Which GitHub Actions are used in projects on the front page of HackerNews with >=10 points?"*
 
 The demo executes the following query across the HackerNews and GitHub APIs and over the YAML-formatted GitHub repository workflow files:
 ```graphql
@@ -43,13 +65,11 @@ The demo executes the following query across the HackerNews and GitHub APIs and
 }
 ```
 
-This demo is part of the ["How to Query (Almost) Everything"
-talk](https://www.hytradboi.com/2022/how-to-query-almost-everything) from the
-[HYTRADBOI 2022](https://www.hytradboi.com/) conference. Instructions for
+Instructions for
 running the demo are available together with the source code in the
 `demo-hytradboi` directory: [link](./demo-hytradboi).
 
-## Example Trustfall implementations
+## Examples of querying real-world data with Trustfall
 
 - [HackerNews APIs](./trustfall/examples/hackernews/), including an overview of the query language
   and an example of querying REST APIs.
@@ -58,6 +78,11 @@ running the demo are available together with the source code in the
 - [airport weather data (METAR)](./trustfall/examples/weather), showing how to query CSV data from
   aviation weather reports.
 
+Trustfall also powers the [`cargo-semver-checks`](https://crates.io/crates/cargo-semver-checks)
+semantic versioning linter.
+More details on the role Trustfall plays in that use case are available in
+[this blog post](https://predr.ag/blog/speeding-up-rust-semver-checking-by-over-2000x/).
+
 ## Using Trustfall over a new data source
 
 The easiest way to plug in a new data source is by implementing

trustfall/src/lib.rs

@@ -1,3 +1,38 @@
+//! # Trustfall
+//!
+//! Trustfall is a query engine for querying any kind of data source, from APIs and databases
+//! to any kind of files on disk — and even AI models.
+//!
+//! ## Try Trustfall in your browser
+//!
+//! The Trustfall Playground supports running queries against public data sources such as:
+//! - the HackerNews REST APIs: <https://play.predr.ag/hackernews>
+//! - the rustdoc JSON of top Rust crates: <https://play.predr.ag/rustdoc>
+//!
+//! For example,
+//! [this link](https://play.predr.ag/hackernews#?f=1&q=IyBDcm9zcyBBUEkgcXVlcnkgKEFsZ29saWEgKyBGaXJlYmFzZSk6CiMgRmluZCBjb21tZW50cyBvbiBzdG9yaWVzIGFib3V0ICJvcGVuYWkuY29tIiB3aGVyZQojIHRoZSBjb21tZW50ZXIncyBiaW8gaGFzIGF0IGxlYXN0IG9uZSBHaXRIdWIgb3IgVHdpdHRlciBsaW5rCnF1ZXJ5IHsKICAjIFRoaXMgaGl0cyB0aGUgQWxnb2xpYSBzZWFyY2ggQVBJIGZvciBIYWNrZXJOZXdzLgogICMgVGhlIHN0b3JpZXMvY29tbWVudHMvdXNlcnMgZGF0YSBpcyBmcm9tIHRoZSBGaXJlYmFzZSBITiBBUEkuCiAgIyBUaGUgdHJhbnNpdGlvbiBpcyBzZWFtbGVzcyAtLSBpdCBpc24ndCB2aXNpYmxlIGZyb20gdGhlIHF1ZXJ5LgogIFNlYXJjaEJ5RGF0ZShxdWVyeTogIm9wZW5haS5jb20iKSB7CiAgICAuLi4gb24gU3RvcnkgewogICAgICAjIEFsbCBkYXRhIGZyb20gaGVyZSBvbndhcmQgaXMgZnJvbSB0aGUgRmlyZWJhc2UgQVBJLgogICAgICBzdG9yeVRpdGxlOiB0aXRsZSBAb3V0cHV0CiAgICAgIHN0b3J5TGluazogdXJsIEBvdXRwdXQKICAgICAgc3Rvcnk6IHN1Ym1pdHRlZFVybCBAb3V0cHV0CiAgICAgICAgICAgICAgICAgICAgICAgICAgQGZpbHRlcihvcDogInJlZ2V4IiwgdmFsdWU6IFsiJHNpdGVQYXR0ZXJuIl0pCgogICAgICBjb21tZW50IHsKICAgICAgICByZXBseSBAcmVjdXJzZShkZXB0aDogNSkgewogICAgICAgICAgY29tbWVudDogdGV4dFBsYWluIEBvdXRwdXQKCiAgICAgICAgICBieVVzZXIgewogICAgICAgICAgICBjb21tZW50ZXI6IGlkIEBvdXRwdXQKICAgICAgICAgICAgY29tbWVudGVyQmlvOiBhYm91dFBsYWluIEBvdXRwdXQKCiAgICAgICAgICAgICMgVGhlIHByb2ZpbGUgbXVzdCBoYXZlIGF0IGxlYXN0IG9uZQogICAgICAgICAgICAjIGxpbmsgdGhhdCBwb2ludHMgdG8gZWl0aGVyIEdpdEh1YiBvciBUd2l0dGVyLgogICAgICAgICAgICBsaW5rCiAgICAgICAgICAgICAgQGZvbGQKICAgICAgICAgICAgICBAdHJhbnNmb3JtKG9wOiAiY291bnQiKQogICAgICAgICAgICAgIEBmaWx0ZXIob3A6ICI%2BPSIsIHZhbHVlOiBbIiRtaW5Qcm9maWxlcyJdKQogICAgICAgICAgICB7CiAgICAgICAgICAgICAgY29tbWVudGVySURzOiB1cmwgQGZpbHRlcihvcDogInJlZ2V4IiwgdmFsdWU6IFsiJHNvY2lhbFBhdHRlcm4iXSkKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBAb3V0cHV0CiAgICAgICAgICAgIH0KICAgICAgICAgIH0KICAgICAgICB9CiAgICAgIH0KICAgIH0KICB9Cn0%3D&v=ewogICJzaXRlUGF0dGVybiI6ICJodHRwW3NdOi8vKFteLl0qXFwuKSpvcGVuYWkuY29tLy4qIiwKICAibWluUHJvZmlsZXMiOiAxLAogICJzb2NpYWxQYXR0ZXJuIjogIihnaXRodWJ8dHdpdHRlcilcXC5jb20vIgp9)
+//! shows the results of the HackerNews query: "Which GitHub or Twitter
+//! users are commenting on stories about OpenAI?"
+//!
+//! In the Playground, Trustfall is configured to run client-side as WASM, performing
+//! all aspects of query processing (parsing, compilation, and execution) within the browser.
+//! While this demo highlights Trustfall's ability to be embedded within a target application,
+//! it is of course able to be used in a more traditional client-server context as well.
+//!
+//! ## Examples of querying real-world data with Trustfall
+//!
+//! - [HackerNews APIs](./trustfall/examples/hackernews/), including an overview of the query language
+//!   and an example of querying REST APIs.
+//! - [RSS/Atom feeds](./trustfall/examples/feeds/), showing how to query structured data
+//!   like RSS/Atom feeds.
+//! - [airport weather data (METAR)](./trustfall/examples/weather), showing how to query CSV data from
+//!   aviation weather reports.
+//!
+//! Trustfall also powers the [`cargo-semver-checks`](https://crates.io/crates/cargo-semver-checks)
+//! semantic versioning linter.
+//! More details on the role Trustfall plays in that use case are available in
+//! [this blog post](https://predr.ag/blog/speeding-up-rust-semver-checking-by-over-2000x/).
+
 use std::{collections::BTreeMap, rc::Rc, sync::Arc};
 
 /// Components needed to implement data providers.

trustfall_core/src/interpreter/basic_adapter.rs

@@ -17,7 +17,7 @@ use super::{
 /// - Automatic handling of the `__typename` special property.
 ///
 /// The easiest way to implement this trait is with the `Vertex` associated type set
-/// to an enum that is [`#[derive(Debug, Clone, TrustfallEnumVertex)]`].
+/// to an enum that is `#[derive(Debug, Clone, TrustfallEnumVertex)]`.
 pub trait BasicAdapter<'vertex> {
     /// The type of vertices in the dataset this adapter queries.
     /// It's frequently a good idea to use an Rc<...> type for cheaper cloning here.