Skip to content

Commit

Permalink
First release of River Ruby bindings
Browse files Browse the repository at this point in the history
A first push of Ruby bindings for River. Meant to be used in conjunction
with a driver like `riverqueue-sequel` [1] to provide an insert-only
client for River. See the README for details on usage.

Overall, I'm happy at how close I was able to keep the API to the Go
version. A lot of syntax in Go just isn't needed due to the more dynamic
and implicit nature of Ruby, but the parts that came through are quite
close. e.g. We have a job args concept, along with `InsertOpts` that can
be added to both jobs and at insert time, just like Go.

Purposely not implemented on this first push (I'll follow up with these
later on):

* Unique jobs.
* Batch insert.

[1] https://github.com/riverqueue/riverqueue-ruby-sequel
  • Loading branch information
brandur committed Mar 4, 2024
1 parent 7437932 commit bd4a615
Show file tree
Hide file tree
Showing 16 changed files with 882 additions and 17 deletions.
80 changes: 80 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
name: CI

env:
# Database to connect to that can create other databases with `CREATE DATABASE`.
ADMIN_DATABASE_URL: postgres://postgres:postgres@localhost:5432

# Just a common place for steps to put binaries they need and which is added
# to GITHUB_PATH/PATH.
BIN_PATH: /home/runner/bin

# A suitable URL for a test database.
TEST_DATABASE_URL: postgres://postgres:postgres@127.0.0.1:5432/riverqueue_ruby_test?sslmode=disable

on:
- push

jobs:
lint:
runs-on: ubuntu-latest
timeout-minutes: 3

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Ruby + `bundle install`
uses: ruby/setup-ruby@v1
with:
ruby-version: "head"
bundler-cache: true # runs 'bundle install' and caches installed gems automatically

- name: Standard Ruby
run: bundle exec standardrb

spec:
runs-on: ubuntu-latest
timeout-minutes: 3

services:
postgres:
image: postgres
env:
POSTGRES_PASSWORD: postgres
options: >-
--health-cmd pg_isready
--health-interval 2s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Install Ruby + `bundle install`
uses: ruby/setup-ruby@v1
with:
ruby-version: "head"
bundler-cache: true # runs 'bundle install' and caches installed gems automatically

# There is a version of Go on Actions' base image, but it's old and can't
# read modern `go.mod` annotations correctly.
- name: Install Go
uses: actions/setup-go@v4
with:
go-version: "stable"
check-latest: true

- name: Create database
run: psql --echo-errors --quiet -c '\timing off' -c "CREATE DATABASE riverqueue_ruby_test;" ${ADMIN_DATABASE_URL}

- name: Install River CLI
run: go install github.com/riverqueue/river/cmd/river@latest

- name: river migrate-up
run: river migrate-up --database-url "$TEST_DATABASE_URL"

- name: Rspec
run: bundle exec rspec
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
*.gem
/*.gem
/coverage/
14 changes: 14 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
source "https://rubygems.org"

gemspec

group :development, :test do
gem "standard"
end

group :test do
gem "debug"
gem "rspec-core"
gem "rspec-expectations"
gem "simplecov", require: false
end
93 changes: 93 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
PATH
remote: .
specs:
riverqueue (0.0.1)

GEM
remote: https://rubygems.org/
specs:
ast (2.4.2)
debug (1.9.1)
irb (~> 1.10)
reline (>= 0.3.8)
diff-lcs (1.5.0)
docile (1.4.0)
io-console (0.7.2)
irb (1.11.2)
rdoc
reline (>= 0.4.2)
json (2.7.1)
language_server-protocol (3.17.0.3)
lint_roller (1.1.0)
parallel (1.24.0)
parser (3.3.0.5)
ast (~> 2.4.1)
racc
psych (5.1.2)
stringio
racc (1.7.3)
rainbow (3.1.1)
rdoc (6.6.2)
psych (>= 4.0.0)
regexp_parser (2.9.0)
reline (0.4.3)
io-console (~> 0.5)
rexml (3.2.6)
rspec-core (3.12.2)
rspec-support (~> 3.12.0)
rspec-expectations (3.12.3)
diff-lcs (>= 1.2.0, < 2.0)
rspec-support (~> 3.12.0)
rspec-support (3.12.1)
rubocop (1.61.0)
json (~> 2.3)
language_server-protocol (>= 3.17.0)
parallel (~> 1.10)
parser (>= 3.3.0.2)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0)
rexml (>= 3.2.5, < 4.0)
rubocop-ast (>= 1.30.0, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 3.0)
rubocop-ast (1.31.1)
parser (>= 3.3.0.4)
rubocop-performance (1.20.2)
rubocop (>= 1.48.1, < 2.0)
rubocop-ast (>= 1.30.0, < 2.0)
ruby-progressbar (1.13.0)
simplecov (0.22.0)
docile (~> 1.1)
simplecov-html (~> 0.11)
simplecov_json_formatter (~> 0.1)
simplecov-html (0.12.3)
simplecov_json_formatter (0.1.4)
standard (1.34.0)
language_server-protocol (~> 3.17.0.2)
lint_roller (~> 1.0)
rubocop (~> 1.60)
standard-custom (~> 1.0.0)
standard-performance (~> 1.3)
standard-custom (1.0.2)
lint_roller (~> 1.0)
rubocop (~> 1.50)
standard-performance (1.3.1)
lint_roller (~> 1.1)
rubocop-performance (~> 1.20.2)
stringio (3.1.0)
unicode-display_width (2.5.0)

PLATFORMS
arm64-darwin-22
x86_64-linux

DEPENDENCIES
debug
riverqueue!
rspec-core
rspec-expectations
simplecov
standard

BUNDLED WITH
2.4.20
8 changes: 0 additions & 8 deletions README.md

This file was deleted.

74 changes: 74 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# River client for Ruby [![Build Status](https://github.com/riverqueue/riverqueue-ruby/workflows/CI/badge.svg)](https://github.com/riverqueue/riverqueue-ruby/actions)

An insert-only Ruby client for [River](https://github.com/riverqueue/river) packaged in the [`riverqueue` gem](https://rubygems.org/gems/riverqueue). Allows jobs to be inserted in Ruby and run by a Go worker, but doesn't support working jobs in Ruby.

## Basic usage

`Gemfile` should contain the core gem and a driver like [`rubyqueue-sequel`](https://github.com/riverqueue/riverqueue-ruby-sequel):

``` yaml
gem "riverqueue"
gem "riverqueue-sequel"
```

Initialize a client with:

```ruby
DB = Sequel.connect("postgres://...")
client = River::Client.new(River::Driver::Sequel.new(DB))
```

Define a job and insert it:

```ruby
class SortArgs
attr_accessor :strings

def initialize(strings:)
self.strings = strings
end

def kind = "sort"

def to_json = JSON.dump({strings: strings})
end

job = client.insert(SimpleArgs.new(strings: ["whale", "tiger", "bear"]))
```

Job args should:

* Respond to `#kind` with a unique string that identifies them in the database, and which a Go worker will recognize.
* Response to `#to_json` with a JSON serialization that'll be parseable as an object in Go.

They may also respond to `#insert_opts` with an instance of `InsertOpts` to define insertion options that'll be used for all jobs of the kind.

### Insertion options

Inserts take an `insert_opts` parameter to customize features of the inserted job:

```ruby
job = client.insert(
SimpleArgs.new(strings: ["whale", "tiger", "bear"]),
insert_opts: River::InsertOpts.new(
max_attempts: 17,
priority: 3,
queue: "my_queue",
tags: ["custom"]
)
)
```

### Inserting with a Ruby hash

`JobArgsHash` can be used to insert with a kind and JSON hash so that it's not necessary to define a class:

```ruby
job = client.insert(River::JobArgsHash.new("hash_kind", {
job_num: 1
}))
```

## Development

See [development](./development.md).
48 changes: 48 additions & 0 deletions docs/development.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# riverqueue-ruby development

## Install dependencies

```shell
$ bundle install
```
## Run tests

Create a test database and migrate with River's CLI:

```shell
$ go install github.com/riverqueue/river/cmd/river
$ createdb riverqueue_ruby_test
$ river migrate-up --database-url "postgres://localhost/riverqueue_ruby_test"
```

Run all specs:

```shell
$ bundle exec rspec spec
```

## Run lint

```shell
$ standardrb --fix
```

## Code coverage

Running the entire test suite will produce a coverage report, and will fail if line and branch coverage is below 100%. Run the suite and open `coverage/index.html` to find lines or branches that weren't covered:

```shell
$ bundle exec rspec spec
$ open coverage/index.html
```

## Publish a new gem

```shell
git checkout master && git pull --rebase
VERSION=v0.0.x
gem build riverqueue.gemspec
gem push riverqueue-$VERSION.gem
git tag $VERSION
git push --tags
```
61 changes: 61 additions & 0 deletions lib/client.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
module River
MAX_ATTEMPTS_DEFAULT = 25
PRIORITY_DEFAULT = 1
QUEUE_DEFAULT = "default"

# Provides a client for River that inserts jobs. Unlike the Go version of the
# River client, this one can insert jobs only. Jobs can only be worked from Go
# code, so job arg kinds and JSON encoding details must be shared between Ruby
# and Go code.
#
# Used in conjunction with a River driver like:
#
# DB = Sequel.connect(...)
# client = River::Client.new(River::Driver::Sequel.new(DB))
#
# River drivers are found in separate gems like `riverqueue-sequel` to help
# minimize transient dependencies.
class Client
def initialize(driver)
@driver = driver
end

# Inserts a new job for work given a job args implementation and insertion
# options (which may be omitted).
#
# Job arg implementations are expected to respond to:
#
# * `#kind`: A string that uniquely identifies the job in the database.
# * `#to_json`: Encodes the args to JSON for persistence in the database.
# Must match encoding an args struct on the Go side to be workable.
#
# They may also respond to `#insert_opts` which is expected to return an
# `InsertOpts` that contains options that will apply to all jobs of this
# kind. Insertion options provided as an argument to `#insert` override
# those returned by job args.
#
# Returns an instance of Job.
def insert(args, insert_opts: InsertOpts.new)
raise "args should respond to `#kind`" if !args.respond_to?(:kind)

# ~all objects in Ruby respond to `#to_json`, so check non-nil instead.
args_json = args.to_json
raise "args should return non-nil from `#to_json`" if !args_json

args_insert_opts = args.respond_to?(:insert_opts) ? args.insert_opts : InsertOpts.new

scheduled_at = insert_opts.scheduled_at || args_insert_opts.scheduled_at

@driver.insert(Driver::JobInsertParams.new(
encoded_args: args_json,
kind: args.kind,
max_attempts: insert_opts.max_attempts || args_insert_opts.max_attempts || MAX_ATTEMPTS_DEFAULT,
priority: insert_opts.priority || args_insert_opts.priority || PRIORITY_DEFAULT,
queue: insert_opts.queue || args_insert_opts.queue || QUEUE_DEFAULT,
scheduled_at: scheduled_at&.utc, # database default to now
state: scheduled_at ? JOB_STATE_SCHEDULED : JOB_STATE_AVAILABLE,
tags: insert_opts.tags || args_insert_opts.tags
))
end
end
end
Loading

0 comments on commit bd4a615

Please sign in to comment.