SQL Source

Kiba Pro SQL source provides an efficient way to read large volumes of rows.

Currently tested against: PostgreSQL 9.2-9.6.

Here is a typical use:

source Kiba::Pro::Sources::SQL,
  database: ENV.fetch('DATABASE_URL'),
  query: "SELECT * FROM items"

You can also specify the query using Sequel, for instance:

source Kiba::Pro::Sources::SQL,
  database: ENV.fetch('DATABASE_URL'),
  query: -> (db) { db[:items].where('updated_at >= ?', last_updated_at).limit(1000) }

Finally, if you are using Kiba inside Sidekiq, you can pass a live connection rather than a database_url:

job = Kiba.parse do
  source Kiba::Pro::Sources::SQL,
    database: db,
    query: "SELECT * FROM items"  
  # SNIP
end

Kiba.run(job)