ActiveRecord::StatementInvalid PG::ProgramLimitExceeded: ERROR: index row size 3296 exceeds btree version 4 maximum 2704 for index

[rgaufman]

I am passing an array of device_ids to good_job, but it's throwing this:

ERROR: index row size 3296 exceeds btree version 4 maximum 2704 for index "index_good_jobs_on_concurrency_key_when_unfinished" DETAIL: Index row references tuple (117364,3) in relation "good_jobs". HINT: Values larger than 1/3 of a buffer page cannot be indexed. Consider a function index of an MD5 hash of the value, or use full text indexing.
/gems/activerecord-7.0.3/lib/active_record/connection_adapters/postgresql_adapter.rb:768

It seems to happen here:

vendor/cache/good_job-bf197d04bfbd/lib/good_job/execution.rb:226 in block in enqueue

Any solution or workaround for this?

[mikereczek]

Hi @rgaufman - it sounds like your concurrency_key value might be too long. Are you passing in the full array of device_ids into key?

If you need to build a concurrency key out of a long or unbounded array, you could try hashing it first:

  # ERROR - I am to reproduce your error with the following
  good_job_control_concurrency_with(
    perform_limit: 1, 
    key: -> { 1000.times.map{|n|n}.join('-') }
  )
  
  # No more error
  good_job_control_concurrency_with(
    perform_limit: 1, 
    key: -> { OpenSSL::Digest.new('md5').base64digest(1000.times.map{|n|n}.join('-')) }
  )

Or sha256 or whatever other hash you'd prefer, of course.

[rgaufman]

Ah, yes, I am doing:

good_job_control_concurrency_with(total_limit: 1, key: -> { arguments.to_json })

I changed it to this:

  good_job_control_concurrency_with(
    total_limit: 1, key: -> { OpenSSL::Digest.new('md5').base64digest(arguments.to_json) }
  )

I will test this but this makes sense to me now, thank you for your help!

[rgaufman]

That fixed it, thank you!