Rails 7.2 on heroku stack24 with ruby 3.3.7 not communicating with s3 bucket #715
Unanswered
danielavdh
asked this question in
Q&A
Replies: 1 comment 1 reply
-
That means the image data wasn't stored in the database for some reason. If something was wrong with S3, you'd get a broken S3 link instead. Can you identify why the uploaded file data is failing to get stored to the attachment column? Does Shrine instrumentation show the S3 upload itself being successful? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
Thank you Janko, adding this at the end: I checked on s3: the cache version gets uploaded (uploadedPhoto)! here my adapted update method. uploadedPhoto and uploadedFile are attachments of Document - and Product has many documents. This is from the products controller before_action :set_product, only: [:show, :edit, :update, :destroy]
def update
Rails.logger.info "Before update - @product_category: #{@product_category.inspect}"
respond_to do |format|
begin
update_successful = @product.update(product_params)
Rails.logger.info "Update result: #{update_successful}"
unless update_successful
Rails.logger.error "Update failed: #{@product.errors.full_messages}"
end
rescue => e
Rails.logger.error "Exception during update: #{e.class} - #{e.message}"
Rails.logger.error e.backtrace.join("\n")
raise
end
if update_successful
#if @product.update(product_params)
if @product.category =~ /(recordings|teaching)/
format.html { redirect_to @product.category + "?id=#{@product.id}", notice: "#{@product_category[0].singularize} was successfully updated." }
elsif @product.category =~ /reviews|biography|photos/
format.html { redirect_to @product.category, notice: "#{@product_category[0].singularize} was successfully updated." }
else
format.html { redirect_to @product.category + "#bm#{@product.id}", notice: "#{@product_category[0].singularize} was successfully updated." }
end
format.json { render :show, status: :ok, location: @product }
else
get_data_for_admin
format.html { render :edit }
format.json { render json: @product.errors, status: :unprocessable_entity }
end
end
end
def set_product
@product = Product.find(params[:id])
categorize(@product.category)
end
def categorize(category)
if category
@product_category = all_frontend_pages.detect{|page| page if category == page[1]} || ["Products", products_path]
else
@product_category = ["Products", products_path]
end
endall_frontend_pages is in application controller def all_frontend_pages
#return Category.frontend_titles_and_paths
return [["News", news_path],
["Compositions", all_compositions_path],
["Concertos", concertos_path],
["Orchestral, Choral & Large Ensemble", orchestral_choral_large_ensemble_path],
["Chamber Music", chamber_music_path],
["Duos", duos_path],
["Songs", songs_path],
["Multiple Guitars", multiple_guitars_path],
["Two Guitars", two_guitars_path],
["Solo Guitar", solo_guitar_path],
["Solo Piano", solo_piano_path],
["Solo Instrument", solo_instrument_path],
["Arrangements & Cadenzas", arrangements_and_cadenzas_path],
["Educational Music", educational_music_path],
["Recordings", recordings_path],
["Articles & Interviews", articles_and_interviews_path],
["Biography", biography_path],
["Photos", photos_path],
["Reviews", reviews_path],
["Teaching", teaching_path],
["Contact", contact_path]]
end
helper_method :all_frontend_pagesHere the adapted code in UploadedPhotoUploader, no log messages at all. require "image_processing/mini_magick"
class UploadedPhotoUploader < Shrine
include ImageProcessing::MiniMagick
ALLOWED_TYPES = %w[image/jpg image/jpeg image/png image/x-png image/gif]
ALLOWED_EXTENSIONS = %w[jpg JPG jpeg JPEG png gif]
MAX_SIZE = 15*1024*1024 # 15 MB
MAX_WIDTH = 2500
MAX_HEIGHT = 2500
plugin :derivatives, create_on_promote: true
plugin :store_dimensions # gem fastimage needed #(for MAX_WIDTH and MAX_HEIGHT)
plugin :default_url
# minimal validations
Attacher.validate do
validate_max_size MAX_SIZE, message: "no more than 15MB please"
if validate_mime_type ALLOWED_TYPES, message: "only image files please"
validate_max_width MAX_WIDTH, message: "no wider than 2500px please"
validate_max_height MAX_HEIGHT, message: "no higher than 2500px please"
end
validate_extension ALLOWED_EXTENSIONS, message: "only .jp(e)g, .png and .gif"
end
Attacher.derivatives do |original|
Dog.poop("Starting derivatives creation for #{original.inspect}")
Dog.poop("File mime type: #{self.file.mime_type}")
if Document.jpg?(self.file.mime_type)
Dog.poop("Processing as JPG")
magick = ImageProcessing::MiniMagick
.source(original)
.strip.background("#fff")
.alpha("remove")
{
square: magick.quality(90).resize_to_fill!(500, 500, gravity: "Center"),
mini_square: magick.quality(90).resize_to_fill!(200, 200, gravity: "Center"),
icon: magick.quality(90).resize_to_fill!(32, 32, gravity: "Center"),
landscape: magick.quality(90).resize_to_limit!(nil, 400){ |cmd| cmd.auto_orient },
portrait: magick.quality(90).resize_to_limit!(400, nil){ |cmd| cmd.auto_orient }
}
else
Dog.poop("Processing as non-JPG")
magick = ImageProcessing::MiniMagick.source(original)
{
square: magick.resize_to_fill!(500, 500, gravity: "Center"),
mini_square: magick.resize_to_fill!(200, 200, gravity: "Center"),
icon: magick.resize_to_fill!(32, 32, gravity: "Center"),
landscape: magick.resize_to_limit!(nil, 400){ |cmd| cmd.auto_orient },
portrait: magick.resize_to_limit!(400, nil){ |cmd| cmd.auto_orient },
}
end
rescue => e
Dog.poop("Derivatives creation failed: #{e.message}")
Rails.logger.error e.backtrace.join("\n")
raise
end
# if a derivative is missing
Attacher.default_url do |derivative: nil, **|
"/images/#{derivative}.png" if derivative
end
endand here the output from the heroku logs --tail when try to update a product with a different photo: and just to be sure: |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Can I please reach out?
I am on a Rails 7.2. app, ruby 3.3.7, on heroku stack 24 and cannot get shrine to work with s3.
I have done this many times in the past and always got it to work, but not with this app.
I can connect to the images and files when I do it from development (both, accessing the production bucket, or development bucket, here commented out), but cannot connect when I run development app in production mode, nor from production app on heroku. Something must be happening, because default_url is being displayed.
This was originally an app with paperclip, started on about rails 4. I have been upgrading it and out of desperation, I now copied the whole app and deployed it to a fresh app on heroku (so there are absolutely no traces of paperclip hanging around). This worked a few times in the past, but not now and not with this app.
shrine.rb:
the s3 bucket has a documents "folder" at the top (and a development folder, I can upload to it and see the files I uploaded, but only from development)
these are the permissions for the user (app) steve:
group permission for all my apps:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowGroupToSeeBucketListAndAlsoAllowGetBucketLocationRequiredForListBucket", "Action": [ "s3:ListAllMyBuckets", "s3:GetBucketLocation" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::*" ] } ] }permissions for this user:
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AllowListBucketIfSpecificPrefixIsIncludedInRequest", "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::steve-paperclip" ], "Condition": { "StringLike": { "s3:prefix": [ "*" ] } } }, { "Sid": "AllowUserToReadWriteObjectDataInDevelopmentFolder", "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:PutObjectAcl", "s3:ListMultipartUploadParts", "s3:AbortMultipartUpload" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::steve-paperclip/*" ] } ] }directly on the s3 bucket, I have these permissions:
{ "Version": "2012-10-17", "Id": "Policy1511378689000", "Statement": [ { "Sid": "AllowPublicReadAccess", "Effect": "Allow", "Principal": "*", "Action": "s3:GetObject", "Resource": "arn:aws:s3:::steve-paperclip/*" } ] }I have 2 uploaders:
and one for images:
Here an example of an uploadedPhoto_data object (first uploaded with shrine, second copied from paperclip):
Here 2 uploadedFile_data cells (first uploaded with shrine, second copied from paperclip when migrating):
document.rb:
the pc_partitions.rb in lib/shrine/plugins
And I have added this CORS policy to the s3 bucket (only json was allowed), just now, though it is meant for direct uploads - not that it makes any difference.
[ { "AllowedHeaders": [ "Authorization", "x-amz-date", "x-amz-content-sha256", "content-type" ], "AllowedMethods": [ "GET", "POST", "PUT" ], "AllowedOrigins": [ "http://steveg-814d17bf33bb.herokuapp.com", "https://steveg-814d17bf33bb.herokuapp.com" ], "ExposeHeaders": [ "ETag" ], "MaxAgeSeconds": 3000 }, { "MaxAgeSeconds": 3000, "AllowedMethods": [ "GET" ], "AllowedOrigins": [ "*" ] } ]I am totally at the end of any ideas. Neither claude.ai nor chatgpt has anything useful to contribute (or stackoverflow or google for that matter).
Thanks for helping!
Beta Was this translation helpful? Give feedback.
All reactions