Page rendering using s3 and cloudfront notably slow

[sg-modlab]

Bug Description

Using the flysystem s3 driver and setting url to the cloudfront domain that was setup is working. Images are loading ... but notably slow. Glide was being used, so I removed those tags from home page in case that caused the slowness. Still notably slow. Cleared all cache and no difference.

I noticed the other issue with assets list in CP being slow with s3. Seeing that as well. It is unusably slow. Similar rendering of front end using s3/cloudfront is unusably slow.

How to Reproduce

Wire up a site with s3 and cloudfront.

Extra Detail

Environment

Statamic 3.0.0 Pro
Laravel 7.25.0
PHP 7.4.9
statamic/seo-pro 2.0.7

Install method (choose one):

• Fresh install from statamic/statamic

[jasonvarga]

Does enabling caching on your s3 disk help?
See here: #1477 (comment)

[sg-modlab]

Have it enabled:

composer.json

filesystems.php - though with correct cloudfront URL. Note that I don't have the AWS_URL set in .env variables.

's3' => [
            'driver' => 's3',
            'key' => env('AWS_ACCESS_KEY_ID'),
            'secret' => env('AWS_SECRET_ACCESS_KEY'),
            'region' => env('AWS_DEFAULT_REGION'),
            'bucket' => env('AWS_BUCKET'),
            'endpoint' => env('AWS_URL'),
            'url' => 'https://#########.cloudfront.net',
            'cache' => true,
        ],

Takes about 6-7 seconds on local to render homepage.

[skylennard]

I am also having a similar problem here but I'm using Digital Ocean Spaces via the same basic s3 setup as @sg-modlab.

[jonatanalvarsson]

Same problem here. Using a S3 compatible cloud storage (no CDN/CloudFront).

Enabling S3 cache (league/flysystem-cached-adapter and 'cache' => true in config/filesystems.php) improves it a lot but it's still way slower than without S3 images. Is this expected behavior?

• 120 ms: rendering html with 9 entries without displaying any images
• 750 ms: rendering html with 9 entries with two images (no Glide) with S3 cache enabled
• 2 500 ms: rendering html with 9 entries with two images (no Glide) without S3 cache

[duncanmcclean]

I wonder if this issue could have the same underlying cause as #2145.

[jonatanalvarsson]

Merely checking the image field is enough to get 750 ms rendering time.

{{ if images }}
    we have images!
{{ /if }}

Without the above code in my Antlers template page rendering time is 120 ms.

Only 6 files in the S3 folder btw.

[jonatanalvarsson]

I noticed the .meta folder is created on the S3 storage. I guess reading those YAML files remotely from S3 is why page rendering becomes so much slower. Maybe (optionally) store the .meta directory locally for remote/S3 storage?

[sg-modlab]

Jonatan, your test is great to see and makes a lot of sense to what I have been experiencing the I was exploring things. Individual pages with a minimal amount s3 assets taking a very long time to render.

…

On Oct 27, 2020, at 9:37 AM, Jonatan Alvarsson ***@***.***> wrote: I noticed the .meta folder is created on the S3 storage. I guess reading those YAML files remotely from S3 is why page rendering becomes so much slower. Maybe (optionally) store the .meta directory locally for remote/S3 storage? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2279 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEBFK7FKLCB72WPVMYLOE3SM3EJ7ANCNFSM4QIZUI2A>.

[github-actions]

This issue has not had recent activity and has been marked as stale — by me, a robot. Simply reply to keep it open and send me away. If you do nothing, I will close it in a week. I have no feelings, so whatever you do is fine by me.

[GioChocolateBro]

This is still a real issue. We build our own wrapper for this that stores all this information (.meta files / and if a file exists) for every glide crop thats made. This way we don't have to be making GET req. to S3.

We clear all the glide crop info from Redis with cache tags if an image is modified.

Is working with an S3 + CDN + Glide still something that is on the horizon 1st party?

[duncanmcclean]

Re-opening.

[jackmcdade]

We have an open issue for this that's on our roadmap: #5143

[github-actions]

This issue has not had recent activity and has been marked as stale — by me, a robot. Simply reply to keep it open and send me away. If you do nothing, I will close it in a week. I have no feelings, so whatever you do is fine by me.

[edalzell]

Please keep open kind robot

[github-actions]

This issue has not had recent activity and has been marked as stale — by me, a robot. Simply reply to keep it open and send me away. If you do nothing, I will close it in a week. I have no feelings, so whatever you do is fine by me.

[nicko170]

Bad bot!

I have about 5 images on a page, that's now taking ~8 seconds to boot.

[jasonvarga]

This may have been resolved by #5143

Try setting up an s3 disk: https://statamic.dev/image-manipulation#custom-disk-cdn

If it's still an issue, please provide a github repo with this problem.

[GioChocolateBro]

Hi @jasonvarga ,

I recently started a new project and decided to ditch my custom glide implementation in favour of the work that was done on #5725. Now with the separate disk & cache store for glide it's perfect!

Sadly I noticed that I got added quite some load for every extra image on the page. With 7 images rolling through glide:generate I'm now up to 800/900 ms (about 100ms per image) for a server response.

After some digging I ended up at Statamic\Imaging\Attribute::from.
$manager->copy("source://{$path}", $destination);

No matter if I have the entry cached, it will copy the image from the cdn to my local disk. Commenting this line will bring me back to a very speedy 50ms server response.

Am I missing something in my configuration or should I go about thing differently? I'd really like to use statamics glide implementation.

[jasonvarga]

Does it still try to copy the image on the second page load?

[GioChocolateBro]

Yes, it does. After a glide:clear it takes longer.
When the cache is build and used it still hits this copy method.

Thing is, this function hits getimagesize eventually. Thats what it's copying the file to local for, after reading out the attributes it deletes this local copy again.

Sadly this kind of defeats the purpose of this whole caching layer if we still hit s3 with a copy for every image on every req. But we do need a local image to read out width/height.

Maybe we can cache these attributes as well? After glide has cropped it I don't see it changing again.

[jasonvarga]

Yeah that looks like an oversight. Seems like the image attributes are being retrieved every time.

[jasonvarga]

Are you using glide as a tag pair?

[GioChocolateBro]

I'm using blade I think that works the same as the tag pair.

I was thinking about making a little wrapper function to do something like {{ glide($asset) }}. There is no way to just do a single asset, you always have to foreach. Is there also maybe something coming to blade for this from statamic?

[GioChocolateBro]

Ah I see, this is because of the tag pair..

I just made a small helper for around glide:index for use in blade

function glide($source, $format = 'webp'): mixed
    {
        $quality = config('statamic.assets.image_manipulation.quality', 90);

        return Statamic::tag('glide:index')->src($source)->format($format)->quality($quality);
    }

No more copies from s3 now. I'm good now. That said, tag pair should probably also benefit from the caching layer, it's the purposed method for using glide with blade.

[jasonvarga]

Yeah it should work. I was just confirming a theory.

It only happens when using the tag pair, because we inject attributes, which does the copying (but could be modified to avoid copying every time).