feat(gatsby-source-contentful): Support storing assets locally

[polarathene]

Downloads and caches Contentful Assets to the local filesystem. Useful for reduced data usage or projects where you want the assets copied locally with builds for deploying without links to Contentful CDN.

Resolves #10587 #5919

[polarathene]

ci/circleci: e2e_tests_path-prefix fails with:

 Production pathPrefix
    ✓ returns 200 on base route (529ms)
    navigation
      ✓ prefixes link with /blog (225ms)
      ✓ can navigate to secondary page (343ms)
      ✓ can navigate back from secondary page (303ms)
      1) can go back


  4 passing (12s)
  1 failing

I'm not sure what's causing that or how to fix it. Navigating my own project works fine. Any advice?

[DSchau]

@polarathene don't worry about that e2e failure--it's unrelated.

I'll create an issue now so that we can track on it.

[DSchau]

Whoops--I already did create that issue.

See #9233, but once more, unrelated to any changes here!

[polarathene]

@DSchau @sidharthachatterjee Would either of you be happy to review and merge this then if the failing e2e test is nothing to worry about? :)

I'm not sure if it will work well if an asset is modified on Contentful, it might not invalidate the cache and either not update the asset locally or create a new file copying over old data while the cache hasn't been cleared? (I have mentioned this in the README.md)

The wordpress plugin uses a modified metadata value to handle this, Contentful isn't providing one for ContentfulAsset afaik, there is internal.contentDigest but I'm not sure if that's useful.

[wardpeet]

@polarathene just to make sure we won't have any regressions in the future could you add a few tests to test your cache invalidation code (https://github.com/gatsbyjs/gatsby/pull/10682/files#diff-caa2171e85ea277bf3075bd98f17c16eR51)?

[polarathene]

@wardpeet I could try, but I've not really written much tests before.

It's been a while since I wrote this, I vaguely remember referencing some existing implementations(although I'm not sure that they had any testing in place either). Any advice(link to a similar test could suffice).

[wardpeet]

It's more or less testing your download function to make sure it's only called when options.localDownload is set and if the correct nodes are created correctly.

I have only found one package that tests sourceNodes https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-source-drupal/src/__tests__/index.js so I understand that this won't help you much as contentful doesn't have a lot of tests either. I don't want to burden you with this so if you're not up for it, no biggy.

[wardpeet]

LGTM! I'm going to test this one and I'll be merging afterwards

[wardpeet]

Tested this on www.gatsbyjs.com and seems to work. Looks identical to https://www.gatsbyjs.org/docs/image-tutorial/ 👍

[gatsbot]

Holy buckets, @polarathene — we just merged your PR to Gatsby! 💪💜

Gatsby is built by awesome people like you. Let us say “thanks” in two ways:

1. We’d like to send you some Gatsby swag. As a token of our appreciation, you can go to the Gatsby Swag Store and log in with your GitHub account to get a coupon code good for one free piece of swag. (Currently we’ve got a couple t-shirts available, plus some socks that are really razzing our berries right now.)
2. We just invited you to join the Gatsby organization on GitHub. This will add you to our team of maintainers. Accept the invite by visiting https://github.com/orgs/gatsbyjs/invitation. By joining the team, you’ll be able to label issues, review pull requests, and merge approved pull requests.

If there’s anything we can do to help, please don’t hesitate to reach out to us: tweet at @gatsbyjs and we’ll come a-runnin’.

Thanks again!