Process Contentful images inlined in markdown #5542

Khaledgarbaya · 2018-05-24T13:57:53Z

Closes #1502

TODO :

Fix Tests
Add a show case page in using-contenful example
Add documentation
Handle configs

gatsbybot · 2018-05-24T14:05:22Z

Deploy preview for using-drupal ready!

Built with commit ea3ff23

https://deploy-preview-5542--using-drupal.netlify.com

gatsbybot · 2018-05-24T14:05:49Z

Deploy preview for gatsbygram ready!

Built with commit ea3ff23

https://deploy-preview-5542--gatsbygram.netlify.com

Khaledgarbaya · 2018-05-30T11:18:09Z

Any reviewer?

pieh · 2018-05-31T15:12:17Z

packages/gatsby-remark-images-contentful/src/index.js

+    backgroundColor: `white`,
+    linkImagesToOriginal: true,
+    showCaptions: false,
+    pathPrefix,


linkImagesToOriginal, showCaptions and pathPrefix aren't really used anywhere? Or I'm missing something?

for some reason my changes were not pushed to the branch 🙈

pieh · 2018-05-31T15:16:42Z

packages/gatsby-remark-images-contentful/src/index.js

+      method: `GET`,
+      url: `https:${node.url}`, // for some reason there is a './' prefix
+      responseType: `stream`,
+    })


I think it would be good idea to implement some sort of caching mechanism so we don't have to redownload images everytime

sqip plugin implemented this locally - https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-transformer-sqip/src/extend-node-type.js#L229-L248 - maybe this can be turned into utility that could be moved to main contentful plugin and exported from it?

I don't actually download the entire file I just get the metadata from the first bits in the stream and then destroy it

Oh, I've missed that, you are of course right. In any case we could cache metadata so we don't do same requests every build I think.

One way to do it is to pass cache helper utility here https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-transformer-remark/src/extend-node-type.js#L181-L188 and use that here - similar to what sqip plugin is doing - https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-transformer-sqip/src/generate-sqip.js#L38-L76 (just note that cache is not namespaced so, you would have to namespace used keys yourself.

If you are not sure how to use it just let me know - I can jump in and do this part.

Hey @pieh That would be nice but I think the gatsby-transfromer-remark is not passing the cache function to it's plugins.
Does it make sense to add a PR that does that ?

pieh · 2018-05-31T15:17:39Z

packages/gatsby-remark-images-contentful/src/index.js

+  const rawHtmlNodes = select(markdownAST, `html`)
+
+  const generateImagesAndUpdateNode = async function(node, resolve) {
+    // Ingonre if it is not contentful image


typo in "Ingonre" :)

pieh · 2018-05-31T15:22:39Z

packages/gatsby-remark-images-contentful/src/index.js

+      node =>
+        new Promise(async (resolve, reject) => {
+          // Ignore gifs as we can't process them,
+          // svgs as they are already responsive by definition


are gifs and svgs actually ignored here? if so explanation that images.ctfassets.net isn't used for those would be helpful

Khaledgarbaya · 2018-06-06T09:33:30Z

@pieh to be able to cache I added the ability to the gatsby-transformer-remark plugin to pass along the cache object to its children plugins.

pieh · 2018-06-07T16:06:30Z

@Khaledgarbaya Is this ready for another round of review?

Khaledgarbaya · 2018-06-07T21:36:50Z

Hey @pieh yes this is ready for another round of review

pieh · 2018-06-08T11:05:35Z

I tried it locally and I couldn't get placeholder images to show - should we actually download those (maybe change from w=40 to w=20 there) and inline it using background-image: url('data:image/jpeg;base64,...')?

And I think we would have to split this into 2 PRs - one is adding gatsby-remark-images-contentful and changes in gatsby-transformer-remark to pass cache and second one are changes to example - we need to publish plugin first otherwise example will not work until we publish new package (I can handle that just before merging)

---edit:
Other stuff look great!

Khaledgarbaya · 2018-06-08T11:15:45Z

Hey, @pieh that make sense. I will split this into 3 PRs

Khaledgarbaya · 2018-06-12T08:29:25Z

@pieh this PR is ready for a review and once it's merged and released I will add the showcase page

pieh · 2018-06-12T11:35:03Z

I updated minor stuff - formatting and failing test (which I'm not super happy how it looks but it works) and I think this is good to go!

pieh · 2018-06-12T14:24:28Z

@Khaledgarbaya yeah, that's a lot nicer way for tests - do you want to add anything more here? If not I'll just fix formatting, rerun tests and merge this in! :)

Khaledgarbaya · 2018-06-12T14:48:11Z

@pieh all good here nothing to add from my side

pieh

Thank You very much @Khaledgarbaya !

* master: (26 commits) Publish [feat] Extract internal plugin for automatically creating pages so we can reuse for other directories (#4490) fix(contentful): properly delete deleted entries and assets (#5756) Fix typo add Linda's post (#5817) [gatsby-source-contentful] Fix prepareJSONNode for array normalization (#5107) Process Contentful images inlined in markdown (#5542) feat(contentful): add traced SVGs to Contentful images (#5659) [SQIP] Add documentation (#5287) Add site to showcase (#5819) [gatsby-remark-copy-linked-files] Support reference-style images (#5818) Update docs for gatsby-plugin-catch-links (#5843) Added ⋅ representing a space to sub-list examples to reflect actual code (#5829) Adding tigerfacilityservices.com to showcase (#5855) fix schema type conflict reporter regressions (#5805) [gatsby-plugin-feed] Support nested output directory (#5820) Make cache available to the plugins (#5840) [gatsby-source-contentful] update data and jest snapshots (#5802) [gatsby-plugin-manifest] Replace all manifest.json with manifest.webmanifest in comments/docs (#5157) Update deploy-gatsby.md (#5800) ... # Conflicts: # docs/tutorial/part-one/index.md # examples/using-sqip/src/components/polaroid.js # packages/gatsby-image/README.md # packages/gatsby-image/package.json # packages/gatsby-plugin-catch-links/package.json # packages/gatsby-plugin-feed/package.json # packages/gatsby-plugin-manifest/package.json # packages/gatsby-plugin-page-creator/src/gatsby-node.js # packages/gatsby-remark-copy-linked-files/package.json # packages/gatsby-source-contentful/package.json # packages/gatsby-source-contentful/src/__tests__/__snapshots__/normalize.js.snap # packages/gatsby-source-contentful/src/extend-node-type.js # packages/gatsby-source-contentful/src/gatsby-node.js # packages/gatsby-source-contentful/src/normalize.js # packages/gatsby-transformer-remark/package.json # packages/gatsby-transformer-remark/src/__tests__/__snapshots__/gatsby-node.js.snap # packages/gatsby-transformer-sqip/package.json # packages/gatsby-transformer-sqip/src/extend-node-type.js # packages/gatsby/package.json # packages/gatsby/src/bootstrap/load-plugins/__tests__/__snapshots__/load-plugins.js.snap # packages/gatsby/src/bootstrap/load-plugins/load.js # www/src/data/sites.yml

Initial commit

e9f8acd

Khaledgarbaya mentioned this pull request May 25, 2018

gatsby-remark-contentful-images #1502

Closed

Khaledgarbaya added 3 commits May 28, 2018 09:47

remove transpiled code

2be0e2a

fix tests

6daf7bc

Add showcase page

a5bc456

Khaledgarbaya changed the title ~~WIP: Process Contentful images inlined in markdown~~ Process Contentful images inlined in markdown May 28, 2018

Khaledgarbaya added 4 commits May 28, 2018 11:58

Fix Image url

a0ba894

Update snapshots

0903089

Format code

d101ce0

Add all options

ba9cef1

pieh self-assigned this May 30, 2018

pieh reviewed May 31, 2018

View reviewed changes

axe312ger mentioned this pull request Jun 4, 2018

feat(contentful): add traced SVGs to Contentful images #5659

Merged

Khaledgarbaya added 3 commits June 4, 2018 12:39

Handle options

db68d6f

fix options

6b55f4d

Add caching

d0d7dd4

Khaledgarbaya added 3 commits June 11, 2018 15:39

Fix base64 image

d9446fe

Revert example changes

2255dd3

Revert chache changes

9c82145

Remove showcase page

f81dbed

Fix linting

955caac

pieh previously approved these changes Jun 12, 2018

View reviewed changes

Fix tests

4ebb446

Khaledgarbaya dismissed pieh’s stale review via 4ebb446 June 12, 2018 12:17

pieh added 2 commits June 12, 2018 16:40

format

716ae68

remove package-lock.json from example

0cf15e9

fix lint (again)

ea3ff23

pieh approved these changes Jun 12, 2018

View reviewed changes

pieh merged commit 7dbfa3f into gatsbyjs:master Jun 12, 2018

Khaledgarbaya deleted the feat/markdown-contentful-images branch June 13, 2018 08:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Process Contentful images inlined in markdown #5542

Process Contentful images inlined in markdown #5542

Khaledgarbaya commented May 24, 2018 •

edited

Loading

gatsbybot commented May 24, 2018 •

edited

Loading

gatsbybot commented May 24, 2018 •

edited

Loading

Khaledgarbaya commented May 30, 2018

pieh May 31, 2018

Khaledgarbaya May 31, 2018

Khaledgarbaya Jun 4, 2018

pieh May 31, 2018

Khaledgarbaya Jun 4, 2018

pieh Jun 5, 2018

Khaledgarbaya Jun 6, 2018

pieh May 31, 2018 •

edited

Loading

pieh May 31, 2018

Khaledgarbaya commented Jun 6, 2018

pieh commented Jun 7, 2018

Khaledgarbaya commented Jun 7, 2018

pieh commented Jun 8, 2018 •

edited

Loading

Khaledgarbaya commented Jun 8, 2018

Khaledgarbaya commented Jun 12, 2018

pieh commented Jun 12, 2018

pieh commented Jun 12, 2018

Khaledgarbaya commented Jun 12, 2018 •

edited

Loading

pieh left a comment

Process Contentful images inlined in markdown #5542

Process Contentful images inlined in markdown #5542

Conversation

Khaledgarbaya commented May 24, 2018 • edited Loading

gatsbybot commented May 24, 2018 • edited Loading

gatsbybot commented May 24, 2018 • edited Loading

Khaledgarbaya commented May 30, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pieh May 31, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Khaledgarbaya commented Jun 6, 2018

pieh commented Jun 7, 2018

Khaledgarbaya commented Jun 7, 2018

pieh commented Jun 8, 2018 • edited Loading

Khaledgarbaya commented Jun 8, 2018

Khaledgarbaya commented Jun 12, 2018

pieh commented Jun 12, 2018

pieh commented Jun 12, 2018

Khaledgarbaya commented Jun 12, 2018 • edited Loading

pieh left a comment

Choose a reason for hiding this comment

Khaledgarbaya commented May 24, 2018 •

edited

Loading

gatsbybot commented May 24, 2018 •

edited

Loading

gatsbybot commented May 24, 2018 •

edited

Loading

pieh May 31, 2018 •

edited

Loading

pieh commented Jun 8, 2018 •

edited

Loading

Khaledgarbaya commented Jun 12, 2018 •

edited

Loading