Delete a node before creating/updating it, so that the related page q… #33285

kszot-ref · 2021-09-22T11:21:24Z

This PR is directly related to this issue - #33284. Deleting a node before updating it forces it properly rerun page queries. I'm not sure if this would be the correct way to fix this issue, but just leaving it as a reference for a potential fix.

Related Issues

Fixes #33284

…ueries are rerun properly

wardpeet · 2021-09-22T11:42:30Z

Hey, thanks for your PR but this actually does not solve the issue. Gatsby will do an update if changes are detected. This means something else is wrong.

cc @KyleAMathews any ideas?

kszot-ref · 2021-09-22T11:46:00Z

Well, the thing is that's not entirely true. I was literally debugging this and manually changing data (like modifying a value on a field, so nothing related to relationships) in the node didn't rerun the queries, only after applying this code it started working on my end. The thing is, I thought that just having a different contentDigest value would be enough to pick up my changes, but that didn't work.

KyleAMathews · 2021-09-22T20:37:54Z

hey @kszot-ref — yeah like Ward said, you don't need to delete a node to trigger page queries to re-run. So the fact this is working points to some other problem e.g. perhaps the contentDigest isn't getting updated. Perhaps you could log out what the node object looks like? I'm working a lot with the Drupal integration right now so happy to jump in and help debug this with you. Don't know where you're at in the world but if any times on here work for you, feel free to schedule a meeting https://calendly.com/kyle-gatsby/30min?back=1&month=2021-09

kszot-ref · 2021-09-23T08:36:42Z

This is how the node looks before it's modified:

{
  "id": "547e16f4-e355-587b-a089-43c39037bc48",
  "drupal_id": "54b6cb8f-cd9c-4297-baaa-d6a12907b3f3",
  "parent": null,
  "children": [],
  "drupal_internal__nid": 8368,
  "langcode": "nl",
  "status": true,
  "title": "ACCENT TRANSPORT & LOGISTICS VILVOORDE Update",
  "default_langcode": true,
  "metatag_normalized": [
    { "tag": "meta", "attributes": { "name": "title", "content": "ACCENT TRANSPORT & LOGISTICS VILVOORDE Update" } },
    { "tag": "meta", "attributes": { "property": "og:type", "content": "Office" } },
    {
      "tag": "meta",
      "attributes": { "property": "og:url", "content": "/nl/onze-kantoren/accent-transport-logistics-vilvoorde-update" }
    },
    {
      "tag": "meta",
      "attributes": { "property": "og:title", "content": "ACCENT TRANSPORT & LOGISTICS VILVOORDE Update" }
    }
  ],
  "path": { "alias": "/onze-kantoren/accent-transport-logistics-vilvoorde-update", "pid": 32042, "langcode": "nl" },
  "publish_on": null,
  "unpublish_on": null,
  "field_address": "Bedrijvenzone Machelen Cargo 741B\r\n1830 Machelen ",
  "field_chat_link": null,
  "field_email": "vilvoorde.logistics@accentjobs.be",
  "field_is_active": true,
  "field_office_description": null,
  "field_office_hours": [
    { "day": 1, "starthours": 730, "endhours": 1215, "comment": "" },
    { "day": 1, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 2, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 2, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 3, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 3, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 4, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 4, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 5, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 5, "starthours": 1315, "endhours": 1730, "comment": "" }
  ],
  "field_office_id": "22d3dc22-d43e-4b1b-904b-01f04d25eba2",
  "field_office_location": { "lat": 50.90462, "lng": 4.444582, "data": [], "value": "50.90462, 4.444582" },
  "field_phone": "+32 2 254 08 52",
  "relationships": {
    "field_paragraphs___NODE": [],
    "field_sidebar_paragraphs___NODE": [],
    "paragraph__vacancies_subset___NODE": ["138d2703-4c10-51ce-8f20-32f05447c181"]
  },
  "internal": {
    "type": "node__office",
    "contentDigest": "da6deadeb1fc57067ebda0b4775ad113",
    "owner": "gatsby-source-drupal",
    "counter": 1177
  }
}

and this is how it looks after the change, this one is passed to the "createNode" method:

{
  "id": "547e16f4-e355-587b-a089-43c39037bc48",
  "drupal_id": "54b6cb8f-cd9c-4297-baaa-d6a12907b3f3",
  "parent": null,
  "children": [],
  "drupal_internal__nid": 8368,
  "langcode": "nl",
  "status": true,
  "title": "ACCENT TRANSPORT & LOGISTICS VILVOORDE Update",
  "default_langcode": true,
  "metatag_normalized": [
    { "tag": "meta", "attributes": { "name": "title", "content": "ACCENT TRANSPORT & LOGISTICS VILVOORDE Update" } },
    { "tag": "meta", "attributes": { "property": "og:type", "content": "Office" } },
    {
      "tag": "meta",
      "attributes": { "property": "og:url", "content": "/nl/onze-kantoren/accent-transport-logistics-vilvoorde-update" }
    },
    {
      "tag": "meta",
      "attributes": { "property": "og:title", "content": "ACCENT TRANSPORT & LOGISTICS VILVOORDE Update" }
    }
  ],
  "path": { "alias": "/onze-kantoren/accent-transport-logistics-vilvoorde-update", "pid": 32042, "langcode": "nl" },
  "publish_on": null,
  "unpublish_on": null,
  "field_address": "Bedrijvenzone Machelen Cargo 741B\r\n1830 Machelen ",
  "field_chat_link": null,
  "field_email": "vilvoorde.logistics@accentjobs.be",
  "field_is_active": true,
  "field_office_description": null,
  "field_office_hours": [
    { "day": 1, "starthours": 730, "endhours": 1215, "comment": "" },
    { "day": 1, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 2, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 2, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 3, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 3, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 4, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 4, "starthours": 1315, "endhours": 1730, "comment": "" },
    { "day": 5, "starthours": 830, "endhours": 1215, "comment": "" },
    { "day": 5, "starthours": 1315, "endhours": 1730, "comment": "" }
  ],
  "field_office_id": "22d3dc22-d43e-4b1b-904b-01f04d25eba2",
  "field_office_location": { "lat": 50.90462, "lng": 4.444582, "data": [], "value": "50.90462, 4.444582" },
  "field_phone": "+32 2 254 08 52",
  "relationships": {
    "field_paragraphs___NODE": [],
    "field_sidebar_paragraphs___NODE": [],
    "paragraph__vacancies_subset___NODE": ["138d2703-4c10-51ce-8f20-32f05447c181"],
    "node__vacancy___NODE": ["0233a7af-fe0b-5029-acfa-cd32aa781e79"]
  },
  "internal": {
    "type": "node__office",
    "contentDigest": "4a16b190cf17de8c0a9f1bc8867b105b",
    "counter": 1177
  }
}

Differences: new relationship, contentDigest was updated, node owner has been removed. So without this PR related page queries wouldn't be rerun. I even tried to modify a field manually in the node (before the digest is generated, using a debugger) and that also didn't trigger page queries rerun.

I've scheduled a meeting so we can do some live debugging on this example if you want to know more.

kszot-ref · 2021-09-28T07:15:27Z

We've went through the code with Kyle, so I've got a few tips where to look to debug this issue. During the call we didn't pinpoint it, but I've tried again the next day.

I've went through the node creation process with a debugger and came to a conclusion that this is indeed an issue inside the gatsby-source-drupal plugin.

To create a backlink for a referenced node, the plugin does the following steps:

use "getNode" method to get the referenced node
it adds the new backlink inplace to that referenced node
it regenerates the content digest inplace
it passes the node to "createNode" function
inside of it, the digests are compared - this is where the issue occurs

gatsby/packages/gatsby/src/utils/nodes.ts

Lines 9 to 16 in 08c5b2a

    
           export const hasNodeChanged = (id: string, digest: string): boolean => { 
        
             const node = getNode(id) 
        
             if (!node) { 
        
               return true 
        
             } else { 
        
               return node.internal.contentDigest !== digest 
        
             } 
        
           }

When the code reaches this point, it gets the node from the datastore, but the issue is that data was modified inplace, the updated values are already in there. So it actually compares the new contentDigest with the already updated contentDigest, which means that the code thinks that the node didn't update at all, as in this scenario they will always be the same.

cc @KyleAMathews

KyleAMathews · 2021-09-28T15:55:09Z

oooo dang. That's bad. I'll add a shallow-clone real quick right now in gatsby-source-drupal. This is a tricky issue to uncover, and an easy footgun for source plugin authors. Thanks for digging deep and figuring it out!

KyleAMathews · 2021-09-28T23:49:21Z

Got a PR up #33328. Huge thanks @kszot-ref for digging in and finding the root cause. Gonna close this PR now in favor of #33328

Delete a node before creating/updating it, so that the related page q…

8607f58

…ueries are rerun properly

gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Sep 22, 2021

wardpeet added topic: source-drupal Related to Gatsby's integration with Drupal and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Sep 22, 2021

wardpeet assigned KyleAMathews Sep 22, 2021

KyleAMathews mentioned this pull request Sep 28, 2021

fix(gatsby-source-drupal): Clone nodes before adding back references so Gatsby knows the node has changed (so knows to re-run queries) #33328

Merged

KyleAMathews closed this Sep 28, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Delete a node before creating/updating it, so that the related page q… #33285

Delete a node before creating/updating it, so that the related page q… #33285

kszot-ref commented Sep 22, 2021

wardpeet commented Sep 22, 2021

kszot-ref commented Sep 22, 2021

KyleAMathews commented Sep 22, 2021

kszot-ref commented Sep 23, 2021 •

edited

Loading

kszot-ref commented Sep 28, 2021 •

edited

Loading

KyleAMathews commented Sep 28, 2021

KyleAMathews commented Sep 28, 2021

Delete a node before creating/updating it, so that the related page q… #33285

Delete a node before creating/updating it, so that the related page q… #33285

Conversation

kszot-ref commented Sep 22, 2021

Related Issues

wardpeet commented Sep 22, 2021

kszot-ref commented Sep 22, 2021

KyleAMathews commented Sep 22, 2021

kszot-ref commented Sep 23, 2021 • edited Loading

kszot-ref commented Sep 28, 2021 • edited Loading

KyleAMathews commented Sep 28, 2021

KyleAMathews commented Sep 28, 2021

kszot-ref commented Sep 23, 2021 •

edited

Loading

kszot-ref commented Sep 28, 2021 •

edited

Loading