[v2] Pages render briefly then turn blank in non-local environments

[jrskerritt]

Description

I'm running into a strange issue where my pages render as expected for a brief moment, but then the content is removed from the DOM within a few milliseconds. This only happens in our CI environment; locally I can test pages fine and they render without the flashing/flickering.

On debugging, it looks like the parent node of all of the gatsby content is being removed for some reason after the initial render:

<div id="___gatsby">
    <div style="outline:none" tabindex="-1" role="group">     <-- this gets removed
        <!-- all page content -->
    </div>
</div>

gatsby build runs as expected without errors.

I've tried changing versions of various packages (gatsby, plugins, webpack) to see if it resolves, with no luck. Also tried commenting out several sections of React code and plugins to try to identify what might be causing this, without any luck.

Any idea what could cause gatsby to remove its content from the mount node?

Environment

System:
OS: Windows 10
CPU: x64 Intel(R) Core(TM) i7-5820K CPU @ 3.30GHz
Binaries:
Yarn: 1.7.0 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
npm: 6.3.0 - C:\Program Files\nodejs\npm.CMD
npmPackages:
gatsby: 2.0.0-beta.98 => 2.0.0-beta.98
gatsby-module-loader: 2.0.0-alpha.3 => 2.0.0-alpha.3
gatsby-plugin-google-tagmanager: 2.0.0-beta.3 => 2.0.0-beta.3
gatsby-plugin-react-helmet: 3.0.0-beta.4 => 3.0.0-beta.4
gatsby-plugin-sass: 2.0.0-beta.10 => 2.0.0-beta.10
gatsby-source-filesystem: 2.0.1-beta.10 => 2.0.1-beta.10
gatsby-transformer-json: 2.1.1-beta.5 => 2.1.1-beta.5
react: 16.4.2
react-dom: 16.4.2
webpack: 4.16.5

File contents (if changed)

gatsby-config.js:

module.exports = {
  pathPrefix: 'path/prefix',
  plugins: [
    'gatsby-plugin-sass',
    'gatsby-plugin-react-helmet',
    'gatsby-transformer-json',
    // gtm plugin, filesystem plugins

package.json:

"scripts": {
  // other scripts ...
  // This is what gets run on TeamCity
  "build": "npm test && npm run teamcity-lint && gatsby build --prefix-paths",
  // other scripts ...
}

gatsby-node.js: N/A

gatsby-browser.js:

const getPolyfills = () => {
  if (!('IntersectionObserver' in window)) {
    require('intersection-observer');
  }
};

exports.onClientEntry = () => {
  getPolyfills();
};

exports.onInitialClientRender = () => {
  new Promise((resolve, reject) =>
    resolve(document.querySelectorAll('img.lazy'))
  ).then(images => lazyLoadImages(images));
};

gatsby-ssr.js: N/A

[m-allanson]

This sounds like the server side rendering is working as expected, but then when the app mounts in the client, it's dropping that div for some reason. You can confirm this by browsing the site in your CI environment with JavaScript disabled.

Is your CI a Windows environment? If it is, can you confirm that you see the error in other non-Windows environments?

Hmm - there's gatsby-module-loader in your list of npm packages, but that doesn't seem to exist any more in the master branch. I wonder where that came from? Can you try ditching your node_modules directory and yarn.lock file and doing a fresh yarn install?

[jrskerritt]

Yep, confirmed that disabling javascript stops the content from getting wiped.

Our CI is Windows, yeah. Confirming non-Windows will take some time but I will get back to you on that.

I think the gatsby-module-loader package was left over from v1, but I'm not sure what it was used for. It wasn't referenced anyhow so I got rid of it. I tried rimraffing node_modules, regenerating the yarn.lock file, and running another build but the issue is still there 🙁

[jrskerritt]

Through more debugging, I've discovered that the wiping of the root node happens at some point after onClientEntry, but before onInitialClientRender. If that helps

[jrskerritt]

This is still happening for me even after I comment out all plugins, gatsby-browser.js content, and update gatsby-node.js to only create a single page containing just a <div />.

[m-allanson]

Huh... any chance you could publish a repo that demos this problem?

[KyleAMathews]

Make sure to look at your browser console. There's almost certainly a JS error.

[jrskerritt]

@m-allanson: I started trying to put one together yesterday but so far have been unable to reproduce the issue starting from a fresh project (using gatsby-cli). Going to keep working on it.

@KyleAMathews: No JS errors 😕

[jrskerritt]

I think I found the problem! The reason it works locally but not when deployed is because we use a pathPrefix. Here's a repo that reproduces the problem:

https://github.com/jrskerritt/gatsby-v2-test

All the public files are committed as they would appear in our CI environment, so don't re-run gatsby build after cloning-- just spin up a web server on /public and fire away.

The pathPrefix used to build is /assets/, and as part of our CI build, we copy all files in public/static/ and js/css in public into an /assets/ folder in the root of where the gatsby pages live. When browsing to a page, there are no errors in the console because all of the js/css is located correctly. But something else then causes it to mount as empty. If you build without a pathPrefix, this doesn't happen and the page renders as expected.

Now that I know what's going on, do you guys think this is a bug, or are we copying our static files incorrectly?

[m-allanson]

Oh good sleuthing! That is definitely an unintended usage of the pathPrefix - interesting that it works enough to to have no JS errors and no failed requests. I think this should output an error of some kind?

Generally you shouldn't need to move files around in Gatsby's generated output as it can result in unusual bugs. Possibly related is the discussion towards the end of this PR.

I'd be interested to hear more about your use-case, particularly if there's something that Gatsby should be supporting? Or if the pathPrefix docs could be made clearer? Let's save other folks from having to go through this debugging process :)

[jrskerritt]

Let's save other folks from having to go through this debugging process :)

Agreed!

Here's the use-case we're building for: we have some existing site directory pages that we thought would be a perfect candidate for building with Gatsby. We have a specific url structure we want to use: /research/{country code}/{category}/{subcategory}/{starts with letter}/{page number}. Currently, we only plan on building these pages for the US, so country code will always be US.

Things would be perfect if we could just drop all of the generated Gatsby files into an S3 bucket in a folder named /research/US and make that our pathPrefix. But the problem is that we have existing pages in an ASP.NET app that also begin with /research/US, so we would have trouble telling Cloudfront how to differentiate between those existing pages vs Gatsby content located at the root of the generated public folder. For example, we would want /research/US/app-0e7d162482d979cfbf53.js to come from Cloudfront, but the existing page /research/US/Country=United_States (don't ask 😅) would now 404 under the same caching behavior (that page isn't in S3).

To solve this, what we attempted to do was to (incorrectly) use the pathPrefix to point all of the js, css, and json generated by Gatsby at /research/assets/directory, and move all of the html files into the existing /research/US structure. So you would go to the page /research/US/Employer and see the flashing content problem because the Gatsby js/css is located somewhere that it doesn't know about, even though the script/link elements seem to have built their urls correctly!

I hope at least some of that made sense!

I've brought this issue up to my team and there are other solutions to get around this, like make a new page for /research/US in Gatsby, or add something to the url structure to separate the new Gatsby content from the old content. Neither of these is ideal, though.

tl;dr:

We thought we could move the js/css assets generated by Gatsby into a special location to avoid Cloudfront behavior collisions that would cause 404s to existing pages.

As things stand, it's difficult for us to introduce Gatsby into our existing (large) website since we don't really get to choose where a lot of these generated files get to live. Having the ability to choose how to organize the output of Gatsby would make it a lot easier for us (and probably others).

In any case, an error would definitely be helpful-- not sure how difficult it would be to implement. Something along the lines of: "you moved files around when you shouldn't have, see the pathPrefix docs for more info."

edit:

I should also mention that the deploy strategy I described above is actually working fine in production right now using v1: https://www.payscale.com/research/US/Employer

[gatsbot]

Old issues will be closed after 30 days of inactivity. This issue has been quiet for 20 days and is being marked as stale. Reply here or add the label "not stale" to keep this issue open!

[gatsbot]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

[dragonza]

We are having the same behavior. We build Gatsby with --prefix-paths, and push to S3 bucket.

The 404 page shows up for any page that does not exist, but if the Url does not have the prefix, the 404 briefly shows up and turns blank.

So,
https://dev-staging.mobify.com/v2.x/this-page-does-not-exist <--- works
https://dev-staging.mobify.com/this-page-does-not-exist <---- does not work