Preload extension preloads URLs but navigating there loads the data again #108

hirasso · 2021-11-26T12:25:28Z

Hi there! I might be confused on how the preload extension is supposed to work. See this screenshot of the chrome dev tools inspecting the network activity when navigating htmx.org:

Starting from the front page I hovered over the docs link in the main navigation. preload.js loaded the URL, as expected. Then I clicked on docs and now htmx.js loaded the URL again from the server, seemingly without using the preloaded data. As soon as the docs page had been rendered, moving the cursor just a little bit (it was still resting on the docs link), triggered preload.js again, resulting in loading the URL again from the server. Same scenario with the reference link. Three server requests instead of one for each link with preload.

Am I missing something here? I would have expected that after preloading a URL, navigating towards it would not need another server request.

P.s.: I don't think the error in the console is related, it just points out a 404 for https://unpkg.com/prismjs@1.20.0/components/prism-ecmascript.min.js:

The text was updated successfully, but these errors were encountered:

benpate · 2021-12-02T23:03:14Z

Hey, thanks for posting this. I'll try to look into it soon. Without doing research, my first thought is that there might be different headers between the two requests, which is why your browser is not using the preloaded/cached content. Do you have anything in your code that changes the request headers?

hirasso · 2021-12-06T14:32:23Z

Hi @benpate , actually I would have expected that HTMX would not send a new request to the server at all, when an URL was preloaded. That's how I usually do it in my setups.

benpate · 2021-12-07T18:17:59Z

For this extension, we're are leaning on the browser's built-in caching mechanism. It does not store a separate copy of the server response, which could get cumbersome really quickly. How do you usually do this, then? Do you have sample code that you could share? If there's a better way, then we should do that :)

I guess this means that other server headers could also be messing with the browser's cache -- for instance, a no-cache header would definitely prevent it from working.

hirasso · 2021-12-13T09:55:21Z

Sorry for my late reply. I understand. If this is out of scope for HTMX, I will continue using my own implementation.

I am using a Map (caniuse) for that kind of functionality, with the URLs as keys and the document content, saved scroll positions etc. as data. I also have a cleanup mechanism running, so that I never have more than a certain amount of cached documents in this map (to prevent RAM issues). Here is the structure of my cache object, maybe this helps if you ever want to implement something like this in the future:

0: {
  key: "https://mysite.test/"
  value: {
    data:
      $document: "<!DOCTYPE html> ...",
      openAccordeonItem: "",
      scrollPositions: {root: 0},
      url: "https://mysite.test/",
    },
    keepInCache: true,
    state: "loaded"
  }
}
1: {"https://mysite.test/news/" => {...}}
2: {"https://mysite.test/about/" => {...}}
3: {"https://mysite.test/members/" => {...}}
4: {"https://mysite.test/rooms/" => {...}}
5: {"https://mysite.test/more-information/" => {...}}
6: {"https://mysite.test/contact/" => {...}}

EDIT: The above object would be the state after navigating 7 links in my site.

benpate · 2021-12-13T18:58:50Z

Hey @hirasso - I have seen some odd examples where the preload extension's HTTP headers don't match those in the final request, leading to duplicate requests. I still want to track those down, along with updating the extension with some of the lessons I've learned from using it. But, I DO want to stick with using the Browser's built-in cache because browser vendors have put so much work into optimizing these tools, and I want to use the platform as much as possible.

But htmx is immensely flexible, and you could certainly make an alternative extension tha stores content in a Map of some kind. Absolutely feel free to use the existing extension as a baseline for migrating your own tool into an extension. :)

hirasso · 2022-08-22T20:59:12Z

Sorry for my late reply. Got caught-up with live ;)

Without doing research, my first thought is that there might be different headers between the two requests, which is why your browser is not using the preloaded/cached content. Do you have anything in your code that changes the request headers?

Today I started investigating this again.

The request headers for the preload request:

Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-DE;q=0.8,en;q=0.7,en-US;q=0.6
Cache-Control: no-cache
Connection: keep-alive
Host: htmx.test
Pragma: no-cache
Referer: https://htmx.test/
sec-ch-ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36

The request headers if I actually click the link:

Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: de-DE,de;q=0.9,en-DE;q=0.8,en;q=0.7,en-US;q=0.6
Cache-Control: no-cache
Connection: keep-alive
Host: htmx.test
HX-Boosted: true
HX-Current-URL: https://htmx.test/
HX-Request: true
HX-Target: container
Pragma: no-cache
Referer: https://htmx.test/
sec-ch-ua: "Chromium";v="104", " Not A;Brand";v="99", "Google Chrome";v="104"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "macOS"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36

There actually is a difference, and it's the HX-* headers for the actual link click!

HX-Boosted: true
HX-Current-URL: https://htmx.test/
HX-Request: true
HX-Target: container

clanofartisans · 2022-10-14T14:39:43Z

I've also been experiencing this with the latest version of Chrome on Windows 10 so I would say chances are good that this affects a large number of users. My findings are the same as @hirasso back in August regarding the headers. Having the preload extension working properly would be a huge boon for me!

benpate · 2022-10-14T16:42:48Z

Absolutely. And, sorry I haven't been able to make much headway on this recently.

The short answer is that it's probably something with the headers being sent -- the extension tries to mimic a regular HTMX request, but there must be differences.

Could you open up the HTTP requests for a typical transaction and pull the headers for a) the preload request and b) the actual request? What headers are different? In hiraso's example above, it was HX-Boosted, HX-Current-URL, HX-Request, and HX-Target. If this is your experience as well, then we can try to improve the "mimicry" that the preload extension does.

One other hack is to try using the Vary header to limit the specific headers that your browser uses to bust the cache. This is a whole science in its own, but for this purpose, if you added a header that only specified Vary:Cookie then your browser should not (in theory) have to reload the URL just because these other headers are different.

Alex-Rafter · 2022-11-03T10:08:44Z

Hi, i'm having the same issues with using the preload extension. Resources are being downloaded twice. The issue seems to be a difference in these headers : hx-target: and hx-trigger.

When id and name are set on the element these differ e.g one one request they are
hx-target: root
hx-trigger: root

And on the other they match those on the element eg
hx-target: tst
hx-trigger: tst

When no id and name are set on the el, the headers are only present on one of the requests, and are set then as
hx-target: root
hx-trigger: root

benpate · 2022-11-03T16:52:27Z

Thanks for this report. Long-term, I think we'll need another way of handling preloads. For now, does the Vary header work for you?

onemoreahmad · 2023-04-12T12:15:31Z

I have same issue.

benpate · 2023-04-14T19:00:20Z

Sorry you're having trouble, @onemoreahmad. Can you provide more information? Have you tried working with the Vary header?

onemoreahmad · 2023-04-15T03:02:28Z

Hi @benpate, thanks for your reply.
I really don't know how this supposed to work, but I did try few things.

1 instantclick.io

I've tried few similar tools to see how it works, first, I checked http://instantclick.io/
The way it works is once I hover to the link, it fires a get request to the page, and when I actually click on the link, it opens fast without issuing anything in the network tab in the dev tool.

Eventhough maybe instantclick doesn't use ajax between pages, just a regular links.

2 quicklink

Then I checked quicklink, in there website when I open network tools, nothings shows up when I hover or click on links, but I tried it with my project which I use HTMX.
With a simple installation

<script src="https://cdnjs.cloudflare.com/ajax/libs/quicklink/2.3.0/quicklink.umd.js"></script>
<script>
window.addEventListener('load', () => {
  quicklink.listen();
});
</script>

It work on my website only on the current links exists on the page when it first loaded, but next time I click on the link again, it just work normally with a full server request.

3 using HTMX Preload extension

when I install the plugin, when I hover the link, it gets loaded from the server, and then when I click on it it loads it again, no cache nothing.

When I check headers requests both request it identical, the only difference is the xsrf token, and both has Vary key

benpate · 2023-05-01T16:46:03Z

Hey.. sorry I couldn't dig into this until now. It's hard to see all the header information on your screenshot, but if your responses have Vary:Accept-Encoding then it's worth it to check the Accept-Encoding header on your requests. They might be different.

Working with the browser's cache IS a little fiddly, and is outside of our control with this extension. I'm hoping that we get more access to the internals in htmx 2.0, which might give us more control over how this caching works.

In the meantime, I'm using a more aggressive version of this code that basically just loads the content on mousedown (without waiting for the mouseup or click events). You're welcome to use this if it makes sense in your app.

jwhitaker-swiftnav · 2023-05-08T11:57:06Z

@onemoreahmad I'm noticing cache-control: no-cache in headers from your server... EDIT: and for OP.

When I hit this, I added an appropriate max-age to my server responses to tell the browser that it's indeed ok to cache requests. Works great..

yokomizor · 2023-09-22T18:16:19Z

I am having this issue when combining hx-boost and preload.

From documentation:

<a href="/server/1" preload>WILL BE requested using a standard XMLHttpRequest() and default options (below)</a>
<button hx-get="/server/2" preload>WILL BE requested with additional htmx headers.</button>

The issue combining hx-boost and preload:

<div hx-boost>
  <a href="/server/1" preload>DOUBLE REQUEST. Preload will not send HX-Request, but the click will</a>
</div>

Here HTMX will make 1 request with HX-Request header and one without it. The response contains Vary: HX-Request, so the browser knows both requests are different and that the response for the preload request can't be used as cache for the following request.

Removing Vary: HX-Request is not an option because the browser needs to know both responses are actually different, otherwise a full refresh could result in it showing only a partial response instead of a full page.

I think the right thing to do would be for this extension to check if the link is inside a hx-boost context.

benpate · 2023-09-22T20:37:16Z

Hey @yokomizor - I'm sorry you're having trouble with the extension. I don't have much experience using hx-boost but it looks like there's an HX-Boosted request header that is also sent to the server. Is this a part of your Vary header? Could it work to resolve the "double request" problem?

yokomizor · 2023-09-23T12:48:00Z

Hey @benpate,

It's all good. The trouble is actually very minor and very easy to patch manually from my side.

About your suggestion, let me try to explain why this is not what I was looking for.

Take a look at this piece:
https://github.com/bigskysoftware/htmx/blob/master/src/ext/preload.js#L45-L69

				// Special handling for HX-GET - use built-in htmx.ajax function
				// so that headers match other htmx requests, then set 
				// node.preloadState = TRUE so that requests are not duplicated
				// in the future
				var hxGet = node.getAttribute("hx-get") || node.getAttribute("data-hx-get")
				if (hxGet) {
					htmx.ajax("GET", hxGet, {
						source: node,
						handler:function(elt, info) {
							done(info.xhr.responseText);
						}
					});
					return;
				}

				// Otherwise, perform a standard xhr request, then set 
				// node.preloadState = TRUE so that requests are not duplicated 
				// in the future.
				if (node.getAttribute("href")) {
					var r = new XMLHttpRequest();
					r.open("GET", node.getAttribute("href"));
					r.onload = function() {done(r.responseText);};
					r.send();
					return;
				}

Here we see that there are 2 different strategies for requesting content for preloading. For <a href=...> we will not send HX-* headers. It would not matter if Vary contains only HX-Boost or only HX-Request.

This is actually expected behavior according to preload extension docs. It seems reasonable to do this, since usually users clicking a <a href=...> will not trigger a AJAX request.

However, using hx-boost makes it so that <a href=...> requests will go via AJAX, and that the server can return only a partial, instead of the whole page. The server knows it has to return a partial based on these HX-Request and/or HX-Boosted headers.

The issue is that preload is not aware of hx-boost. So it asks for the server for a full page (by not adding HX-* headers), which ultimately preloads a request that will not be used, since boost will make a AJAX call, and not a "full reload".

Does that make more sense?

fbinz · 2024-02-04T13:17:38Z

It's all good. The trouble is actually very minor and very easy to patch manually from my side

Hey @yokomizor,
I'm running into the same issue of combining hx-boost with the preload extension. May I ask how you decided to solve the problem?

hlozancic · 2024-09-27T13:41:17Z

Same problem here... hx-boost and preload extension are not working together if you have partial renders.

marisst · 2024-11-06T08:48:37Z

@hlozancic @fbinz @yokomizor I have implemented a change which fixes this issue in PR#106 of the extensions repository. When this change is merged, hx-boosted elements will include HX-Request headers in the request. If you would like to use the updated extension already now, you can find it in my fork.

Telroshan · 2024-12-06T08:07:42Z

Fixed in #106

hirasso changed the title ~~Preload preloads urls but clicking them loads them again~~ Preload extension preloads URLs but navigating there loads the data again Nov 26, 2021

bencroker mentioned this issue Aug 29, 2023

Update preload extension docs bigskysoftware/htmx#1741

Merged

marisst mentioned this issue Nov 5, 2024

Preload extension: hx-boost, no side effects, HX-Preloaded header, form support, tests and JSDocs #106

Merged

Telroshan transferred this issue from bigskysoftware/htmx Nov 7, 2024

Telroshan closed this as completed Dec 6, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Preload extension preloads URLs but navigating there loads the data again #108

Preload extension preloads URLs but navigating there loads the data again #108

hirasso commented Nov 26, 2021 •

edited

Loading

benpate commented Dec 2, 2021

hirasso commented Dec 6, 2021

benpate commented Dec 7, 2021

hirasso commented Dec 13, 2021 •

edited

Loading

benpate commented Dec 13, 2021

hirasso commented Aug 22, 2022

clanofartisans commented Oct 14, 2022

benpate commented Oct 14, 2022

Alex-Rafter commented Nov 3, 2022

benpate commented Nov 3, 2022

onemoreahmad commented Apr 12, 2023

benpate commented Apr 14, 2023

onemoreahmad commented Apr 15, 2023 •

edited

Loading

benpate commented May 1, 2023

jwhitaker-swiftnav commented May 8, 2023 •

edited

Loading

yokomizor commented Sep 22, 2023

benpate commented Sep 22, 2023

yokomizor commented Sep 23, 2023 •

edited

Loading

fbinz commented Feb 4, 2024

hlozancic commented Sep 27, 2024

marisst commented Nov 6, 2024

Telroshan commented Dec 6, 2024

Preload extension preloads URLs but navigating there loads the data again #108

Preload extension preloads URLs but navigating there loads the data again #108

Comments

hirasso commented Nov 26, 2021 • edited Loading

benpate commented Dec 2, 2021

hirasso commented Dec 6, 2021

benpate commented Dec 7, 2021

hirasso commented Dec 13, 2021 • edited Loading

benpate commented Dec 13, 2021

hirasso commented Aug 22, 2022

clanofartisans commented Oct 14, 2022

benpate commented Oct 14, 2022

Alex-Rafter commented Nov 3, 2022

benpate commented Nov 3, 2022

onemoreahmad commented Apr 12, 2023

benpate commented Apr 14, 2023

onemoreahmad commented Apr 15, 2023 • edited Loading

1 instantclick.io

2 quicklink

3 using HTMX Preload extension

benpate commented May 1, 2023

jwhitaker-swiftnav commented May 8, 2023 • edited Loading

yokomizor commented Sep 22, 2023

benpate commented Sep 22, 2023

yokomizor commented Sep 23, 2023 • edited Loading

fbinz commented Feb 4, 2024

hlozancic commented Sep 27, 2024

marisst commented Nov 6, 2024

Telroshan commented Dec 6, 2024

hirasso commented Nov 26, 2021 •

edited

Loading

hirasso commented Dec 13, 2021 •

edited

Loading

onemoreahmad commented Apr 15, 2023 •

edited

Loading

jwhitaker-swiftnav commented May 8, 2023 •

edited

Loading

yokomizor commented Sep 23, 2023 •

edited

Loading