Various breakages when formatting json heredocs

[dbaynard]

I've hit various issues, with 2.7.5 and earlier versions of caddy, when it comes to heredocs.

caddy fmt introduces spacing which means the resulting file is invalid.

See the caddyfile in https://github.com/dbaynard/caddy-fmt-heredoc-example — I recommend you step through the commits.

0. Empty commit

1. I created a sample, unformatted caddyfile, which works.

   	handle_path /never {
   		respond <<JSON
   		\{"response":"nope"\}
   		JSON 200
   	}
   	handle_path /json {
   		respond <<JSON
   		{"response":"nope"}
   		JSON 200
   	}

   The first appears to be returned as text/plain.

2. I adapted it, and it fails to load

   	handle_path /never {
   		respond <<JSON \{
   		"response":"nope"\}
   		JSON 200
   	}
   	handle_path /json {
   		respond <<JSON 
   		{"response":"nope"}
   		JSON 200
   	}

   Note the space at the end of the JSON for /json.

   Error: adapting config using caddyfile: parsing caddyfile tokens for 'handle_path': unrecognized directive: response - are you sure your Caddyfile structure (nesting and braces) is correct?, at Caddyfile:18

3. I removed the handle_path for /nope with escaped braces

Error: adapting config using caddyfile: parsing caddyfile tokens for 'handle_path': unrecognized directive: {"response":"nope"} - are you sure your Caddyfile structure (nesting and braces) is correct?, at Caddyfile:18

4. I removed the space after the heredoc market, and it's fine, again.

I tried with various different sigils, and saw no difference. In the repo, I have an example with a placeholder (that works correctly). And I have only tried the json formats.

Incidentally, the correct version of this would be helpful as an example.

[francislavoie]

Yeah, unfortunately caddy fmt doesn't handle heredocs at all right now. The problem is the Caddyfile lexer is completely separate code from the formatter. We'd have to re-implement heredoc lexing in the formatter as well, which is tedious. I want to rewrite the lexer to make it retain whitespace so that it can also be used for formatting, but that's a significant undertaking.

For now, unfortunately you'll need to avoid using caddy fmt if you're using heredocs.

[dbaynard]

Thanks for confirming. I'll skim the documentation — there may be a tweak I can make, and then I can close this.

[francislavoie]

Oh I missed this in your post:

The first appears to be returned as text/plain.

Yeah the heredoc tag is purely textual and doesn't affect mime parsing because it's just for the Caddyfile. You'll need to add a header Content-Type directive alongside the respond to send the right content type.

[dbaynard]

Thanks — yes, I did know about that. As the docs on the respond directive says,

If the body is non-empty, this directive sets the Content-Type header if it is not already set. The default value is text/plain; utf-8 unless the body is a valid JSON object or array, in which case it is set to application/json. For all other types of content, set the proper Content-Type explicitly using the header directive.

I thought this automatic detection was working correctly in the example with the non-escaped braces. Where I escaped the braces, I did need to add the header, contrary to the impression from the docs.

I only escaped the braces because in my example I had placeholders, and I initially thought there was an issue parsing the placeholder from the JSON (I reported a minimal repro; there is an example with a placeholder in the repro repo). Then I discovered caddy fmt was breaking the heredoc.

[francislavoie]

Ah I see, yeah it's not necessary to escape the JSON here. So your second handle-path should be what you use, and just avoid caddy fmt for now.

[francislavoie]

FYI @dbaynard, thanks to @bbaa-bbaa this should now be properly fixed 🎉

Also FYI @matthewpi since the vscode extension will be improved by this 😅

[dbaynard]

This is great — thanks @bbaa-bbaa

caddyserver/website#352 will need a change — I can note the first caddy version for which this feature works?

[francislavoie]

We'll just remove the note from caddyserver/website#352 when we release the next version.