Pandoc error when rendering Rmd included in roxygen documentation during devtools::document()

[cderv]

We are reporting this here for now. cc @EmilHvitfeldt

devtools::document()
#> ℹ Updating parsnip documentation
#> ℹ Loading parsnip
#> Could not fetch https://latex.codecogs.com/png.image?%5Cdpi%7B110%7D&space;%5Cbg_white&space;%0A%5Ceta_%7Bi%7D%20%3D%20%28%5Cbeta_0%20%2B%20b_%7B0i%7D%29%20%2B%20%5Cbeta_1x_%7Bi1%7D%0A
#> HttpExceptionRequest Request {
#>   host                 = "latex.codecogs.com"
#>   port                 = 443
#>   secure               = True
#>   requestHeaders       = []
#>   path                 = "/png.image"
#>   queryString          = "?%5Cdpi%7B110%7D&space;%5Cbg_white&space;%0A%5Ceta_%7Bi%7D%20%3D%20%28%5Cbeta_0%20%2B%20b_%7B0i%7D%29%20%2B%20%5Cbeta_1x_%7Bi1%7D%0A"
#>   method               = "GET"
#>   proxy                = Nothing
#>   rawBody              = False
#>   redirectCount        = 10
#>   responseTimeout      = ResponseTimeoutDefault
#>   requestVersion       = HTTP/1.1
#> }
#> (InternalException (HostCannotConnect "latex.codecogs.com" [Network.Socket.connect: <socket: 24>: does not exist (Connection refused)]))
#> Error: pandoc document conversion failed with error 61


traceback()
#> 16: stop(..., call. = FALSE)
#> 15: stop2("pandoc document conversion failed with error ", result)
#> 14: pandoc_convert(input = output_file, to = "html", from = variant, 
#>                    output = preview_file, options = args, verbose = verbose)
#> 13: output_format$post_processor(front_matter, input, output_file, 
#>                                  clean, !quiet)
#> 12: rmarkdown::render(rmd_path, output_format = rmarkdown::github_document(), 
#>                       output_file = md_path, quiet = TRUE, envir = new_environment(parent = global_env()))
#> 11: roxy_tag_rd.roxy_tag_includeRmd(tag, env = env, base_path = base_path)
#> 10: roxy_tag_rd(tag, env = env, base_path = base_path)
#> 9: inherits(x, "RoxyTopic")
#> 8: rd$add(roxy_tag_rd(tag, env = env, base_path = base_path))
#> 7: block_to_rd.roxy_block(block, base_path, env)
#> 6: block_to_rd(block, base_path, env)
#> 5: roclet_process.roclet_rd(X[[i]], ...)
#> 4: FUN(X[[i]], ...)
#> 3: lapply(roclets, roclet_process, blocks = blocks, env = env, base_path = base_path)
#> 2: roxygen2::roxygenise(pkg$path, roclets)
#> 1: devtools::document()

Reprex repo by @EmilHvitfeldt: https://github.com/EmilHvitfeldt/codecogsreprex

This happens because we made math_method: "webtex" the default in github_document(), which is called by roxygen @includeRmd:
https://github.com/r-lib/roxygen2/blob/7b706c9f7f2e4227c98928a67c5dcb51d88ec15d/R/rd-include-rmd.R#L46-L52

This happens when math are using in the Rmd or md file which is included.

It seems like codecog is not responded well when the rendering is done within the context of devtools::document().
Maybe something related to User-Agent being checked. From @gaborcsardi investigation

curl -o /dev/null -v -H 'User-Agent: Firefox' 'https://latex.codecogs.com/png.image?%5Cdpi%7B110%7D&space;%5Cbg_white&space;%0A%5Ceta_%7Bi%7D%20%3D%20%28%5Cbeta_0%20%2B%20b_%7B0i%7D%29%20%2B%20%5Cbeta_1x_%7Bi1%7D%0A'

cc @juliasilge

[cderv]

I can't reproduce using Pandoc 2.17.1.1 on Windows and Linux. It seems like it could be related to environment-related thing. 🤔

[cderv]

From @EmilHvitfeldt:

the issue happens without roxygen2

> rmarkdown::render("man/rmd/foo.md", output_format = rmarkdown::github_document())
/Applications/RStudio.app/Contents/MacOS/quarto/bin/pandoc +RTS -K512m -RTS foo.md --to gfm-yaml_metadata_block --from markdown+autolink_bare_uris+tex_math_single_backslash --output foo.md --template /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/rmarkdown/rmarkdown/templates/github_document/resources/default.md '--webtex=https://latex.codecogs.com/png.image?%5Cdpi%7B110%7D&space;%5Cbg_white&space;' 
/Applications/RStudio.app/Contents/MacOS/quarto/bin/pandoc +RTS -K512m -RTS foo.md --to html4 --from gfm --output foo.html --standalone --self-contained --highlight-style pygments --template /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/rmarkdown/rmarkdown/templates/github_document/resources/preview.html --variable 'github-markdown-css:/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/rmarkdown/rmarkdown/templates/github_document/resources/github.css' --metadata pagetitle=PREVIEW '--webtex=https://latex.codecogs.com/png.image?%5Cdpi%7B110%7D&space;%5Cbg_white&space;' 
Could not fetch https://latex.codecogs.com/png.image?%5Cdpi%7B110%7D&space;%5Cbg_white&space;%0A%5Ceta_%7Bi%7D%20%3D%20%28%5Cbeta_0%20%2B%20b_%7B0i%7D%29%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%0A
HttpExceptionRequest Request {
  host                 = "latex.codecogs.com"
  port                 = 443
  secure               = True
  requestHeaders       = []
  path                 = "/png.image"
  queryString          = "?%5Cdpi%7B110%7D&space;%5Cbg_white&space;%0A%5Ceta_%7Bi%7D%20%3D%20%28%5Cbeta_0%20%2B%20b_%7B0i%7D%29%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%2B%20%5Cbeta_1x_%7Bi1%7D%20%0A"
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (InternalException (HostCannotConnect "latex.codecogs.com" [Network.Socket.connect: <socket: 69>: does not exist (Connection refused)]))
Error: pandoc document conversion failed with error 61

[cderv]

It seems I can reproduce intermittently in some situations. Still looking into this to understand the why and the correct solution.

[cderv]

After looking deeper into this, I can reproduce sometimes on my side. I believe this comes from the fact that by default github_document() has html_preview = TRUE and build a HTML self contained preview in addition to the md file. This would explain the need for Pandoc to reach out to codecogs with a query. When building the md file, it should only build the URL inside the md and the request would be done, when HTML file load.

Regarding this new default feature, I am still wondering if we should really use webtex by default. First it seems that this does not work for Rmd used in roxygen2 - I don't know how images from the web are ok to used in Help Page. I am looking at that currently, and report. Then, as I have mentioned initially in the PR (#2301), there is a Fair Usage limitation https://editor.codecogs.com/docs/3-fair_usage.php and I wonder if this does not comply to it.

So maybe we should make math_method = NULL the default, and just allow users to set to webtex (and maybe r-katex) so that then can render in Github document Markdown some equation that don't do well in Plain text.

Thanks for your repro reprex @EmilHvitfeldt it is helpful !

[github-actions]

This old thread has been automatically locked. If you think you have found something related to this, please open a new issue by following the issue guide (https://yihui.org/issue/), and link to this old issue if necessary.