Skip to content

Commit

Permalink
Support prerendering of code blocks.
Browse files Browse the repository at this point in the history 
Passing `prerender=true` to `Documenter.HTML` now renders syntax
highlighting of code blocks when building the page rather than
in the browser when the page is loaded. This requires a `node`
(NodeJS) executable in PATH or that a path to an executable is
passed as the `node` keyword argument. The `highlightjs` keyword
argument can be used to give a (local) file path to a custrom
hightlight.js library.

Example configuration:

  using Documenter, NodeJS_16_jll

  makedocs(;
      format = Documenter.HTML(
          prerender = true,            # enable prerendering
          node = NodeJS_16_jll.node(), # specify node executable (required if not available in PATH)
          # ...
      )
      # ...
  )

This feature is marked as experimental for now.
  • Loading branch information
@fredrikekre
fredrikekre committed Jul 7, 2021
1 parent 7cd120e commit a974c49
Show file tree
Hide file tree
Showing 2 changed files with 110 additions and 10 deletions.
16 changes: 16 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,21 @@

* ![Feature][badge-feature] `@example`- and `@repl`-blocks now support colored output by mapping ANSI escape sequences to HTML. This requires Julia >= 1.6 and passing `ansicolor=true` to `Documenter.HTML` (e.g. `makedocs(format=Documenter.HTML(ansicolor=true, ...), ...)`). In Documenter 0.28.0 this will be the default so to (preemptively) opt-out pass `ansicolor=false`. ([#1441][github-1441], [#1628][github-1628])

* ![Experimental][badge-experimental] ![Feature][badge-feature] Documenter's HTML output can now prerender syntax highlighting of code blocks, i.e. syntax highlighting is applied when generating the HTML page rather than on the fly in the browser after the page is loaded. This requires (i) passing `prerender=true` to `Documenter.HTML` and (ii) a `node` (NodeJS) executable available in `PATH`. A path to a `node` executable can be specified by passing the `node` keyword argument to `Documenter.HTML` (for example from the `NodeJS_16_jll` Julia package). In addition, the `highlightjs` keyword argument can be used to specify a file path to a highlight.js library (if this is not given the release used by Documenter will be used). Example configuration:
```julia
using Documenter, NodeJS_16_jll

makedocs(;
format = Documenter.HTML(
prerender = true, # enable prerendering
node = NodeJS_16_jll.node(), # specify node executable (required if not available in PATH)
# ...
)
# ...
)
```
_This feature is experimental and subject to change in future releases._ ([#1627][github-1627])

* ![Bugfix][badge-bugfix] Dollar signs in the HTML output no longer get accidentally misinterpreted as math delimiters in the browser. ([#890](github-890), [#1625](github-1625))

## Version `v0.27.3`
Expand Down Expand Up @@ -852,6 +867,7 @@
[github-1616]: https://github.com/JuliaDocs/Documenter.jl/pull/1616
[github-1617]: https://github.com/JuliaDocs/Documenter.jl/pull/1617
[github-1625]: https://github.com/JuliaDocs/Documenter.jl/pull/1625
[github-1627]: https://github.com/JuliaDocs/Documenter.jl/pull/1627
[github-1628]: https://github.com/JuliaDocs/Documenter.jl/pull/1628

[julia-38079]: https://github.com/JuliaLang/julia/issues/38079
Expand Down
104 changes: 94 additions & 10 deletions src/Writers/HTMLWriter.jl
View file
Original file line number Diff line number Diff line change
Expand Up @@ -325,7 +325,7 @@ Defaults to `true`.
**`highlights`** can be used to add highlighting for additional languages. By default,
Documenter already highlights all the ["Common" highlight.js](https://highlightjs.org/download/)
languages and Julia (`julia`, `julia-repl`). Additional languages must be specified by"
languages and Julia (`julia`, `julia-repl`). Additional languages must be specified by
their filenames as they appear on [CDNJS](https://cdnjs.com/libraries/highlight.js) for the
highlight.js version Documenter is using. E.g. to include highlighting for YAML and LLVM IR,
you would set `highlights = ["llvm", "yaml"]`. Note that no verification is done whether the
Expand All @@ -349,6 +349,16 @@ and the [Julia Programming Language](https://julialang.org/)."`.
**`warn_outdated`** inserts a warning if the current page is not the newest version of the
documentation.
## Experimental options
**`prerender`** a boolean (`true` or `false` (default)) for enabling prerendering/build
time application of syntax highlighting of code blocks. Requires a `node` (NodeJS)
executable to be available in `PATH` or to be passed as the `node` keyword.
**`node`** path to a `node` (NodeJS) executable used for prerendering.
**`highlightjs`** file path to custom highglight.js library to be used with prerendering.
# Default and custom assets
Documenter copies all files under the source directory (e.g. `/docs/src/`) over
Expand Down Expand Up @@ -393,6 +403,9 @@ struct HTML <: Documenter.Writer
mathengine :: Union{MathEngine,Nothing}
footer :: Union{Markdown.MD, Nothing}
ansicolor :: Bool
prerender :: Bool
node :: Union{Cmd,String,Nothing}
highlightjs :: Union{String,Nothing}
lang :: String
warn_outdated :: Bool

Expand All @@ -409,12 +422,53 @@ struct HTML <: Documenter.Writer
mathengine :: Union{MathEngine,Nothing} = KaTeX(),
footer :: Union{String, Nothing} = "Powered by [Documenter.jl](https://github.com/JuliaDocs/Documenter.jl) and the [Julia Programming Language](https://julialang.org/).",
ansicolor :: Bool = false, # true in 0.28
prerender :: Bool = false,
node :: Union{Cmd,String,Nothing} = nothing,
highlightjs :: Union{String,Nothing} = nothing,
lang :: String = "en",
warn_outdated :: Bool = true,

# deprecated keywords
edit_branch :: Union{String, Nothing, Default} = Default(nothing),
lang :: String = "en",
warn_outdated :: Bool = true
)
collapselevel >= 1 || throw(ArgumentError("collapselevel must be >= 1"))
if prerender
node = node === nothing ? Sys.which("node") : node
if node === nothing
@error "HTMLWriter: no node executable given or found on the system. Setting `prerender=false`."
prerender = false
else
if !success(`$node --version`)
@error "HTMLWriter: bad node executable at $node. Setting `prerender=false`."
prerender = false
end
end
end
if prerender && highlightjs === nothing
# Try to download
curl = Sys.which("curl")
if curl === nothing
@error "HTMLWriter: no highlight.js file given and no curl executable found " *
"on the system. Setting `prerender=false`."
prerender = false
else
@debug "HTMLWriter: downloading highlightjs"
r = Utilities.JSDependencies.RequireJS([])
RD.highlightjs!(r, highlights)
libs = sort!(collect(r.libraries); by = first) # puts highlight first
key = join((x.first for x in libs), ',')
highlightjs = get!(HLJSFILES, key) do
path, io = mktemp()
for lib in libs
println(io, "// $(lib.first)")
run(pipeline(`$(curl) -fsSL $(lib.second.url)`; stdout=io))
println(io)
end
close(io)
return path
end
end
end
assets = map(assets) do asset
isa(asset, HTMLAsset) && return asset
isa(asset, AbstractString) && return HTMLAsset(assetclass(asset), asset, true)
Expand All @@ -440,10 +494,14 @@ struct HTML <: Documenter.Writer
end
isa(edit_link, Default) && (edit_link = edit_link[])
new(prettyurls, disable_git, edit_link, canonical, assets, analytics,
collapselevel, sidebar_sitename, highlights, mathengine, footer, ansicolor, lang, warn_outdated)
collapselevel, sidebar_sitename, highlights, mathengine, footer,
ansicolor, prerender, node, highlightjs, lang, warn_outdated)
end
end

# Cache of downloaded highlight.js bundles
const HLJSFILES = Dict{String,String}()

"Provides a namespace for remote dependencies."
module RD
using JSON
Expand Down Expand Up @@ -485,7 +543,7 @@ module RD
"highlight",
"https://cdnjs.cloudflare.com/ajax/libs/highlight.js/$(hljs_version)/highlight.min.js"
))
prepend!(languages, ["julia", "julia-repl"])
languages = ["julia", "julia-repl", languages...]
for language in languages
language = jsescape(language)
push!(r, RemoteLibrary(
Expand Down Expand Up @@ -663,7 +721,9 @@ function render(doc::Documents.Document, settings::HTML=HTML())
RD.jquery, RD.jqueryui, RD.headroom, RD.headroom_jquery,
])
RD.mathengine!(r, settings.mathengine)
RD.highlightjs!(r, settings.highlights)
if !settings.prerender
RD.highlightjs!(r, settings.highlights)
end
for filename in readdir(joinpath(ASSETS, "js"))
path = joinpath(ASSETS, "js", filename)
endswith(filename, ".js") && isfile(path) || continue
Expand Down Expand Up @@ -1364,7 +1424,7 @@ end

function domify(ctx, navnode, node)
fixlinks!(ctx, navnode, node)
mdconvert(node, Markdown.MD(); footnotes=ctx.footnotes)
mdconvert(node, Markdown.MD(); footnotes=ctx.footnotes, settings=ctx.settings)
end

function domify(ctx, navnode, anchor::Anchors.Anchor)
Expand Down Expand Up @@ -1694,15 +1754,17 @@ mdconvert(b::Markdown.BlockQuote, parent; kwargs...) = Tag(:blockquote)(mdconver

mdconvert(b::Markdown.Bold, parent; kwargs...) = Tag(:strong)(mdconvert(b.text, parent; kwargs...))

function mdconvert(c::Markdown.Code, parent::MDBlockContext; kwargs...)
function mdconvert(c::Markdown.Code, parent::MDBlockContext; settings::HTML=HTML(), kwargs...)
@tags pre code
language = Utilities.codelang(c.language)
class = isempty(language) ? "nohighlight" : "language-$(language)"
if language == "documenter-ansi" # From @repl blocks (through MultiCodeBlock)
return pre(domify_ansicoloredtext(c.code, "nohighlight"))
else
return pre(code[".$class"](c.code))
elseif settings.prerender && !(isempty(language) || language == "nohighlight")
r = hljs_prerender(c, settings)
r !== nothing && return r
end
return pre(code[".$class"](c.code))
end
function mdconvert(mcb::Documents.MultiCodeBlock, parent::MDBlockContext; kwargs...)
@tags pre br
Expand All @@ -1723,6 +1785,28 @@ function mdconvert(mcb::Documents.MultiCodeBlock, parent::MDBlockContext; kwargs
end
mdconvert(c::Markdown.Code, parent; kwargs...) = Tag(:code)(c.code)

function hljs_prerender(c::Markdown.Code, settings::HTML)
@assert settings.prerender "unreachable"
@tags pre code
lang = Utilities.codelang(c.language)
hljs = settings.highlightjs
js = """
const hljs = require('$(hljs)');
console.log(hljs.highlight($(repr(c.code)), {'language': "$(lang)"}).value);
"""
out, err = IOBuffer(), IOBuffer()
try
run(pipeline(`$(settings.node) -e "$(js)"`; stdout=out, stderr=err))
str = String(take!(out))
# prepend nohighlight to stop runtime highlighting
# return pre(code[".nohighlight $(lang) .hljs"](Tag(Symbol("#RAW#"))(str)))
return pre(code[".language-$(lang) .hljs"](Tag(Symbol("#RAW#"))(str)))
catch e
@error "HTMLWriter: prerendering failed" exception=e stderr=String(take!(err))
end
return nothing
end

mdconvert(h::Markdown.Header{N}, parent; kwargs...) where {N} = DOM.Tag(Symbol("h$N"))(mdconvert(h.text, h; kwargs...))

mdconvert(::Markdown.HorizontalRule, parent; kwargs...) = Tag(:hr)()
Expand Down

0 comments on commit a974c49

Please sign in to comment.