Error: Unknown language: "plaintext code-output"

[Seelengrab]

I'm running into this error when trying to display some textoutput of my code while fully prerendering with highlight.js, i.e. serve(prerender=true). Full error here:

Could not find the language 'plaintext code-output', did you forget to load/include a language module?
/home/sukera/Documents/projects/blog/node_modules/highlight.js/lib/core.js:2093
      throw new Error('Unknown language: "' + languageName + '"');
      ^

Error: Unknown language: "plaintext code-output"
    at _highlight (/home/sukera/Documents/projects/blog/node_modules/highlight.js/lib/core.js:2093:13)
    at Object.highlight (/home/sukera/Documents/projects/blog/node_modules/highlight.js/lib/core.js:1718:9)
    at [eval]:1:6326
    at Script.runInThisContext (vm.js:120:18)
    at Object.runInThisContext (vm.js:309:38)
    at Object.<anonymous> ([eval]-wrapper:10:26)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at evalScript (internal/process/execution.js:94:25)
    at internal/main/eval_string.js:23:3

...long Franklin warning with all generated JS...

stacktrace:

Stacktrace:
  [1] pipeline_error
    @ ./process.jl:561 [inlined]
  [2] run(::Base.CmdRedirect; wait::Bool)
    @ Base ./process.jl:476
  [3] run
    @ ./process.jl:474 [inlined]
  [4] js2html(hs::String, jsbuffer::IOBuffer, matches::Vector{RegexMatch}, splitter::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/converter/html/prerender.jl:88
  [5] js_prerender_highlight(hs::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/converter/html/prerender.jl:75
  [6] postprocess_page(pg::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/write_page.jl:50
  [7] write_page(output_path::String, content::String; head::String, pgfoot::String, foot::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/write_page.jl:147
  [8] convert_and_write(root::String, file::String, head::String, pgfoot::String, foot::String, output_path::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/write_page.jl:197
  [9] process_file_err(case::Symbol, fpair::Pair{String, String}, head::String, pgfoot::String, foot::String, t::Float64)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/file_utils.jl:153
 [10] process_file(::Symbol, ::Pair{String, String}, ::String, ::Vararg{Any})
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/file_utils.jl:104
 [11] fd_fullpass(watched_files::NamedTuple{(:other, :infra, :md, :html, :literate), NTuple{5, Dict{Pair{String, String}, Float64}}}, join_to_prepath::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/franklin.jl:260
 [12] serve(; clear::Bool, verb::Bool, port::Int64, single::Bool, prerender::Bool, nomess::Bool, is_final_pass::Bool, no_fail_prerender::Bool, eval_all::Bool, silent::Bool, cleanup::Bool, on_write::Franklin.var"#247#250", log::Bool, host::String, show_warnings::Bool, fail_on_warning::Bool, launch::Bool, no_set_paths::Bool, join_to_prepath::String)
    @ Franklin ~/.julia/packages/Franklin/VHeTZ/src/manager/franklin.jl:124
 [13] top-level scope
    @ REPL[27]:1

Best I can tell, this is caused by this thing that's being passed to highlight.js:

console.log(\"<pre><code class=\\"plaintext code-output hljs\\">\" + hljs.highlight(\"SOME LONG STRING\", {'language': \"plaintext code-output\"}).value + \"</code></pre>\")

which is all autogenerated. I haven't figured out though where the language is coming from, all I know is that the way I generated that text was via some julia:./code/output.jl followed by \output{./code/output.jl}. So I suspect the culprit is either there or in the thing that's passing this to highlight.js, which should be

Franklin.jl/src/converter/html/prerender.jl

Lines 46 to 76 in 469e6fe

	function js_prerender_highlight(hs::String)::String
	# look for "<pre><code" and "</code></pre>" these will have been automatically generated
	# and therefore the regex can be fairly strict with spaces etc
	matches = collect(eachmatch(r"<pre><code\s*(class=\"?(?:language-)?(.*?)\"?)?\s*>|<\/code><\/pre>", hs))
	isempty(matches) && return hs
	# buffer to write the JS script
	jsbuffer = IOBuffer()
	write(jsbuffer, """const hljs = require('$(HIGHLIGHTJS[])');""")
	# string to separate the output of the different blocks
	splitter = "_>fdsplit<_"
	# go over each match and add the content to the jsbuffer
	for i ∈ 1:2:length(matches)-1
	# tokens are paired, no nesting
	co, cc = matches[i:i+1]
	# core code
	cs = subs(hs, nextind(hs, matchrange(co).stop), prevind(hs, matchrange(cc).start))
	# cs = escape_string(cs)
	# lang
	lang = co.captures[2]
	# un-escape code string
	cs = html_unescape(cs) |> escape_string
	# add to content of jsbuffer
	write(jsbuffer, """console.log("<pre><code class=\\"$lang hljs\\">" + hljs.highlight("$cs", {'language': "$lang"}).value + "</code></pre>");""")
	# in between every block, write $splitter so that output can be split easily
	i == length(matches)-1 || write(jsbuffer, """console.log('$splitter');""")
	end
	return js2html(hs, jsbuffer, matches, splitter)
	end

[tlienart]

Hello @Seelengrab, thanks for reporting, as mentioned on Slack recently and consequently added in the README, the prerender=true path will be deprecated and it's recommended to use prerender=false, minify=false.

There were some significant changes in the latest minor release of highlight js which I didn't have the time to look into.

For the next version of Franklin (~ around the corner), the logic to pre-render will be removed from Franklin and left to the user or to a separate package/plugin to make maintenance easier.

Hope that makes sense

[Seelengrab]

That's a shame - as far as I could tell, this was the only way so far to get patch highlighting in code blocks :/ Do you have some links for an alternative for prerendered highlighting I could look into and how I could set that up as a postprocessing step? I'm thinking of some automatically running postprocessing I could hook into when the HTML is fully generated. I'm not familiar with highlight.js, so I'm not sure if just shipping the full highlight.js is an option for me.

[tlienart]

In 88ec5b9 I added the possibility to specify the code-output class via a page variable code_output_class and, in particular, you can set it to empty. Consider the following:

# default (as before)
julia> fd2html("""
       ```!
       5+7
       ```
       """) |> println
→ evaluating code [_ceval_1] in (index.md)
<pre><code class="language-julia">5&#43;7</code></pre>
<pre><code class="plaintext code-output">12</code></pre>

# setting the code_output_class to empty (what I think you want)
julia> fd2html("""
       +++
       code_output_class=""
       +++

       ```!
       5+7
       ```
       """) |> println
→ evaluating code [_ceval_1] in (index.md)                                      
<pre><code class="language-julia">5&#43;7</code></pre>
<pre><code class="plaintext">12</code></pre>

In the second one the output only has class plaintext.

Could you try (by adding Franklin master) and let me know whether that solves your issue? if so I'll do a patch release with it.

[Seelengrab]

I think I already have a better fix:

diff --git a/src/converter/html/prerender.jl b/src/converter/html/prerender.jl
index dbbc819e..5f6b6bd8 100644
--- a/src/converter/html/prerender.jl
+++ b/src/converter/html/prerender.jl
@@ -46,7 +46,7 @@ highlight.js to pre-render them to HTML.
 function js_prerender_highlight(hs::String)::String
     # look for "<pre><code" and "</code></pre>" these will have been automatically generated
     # and therefore the regex can be fairly strict with spaces etc
-    matches = collect(eachmatch(r"<pre><code\s*(class=\"?(?:language-)?(.*?)\"?)?\s*>|<\/code><\/pre>", hs))
+    matches = collect(eachmatch(r"<pre><code\s*(class=\"?(?:(?:language-)?(?<lang>[^\ \">]+))?(?:\s+([^\ \">]+)?)*\"?)?\s*>|<\/code><\/pre>", hs))
     isempty(matches) && return hs
 
     # buffer to write the JS script
@@ -64,7 +64,7 @@ function js_prerender_highlight(hs::String)::String
         cs = subs(hs, nextind(hs, matchrange(co).stop), prevind(hs, matchrange(cc).start))
         # cs = escape_string(cs)
         # lang
-        lang = co.captures[2]
+        lang = co[:lang]
         # un-escape code string
         cs = html_unescape(cs) |> escape_string
         # add to content of jsbuffer

This regex extracts either the first class or whatever is written after language-. It should be compatible with the old regex that just naively assumed there only was one class, while still keeping all classes in the final output. With this, full highlighting during prerendering works again for me.

I've chosen [^\ \">] as the "catch all" match because that prevents over-eagerly matching of the rest of the string. If that patch looks good to you, I can make a PR so you can make a patch release.

In 88ec5b9 I added the possibility to specify the code-output class via a page variable code_output_class and, in particular, you can set it to empty. Consider the following:

Having that as part of the \output command itself would be very useful, to get syntax highlighting for included, but generated code :D Not pressing though, I'll work around it by defining my own lx_.

[tlienart]

nice work and that makes a lot of sense, I'll review the PR and patch release, thank you!

[tlienart]

btw I'd be curious to see what you meant with patch earlier? if you have a demo somewhere, I'll be glad to have a look

[Seelengrab]

Oh I was just using patch as a language for the code block, which highlights a block of text produced by diff or git diff. It looks like the one in the comment above, with the +/- lines in color. It's usually hard to read patches without color, which is why I added that & wanted to prerender it.

[tlienart]

Ah right, so an alternative for you would also have been to generate a bundle with that. But it's great that we fixed the pre-rendering, thanks for your PR!

[Seelengrab]

Yes, it would have been my alternative :) I prefer prerendering, since it keeps download/runtime dependencies down (also prevents reflowing due to DOM manipulations through JS - greatly enhances website rendering). Images & embedded video already take up a lot of data, adding more than necessary just adds to that :/