Raw HTML doesn't work in Markdown

[srid]

Bug: add raw HTML to Markdown, and it will render as-is with everything escaped.

Possible culprit:: https://github.com/snapframework/xmlhtml/blob/54463f1691c7b31cc3c4c336a6fe328b1f0ebb95/src/Text/Blaze/Renderer/XmlHtml.hs#L27

This will require a good bit of debugging.

[hi-learn-triz-and-usit]

Slowly going through and learning about emanote while ruling out sources of the error. Dropping breadcrumbs here in case someone else wants to pick it up.

Using debug tracing, the following Note gets inserted into the model:

Note {_noteDoc = Pandoc (Meta {unMeta = fromList []}) [Header 1 ("emanote",[],[]) [Str "Emanote"],Para [Str "Welcome",Space,Str "to",Space,Link ("",[],[]) [Str "Emanote"] ("https://github.com/srid/emanote",""),Str "."],Para [Str "As",Space,Str "Emanote",Space,Str "is",Space,Str "still",Space,Str "a",Space,Strong [Str "work",Space,Str "in",Space,Str "progress"],Space,Str "there",Space,Str "is",Space,Str "no",Space,Str "documentation",Space,Str "here,",Space,Str "yet."],Para [RawInline (Format "html") "<b>",Str "b",Space,RawInline (Format "html") "</b>"]], _noteMeta = Null, _noteRoute = R[LMLType Md]:index}

Notably, RawInline (Format "html") "<b>" correctly occurs.

Next steps, see how models get rendered.

[srid]

Rendering of Pandoc AST happens in Heist.Extra.Splices.Pandoc. The raw HTML node (there are two versions: inline and block) in particular is handled here (the inline one),

emanote/src/Heist/Extra/Splices/Pandoc.hs

Lines 187 to 190 in e1a8e59

	B.RawInline (B.Format fmt) s ->
	if fmt == "html"
	then H.unsafeByteString $ encodeUtf8 s
	else H.pre ! A.class_ ("pandoc-raw-" <> show fmt) $ H.toHtml s

According to blaze-html's docs, the use of H.unsafeByteString should in theory render the HTML without escaping it. But the bug here is that it escapes it when rendered through pandoc splice ... due to the way xmlhtml handles it I guess, because we use that library's Text.Blaze.Renderer.XmlHtml.renderHtmlNodes in the pandoc splice:

emanote/src/Heist/Extra/Splices/Pandoc.hs

Line 34 in e1a8e59

pure $ RX.renderHtmlNodes $ renderPandocWith ctx doc

[hi-learn-triz-and-usit]

Tracing line 189 using (\x -> traceShow (H.renderHtml x) x) $ H.unsafeByteString $ encodeUtf8 s yields

[Debug#WS.Client.01] <~~ ERNoteHtml R[Html]:index
"<b>"
"</b>"

So H.unsafeByteString seems to be doing what it claims.

The bug seems to be in RX.renderHtmlNodes: it turns RawInline (Format "html") "<b>" into TextNode "<b>".

[hi-learn-triz-and-usit]

There seems to be a related SO question from years ago: https://stackoverflow.com/questions/19190125/how-can-i-embed-a-raw-html-string-into-a-text-xmlhtml-node-structure

[hi-learn-triz-and-usit]

How does neuron handle this, anyway?

[srid]

How does neuron handle this, anyway?

I don't understand this question. Can you rephrase? EDIT: Oh, you are asking about neuron. Neuron uses reflex-dom, and doesn't even use Heist or xmlhtml.

The SO question is a good find,

If you want a node to be treated as raw, just add the xmlhtmlRaw attribute to the tag.

So would this be a fix for this bug? To add xmlHtmlRaw in

emanote/src/Heist/Extra/Splices/Pandoc.hs

Lines 187 to 190 in e1a8e59

	B.RawInline (B.Format fmt) s ->
	if fmt == "html"
	then H.unsafeByteString $ encodeUtf8 s
	else H.pre ! A.class_ ("pandoc-raw-" <> show fmt) $ H.toHtml s

?

[hi-learn-triz-and-usit]

So would this be a fix for this bug? To add xmlHtmlRaw in

I haven't been able to get that to work without also wrapping it in a span tag, which I don't like, because it results in weird HTML like

<span><b></span>ab<span></b></span>

[srid]

That weird HTML thing you observed might be specific to blaze-html, because using xmlHtmlRaw in #43 (which uses xmlhtml directly instead of blaze) doesn't suffer from the same issue.