MediaWiki preformatted text as GitHub Flavour Markdown

[peterjc]

I've found a problem with what MediaWiki terms "Preformatted text" where each line starts with a space http://www.mediawiki.org/wiki/Help:Formatting once converted to GitHub Flavour Markdown (GFM) and viewed on GitHub.com or with GitHub Pages / Jekyll (on github.io).

This test case is using the example from http://www.mediawiki.org/wiki/Help:Formatting saved as preformatted.mediawiki

$ cat preformatted.mediawiki
The following example is intended to be pre-formatted text,
meaning the line breaks should be preserved in the rendered
output, but that italics and bold etc are still allowed:

 Start each line with a space.
 Text is '''preformatted''' and
 ''markups'' '''''can''''' be done.

End of three line pre-formatted block.

I converted it to GitHub Flavour Markdown using pandoc version 1.13.1,

$ pandoc -f mediawiki -t markdown_github preformatted.mediawiki 
The following example is intended to be pre-formatted text, meaning the line breaks should be preserved in the rendered output, but that italics and bold etc are still allowed:

`Start each line with a space.`
`Text is `**`preformatted`**` and`
*`markups`*` `***`can`***` be done.`

End of three line pre-formatted block.

For now, you can view these files here:

• https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted.mediawiki
• https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted.md

In MediaWiki we get a three line paragraph in a box.

In GFM we get three boxes all on the same line.

Note that how line breaks are handled is one of the differences in GFM to plain markdown.

It is unclear to me how best to map MediaWiki performatted block to GitHub Flavoured Markdown (GFM). My interpretation of the role of these blocks is a way to pre-set the line breaks, while still using formatting (italics etc). Presumably it was intended for poetry etc but our wiki instance renders it in a box, and so it has also been used for code/terminal output snippets.

Adding explicit <br /> tags works quite well, but does not give any block-level visual styling (which MediaWiki did give us -- and may be the author's purpose in picking this markup, but may not reflect the original meaning of the performatted block):

• https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted_alt.md

Leaving blank lines in the raw markdown results in vertically separated lines, not great:

• https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted_alt2.md

Using a GFM literal block fails to render any markup like italics etc, so that won't do in general (but would work on the specific wiki pages where I noticed this problem, since it was being used for terminal output snippets etc):

• https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted_alt3.md

The best idea I've had so far (for general usage) is to combine a block quote with manual line breaks:

• https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted_alt4.md

[zudov]

Representing mediawiki syntax with pandoc AST or markdown syntax looks tricky to me.

An interesting thing that I noticed, that the preformatted.md which you linked is renderd with three boxes at the same line, but if I copy paste the raw into the comment box, it gives me three boxes on three lines (that's what pandoc outputs). Like this:

Start each line with a space.
Text ispreformattedand
markups can be done.

I haven't yet figured out why is that.

[peterjc]

I'd noticed that too - and assumed it was a quirk with the GitHub comment rendering versus their rendering when viewing a markdown file, or on GitHub pages via Jekyll. Strange isn't it?

[zudov]

Yep, it's strange.

In fact most markdown implementations do not render this newlines. see

But the linebreak can be forced by inserting the spaces at the end of the line.

Spaces also work for GFM, see . Would it help you (seems better than <br>) ?

I am just trying to understand if it is an issue with pandoc's GFM writer. If it is, then appending spaces is one possible solution that can fix the writer.

[peterjc]

Trailing spaces does seem more elegant than <br />.

If you can get this to look good in multiple markdown flavours, that's great. I've only focussed on the GFM output and display (but long term hope to just use CommonMark instead).

[zudov]

Hmm. Here is an interesting thing I found github/markup#208 (comment)

Basically it says that GFM is only used in comments, PRs and other things like that.
And the actual markdown files in repos are using normal markdown.

[zudov]

Here is a better link https://help.github.com/articles/github-flavored-markdown/

GitHub uses "GitHub Flavored Markdown," or GFM, across the site--in issues, comments, and pull requests.

[jgm]

More accurately, I think, there are two kinds of github-flavored markdown.
There's the variant for comment boxes and the variant for READMEs, wiki pages, etc.
Neither is really "standard"; both have significant divergences from and extensions to original Markdown.

+++ Konstantin Zudov [Mar 05 15 08:34 ]:

Hmm. Here is an interesting thing I found [1]github/markup#208
(comment)

Basically it says that GFM is only used in comments, PRs and other
things like that.
And the actual markdown files in repos are using normal markdown.

—
Reply to this email directly or [2]view it on GitHub.

References

1. Task lists do not render in README.md github/markup#208 (comment)
2. MediaWiki preformatted text as GitHub Flavour Markdown #1993 (comment)

[zudov]

So this explains a lot. Pandoc generated GFM, renders correctly when you past it into a comment. If you want to generate the file which would be viewed as a plain markdown file you shouldn't use GFM.

[zudov]

@jgm Indeed, for example README's support the tables, fenced blocks, etc.

This line confused me a little:

It differs from standard Markdown (SM) in a few significant ways

I guess they just call markdown used in README's "standard"

[jgm]

You can always use markdown_github-hard_line_breaks.

So, can this issue be closed?

[peterjc]

I will try using markdown_github-hard_line_breaks with pandoc and get back to you.

I want to turn MediaWiki markup into whatever GFM GitHub is using to show README.md and other example.md files on GitHub.com, and have my GitHub Pages / Jekyll configuration setup to use the same rendering.

I'm not wanting to use GFM as shown in GitHub issues comments (if this is subtly different).

[zudov]

I want to turn MediaWiki markup into whatever GFM GitHub is using to show README.md and other example.md files on GitHub.com

As I said, that is tricky because MediaWiki isn't really compatible with markdown and it becomes even harder to make it alike when you can't control the stylesheet of the rendered html.

[peterjc]

Trying @jgm's suggestion,

$ pandoc -f mediawiki -t markdown_github-hard_line_breaks preformatted.mediawiki
The following example is intended to be pre-formatted text, meaning the
line breaks should be preserved in the rendered output, but that italics
and bold etc are still allowed:

`Start each line with a space.`  
`Text is `**`preformatted`**` and`  
*`markups`*` `***`can`***` be done.`

End of three line pre-formatted block.

Output recorded here in order to show the GitHub rendering: https://github.com/peterjc/peterjc.github.io/blob/preformatting/preformatted_gh_hlb.md

The key difference is two trailing spaces on the lines in the problematic block, exactly the suggestion from @zudov

This appears to solve my problem, closing issue. Thank you everyone for your input.