dollarmath: Add `allow_blank_lines` option #46

eric-wieser · 2022-06-29T14:35:35Z

Closes #45 behind a non-default allow_blank_lines=False flag.

Note this means that the default behavior of the javascript and python implementations has diverged.

welcome · 2022-06-29T14:35:37Z

Thanks for submitting your first pull request! You are awesome! 🤗

If you haven't done so already, check out EBP's Code of Conduct and our Contributing Guide, as this will greatly help the review process.

Welcome to the EBP community! 🎉

tests/fixtures/dollar_math.md

eric-wieser · 2022-06-30T08:04:45Z

I think a maintainer (@chrisjsewell?) has to click some sort of approval button in order for CI to run, to verify that I'm not mining cryptocurrency in this PR?

chrisjsewell · 2022-06-30T08:17:36Z

mdit_py_plugins/dollarmath/index.py

@@ -302,15 +302,14 @@ def _math_block_dollar(
                start = state.bMarks[nextLine] + state.tShift[nextLine]
                end = state.eMarks[nextLine]

-                if end - start < 2:


I think this should stay (but break not continue) to safeguard against any index errors

This line doesn't exist any more in the javascript version, why would you want to keep it in the Python version?

Because JavaScript does not act the same as Python; if you index a JavaScript array with "bad indexes" it will simply return null, whereas in python it will raise an indexerror.
There has been multiple bug fixed for this in markdown-it-py

The only indexing that takes place is state.src[start:end], and that will never give an IndexError. If start=end then it will return "", but that's absolutely fine.

@rowanc1: what can we do to progress here?

eric-wieser · 2022-06-30T08:30:36Z

@chrisjsewell: this is what I am referring to:

codecov · 2022-07-03T19:10:45Z

Codecov Report

Patch coverage: 100.00% and project coverage change: +0.05 🎉

Comparison is base (be14587) 92.84% compared to head (65d763a) 92.90%.

❗ Current head 65d763a differs from pull request most recent head 5412706. Consider uploading reports for the commit 5412706 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #46      +/-   ##
==========================================
+ Coverage   92.84%   92.90%   +0.05%     
==========================================
  Files          30       30              
  Lines        1748     1748              
==========================================
+ Hits         1623     1624       +1     
+ Misses        125      124       -1

Flag	Coverage Δ
pytests	`92.90% <100.00%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mdit_py_plugins/dollarmath/index.py	`91.07% <100.00%> (+0.59%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

eric-wieser · 2022-07-03T20:52:25Z

I assume the CI failure due to a bad interaction between click and black is bogus? Thanks for triggering the CI run.

eric-wieser · 2023-03-03T20:44:39Z

Can I do anything to help this to go through?

chrisjsewell · 2023-03-03T21:01:32Z

Can I do anything to help this to go through?

I feel this behaviour needs to be under a options flag, (a) because it is a breaking change and (b) because

This matches the behavior of LaTeX (and of the stackexchange and github markdown parsers), and prevents runaway parsing if a $$ is opened but not closed.

I would suggest this is not always the desired behaviour in MyST
Firstly because in MyST they are partially intended to be a shorthand for: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-math, which can have blank lines (and quite probably there is already people using this feature),
and (b) "runaway parsing" is actually a feature of Markdown not a bug, both fence blocks https://spec.commonmark.org/0.30/#example-126, and (extended) markdown div blocks https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#div are both "runaway" (and there are design reasons for this)

chrisjsewell · 2023-03-03T21:21:18Z

I think there is also a potentially critical difference between block level $$ and inline $$;
inline $$ already does not allow blank lines

eric-wieser · 2023-03-03T22:19:25Z

Does this mean you would advocate for the change to be reverted on the javascript side?

chrisjsewell · 2023-03-03T22:21:12Z

Does this mean you would advocate for the change to be reverted on the javascript side?

Oh I'm not touching lol. Just let's add it in here as a feature flag, and I think that's a good compromise

eric-wieser · 2023-03-03T22:27:42Z

I would suggest this is not always the desired behaviour in MyST Firstly because in MyST they are partially intended to be a shorthand for: https://www.sphinx-doc.org/en/master/usage/restructuredtext/directives.html#directive-math, which can have blank lines (and quite probably there is already people using this feature)

The semantics of .. math:: is to auto-insert an align environment when multiple lines are provided. This is pretty non-standard for a latex parser, possibly suggesting that the MyST behavior ought to be behind a flag

and (b) "runaway parsing" is actually a feature of Markdown not a bug, both fence blocks https://spec.commonmark.org/0.30/#example-126, and (extended) markdown div blocks https://htmlpreview.github.io/?https://github.com/jgm/djot/blob/master/doc/syntax.html#div are both "runaway" (and there are design reasons for this)

I agree; but on the other hand, it seems more valuable to match what the consensus seems to be among major implementations, rather than extrapolate from a spec that doesn't actually specify the behavior here!

Can I convince you of a feature flag where the default is the new behavior, or is that going to break too many things?

chrisjsewell · 2023-03-03T22:30:40Z

Can I convince you of a feature flag where the default is the new behavior, or is that going to break too many things?

Yeh just to much possibility for breakage for the outset. Of course once it is in there then it's easy to control, if defaults need to be changed

for more information, see https://pre-commit.ci

eric-wieser · 2023-03-03T22:37:24Z

Alright, I've added the flag to default to the old behavior.

eric-wieser · 2023-03-03T22:54:37Z

Note that the tests won't run until you approve them (not the PR, just the "yes you didn't mine crypto in CI" button):

for more information, see https://pre-commit.ci

chrisjsewell

Cheers Eric

welcome · 2023-03-06T18:15:21Z

Congrats on your first merged pull request in this project! 🎉

Thank you for contributing, we are very proud of you! ❤️

port executablebooks/markdown-it-dollarmath#8

094d58c

Copy across the tests from upstream

1879f17

eric-wieser commented Jun 29, 2022

View reviewed changes

tests/fixtures/dollar_math.md Outdated Show resolved Hide resolved

chrisjsewell requested changes Jun 30, 2022

View reviewed changes

chrisjsewell and others added 3 commits July 4, 2022 00:36

Merge branch 'master' into patch-1

2f3f3b8

Revert removal

3318fb1

Remove stray extra line

f76fe8c

eric-wieser mentioned this pull request Jul 25, 2022

display automatic translations to informal math leanprover-community/doc-gen#168

Open

This was referenced Mar 3, 2023

✨ NEW: Do not allow blank lines within $$ executablebooks/markdown-it-dollarmath#8

Merged

Switch to using markdownit for rendering leanprover-community/doc-gen#177

Draft

eric-wieser and others added 3 commits March 3, 2023 22:32

Add a feature flag to restore the old behavior

e74277d

[pre-commit.ci] auto fixes from pre-commit.com hooks

3d8489a

for more information, see https://pre-commit.ci

Update index.py

4312f81

eric-wieser and others added 2 commits March 3, 2023 23:28

Update tests

6dcc804

[pre-commit.ci] auto fixes from pre-commit.com hooks

65d763a

for more information, see https://pre-commit.ci

chrisjsewell approved these changes Mar 6, 2023

View reviewed changes

Merge branch 'master' into patch-1

5412706

chrisjsewell changed the title ~~port executablebooks/markdown-it-dollarmath#8~~ dollarmath: Add allow_blank_lines option Mar 6, 2023

chrisjsewell merged commit 061a544 into executablebooks:master Mar 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dollarmath: Add `allow_blank_lines` option #46

dollarmath: Add `allow_blank_lines` option #46

eric-wieser commented Jun 29, 2022 •

edited

Loading

welcome bot commented Jun 29, 2022

eric-wieser commented Jun 30, 2022

chrisjsewell Jun 30, 2022

eric-wieser Jun 30, 2022 •

edited

Loading

chrisjsewell Jun 30, 2022

eric-wieser Jun 30, 2022 •

edited

Loading

eric-wieser Jul 25, 2022

eric-wieser commented Jun 30, 2022

codecov bot commented Jul 3, 2022 •

edited

Loading

eric-wieser commented Jul 3, 2022

eric-wieser commented Mar 3, 2023

chrisjsewell commented Mar 3, 2023 •

edited

Loading

chrisjsewell commented Mar 3, 2023

eric-wieser commented Mar 3, 2023

chrisjsewell commented Mar 3, 2023

eric-wieser commented Mar 3, 2023

chrisjsewell commented Mar 3, 2023

eric-wieser commented Mar 3, 2023

eric-wieser commented Mar 3, 2023 •

edited

Loading

chrisjsewell left a comment

welcome bot commented Mar 6, 2023

dollarmath: Add allow_blank_lines option #46

dollarmath: Add allow_blank_lines option #46

Conversation

eric-wieser commented Jun 29, 2022 • edited Loading

welcome bot commented Jun 29, 2022

eric-wieser commented Jun 30, 2022

chrisjsewell Jun 30, 2022

Choose a reason for hiding this comment

eric-wieser Jun 30, 2022 • edited Loading

Choose a reason for hiding this comment

chrisjsewell Jun 30, 2022

Choose a reason for hiding this comment

eric-wieser Jun 30, 2022 • edited Loading

Choose a reason for hiding this comment

eric-wieser Jul 25, 2022

Choose a reason for hiding this comment

eric-wieser commented Jun 30, 2022

codecov bot commented Jul 3, 2022 • edited Loading

Codecov Report

eric-wieser commented Jul 3, 2022

eric-wieser commented Mar 3, 2023

chrisjsewell commented Mar 3, 2023 • edited Loading

chrisjsewell commented Mar 3, 2023

eric-wieser commented Mar 3, 2023

chrisjsewell commented Mar 3, 2023

eric-wieser commented Mar 3, 2023

chrisjsewell commented Mar 3, 2023

eric-wieser commented Mar 3, 2023

eric-wieser commented Mar 3, 2023 • edited Loading

chrisjsewell left a comment

Choose a reason for hiding this comment

welcome bot commented Mar 6, 2023

dollarmath: Add `allow_blank_lines` option #46

dollarmath: Add `allow_blank_lines` option #46

eric-wieser commented Jun 29, 2022 •

edited

Loading

eric-wieser Jun 30, 2022 •

edited

Loading

eric-wieser Jun 30, 2022 •

edited

Loading

codecov bot commented Jul 3, 2022 •

edited

Loading

chrisjsewell commented Mar 3, 2023 •

edited

Loading

eric-wieser commented Mar 3, 2023 •

edited

Loading