Add --max-syntax-highlighting-length, set to 400

[th1000s]

• Add --max-syntax-highlighting-length, set to 400

  --max-line-length increased to 3000, highlighting now stops after 400
  characters. In that case the highlighting may be incorrect until it
  is reset for the next hunk.

• Add truncate_str_short() which always returns a prefix of the input

  Document that the input of truncate_str() may be cut on a double width
  char ("fullwidth") - then the output is no longer a prefix of the input.

  truncate_str_short() could be faster by not constructing the result and
  instead only returning an index, but it will be used in rare edge cases so
  reusing existing logic is good enough.

  Common code moved to truncate_str_impl().

  Assume (with fallback) that graphemes are at most 2 wide.
  Fewer allocations.
  Tests.

---

See #290

Some rough benchmarking shows it is faster when truncating lines. But note that this has a good chance of ruining the highlighting of the following lines.

[dandavison]

Suggested change

	/// To always highlighting entire lines set to zero - but note that delta will be slow on very
	/// To always highlight entire lines set to zero - but note that delta will be slow on very

[dandavison]

(Ignore, I made this change and force-pushed to your last commit)

[dandavison]

Great, thanks! I tried it out a bit manually and I see (as you said somewhere else) that incorrect highlighting after a truncated line is quite common both on this branch and released main. (Is it feasible to fix that by resetting the highlighter manually in such cases?)

[th1000s]

HighlightLines itself doesn't provide a reset option, so the between-hunks resetting has to be made available in the relevant function. And it could still be wrong if an opening quote is never closed. Other options would be to tolerate one or two overly long lines here and there, and/or to not even start highlighting those lines. I'd wait and see for now, I think these lines are quite rare :)

[CGamesPlay]

Love this change! I hit this issue surprisingly frequently.

Is there any reason that max-line-length should not be 0 by default? What are the drawbacks of setting that?

[th1000s]

Since this PR solves the slow-to-highlight problem, the idea behind keeping the limit (increased to 3000) is to make it easy (or even possible at all) to scroll past such a diff line. Or worse, an entire block of those.

What kind of diff do you have in mind where a longer limit would help and still show useful information? I myself see the usual minified js/css files, but also in-line blobs const char* dat = "..."; or entire data files (which are valid ASCII/UTF-8, so git will print them).

[CGamesPlay]

Oh, I typically run into this when I turn off line wrapping and scroll right, trying to find the single-word difference in two long lines. Most frequently, this is either in a Markdown file or an NDJSON file. It feels very silly to have a maximum line length when word wrapping is disabled, but if I understand correctly, Delta is actually passing formatted output to less, so it doesn't really know that I have line-wrapping off ("chop long lines" / less -S / also you can toggle it by typing -S in a live less session).

[append]
I guess I could also add to this to say... an ideal tool would elide some of the identical prefix of the line to show the actual diff.