base16 theme: number values have the same color as keys in YAML files

[antoineco]

What version of bat are you using?

0.15.4

Describe the bug you encountered:

Number values have the same color as keys in YAML files:

Describe what you expected to happen?

A different color. Here's how Vim displays the same file with Base16 Vim (from the author of Base16):

How did you install bat?

From the bat-v0.15.4-x86_64-unknown-linux-gnu.tar.gz archive on the release page.

[sharkdp]

Thank you for reporting this.

The base16 theme is - as the name indicates - severely limited in the choice of colors.

Try one of the other color schemes and you should see a difference in color.

[sharkdp]

That being said, you can try to copy the base16.tmTheme to your bat --config-dir, modify it, and run bat --cache --build (see README for details).

If you think that there is a way to fix this without introducing "color clashes" in other languages, I'm glad to change it.

[antoineco]

Sounds good, I'll give it a try 👍 I don't mind the restricted choice of colors, as long as the values are easy to distinguish from the keys.

I guess I also have to inspect the YAML parser to see what tags it generates for numbers. You use https://docs.rs/syntect/3.2.0/syntect/parsing/index.html for that in bat, right?

[sharkdp]

Yes

[antoineco]

I ended up downloading the base16-default-dark theme from https://github.com/chriskempson/base16-textmate/tree/master/Themes and I'm happy-ish with the result: numbers are printed in a different color just like I expected.

I decided against modifying the built-in theme contributed by @mk12 because it is very different from the "original" tmTheme linked above, and I'm not familiar enough with the tmTheme format to really understand the implications of a change on other languages.

Closing

[mk12]

... because it is very different from the "original" tmTheme ...

The file looks different because of how base16 works. Here's how I think of it:

• A base is a number from 0 to 15.
• A color is a #rrggbb color.
• A scheme is a function base → color.
• A template_A is a function scheme → theme_A.
• A theme_A is a color theme config file for the app A.
• An ansi code is a number from 0 to 15.
• A theme_terminal can be represented as a function ansi code → color.

Therefore:

    template_terminal
= scheme → theme_terminal
= scheme → (ansi code → color)
= (base → color) → (ansi code → color)

Then,

• Let T be a template_terminal, f be a scheme, a be an ansi code
• Then T(f) is a theme_terminal and T(f)(a) is a color
• We can define T by T(f)(a) = f(g(a)) where g is a function ansi code → base

So terminal templates are defined by mappings from ansi code to base. If you look at the templates for iterm2, kitty, or any other terminal, you'll find the same mapping:

a        g(a)        a        g(a)
=======  ======      =======  ======
color00  base00      color08  base03
color01  base08      color09  base09
color02  base0B      color10  base01
color03  base0A      color11  base02
color04  base0D      color12  base04
color05  base0E      color13  base06
color06  base0C      color14  base0F
color07  base05      color15  base07

This is because the styling guidelines are fairly precise, e.g., base08 is "diff deleted", so schemes always use a red-ish color, and terminal templates map color1 (red) to base08.

Now, bat uses TextMate theme files, so I based my theme_bat on template_TextMate. I replaced the base placeholders in that file with ansi code values (actually specially encoded #rrggbbaa colors of the form #kk000000 which indicate the ansi code kk). Specifically, I replaced each base b with g⁻¹(b), to match how the terminal templates work.

The upshot is that if you use a base16 theme in your terminal derived from base16 scheme S, then bat with its base16 theme will highlight code using S. A downside, as you've noticed in the YAML file, is that these levels of indirection (highlight group → base16 base → ansi code → color) can result in less precise highlighting compared to downloading and using a purpose-built theme (highlight group → color). So by instead downloading a concrete base16 theme for TextMate and using that with bat, you might like the colors better, but you're losing the whole purpose of bat's base16 theme — when you change your terminal colors (e.g. alternating light/dark themes), bat's highlighting will not change with it.

[antoineco]

Wow @mk12 that's a fantastic explanation! Thanks a lot for taking the time to go into so much detail! 🙌

[antoineco]

BTW, I forgot to mention that I was indeed using a base16 theme in my terminal.

What I still don't get is why the highlighting of one group (YAML: numbers) is an entirely different color (let's call them "red" vs "orange" although I know it's based on my term's colors) in the bat theme compared to the theme I downloaded? Is it because highlight groups and colors were matched in a different fashion?

[mk12]

Hm, yeah that is odd.

In my setup (kitty terminal, base16-kitty template, base16-solarized scheme, bat --theme base16), bat shows YAML keys in red and number values in orange. There was an issue #979 in 0.14.0 where base16 would only use non-bright colors, which would cause orange to become red. Except you said you're on 0.15.4... The only other thing I can think of is maybe the base16 template for your terminal does not follow the pattern I described — maybe it doesn't map color9 (ansi bright red) to base09 (usually orange-ish) like the kitty and iterm2 ones do.

[antoineco]

I'm using base16-default-dark from base16-shell, and base09 is indeed the orange from my first Vim screenshot (terminal Vim, not GUI Vim).

I'm not sure whether that could be an issue, but I didn't refresh the bat cache after updating it from 0.14 to 0.15. I'll give it another try, also with different base16 terminal themes.

[mk12]

Ah, I think base16-shell would be the problem. A downside of mapping 16 ansi codes to base16 is that the bright colors don’t really make sense — bright red = orange is fine, but for example bright green = dark gray. This is because base16 needs a range of monochrome shades (comments, status bar, selection background, etc.) and it has to put them somewhere. So terminal programs assuming bright green really is “bright green” will have odd looking (or worse, unreadable) output. Base16-shell attempts to solve this by mapping the bright colors to be the same as the non-bright ones (so bright green is the same as green) and instead placing the other 8 base16 colors elsewhere in the 256 terminal color palette. I’ve argued here that this defeats the purpose of base16: https://github.com/chriskempson/base16/issues/174 In my experience the problems of terminal programs using unreadable bright colors is pretty rare. When it does happen, I have a script that converts output to replace bright colors with non-bright colors: https://github.com/mk12/scripts/blob/master/unbrighten.py

…

On Sun, Jul 19, 2020 at 12:35 PM Antoine Cotten ***@***.***> wrote: I'm using base16-default-dark from base16-shell <https://github.com/chriskempson/base16-shell>, and base09 is indeed the orange from my first Vim screenshot (terminal Vim, not GUI Vim). I'm not sure whether that could be an issue, but I didn't refresh the bat cache after updating it from 0.14 to 0.15. I'll give it another try, also with different base16 terminal themes. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1033 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACNKAPUPDQJABMF5PBYXA3R4MOFBANCNFSM4NN3IPNQ> .

[antoineco]

Yeah, I've been there with Solarized too. I used to force myself to use it the way it should be used, and configure my tools to output colors in the 256 colors palette instead of the bright ANSI colors... until I got tired of it and switched to base16-shell for it's simplicity and better compatibility.

So far the experience has been amazing, and I've had everything you described in the linked issue: change the terminal theme and everything matches instantly in Vim and tmux. I love it.

I'm a bit reluctant to mapping 16 colors instead of 8 in my terminal just to be able to use bat's built-in base16 theme. I know that's how it should be, but the reality is that a few of the tools I use regularly assume bright colors are really the bright equivalent to the 8 ANSI colors.

I may try your script though, eventually 👍

[mk12]

Got it, makes sense.

In this case, I think a better solution for you would be to create a base16-256 bat theme which is like base16 but uses ANSI codes 16-21 instead of the bright colors, to match how base16-shell works. I've just done this in #1111 — feel free to try it out before it's released by copying the file to ~/.config/bat/themes and running bat cache --build.