Chat format: Recognize specified language and offloaded lexguessing to every newline

[SinanAkkoyun]

#71 (comment)

It also looks like calling guess_lexer on every token is a little slow, so maybe it makes more sense to call it at the end of every line instead?

#71 (comment)

Sometimes the model specifies the language, use that (and/or incorporate a small prompt to ensure lang specification)

I found some time :) Now it's detecting the specified language (I like to finetune my models to always do that) and if not, it only lexguesses each newline.
The PR is nothing too big, but I thought this might help nonetheless.

Here is the prompt I found most effective for Llama2-7B-chat 4.0bpw to specify the language:
-sp "You are a helpful coding assistant. Always answer as helpfully as possible. Specify the language after starting a codeblock like: ```python\nprint('hello')\n"

#71 (comment)

So that won't really work for code snippets. Also it supports a lot of obscure formats so maybe that could be narrowed down a bit for more accurate results. I'm looking into it.

You said you wanted to improve the lexguesser, but still, if I can somehow help or if you want to spend your time on other problems, please let me know and I'll try to take care of it.

[SinanAkkoyun]

I just tested Mistral 7B's coding skills and noticed that the current chat code only treats ``` when it's one chunk/token. Mistral outputs it as `` and ` or so. Fix commit ahead

[SinanAkkoyun]

@turboderp The Mistral 7B chunking has been fixed! If you find any bug with that please let me know