crash reading file with UTF-8 BOM + "# coding: utf-8" (supported since python 2.5)

[lepistone]

Reading such a file fails at

https://github.com/mitsuhiko/babel/blob/master/babel/util.py#L81

however, those files seem to be supported since 2.5:

https://bugs.python.org/issue1378022

Would you consider a patch to allow such files in babel?

Thanks!

[sils]

Sounds great, you plan to do a PR?

[lepistone]

Sure. I gave a look at what cpython does now, and it looks like it compares the BOM and the comment, which sounds reasonable to me.

https://github.com/python/cpython/blob/master/Parser/tokenizer.c#L333-L339

[sils]

Ah note that the code will also be tested against pypy so if they don't support that that will be a problem. We could make an interpreter branch though and only support that for CPython...

[lepistone]

I tried, and pypy seems to have the same behavior (crashes only if there is a BOM but the comment says latin1).

[sils]

Cool :)

2015-08-05 18:48 GMT+02:00 Leonardo Pistone [email protected]:

I tried, and pypy seems to have the same behavior (crashes only if there
is a BOM but the comment says latin1).

—
Reply to this email directly or view it on GitHub
#189 (comment).