More 'asciify' documentation

[gwern]

The documentation for the asciify_paths option says

Convert all non-ASCII characters in paths to ASCII equivalents. For example, if your path template for singletons is singletons/$title and the title of a track is “Café”, then the track will be saved as singletons/Cafe.mp3.

This is clear enough for a Latin script (one would expect 'é' to be converted to 'e'), but it's unclear what this command would do for anything written in entirely different scripts or writing systems like Japanese kanji. (It says 'all' - does that mean they would be deleted since they have no ASCII equivalents?) I have no idea what it might do, and am too terrified to let this option anywhere near my files to figure it out empirically, so more documentation would be helpful for deciding whether to use this option.

[andriykohut]

You can check out unidecode description, it's just doing transliteration:

function unidecode() takes Unicode data and tries to represent it in ASCII characters (i.e., the universally displayable characters between 0x00 and 0x7F), where the compromises taken when mapping between two character sets are chosen to be near what a human with a US keyboard would choose.

Something like:

from unidecode import unidecode

unidecode.unidecode(u'パイソン') # 'paison'
unidecode.unidecode(u'蠎') # 'mang'

[sampsyo]

Thanks for chiming in, @andriykohut.

@gwern, if you do some investigation here, could you please consider adding what you find to the docs? The Unidecode mapping is pretty straightforward and there's no reason we shouldn't spend a sentence or two giving more detail.