Stop changing 'encoding'

[k-takata]

Changing 'encoding' has many side effects and that should be avoided.
Strip BOM manually.

Fix #144
Fix #145
Fix #147

[cxw42]

Thanks for this PR! Would you be willing to add some test cases of files with different encodings? I can help with the test-case plumbing if that would be helpful.

[cxw42]

Will the first three bytes always come through as the UTF-8 BOM values? E.g., what happens when the file is UTF-16?

[k-takata]

This part strips the UTF-8 BOM only when it exists. Otherwise it is kept as is.

(The editorconfig spec says that "EditorConfig files should be UTF-8 encoded", so I don't think we need to care about UTF-16.)

[k-takata]

Would you be willing to add some test cases of files with different encodings?

Do we need to add some test cases with a .editorconfig file with UTF-8 BOM? Then, should https://github.com/editorconfig/editorconfig-plugin-tests be updated?

[cxw42]

Would you be willing to add some test cases of files with different encodings?

Do we need to add some test cases with a .editorconfig file with UTF-8 BOM? Then, should https://github.com/editorconfig/editorconfig-plugin-tests be updated?

@k-takata Good question! I don't think we need to update the main plugin tests, since this problem is specific to this plugin. If you or @h3xx can send me some test cases, I'll figure out where to put them.

[k-takata]

How about creating a copy of https://github.com/editorconfig/editorconfig-plugin-tests/tree/master/test_files with just adding a UTF-8 BOM to test_files/.editorconfig?

[h3xx]

+1 for this PR. Tested these changes against the issue in my PR #194, and it in fact fixes the issue I described.

This PR seems more comprehensive than my change so I closed #194.

Summary of issue from #194 (~12 lines)

For some reason, this plugin is causing :e to change the file's encoding option (&fileencoding) to utf-8, overriding whatever it's currently set to. This doesn't happen upon first opening the file, but only upon re-opening it with :e.

Steps to reproduce:

1. Open and save a file in latin1 mode:

export LANG=en_US LC_ALL=en_US LC_CTYPE=en_US
vim foo.txt
:set fenc=latin1
:w

2. Without closing Vim, reload the file:

:e

3. Check the file encoding hasn't changed:

:echo &fenc

&fenc should still be latin1. If it was reset to utf-8 the issue persists.

[alkim0]

This will also fix #196

[cxw42]

Merged --- let me know if you experience any issues!