Change default charset and collation from utf8 to utf8mb4

[winkyao]

TiDB default charset is utf8 and collation is utf8_bin, in some cases, if the Unicode string length is more than 3 bytes(4 bytes), insert this string into a column with utf8 charset will report error like:

ERROR 1366 (HY000): Incorrect string value: '\xF0\xA4\x8B\xAE' for column 'v' at row 1

Maybe we should consider to change TiDB default charset from utf8 to utf8mb4?

@morgo what's your opinion？

[morgo]

MySQL 5.7 default is latin1. MySQL 8.0 is utf8mb4.

Tidb being UTF8 is weird - because it matches neither. So +1 for changing to utf8mb4 :)

[shenli]

Actually, TiDB treats all the data as utf8mb4. So I think we could return utf8mb4 in the show create table result.

[winkyao]

@shenli return utf8mb4 anyway in the show create table, ignore the original charset?

[morgo]

I think it is too nuanced that tidb treats UTF8 the same as utf8mb4. There are some apps that look for utf8mb4 specifically.

[winkyao]

We have a discussion about this issue, and come to a conclusion:

1. show create table always return the charset and collation withutf8mb4 and utf8mbr_bin, because TiDB treats all the data as utf8mb4 actually.
2. Support alter other charsets to utf8 or utf8mb4(TiDB forbid altering the charset now.)
3. create a table with charsets other than utf8 or utf8mb4, we create it normally and return with a warning like "WARNING: TiDB treat all the data as utf8mb4"

@morgo FYI

[morgo]

LGTM

[gregwebs]

I have a pull request that does this (for new users): #7757
Originally I created a command line switch, but we came to the conclusion that instead of a switch we should just make 'utf8mb4' the new default charset.