Add two buildin function ( decode and encode) #7622

lanjingquan · 2018-09-05T15:27:55Z

Add two buildin function ( decode and encode)：

add crypt.go to util/encrypt
modify buildin_encryption.go

What problem does this PR solve?

What is changed and how it works?

Check List

Tests

Unit test
Integration test
Manual test (add detailed scripts or steps below)
No code

Code changes

Has exported function/method change
Has exported variable/fields change
Has interface methods change
Has persistent data change

Side effects

Possible performance regression
Increased code complexity
Breaking backward compatibility

Related changes

Need to cherry-pick to the release branch
Need to update the documentation
Need to update the tidb-ansible repository
Need to be included in the release note

Add two buildin function ( decode and encode)： 1) add crypt.go to util/encrypt 2) modify buildin_encryption.go

sre-bot · 2018-09-05T15:27:59Z

Hi contributor, thanks for your PR.

This patch needs to be approved by someone of admins. They should reply with "/ok-to-test" to accept this PR for running test automatically.

CLAassistant · 2018-09-05T15:28:04Z

All committers have signed the CLA.

mysql> select decode("pingancap","pass1234"); +--------------------------------+ | decode("pingancap","pass1234") | +--------------------------------+ | .�[P�n�� | +--------------------------------+ 1 row in set (0.32 sec) mysql> select encode(decode("pingancap","pass1234"),"pass1234"); +---------------------------------------------------+ | encode(decode("pingancap","pass1234"),"pass1234") | +---------------------------------------------------+ | pingancap | +---------------------------------------------------+ 1 row in set (0.00 sec) mysql>

morgo · 2018-09-05T15:58:18Z

Note that these functions are deprecated in 5.7 and removed in MySQL 8.0, as the crypto is now considered weak.

mysql> select decode("pingancap","pass1234"); +--------------------------------+ | decode("pingancap","pass1234") | +--------------------------------+ | .�[P�n�� | +--------------------------------+ 1 row in set (0.32 sec) mysql> select encode(decode("pingancap","pass1234"),"pass1234"); +---------------------------------------------------+ | encode(decode("pingancap","pass1234"),"pass1234") | +---------------------------------------------------+ | pingancap | +---------------------------------------------------+ 1 row in set (0.00 sec) mysql>

zz-jason · 2018-09-05T16:58:15Z

expression/builtin_encryption.go

+		return "", true, errors.Trace(err)
+	}
+
+	decodeStr, _ := encrypt.SQLDecode(dataStr, passwordStr)


I think we should check the returned error

zz-jason · 2018-09-05T16:58:39Z

@lanjingquan Please fix CI.

lanjingquan · 2018-09-05T22:46:39Z

how to run ci check again after i fix it ?

mccxj · 2018-09-06T00:45:49Z

@lanjingquan when you push new code, ci will rerun automatically. You can cilck the detail link to check what is going wrong.

winkyao · 2018-09-06T08:09:50Z

Do you port the code from MySQL Item_func_encode?

zz-jason · 2018-09-06T09:44:21Z

@lanjingquan You can run make dev in your local machine.

…into lanjingquan/tidb

lanjingquan · 2018-09-06T14:22:06Z

@winkyao I looked at the implementation code of MySQL and transferred them from C++ to Go to ensure that the same results could be obtained.

zz-jason · 2018-09-07T04:54:52Z

util/encrypt/crypt.go

+	seed1 := nr & ((uint32(1) << 31) - uint32(1))
+	seed2 := nr2 & ((uint32(1) << 31) - uint32(1))
+
+	fmt.Println(seed1)


remove the debug code?

done! Thanks!

zz-jason · 2018-09-07T04:55:36Z

@XuHuaiyu PTAL

spongedu · 2018-09-07T10:21:53Z

util/encrypt/crypt.go

+}
+
+// SQLCrypt use to store initialization results
+type SQLCrypt struct {


We'd better not export struct SQLCrypt if it's not used outside module util/crypt

…into lanjingquan/tidb

spongedu · 2018-09-07T13:53:47Z

util/encrypt/crypt_test.go

+		isError bool
+	}{
+		{"", "", "", false},
+		{"pingcap", "1234567890123456", "2C35B5A4ADF391", false},


spongedu · 2018-09-07T13:53:55Z

util/encrypt/crypt_test.go

+		isError bool
+	}{
+		{"", "", "", false},
+		{"pingcap", "1234567890123456", "pingcap", false},


spongedu · 2018-09-07T13:54:24Z

expression/builtin_encryption_test.go

+}{
+	{"", "", ""},
+	{"pingcap", "1234567890123456", "2C35B5A4ADF391"},
+	{"pingcap", "asdfjasfwefjfjkj", "351CC412605905"},


Please add some cases for NULL inputs and other conditions, like:

mysql> select hex(decode("数据库", "asdfjasfwefjfjkj")); +----------------------------------------------+ | hex(decode("数据库", "asdfjasfwefjfjkj")) | +----------------------------------------------+ | 8A5F1D65A3BECDD210 | +----------------------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql> select hex(decode(12355.5555, "asdfjasfwefjfjkj")); +---------------------------------------------+ | hex(decode(12355.5555, "asdfjasfwefjfjkj")) | +---------------------------------------------+ | E2359202FA4CDD5AB4F2 | +---------------------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql> select hex(decode("分布式データベース", "asdfjasfwefjfjkj")); +----------------------------------------------------------------+ | hex(decode("分布式データベース", "asdfjasfwefjfjkj")) | +----------------------------------------------------------------+ | A9F2E51CEA86CC75EB4A579CA8133E9B346D5B4B610FB575637F34 | +----------------------------------------------------------------+ 1 row in set, 1 warning (0.00 sec) mysql> select hex(decode("数据库", null)); +--------------------------------+ | hex(decode("数据库", null)) | +--------------------------------+ | NULL | +--------------------------------+ 1 row in set (0.00 sec) mysql> select hex(decode(null, "asdfjasfwefjfjkj")); +---------------------------------------+ | hex(decode(null, "asdfjasfwefjfjkj")) | +---------------------------------------+ | NULL | +---------------------------------------+ 1 row in set (0.00 sec) mysql> select hex(decode("pingcap", "密匙")); +----------------------------------+ | hex(decode("pingcap", "密匙")) | +----------------------------------+ | CE5C02A5010010 | +----------------------------------+ 1 row in set, 1 warning (0.01 sec)

…into lanjingquan/tidb

spongedu · 2018-09-08T02:25:58Z

util/encrypt/crypt_test.go

+		{"pingcap#%$%^", "*^%YTu1234567", "8634B9C55FF55E5B6328F449", false},
+		{"pingcap", "", "4A77B524BD2C5C", false},
+		{"分布式データベース","pass1234@#$%%^^&","80CADC8D328B3026D04FB285F36FED04BBCA0CC685BF78B1E687CE",false},
+        	{"分布式データベース","分布式7782734adgwy1242","0E24CFEF272EE32B6E0BFBDB89F29FB43B4B30DAA95C3F914444BC",false},


Pls fix format

spongedu · 2018-09-08T02:26:08Z

util/encrypt/crypt_test.go

+		{"pingcap#%$%^", "*^%YTu1234567", "pingcap#%$%^", false},
+		{"pingcap", "", "pingcap", false},
+	        {"分布式データベース","pass1234@#$%%^^&","分布式データベース",false},
+        	{"分布式データベース","分布式7782734adgwy1242","分布式データベース",false},


spongedu · 2018-09-08T02:26:26Z

expression/builtin_encryption_test.go

+	{"pingcap#%$%^", "*^%YTu1234567", "8634B9C55FF55E5B6328F449"},
+	{"pingcap", "", "4A77B524BD2C5C"},
+        {"分布式データベース","pass1234@#$%%^^&","80CADC8D328B3026D04FB285F36FED04BBCA0CC685BF78B1E687CE"},
+        {"分布式データベース","分布式7782734adgwy1242","0E24CFEF272EE32B6E0BFBDB89F29FB43B4B30DAA95C3F914444BC"},


Pls fix fmt

spongedu

Thanks a lot! LGTM. @zz-jason PTAL

zz-jason

LGTM

zz-jason · 2018-09-10T09:48:49Z

/run-all-tests

Add two buildin function ( decode and encode)

b1b66b7

Add two buildin function ( decode and encode)： 1) add crypt.go to util/encrypt 2) modify buildin_encryption.go

zz-jason added type/enhancement The issue or PR belongs to an enhancement. contribution This PR is from a community contributor. component/expression labels Sep 5, 2018

zz-jason reviewed Sep 5, 2018

View reviewed changes

Merge branch 'master' into lanjingquan/tidb

93f593a

lanjingquan added 6 commits September 6, 2018 20:35

fix ci

4e4459a

Merge branch 'lanjingquan/tidb' of https://github.com/lanjingquan/tidb …

4356579

…into lanjingquan/tidb

fix make error

bdf038b

fix again

0a20e3b

add comment to functions

a1af035

add comment again

86ff091

Merge branch 'master' into lanjingquan/tidb

c3c5de5

zz-jason reviewed Sep 7, 2018

View reviewed changes

spongedu reviewed Sep 7, 2018

View reviewed changes

lanjingquan added 2 commits September 7, 2018 20:08

remove debug code, and rename SQLCrypt struct

5c71833

Merge branch 'lanjingquan/tidb' of https://github.com/lanjingquan/tidb …

b69fb8d

…into lanjingquan/tidb

spongedu reviewed Sep 7, 2018

View reviewed changes

lanjingquan added 5 commits September 8, 2018 08:32

add some cases for NULL inputs and other conditions

761a352

Merge branch 'master' into lanjingquan/tidb

6f2fa41

fix err

f9f7c97

Merge branch 'lanjingquan/tidb' of https://github.com/lanjingquan/tidb …

304ab49

…into lanjingquan/tidb

fix err again

9e8b06f

spongedu reviewed Sep 8, 2018

View reviewed changes

fix fmt

d77eec7

spongedu approved these changes Sep 8, 2018

View reviewed changes

zz-jason approved these changes Sep 10, 2018

View reviewed changes

Merge branch 'master' into lanjingquan/tidb

70a2ce9

zz-jason added the status/LGT2 Indicates that a PR has LGTM 2. label Sep 10, 2018

zz-jason merged commit 8b1feeb into pingcap:master Sep 10, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add two buildin function ( decode and encode) #7622

Add two buildin function ( decode and encode) #7622

lanjingquan commented Sep 5, 2018

sre-bot commented Sep 5, 2018

CLAassistant commented Sep 5, 2018 •

edited

Loading

morgo commented Sep 5, 2018

zz-jason Sep 5, 2018

zz-jason commented Sep 5, 2018

lanjingquan commented Sep 5, 2018

mccxj commented Sep 6, 2018 •

edited

Loading

winkyao commented Sep 6, 2018

zz-jason commented Sep 6, 2018

lanjingquan commented Sep 6, 2018

zz-jason Sep 7, 2018

lanjingquan Sep 7, 2018

zz-jason commented Sep 7, 2018

spongedu Sep 7, 2018

lanjingquan Sep 7, 2018

spongedu Sep 7, 2018

lanjingquan Sep 8, 2018

spongedu Sep 7, 2018

lanjingquan Sep 8, 2018

spongedu Sep 7, 2018

lanjingquan Sep 8, 2018

spongedu Sep 8, 2018

spongedu Sep 8, 2018

spongedu Sep 8, 2018

lanjingquan Sep 8, 2018

spongedu left a comment

zz-jason left a comment

zz-jason commented Sep 10, 2018

Add two buildin function ( decode and encode) #7622

Add two buildin function ( decode and encode) #7622

Conversation

lanjingquan commented Sep 5, 2018

What problem does this PR solve?

What is changed and how it works?

Check List

sre-bot commented Sep 5, 2018

CLAassistant commented Sep 5, 2018 • edited Loading

morgo commented Sep 5, 2018

Choose a reason for hiding this comment

zz-jason commented Sep 5, 2018

lanjingquan commented Sep 5, 2018

mccxj commented Sep 6, 2018 • edited Loading

winkyao commented Sep 6, 2018

zz-jason commented Sep 6, 2018

lanjingquan commented Sep 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zz-jason commented Sep 7, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

spongedu left a comment

Choose a reason for hiding this comment

zz-jason left a comment

Choose a reason for hiding this comment

zz-jason commented Sep 10, 2018

CLAassistant commented Sep 5, 2018 •

edited

Loading

mccxj commented Sep 6, 2018 •

edited

Loading