Have issue when insert data with double qoute and use_prepared_statements=False

[p0m1d0rka]

`
import vertica_python

conn_info = {'host': '127.0.0.1',
'port': 5433,
'user': 'dbadmin',
'password': '',
'database': 'VMart',
'session_label': 'some_label',
'unicode_error': 'strict',
'ssl': False,
'autocommit': False,
'use_prepared_statements': False,
'connection_timeout': 5}

data = [('some text1', 'some test2'), ('some test3', '"some" text4')]
conn = vertica_python.connect(**conn_info)
crsr = conn.cursor()
crsr.executemany("INSERT INTO test1 (column1, column2) VALUES (%s, %s)", data)

conn.commit()
`

Will have error
vertica_python.errors.CopyRejected: Severity: ERROR, Message: COPY: Input record 2 has been rejected (Too many columns found), Sqlstate: 22V04, Routine: reject, File: /data/jenkins/workspace/RE-ReleaseBuilds/RE-Jackhammer/server/vertica/EE/UDL/RejectRows.cpp, Line: 286, Error Code: 2035, SQL: 'COPY test1 (column1,column2) FROM STDIN DELIMITER \',\' ENCLOSED BY \'"\' ENFORCELENGTH ABORT ON ERROR NO COMMIT'

Tried to enclose " with \ - same result.

plz geve me a tip.

TY

[h-serizawa]

Tested on vsql. It worked with the double-quote escaped by the backslash.

=> COPY test1 (column1,column2) FROM STDIN DELIMITER ',' ENCLOSED BY '"' ENFORCELENGTH ABORT ON ERROR NO COMMIT;
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> "some test3",""some" text4"
>> \.
ERROR 2035:  COPY: Input record 1 has been rejected (Too many columns found)

=> COPY test1 (column1,column2) FROM STDIN DELIMITER ',' ENCLOSED BY '"' ENFORCELENGTH ABORT ON ERROR NO COMMIT;
Enter data to be copied followed by a newline.
End with a backslash and a period on a line by itself.
>> "some test3","\"some\" text4"
>> \.
=>

Checked the data sent by vertica-python to the database. In the case of the following data with the single backslash, the database received the data without the backslash.

data = [('some text1', 'some test2'), ('some test3', '\"some\" text4')]

$ cat test1-STDIN-copy-from-rejected-data.28
"some\ test3",""some"\ text4"

In the case of the data with the double backslashes, the database received the data with the double backslashes.

data = [('some text1', 'some test2'), ('some test3', '\\"some\\" text4')]

$ cat test1-STDIN-copy-from-rejected-data.29
"some\ test3","\\"some\\"\ text4"

format_quote() in cursor.py escapes the characters in the data.

def format_quote(self, param, is_copy_data):
    if is_copy_data:
        return u'"{0}"'.format(re.escape(param))

Since Python 3.7, the double quote is not escaped.

re.escape(pattern)
Changed in version 3.7: Only characters that can have special meaning in a regular expression are escaped. As a result, '!', '"', '%', "'", ',', '/', ':', ';', '<', '=', '>', '@', and "`" are no longer escaped.

https://docs.python.org/3/library/re.html

I guess it needs the additional step to escape the double quote in Python 3.7 or later.

[sitingren]

Agreed with @h-serizawa that this issue happens in Python 3.7 or later. I'll fix that soon.

[sitingren]

The fix goes into release v1.0.4.

[p0m1d0rka]

ty for quick fix