Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proxysql crashes after opening eventlog mode in Proxysql 2.0.7 #2356

Closed
zhanghaibo-BJ opened this issue Oct 31, 2019 · 3 comments
Closed

Proxysql crashes after opening eventlog mode in Proxysql 2.0.7 #2356

zhanghaibo-BJ opened this issue Oct 31, 2019 · 3 comments

Comments

@zhanghaibo-BJ
Copy link

##Problem Description：

Proxysql crashes after opening eventlog mode in Proxysql 2.0.7

##ProxySQL version：

2.0.7-80-g4dd4ef5

##Proxysql package：

proxysql-2.0.7-1-centos67.x86_64.rpm

##Operating system version：

Linux version 2.6.32-642.el6.x86_64 ([email protected]) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) ) #1 SMP Tue May 10 17:27:01 UTC 2016

##Proxysql variables：

+--------------------------------------------------------------+-----------------------------------------+
| Variable_name | Value |
+--------------------------------------------------------------+-----------------------------------------+
| admin-admin_credentials | admin:;cluster: |
| admin-checksum_mysql_query_rules | true |
| admin-checksum_mysql_servers | true |
| admin-checksum_mysql_users | true |
| admin-cluster_check_interval_ms | 15000 |
| admin-cluster_check_status_frequency | 2 |
| admin-cluster_mysql_query_rules_diffs_before_sync | 3 |
| admin-cluster_mysql_query_rules_save_to_disk | true |
| admin-cluster_mysql_servers_diffs_before_sync | 3 |
| admin-cluster_mysql_servers_save_to_disk | true |
| admin-cluster_mysql_users_diffs_before_sync | 3 |
| admin-cluster_mysql_users_save_to_disk | true |
| admin-cluster_password | 1qaz@wsx#EDC |
| admin-cluster_proxysql_servers_diffs_before_sync | 3 |
| admin-cluster_proxysql_servers_save_to_disk | true |
| admin-cluster_username | cluster |
| admin-hash_passwords | true |
| admin-mysql_ifaces | 0.0.0.0:6032 |
| admin-read_only | false |
| admin-refresh_interval | 2000 |
| admin-stats_credentials | stats:stats |
| admin-stats_mysql_connection_pool | 60 |
| admin-stats_mysql_connections | 60 |
| admin-stats_mysql_query_cache | 60 |
| admin-stats_system_cpu | 60 |
| admin-stats_system_memory | 60 |
| admin-telnet_admin_ifaces | (null) |
| admin-telnet_stats_ifaces | (null) |
| admin-vacuum_stats | true |
| admin-version | 2.0.7-80-g4dd4ef5 |
| admin-web_enabled | false |
| admin-web_port | 6080 |
| mysql-add_ldap_user_comment | |
| mysql-auditlog_filename | |
| mysql-auditlog_filesize | 104857600 |
| mysql-auto_increment_delay_multiplex | 5 |
| mysql-autocommit_false_is_transaction | true |
| mysql-autocommit_false_not_reusable | false |
| mysql-binlog_reader_connect_retry_msec | 3000 |
| mysql-client_found_rows | true |
| mysql-client_multi_statements | true |
| mysql-client_session_track_gtid | false |
| mysql-commands_stats | true |
| mysql-connect_retries_delay | 200 |
| mysql-connect_retries_on_failure | 10 |
| mysql-connect_timeout_server | 3000 |
| mysql-connect_timeout_server_max | 10000 |
| mysql-connection_delay_multiplex_ms | 0 |
| mysql-connection_max_age_ms | 14400000 |
| mysql-connpoll_reset_queue_length | 50 |
| mysql-default_character_set_results | NULL |
| mysql-default_charset | utf8 |
| mysql-default_collation_connection | |
| mysql-default_isolation_level | READ REPEATABLE |
| mysql-default_max_join_size | 18446744073709551615 |
| mysql-default_max_latency_ms | 1000 |
| mysql-default_net_write_timeout | 60 |
| mysql-default_query_delay | 0 |
| mysql-default_query_timeout | 86400000 |
| mysql-default_reconnect | true |
| mysql-default_schema | information_schema |
| mysql-default_session_track_gtids | OFF |
| mysql-default_sql_auto_is_null | OFF |
| mysql-default_sql_mode | |
| mysql-default_sql_safe_updates | OFF |
| mysql-default_sql_select_limit | DEFAULT |
| mysql-default_time_zone | SYSTEM |
| mysql-default_transaction_read | WRITE |
| mysql-default_tx_isolation | READ-REPEATABLE |
| mysql-enforce_autocommit_on_reads | false |
| mysql-eventslog_default_log | 1 |
| mysql-eventslog_filename | querieslog |
| mysql-eventslog_filesize | 104857600 |
| mysql-eventslog_format | 2 |
| mysql-forward_autocommit | false |
| mysql-free_connections_pct | 10 |
| mysql-have_compress | true |
| mysql-have_ssl | false |
| mysql-hostgroup_manager_verbose | 1 |
| mysql-init_connect | |
| mysql-interfaces | 0.0.0.0:6033 |
| mysql-keep_multiplexing_variables | tx_isolation,version |
| mysql-kill_backend_connection_when_disconnect | true |
| mysql-ldap_user_variable | |
| mysql-long_query_time | 1000 |
| mysql-max_allowed_packet | 4194304 |
| mysql-max_connections | 1000 |
| mysql-max_stmts_cache | 10000 |
| mysql-max_stmts_per_connection | 20 |
| mysql-max_transaction_time | 14400000 |
| mysql-min_num_servers_lantency_awareness | 1000 |
| mysql-mirror_max_concurrency | 16 |
| mysql-mirror_max_queue_length | 32000 |
| mysql-monitor_connect_interval | 120000 |
| mysql-monitor_connect_timeout | 15000 |
| mysql-monitor_enabled | true |
| mysql-monitor_galera_healthcheck_interval | 30000 |
| mysql-monitor_galera_healthcheck_max_timeout_count | 3 |
| mysql-monitor_galera_healthcheck_timeout | 10000 |
| mysql-monitor_groupreplication_healthcheck_interval | 5000 |
| mysql-monitor_groupreplication_healthcheck_max_timeout_count | 3 |
| mysql-monitor_groupreplication_healthcheck_timeout | 800 |
| mysql-monitor_history | 600000 |
| mysql-monitor_password | ******* |
| mysql-monitor_ping_interval | 15000 |
| mysql-monitor_ping_max_failures | 5 |
| mysql-monitor_ping_timeout | 14000 |
| mysql-monitor_query_interval | 60000 |
| mysql-monitor_query_timeout | 5000 |
| mysql-monitor_read_only_interval | 2400 |
| mysql-monitor_read_only_max_timeout_count | 3 |
| mysql-monitor_read_only_timeout | 800 |
| mysql-monitor_replication_lag_interval | 10000 |
| mysql-monitor_replication_lag_timeout | 1000 |
| mysql-monitor_replication_lag_use_percona_heartbeat | |
| mysql-monitor_slave_lag_when_null | 60 |
| mysql-monitor_threads_max | 128 |
| mysql-monitor_threads_min | 8 |
| mysql-monitor_threads_queue_maxsize | 128 |
| mysql-monitor_username | proxysql_monitor |
| mysql-monitor_wait_timeout | true |
| mysql-monitor_writer_is_also_reader | true |
| mysql-multiplexing | true |
| mysql-ping_interval_server_msec | 15000 |
| mysql-ping_timeout_server | 5000 |
| mysql-poll_timeout | 2000 |
| mysql-poll_timeout_on_failure | 100 |
| mysql-query_cache_size_MB | 256 |
| mysql-query_cache_stores_empty_result | true |
| mysql-query_digests | true |
| mysql-query_digests_lowercase | false |
| mysql-query_digests_max_digest_length | 2048 |
| mysql-query_digests_max_query_length | 65000 |
| mysql-query_digests_no_digits | false |
| mysql-query_digests_normalize_digest_text | false |
| mysql-query_digests_replace_null | false |
| mysql-query_digests_track_hostname | false |
| mysql-query_processor_iterations | 0 |
| mysql-query_processor_regex | 1 |
| mysql-query_retries_on_failure | 1 |
| mysql-reset_connection_algorithm | 2 |
| mysql-server_capabilities | 569899 |
| mysql-server_version | 5.7.26 |
| mysql-servers_stats | true |
| mysql-session_idle_ms | 1000 |
| mysql-session_idle_show_processlist | true |
| mysql-sessions_sort | true |
| mysql-set_query_lock_on_hostgroup | 1 |
| mysql-show_processlist_extended | 0 |
| mysql-shun_on_failures | 5 |
| mysql-shun_recovery_time_sec | 10 |
| mysql-ssl_p2s_ca | |
| mysql-ssl_p2s_cert | |
| mysql-ssl_p2s_cipher | |
| mysql-ssl_p2s_key | |
| mysql-stacksize | 1048576 |
| mysql-stats_time_backend_query | false |
| mysql-stats_time_query_processor | false |
| mysql-tcp_keepalive_time | 0 |
| mysql-threads | 32 |
| mysql-threshold_query_length | 524288 |
| mysql-threshold_resultset_size | 4194304 |
| mysql-throttle_connections_per_sec_to_hostgroup | 1000000 |
| mysql-throttle_max_bytes_per_second_to_client | 0 |
| mysql-throttle_ratio_server_to_client | 0 |
| mysql-use_tcp_keepalive | 0 |
| mysql-verbose_query_error | false |
| mysql-wait_timeout | 14400000 |
+--------------------------------------------------------------+-----------------------------------------+
168 rows in set (0.03 sec)

##Proxysql error log:

2019-10-31 09:34:33 MySQL_Session.cpp:5308:handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_qpo(): [WARNING] Unable to parse multi-statements com
mand with SET statement: setting lock hostgroup . Command: SET PROFILING=1;
2019-10-31 09:35:17 MySQL_Session.cpp:5308:handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_qpo(): [WARNING] Unable to parse multi-statements com
mand with SET statement: setting lock hostgroup . Command: SET PROFILING=1;
2019-10-31 09:36:31 MySQL_Session.cpp:5308:handler___status_WAITING_CLIENT_DATA___STATE_SLEEP___MYSQL_COM_QUERY_qpo(): [WARNING] Unable to parse multi-statements com
mand with SET statement: setting lock hostgroup . Command: SET PROFILING=1;
terminate called after throwing an instance of 'nlohmann::detail::type_error'
what(): [json.exception.type_error.316] invalid UTF-8 byte at index 27: 0xBA
Error: signal 6:
proxysql(_Z13crash_handleri+0x1a)[0x49a2ea]
/lib64/libc.so.6(+0x32570)[0x7f1bde780570]
/lib64/libc.so.6(gsignal+0x35)[0x7f1bde7804f5]
/lib64/libc.so.6(abort+0x175)[0x7f1bde781cd5]
/usr/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x12d)[0x7f1bdf03aa8d]
/usr/lib64/libstdc++.so.6(+0xbcbe6)[0x7f1bdf038be6]
/usr/lib64/libstdc++.so.6(+0xbcc13)[0x7f1bdf038c13]
/usr/lib64/libstdc++.so.6(+0xbcd32)[0x7f1bdf038d32]
proxysql(_ZN8nlohmann6detail10serializerINS_10basic_jsonISt3mapSt6vectorSsblmdSaNS_14adl_serializerEEEE12dump_escapedERKSsb+0x59e)[0x4c6c2e]
proxysql(_ZN8nlohmann6detail10serializerINS_10basic_jsonISt3mapSt6vectorSsblmdSaNS_14adl_serializerEEEE4dumpERKS6_bbjj+0x3b6)[0x4c9836]
proxysql(_ZN8nlohmann6detail10serializerINS_10basic_jsonISt3mapSt6vectorSsblmdSaNS_14adl_serializerEEEE4dumpERKS6_bbjj+0x850)[0x4c9cd0]
proxysql(_ZN11MySQL_Event25write_query_format_2_jsonEPSt13basic_fstreamIcSt11char_traitsIcEE+0x974)[0x550d14]
proxysql(_ZN12MySQL_Logger11log_requestEP13MySQL_SessionP17MySQL_Data_Stream+0x253)[0x551843]
proxysql(_ZN13MySQL_Session10RequestEndEP17MySQL_Data_Stream+0x9d)[0x4e81ed]
proxysql(_ZN13MySQL_Session7handlerEv+0x2b74)[0x4faaa4]
proxysql(_ZN12MySQL_Thread20process_all_sessionsEv+0x2bc)[0x4d55cc]
proxysql(_ZN12MySQL_Thread3runEv+0xe06)[0x4df386]
proxysql(_Z24mysql_worker_thread_funcPv+0x6c)[0x496bcc]
/lib64/libpthread.so.0(+0x7aa1)[0x7f1bdf8abaa1]
/lib64/libc.so.6(clone+0x6d)[0x7f1bde836c4d]
2019-10-31 09:37:58 main.cpp:1395:ProxySQL_daemonize_phase3(): [ERROR] ProxySQL crashed. Restarting!
2019-10-31 09:37:58 [INFO] ProxySQL version 2.0.7-80-g4dd4ef5
2019-10-31 09:37:58 [INFO] ProxySQL SHA1 checksum: 296aae73f40f213199a2abb425bef2bb19b7a535
2019-10-31 09:37:58 [INFO] Angel process started ProxySQL process 28021
Standard ProxySQL MySQL Logger rev. 2.0.0714 -- MySQL_Logger.cpp -- Fri Sep 27 09:10:35 2019
Standard ProxySQL Cluster rev. 0.4.0906 -- ProxySQL_Cluster.cpp -- Fri Sep 27 09:10:35 2019
Standard ProxySQL Statistics rev. 1.4.1027 -- ProxySQL_Statistics.cpp -- Fri Sep 27 09:10:35 2019
Standard ProxySQL HTTP Server Handler rev. 1.4.1031 -- ProxySQL_HTTP_Server.cpp -- Fri Sep 27 09:10:35 2019
2019-10-31 09:38:00 [INFO] Switching query logging format from 1 to 2
Standard ProxySQL Admin rev. 2.0.6.0805 -- ProxySQL_Admin.cpp -- Fri Sep 27 09:10:35 2019
2019-10-31 09:38:00 [INFO] ProxySQL SHA1 checksum: 296aae73f40f213199a2abb425bef2bb19b7a535
Standard MySQL Threads Handler rev. 0.2.0902 -- MySQL_Thread.cpp -- Mon Sep 30 19:01:56 2019
Standard MySQL Authentication rev. 0.2.0902 -- MySQL_Authentication.cpp -- Fri Sep 27 09:10:35 2019
2019-10-31 09:38:00 [INFO] Dumping mysql_servers_incoming
@yakirgb
Copy link
Contributor

yakirgb commented Oct 31, 2019

I think the issue fixed in commit: 3ad899c
and will be in version 2.0.8

We can update the event log to standard output (not JSON) or upgrade.

SET mysql-eventslog_format=1;
LOAD MYSQL VARIABLES TO RUNTIME;
SAVE MYSQL VARIABLES TO DISK;

@zhanghaibo-BJ
Copy link
Author

@yakirgb thank you very much

@renecannao
Copy link
Contributor

Closing because fixed in 2.0.8 . Thank you @yakirgb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@renecannao @yakirgb @zhanghaibo-BJ