-
Notifications
You must be signed in to change notification settings - Fork 892
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ndpiReader crash while processiong the live traffic #423
Comments
Thanks for reporting. I'll check asap |
@lynn1050 have you found that crash reason ? I encounter this problem too, but I don't understand the program deeply, so it's hard for me to do a deep diagnostic. |
@kYroL01 any update on this questions ? I can do some test |
Maybe related to this #499 |
@qianguozheng @lynn1050 Could you please do more test after latest commits ? For what I understand, the problem occurs after many time and many traffic, right ? Any help is appreciated to better understand, thanks. |
@kYroL01 & @qianguozheng, Reasons were described in the above comment. Have you got it? |
@lynn1050 have you got any method to fix such things ? |
Yes. I modified the function get_ndpi_flow_info(...) in ndpi_util.c. When a new packet comes in, it's requested to get the flow from ndpi_flows_root. If the flow cannot be got, swap the source and the desination, and then try to get the flow. If neither of them cannot work, then create a new flow(a new node in ndpi_flows_root). Such modification can lead to a unique node in ndpi_flows_root. ...... /************************************** newly added code, start ****************************************/
} I hope that would be useful for you. Good luck! |
@lynn1050 That's great, thanks a lot. |
The variable is_changed is related to the last output parameter of get_ndpi_flow_info. And my code is written as below,
} else {
} //the end of get_ndpi_flow_info function. Anyway, I found the output parameter had been used nowhere. |
Thanks a lot. @lynn1050 |
Guys, if you find the problem and you wanna contribute to fix this issue definetley, send us a pull request, so we can test and integrate inside the code. Is the best way for all the community :) Thank you |
@kYroL01 Ok, I will have a try. |
It's the easiest things to discuss togheter and test better. Thanks ;) |
@qianguozheng I need help to send a pull request. My qq number is: 928335321. |
Solved by last commit |
ndpiReader crash on one server, but not on another server, when it is processing the live traffic.
The backtrace is as follows:
#0 0x000000308287b81c in free () from /lib64/libc.so.6
#1 0x000000000040a32a in ndpi_free_flow_info_half (flow=0x7ffff220d8b0, func=0x438480 "node_idle_scan_walker") at ndpi_util.c:79
#2 0x00000000004071fc in node_idle_scan_walker (node=, which=, depth=, user_data=) at ndpiReader.c:1099
#3 0x000000000040b028 in ndpi_trecurse (root=0x7fffeb5889e0, action=0x407140 <node_idle_scan_walker>, level=9, user_data=0x7ffff7d64b3e) at ndpi_main.c:143
#4 0x000000000040b06a in ndpi_trecurse (root=0x7fffe855fcd0, action=0x407140 <node_idle_scan_walker>, level=8, user_data=0x7ffff7d64b3e) at ndpi_main.c:148
#5 0x000000000040b041 in ndpi_trecurse (root=0x7ffff1e6e290, action=0x407140 <node_idle_scan_walker>, level=7, user_data=0x7ffff7d64b3e) at ndpi_main.c:145
#6 0x000000000040b06a in ndpi_trecurse (root=0x7ffff0752120, action=0x407140 <node_idle_scan_walker>, level=6, user_data=0x7ffff7d64b3e) at ndpi_main.c:148
#7 0x000000000040b041 in ndpi_trecurse (root=0x7fffea32fc70, action=0x407140 <node_idle_scan_walker>, level=5, user_data=0x7ffff7d64b3e) at ndpi_main.c:145
#8 0x000000000040b041 in ndpi_trecurse (root=0x7ffff088fcd0, action=0x407140 <node_idle_scan_walker>, level=4, user_data=0x7ffff7d64b3e) at ndpi_main.c:145
#9 0x000000000040b06a in ndpi_trecurse (root=0x7ffff2197a90, action=0x407140 <node_idle_scan_walker>, level=3, user_data=0x7ffff7d64b3e) at ndpi_main.c:148
#10 0x000000000040b06a in ndpi_trecurse (root=0x7ffff16cb800, action=0x407140 <node_idle_scan_walker>, level=2, user_data=0x7ffff7d64b3e) at ndpi_main.c:148
#11 0x000000000040b06a in ndpi_trecurse (root=0x7ffff0cbad30, action=0x407140 <node_idle_scan_walker>, level=1, user_data=0x7ffff7d64b3e) at ndpi_main.c:148
#12 0x000000000040b041 in ndpi_trecurse (root=0x7ffff1ff9390, action=0x407140 <node_idle_scan_walker>, level=0, user_data=0x7ffff7d64b3e) at ndpi_main.c:145
#13 0x0000000000408910 in pcap_process_packet (args=, header=0x7ffff7d64bd0, packet=0x7ffff7edd046 "\350\261\374\311'\360\220\261\034\006\254\300\201") at ndpiReader.c:2020
#14 0x00000033b7808ace in ?? () from /usr/lib64/libpcap.so.1
#15 0x00000033b781137d in pcap_loop () from /usr/lib64/libpcap.so.1
#16 0x0000000000408d5f in runPcapLoop (_thread_id=0x0) at ndpiReader.c:2117
#17 processing_thread (_thread_id=0x0) at ndpiReader.c:2144
#18 0x0000003082c079d1 in start_thread () from /lib64/libpthread.so.0
#19 0x00000030828e8b6d in clone () from /lib64/libc.so.6
I found some clues.
In the idle flow cleanup, some nodes cannot be deleted from the "ndpi_flow_root" when calling function ndpi_tdelete, and then the flow, which is the key of the node, is free.
In the next loop, ndpi_twalk finds the node, and calls node_idle_scan_walker, and then calls ndpi_free_flow_info_half. The memory is double free here and finally crash.
And then I found there was some porblems in the sorting function of the binary tree.
For example, there are three nodes, node A, node B and node C.
A: 0x7fffe9079520(hashval: 3484336491, vlan_id:401, protocol:6, lower_ip: 3534356490, lower_port: 52459, upper_ip: 4244875456, upper_port: 18975)
B: 0x7ffff04b3c60(hashval: 3484336491, vlan_id:401, protocol:6, lower_ip: 3534356490, lower_port: 52971, upper_ip: 4244875456, upper_port: 18463)
C: 0x7fffe93c2f50(hashval: 3484336491, vlan_id:401, protocol:6, lower_ip: 4244875456, lower_port: 18975, upper_ip: 3534356490, upper_port: 52459)
We call funcion ndpi_workflow_node_cmp, and get this result: A<B, C>B, A==C.
This results the failure of deleting node from "ndpi_flow_root".
But it doesn't happen frequently, so crash happened only on some server.
The text was updated successfully, but these errors were encountered: