New IPv4 filter implementation that isn't a memory hog -- fixes rtorrent issue #106 and #71 #109
Conversation
|
so with this added it is possible to use p2p format files? with comments? |
|
when parsing file line everything up to colon ':' is ignored. if you mean comments that start with '#' then handling is not ideal as parser will still try to parse line so '# abc def: 1.2.3.4.5-1.2.3.4.10' will still get added to list even though it is commented out. However, it will not crash on comments so if nothing that could be parsed as ip address is in comments they will not prevent program from running. |
|
good! |
|
mm... seems like you have pulled from your master... latest commit is unrelated to this issue. |
|
ugh. sorry. I am familiar with git but new to github. do you know what would be the easiest way to fix it now that pull request is sent and second commit is pushed to the same branch? |
|
yes... did the same thing in the beginning... you need to do a rebase and then a force push something like this if you only did one commit i think... but if you want to repull from a branch it is easier to close this delete your repo and refork... or you probably could rebase back to beginning... |
|
the github guides are a little messy... I have found this one really useful: https://code.djangoproject.com/wiki/CollaborateOnGithub |
|
doing rebase would just combine commits though. not what I want. instead [1] checkout last commit in rakshasa master [2] create new branch from there [3] cherry-pick latest commit into new branch [4] revert commit in master. problem solved. |
|
no... you want to do rebase with the f option f, fixup = like "squash", but discard this commit's log message then you can branch and add it to a branch but you will still have this commit thats not in rakshasas master yet... |
|
or if you can rebase in to a new branch... but i dont know hove to do that... |
|
I tested it joust now you probably had to do that revert first yes... |
|
ok. all fixed now. |
|
but you should do a rebase to fixup all the commit messages... else al commit messages will go into rtorrent... |
|
yes. I just did a rebase. only one commit is left. |
|
a sorry... had to reload the page... <:-) |
|
Now the waiting being for the mighty @rakshasa |
|
Tried the new version with the ipfilter corrections. Loaded a 5.6 MB CIDR format list. Memory usage appears to be minimal: 1358359195 I Loaded 338529 unwanted address blocks (3967 kb in-memory) from '~/ipfilter.dat'. Not sure if it is actually blocking though, as I loaded a torrent and let it run for a few minutes, but I didn't see any messages in the logs about connections being blocked. I'll continue to run the service with logging over the next few days to see if anything pops up. |
|
Thanks for the work. I'll do a more thorough review later, however I'd like to note that C-style comments are not accepted in my codebase. |
|
sorry for not matching preferred style. I will switch comments to C++ style. |
…handle p2p lists from bluetack [3] still has quick log n lookup time for extents data structure.
|
all comments are now changed to C++ style in both rtorrent and libtorrent. I also made two minor improvements. I added code to handle comments "#" when parsing filter file lines and added logging when connection to unwanted peer is prevented. this addresses issue pointed out by DugieHowsa above. wihtout logging it is hard to know when filter is working. log level is set to LOG_PEER_INFO. @rakshasa if you need more information when reviewing my code please ask and I will be happy to help. if you want any further changes beyond c++ comments just ask. |
|
Is it not time to implement this and make a new release son... I have been run this in my OpenELEC rTorrent service add-on for a while and it seams to work good. |
|
Ping @rakshasa |
|
Currently not in a position to handle this and other issues that have to be done before a release. |
|
Can you make it compatible with the standard ipfilter.dat files? |
|
@rakshasa I'm using @sallyswiss fork for more than 1 year now and it works like a charm with IP filters, without memory leak. Now that you have bumped the new release, maybe could you merge it ? |
|
Any news about a possible merge ? |
|
I'll take a proper look later. |
|
Still no merge ? That's a 4 years PR and I'm stuck on the fork. Is the issue fixed on master ? |
|
@gauthier-delacroix : I'll rebase this on current master branches today, so you can try it out. |
|
Done, rebased on current master branch in: #671
Enjoy! |
|
This can be closed since the other one is merged: #671 |
New extents.h data structure in libtorrent so that it does not waste memory. it is now an ordered map instead of a crazy sparse hash, much better, still log n access time. it also stores ranges as in p2p file format so it can handle p2p data from bluetack lists as well as cidr notation which is now only thing supported. There is corresponding commit and pull request for rtorrent that makes the necessary changes there too.