Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please provide this list in RPZ format #13

Closed
ppaeps opened this issue Jul 23, 2021 · 11 comments
Closed

Please provide this list in RPZ format #13

ppaeps opened this issue Jul 23, 2021 · 11 comments

Comments

@ppaeps
Copy link
Contributor

ppaeps commented Jul 23, 2021

This list is very useful; many thanks!

It would be even more useful if it were in RPZ format. This (horrible) awk hack does exactly that:

awk '{ if ($0 !~ /^#/) print $0 " CNAME ." ; else { sub(/#/, ";"); print $0 }}' combined_disguised_trackers_justdomains.txt

Please consider providing an RPZ version of this list in addition to the AdBlock formatted version.

@dnmTX
Copy link

dnmTX commented Jul 23, 2021

@ppaeps sed will do this in a instant on your end. No need to bother the @AdguardTeam fellas for that 😉 :
sed -i '/^[^#]/s/$/ CNAME \./g' combined_disguised_trackers_justdomains.txt
Result(partial sample):

# Company name: A8.net
#
# a8.net disguised trackers
#
a8.01cloud.jp CNAME .
a8.cyclemarket.jp CNAME .
a8.denwa-hikari.com CNAME .
a8.eonet.jp CNAME .
a8.haptic.co.jp CNAME .

👍

@ppaeps
Copy link
Contributor Author

ppaeps commented Jul 23, 2021

Hah! You are not wrong. sed would be easier than awk. I put together that one-liner from a larger (more gruesome) awk script that converts and combines a bunch of blocklists into an RPZ for me. :)

Since the nice folks of @AdguardTeam are already providing a list in AdBlock format, it shouldn't be too much of a hardship for them to add the additional one-liner to produce RPZ too. More formats, more better. The easier we can make it for everyone to block these cretins, the less incentivised more cretins will be to join the party and the slower these lists grow.

@dnmTX
Copy link

dnmTX commented Jul 23, 2021

@ppaeps as i'm really not familar with RPZ format i see in your script you're substituting # with ;.
Is that how comments are marked there?
I agree with you about your request here,more formats are allways better 👍

@TPS
Copy link

TPS commented Jul 23, 2021

An example in the RFC @ https://tools.ietf.org/id/draft-vixie-dnsop-dns-rpz-00.html#rfc.appendix.A does show such commenting.

@dnmTX
Copy link

dnmTX commented Jul 23, 2021

Thanks @TPS 👍 For some reason i couldn't find it. Anyway,here is reworked final sed command(in case is needed):
sed -i '/^#$/d; s/^#/;/g; /^[^;]/s/$/ CNAME \./g' combined_disguised_trackers_justdomains.txt

Result(partial sample):

; Title: AdGuard CNAME disguised trackers list
; Description: The list of trackers that disguise the real trackers by using CNAME records.
; Homepage: https://github.com/AdguardTeam/cname-trackers
; Company name: A8.net
; a8.net disguised trackers
a8.01cloud.jp CNAME .
a8.cyclemarket.jp CNAME .
a8.denwa-hikari.com CNAME .
a8.eonet.jp CNAME .
a8.haptic.co.jp CNAME .
a8.lavie-official.jp CNAME .
a8.lens-labo.com CNAME .

@ppaeps
Copy link
Contributor Author

ppaeps commented Jul 28, 2021

@dnmTX You'll want to add a ;/^$/d;$G or something similar (;$a\, ;$s/$/\'$'\n'$'/, depending on your mood) to ensure there's a newline at the end of the output file. Some DNS servers (e.g. NSD) will cry about files that don't end with a newline.

This is all academic though. I just submitted pull request #14 which generates the RPZ formatted files directly. It would be great if @AdguardTeam could merge this. 🙏

@dnmTX
Copy link

dnmTX commented Aug 12, 2021

Philip @ppaeps your wish came through(congrats 👏 😄 ) so you can close the issue here if you want 👍

@ppaeps
Copy link
Contributor Author

ppaeps commented Aug 12, 2021

Hooray! Thanks for the merge @adguard!

@ppaeps ppaeps closed this as completed Aug 12, 2021
@alsyundawy
Copy link

what diff use IN and without IN in cname
sample
a.b.c.d CNAME .
a.b.c.d IN CNAME .

@alsyundawy
Copy link

This list is very useful; many thanks!

It would be even more useful if it were in RPZ format. This (horrible) awk hack does exactly that:

awk '{ if ($0 !~ /^#/) print $0 " CNAME ." ; else { sub(/#/, ";"); print $0 }}' combined_disguised_trackers_justdomains.txt

Please consider providing an RPZ version of this list in addition to the AdBlock formatted version.

How about this command?

awk '{print $1" IN CNAME ."}' combined_disguised_trackers_justdomains.txt >> combined_disguised_trackers_justdomains.zone

@ppaeps
Copy link
Contributor Author

ppaeps commented Aug 13, 2021

No difference. Check section 5.1 of RFC 1035. "Omitted class [...] default to the last explicitly stated values". In practice, every DNS implementation I've encountered, defaults them to IN even when they don't appear anywhere in a zone file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants