$dnsrewrite modifier

[ameshkov]

Where the idea comes from: #2098

Example: ||example.org^$dnsrewrite=127.0.0.1

The idea is to benefit from rules syntax and other modifiers. For instance, we'll be able to combine these rules with other modifiers.

We'll need to carefully think it through, edge cases, etc.

Spec:

dnsrewrite

The $dnsrewrite modifier allows specifying the exact DNS records that will be returned as a response to a DNS query. This is a very powerful and flexible modifier so please be cautious with it.

The $dnstype modifier allows specifying one or several DNS query types the rule will be working for.

The syntax is:

$dnsrewrite=RCODE;RR;response_content

Alternatively, you can specify just the status to respond with. In this case, AdGuard Home will respond with an empty DNS response with the specified status code.

$dnsrewrite=KEYWORD
$dnsrewrite=SHORTHAND

Where:

• response_content is the contents of a DNS record.
• KEYWORD is an all-caps keyword. For example, REFUSED.
• SHORTHAND is either an IPv4 (A record), an IPv6 (AAAA record), or a hostname (CNAME rewrite, see below).
• RCODE is a DNS response code.
• RR is a DNS record type.
  • Queries with DNS type equal to RR will be rewritten to the specified response_content response.
  • Queries with DNS type different from RR will be rewritten to an empty NOERROR response (unless they're modified by a different $dnsrewrite rule).
  • CNAME is a special case that redirects both A and AAAA queries to the specified domain. If there's a rule with the CNAME type matching the domain name, all other rules will be ignored.

Multiple rules matching a single request

1. In the case of multiple $dnsrewrite rules matching a single request, we will apply each of them.
2. For example, |example.org|$dnsrewrite=127.0.0.1 and |example.org|$dnsrewrite=127.0.0.2 will result in a DNS response with two A records: 127.0.0.1 and 127.0.0.2.
3. If any of the matching rules implies that the response should be empty, other matching rules are ignored. For example, |example.org|$dnsrewrite=REFUSED and |example.org|$dnsrewrite=127.0.0.1 will result in a DNS response with no resource records and status REFUSED.

Disabling $dnsrewrite rules

• @@||example.org^$dnsrewrite will disable all $dnsrewrite rules matching ||example.org^ pattern.
• @@||example.org^$dnsrewrite=127.0.0.1 will disable rules that are trying to rewrite ||example.org^ to 127.0.0.1

Examples

• Simple rewrite
  |example.org|$dnsrewrite=127.0.0.1 — rewrites example.org (exact match) to 127.0.0.1. Note that this rule is actually the same as |example.org|$dnsrewrite=NOERROR;A;127.0.0.1 as the type A is inferred automatically. Therefore, dig -t a example.org will return 127.0.0.1, other queries will return an empty NOERROR

• CNAME rewrite — this is a special case that rewrites both A and AAAA queries.
  |example.org|$dnsrewrite=example.net — rewrites example.org to example.net. Note that the CNAME type is inferred since response_content is a valid domain name and not an IP address.

• Block HTTPS queries
  ||*^$dnstype=HTTPS,dnsrewrite=REFUSED — responds with an empty REFUSED response to all HTTPS queries. Note, that we specified the dnstype modifier in this rule. Without it, we would've broken all queries (as they would've been rewritten to an empty REFUSED response).

• Multiple A and AAAA records

  • |example.org|$dnsrewrite=127.0.0.1
  • |example.org|$dnsrewrite=127.0.0.2
  • |example.org|$dnsrewrite=::1
  • |example.org|$dnsrewrite=::2

[emlimap]

This also opens up the possibility of being able to subscribe to DNS rewrites like blocklist/allowlists which makes it easier when you run 2 instances of Adguard and have to keep the DNS rewrite rules in sync.

[szolin]

As an example, this is the rule chain that user asks in #2105:

pass A *.cloudfront.net
rewrite AAAA *.cloudfront.net -> dd9uw8v41wo40.cloudfront.net
pass A/AAAA dd9uw8v41wo40.cloudfront.net

[emlimap]

One of the edge case I could think of is how would you distinguish whether a rewrite is for example.org or *.example.org in the above example? As far as I understand, when it comes to allowlist or blocklist, || would cover the domain and any of its subdomains.

Couple of suggestions to make DNS rewrite more powerful

1. Ability to limit the level of a wildcard rewrite. For example, currently a rewrite for *.example.com would apply to both level1.example.com and level2.level1.example.com. It would be good to be able to specify the level it applies. This could be done using a modifier. For example, ||example.org^$dnsrewrite=127.0.0.1,rewritelevel=1. Modifier name is up for discussion.

2. To be able to prevent a DNS rewrite being applied if the rewrite domain is a CNAME record to another domain. For example, if there is a DNS rewrite for abc.example.com, the rewrite shouldn't apply when resolving example.net which has abc.example.com as CNAME. To keep it consistent, we could use whatever modifier is chosen as part of this ticket (Add support for uBO's new "$cname" feature #1570) to exclude cname.

[ameshkov]

1. Ability to limit the level of a wildcard rewrite

Well, for the cases that complicated I'd suggest using regular expressions instead:
/[^.]*\.example.org/$dnsrewrite=127.0.0.1

2. To be able to prevent a DNS rewrite being applied if the rewrite domain is a CNAME record to another domain

Could you please explain why this is useful?

[emlimap]

The use case for me is to add AAAA record for *.github.io to enable IPv6 for Github pages by taking advantage of their CDN provider who support IPv6 as Github don't publish that record at the moment. However, Github pages on custom domain will fail with certificate mismatch as the load balancer on IPv6 address doesn't have certs for custom domains.

Custom domains are served by another load balancer that isn't accessible over IPv6 or its IPv6 address hasn't been publicly documented so far. https://github.com/isaacs/github/issues/354#issuecomment-412956515

Custom domains point to username.github.io and it will get rewritten by AGH when it receives a AAAA query.

Steps to reproduce:

Step 0: Requires you are on IPv6 enabled network
Step 1. Setup rewrite for*.github.io pointing to 2a04:4e42::133
Step 2: Visit https://adguardteam.github.io which should work
Step 3: Visit https://www.zigbee2mqtt.io - Will throw a cert error and needs DNS rewrite disabled

[ameshkov]

That's not a problem, there will be no additional query for the CNAME anyway so we won't rewrite it (CNAME resolution is done on the resolver side, and it returns both A/AAAA and CNAME records right away).

[ameshkov]

@ainar-g check the spec in the issue description, I've finally updated it.

[emlimap]

@ameshkov So came across this thread https://old.reddit.com/r/firefox/comments/jysc04/clearurls_remove_url_tracking_elements_should_be/gd6rxtb/ where OP mentions that Ublock origin dev declined to support redirects as it could potentially be abused by malicious lists.

This made me think, should we have an option to disable importing rewrite rules from remote lists. It could be an option when adding new/editing an existing list or in YAML config file for now and later surfaced to UI should there be a demand. Personally I would prefer to exclude all lists except my own when it comes to importing rewrite rules.

Took a look at the update spec and have a couple of questions,

Simple rewrite
|example.org|$dnsrewrite=127.0.0.1— rewrites example.org (exact match) to 127.0.0.1. Note that this rule is actually the same as |example.org|$dnsrewrite=A:127.0.0.1 as the type A is inferred automatically. Therefore, dig -t a example.org will return 127.0.0.1, other queries will return an empty NOERROR

Unless I am missing something, I don't see any plans to keep the special A and AAAA values available in current rewrite where you can specify to keep the A or AAAA records as received from upstream. In the example, adding A rewrite would cause it to return NOERROR for other records including AAAA.
Really useful when you just want to rewrite AAAA record but keep A or vice versa. Kind of covers this case where just AAAA record needs to be rewritten #2102 (comment)

Don't see the option to ignore a rewrite if the rewrite domain is a CNAME record for another domain. Is it something you prefer to address on a separate feature request?

[ameshkov]

Well, we could do what we do in other AdGuard products.

In AdGuard when you add a custom filter list, you have a choice whether you trust the list to use potentially dangerous rules or not, and then the list will be limited to simple rules only. This should be a separate feature request, though.

Unless I am missing something, I don't see any plans to keep the special A and AAAA values available in current rewrite where you can specify to keep the A or AAAA records as received from upstream.

Here's how this could be achieved:

• |example.org|$dnsrewrite=127.0.0.1 - rewrite A records to 127.0.0.1
• @@|example.org|$dnsrewrite,dnstype=~A - disable "dnsrewrite" rules for all request types save for A, therefore keep the original records.

Don't see the option to ignore a rewrite if the rewrite domain is a CNAME record for another domain. Is it something you prefer to address on a separate feature request?

Could you please elaborate?

[emlimap]

Will raise a separate feature request for the option to trust dangerous rules.

Thank you for the example. That helps.

Could you please elaborate?

You can ignore this request as I got it mixed up with how Firefox DoH deals when it encounters domain names with CNAME but to give you a context, what I was trying to do is add AAAA records for *.github.io domains using rewrite to enable IPv6 on Github pages but keep A record as it is.

rewrite AAAA *.github.io -> 2a04:4e42::133
pass A *.github.io

This works great as long as you are accessing any github.io domain directly but doesn't work for custom domains as the IPv6 load balancer don't have certs for them.

Below is a github hosted domain

❯ dig +short www.zigbee2mqtt.io
koenkk.github.io.
185.199.111.153
<removed for brevity>

Initially I thought that it was AGH rewriting the domain www.zigbee2mqtt.io to include AAAA record since the CNAME matched an existing rewrite rule. Hence, my request wondering if we can ignore rewrite should the rewrite domain be a CNAME to another domain. But it turns out, it was Firefox DNS over HTTPS that was chasing CNAME.

So what actually happened was Firefox made a DNS resolution request for www.zigbee2mqtt.io and then when it came across the CNAME koenkk.github.io, it made a separate request for it thereby triggering the rewrite in AGH. It then merged the results from koenkk.github.io resolution as well as www.zigbee2mqtt.io results as *.github.io would not have returned any A records due to current DNS rewrite running into an edge case it can't handle. It then used IPv6 address to connect because of Happy Eyeballs algorithm which prioritizes v6 over v4 on a dual stacked network.

This can be confirmed from logs

as well as DNS section in about:networking

Querying AGH directly doesn't rewrite. Returns no AAAA records as expected and also confirms the theory

❯ dig +short AAAA www.zigbee2mqtt.io
koenkk.github.io.

This might be intended behaviour on Firefox side. Unbound does something similar. See Forward Zone Options section in https://www.nlnetlabs.nl/documentation/unbound/unbound.conf/

CNAMEs are chased by unbound itself, asking the remote server for every name in the indirection chain, to protect the local cache from illegal indirect referenced items

[ainar-g]

I think that the currently proposed syntax is a bit tricky to parse and has a few unwanted properties. For example, if there is a host named refused in the network, then $dnsrewrite=refused and $dnsrewrite=cname:refused have different semantics. I also think that @emlimap's concerns about rewriting only CNAME records are valid. I propose a more explicit and easier for a machine to parse syntax:

$dnsrewrite=RCODE[:RR:arg]

And, for full A, AAAA, and CNAME rewrites:

$dnsrewrite=NEWCNAME:host

(Alternatively, we could call this one ALIAS or some other, better name.)

@ameshkov's examples from the OP thus become:

|example.org^$dnsrewrite=NOERROR:A:127.0.0.1
|example.org^$dnsrewrite=NEWCNAME:example.net
||*^$dnstype=HTTPS,dnsrewrite=REFUSED
|example.org^$dnsrewrite=NOERROR:A:127.0.0.1
|example.org^$dnsrewrite=NOERROR:A:127.0.0.2
|example.org^$dnsrewrite=NOERROR:AAAA:[::1]
|example.org^$dnsrewrite=NOERROR:AAAA:[::2]

This syntax also seems more easily extensible, at least to me. What do you all think?

[ameshkov]

@ainar-g I have a couple of questions about the proposed syntax.

First, what would be the difference between NEWCNAME/ALIAS and CNAME?

Also, why can't we simplify the life of filter maintainers and allow using shorter syntax for the most popular use cases?

These rules won't introduce any extra ambiguity and are perfectly readable:

|example.org^$dnsrewrite=127.0.0.1
|example.org^$dnsrewrite=[::1]

Regarding domain names, the example with refused is valid and there is an ambiguity with domain names indeed. But still, it nags me that we're making people use the full syntax when it's trivial to infer what exactly they want from it.

[ainar-g]

@ameshkov

First, what would be the difference between NEWCNAME/ALIAS and CNAME?

NEWCNAME:example.org is exactly like the CNAME behaviour in the OP: rewrite A, AAAA, and CNAME records to those of example.org. NOERROR:CNAME:example.org will leave A and AAAA as is but replace the original CNAME with example.org.

Also, why can't we simplify the life of filter maintainers and allow using shorter syntax for the most popular use cases?

These rules won't introduce any extra ambiguity and are perfectly readable:

|example.org^$dnsrewrite=127.0.0.1
|example.org^$dnsrewrite=[::1]

Just to be clear, you are proposing this shorthand form only for A and AAAA records as opposed to the NEWCNAME behaviour, right? I think we could do that, although I personally would not use that form.

[ameshkov]

I just don't really understand what's the necessity in rewriting just the CNAME without actually resolving it and replacing A/AAAA records.

Just to be clear, you are proposing this shorthand form only for A and AAAA records as opposed to the NEWCNAME behaviour, right? I think we could do that, although I personally would not use that form.

Why, it's not opposed to anything.

The full syntax would be: |example.org^$dnsrewrite=NOERROR:A:127.0.0.1.

Also, I am still not convinced that making people write |example.org^dnsrewrite=CNAME:example.net instead of just |example.org^dnsrewrite=example.net is justified. The easy way to solve the ambiguity would be to limit the shorthand strictly to TLD+N domains (i.e. you can't rewrite anything to refused) and mention this in the spec.

I fully support the full syntax, though. It's clearer that what I've written in the first place. My only concern is lack of shorthands.

[ainar-g]

After some discussion, here is my updated proposal:

1. The full form stays, but we use another separator, like ;.
2. If the rule has two separators, then it is a full-form rule, otherwise it's a shorthand rule.
3. The full-form CNAME only replaces CNAME records;
4. The shorthand rules are basically what @ameshkov described initially:
   1. IPv4 means replacing A records, so $dnsrewrite=127.0.0.1 equals $dnsrewrite=NOERROR;A;127.0.0.1;
   2. IPv6 means replacing AAAA records, so $dnsrewrite=::1 equals $dnsrewrite=NOERROR;AAAA;::1;
   3. uppercase words are keywords, like REFUSED;
   4. lowercase words are domains for the “smart CNAME rewrite” that rewrites A, AAAA, and CNAME records.

Obviously, all of that shall be documented with lots of examples based on real-life use-cases.

[ameshkov]

Works for me. Could you please change the spec in the OP?

[ameshkov]

Wait, not everything works for me:

The full-form CNAME only replaces CNAME records;

I still think that CNAME should make AGH chase the first CNAME and add it's records.

If we ever need to specify CNAME that won't chase it, we can add a new type. However, I really doubt we will ever need this.

[ainar-g]

@ameshkov, okay, but you owe me one if there is a legit case for it :-) .

[ainar-g]

As an update: we've just merged the first batch into AdGuardHome's master branch. The next steps are:

1. Documentation.
   1. This issue's OP.
   2. AdGuardHome Wiki.
2. Support for MX, PTR, and HTTPS/SVCB.

[ainar-g]

@ameshkov, I think this one is basically done? I propose we close this issue.

[ameshkov]

Yep