Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Server discovery via SRV records for mtop-client #107

Closed
4 of 5 tasks
56quarters opened this issue Jan 25, 2024 · 1 comment
Closed
4 of 5 tasks

Server discovery via SRV records for mtop-client #107

56quarters opened this issue Jan 25, 2024 · 1 comment
Assignees
@56quarters
Labels
area:client Relating to the Memcached client area:dns Relating to the DNS client enhancement New feature or request

Comments

@56quarters
Copy link
Owner

56quarters commented Jan 25, 2024 •
edited
Loading

The client currently has basic "discovery" functionality, DiscoveryDefault will create Server instances based on A or AAAA records. Being able to resolve SRV records is the next missing bit of functionality. Let's try to do it without adding a dependency, how hard could DNS be? 😬

The work will be split into the following chunks to get something available sooner.
@56quarters 56quarters added enhancement New feature or request area:client Relating to the Memcached client labels Jan 25, 2024
56quarters added a commit that referenced this issue Jan 29, 2024
@56quarters
Implementation of jump consistent hashing for selecting servers
2e2e53e 
This change adds new logic for selecting which server owns a particular
key based on the jump consistent hash. This logic is much faster than
the existing rendezvous hashing but is more operationally fragile.

Using jump consistent hashing requires that servers are always in the
same order and that servers are added and removed from the end of this
list, think how statefulsets in Kubernetes work.

It's not currently possible to make use of this in mtop since we have
no way to look up the names of servers until we add support for SRV
DNS records. Until then, servers are ordered by their SocketAddr which
means they would end up being removed from the middle of the list
which completely breaks jump consistent hashing.

Depends on #107
@56quarters 56quarters mentioned this issue Jan 29, 2024
Closed
@56quarters 56quarters self-assigned this Mar 8, 2024
56quarters added a commit that referenced this issue Mar 19, 2024
@56quarters
Implement a rudimentary DNS client for SRV record lookup
852b6bc 
This change creates a very basic DNS client capable of performing A, AAAA
CNAME, NS, SOA, TXT, and SRV types of queries. This is the first step in
supporting DNS lookup of SRV records for service discovery in `mtop` and
`mc`. It also adds a `dns` executable capable of making queries and parsing
binary DNS messages.

This client has undergone very little testing and should be considered
both experimental and internal to mtop.

See #107
@56quarters 56quarters mentioned this issue Mar 19, 2024
Merged
56quarters added a commit that referenced this issue Mar 19, 2024
@56quarters
Implement a rudimentary DNS client for SRV record lookup
443c50e 
This change creates a very basic DNS client capable of performing A, AAAA,
CNAME, NS, SOA, TXT, and SRV types of queries. This is the first step in
supporting DNS lookup of SRV records for service discovery in `mtop` and
`mc`. It also adds a `dns` executable capable of making queries and parsing
binary DNS messages.

This client has undergone very little testing and should be considered
both experimental and internal to mtop.

See #107
56quarters added a commit that referenced this issue Mar 20, 2024
@56quarters
Implement a rudimentary DNS client for SRV record lookup
d946372 
This change creates a very basic DNS client capable of performing A, AAAA,
CNAME, NS, SOA, TXT, and SRV types of queries. This is the first step in
supporting DNS lookup of SRV records for service discovery in `mtop` and
`mc`. It also adds a `dns` executable capable of making queries and parsing
binary DNS messages.

This client has undergone very little testing and should be considered
both experimental and internal to mtop.

See #107
56quarters added a commit that referenced this issue Mar 20, 2024
@56quarters
Implement a rudimentary DNS client for SRV record lookup
b6d6f3b 
This change creates a very basic DNS client capable of performing A, AAAA,
CNAME, NS, SOA, TXT, and SRV types of queries. This is the first step in
supporting DNS lookup of SRV records for service discovery in `mtop` and
`mc`. It also adds a `dns` executable capable of making queries and parsing
binary DNS messages.

This client has undergone very little testing and should be considered
both experimental and internal to mtop.

See #107
56quarters added a commit that referenced this issue Mar 21, 2024
@56quarters
Implement a rudimentary DNS client for SRV record lookup
3221a68 
This change creates a very basic DNS client capable of performing A, AAAA,
CNAME, NS, SOA, TXT, and SRV types of queries. This is the first step in
supporting DNS lookup of SRV records for service discovery in `mtop` and
`mc`. It also adds a `dns` executable capable of making queries and parsing
binary DNS messages.

This client has undergone very little testing and should be considered
both experimental and internal to mtop.

See #107
56quarters added a commit that referenced this issue Apr 4, 2024
@56quarters
Allow DNS SRV records to be use to disover cache servers
0a979a4 
Adds support for disovering Memcached servers using DNS SRV records
via the `dnssrv+` prefix on hostnames supplied for `mtop` and `mc`.

Part of #107
@56quarters 56quarters mentioned this issue Apr 4, 2024
Merged
56quarters added a commit that referenced this issue Apr 4, 2024
@56quarters
Allow DNS SRV records to be use to disover cache servers
f22c927 
Adds support for disovering Memcached servers using DNS SRV records
via the `dnssrv+` prefix on hostnames supplied for `mtop` and `mc`.

Part of #107
56quarters added a commit that referenced this issue Apr 4, 2024
@56quarters
Allow DNS SRV records to be use to disover cache servers
ef81989 
Adds support for disovering Memcached servers using DNS SRV records
via the `dnssrv+` prefix on hostnames supplied for `mtop` and `mc`.

Part of #107
56quarters added a commit that referenced this issue Apr 5, 2024
@56quarters
Allow DNS SRV records to be use to discover cache servers
4fd1377 
Adds support for discovering Memcached servers using DNS SRV records
via the `dnssrv+` prefix on hostnames supplied for `mtop` and `mc`.

Part of #107
56quarters added a commit that referenced this issue May 17, 2024
@56quarters
Read DNS settings from a resolv.conf file
dfa5fcb 
Allow some DNS settings to be read from a system's /etc/resolv.conf
file. Noteably, only the `nameserver` and a few `option`s are supported
for now.

Part of #107
@56quarters 56quarters mentioned this issue May 17, 2024
Merged
56quarters added a commit that referenced this issue May 17, 2024
@56quarters
Read DNS settings from a resolv.conf file
afbfa60 
Allow some DNS settings to be read from a system's /etc/resolv.conf
file. Noteably, only the `nameserver` and a few `option`s are supported
for now. This change also switches resolution of A and AAAA names to
our DNS client instead of using the system resolver via `lookup_host`.

Part of #107
56quarters added a commit that referenced this issue May 19, 2024
@56quarters
Add TCP support to DNS client
c3ac0e0 
Add fallback to TCP for DNS client when the UDP message is trunctated.

Part of #107
@56quarters 56quarters mentioned this issue May 19, 2024
Merged
56quarters added a commit that referenced this issue May 19, 2024
@56quarters
Add TCP support to DNS client
c5ae501 
Add fallback to TCP for DNS client when the UDP message is trunctated.

Part of #107
56quarters added a commit that referenced this issue May 19, 2024
@56quarters
Add TCP support to DNS client
49615b8 
Add fallback to TCP for DNS client when the UDP message is trunctated.

Part of #107
56quarters added a commit that referenced this issue May 19, 2024
@56quarters
Add TCP support to DNS client
1c4243d 
Add fallback to TCP for DNS client when the UDP message is trunctated.

Part of #107
56quarters added a commit that referenced this issue May 21, 2024
@56quarters
Refactor DnsClient and introduce unit tests
073ca67 
Part of #107
@56quarters 56quarters mentioned this issue May 21, 2024
Merged
56quarters added a commit that referenced this issue Jun 15, 2024
@56quarters
dns: initial opt record support
ee4943b 
This change introduces the ability to parse OPT records but
does not yet add them to requests or uses them when they are
part of a response. A future PR will start making use of OPT
records.

This change also does some unrelated cleanup, removing `--dns-local`
flags from each of the binaries in favor of picking this value
automatically. Additionally, the `--timeout-secs` option is removed
from the `dns` binary since it has been unused since resolv.conf
support was added.

Part of #107
@56quarters 56quarters mentioned this issue Jun 15, 2024
Merged
@56quarters 56quarters added the area:dns Relating to the DNS client label Jun 15, 2024
@56quarters
Copy link
Owner Author

Opened #155 to finish OPT support. Closing this as complete.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@56quarters 56quarters

Labels
area:client Relating to the Memcached client area:dns Relating to the DNS client enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@56quarters