How to use OctoDNS and DNS split authoritative configuration

Introduction

Building a robust system requires designing for failure. As Site Reliability Engineers (SREs) at GitHub, we're always looking to help mitigate issues through redundancy, and today we'll discuss recent work we've done to support you finding our servers via DNS.

Large DNS providers have multiple levels of redundancy built into their services, and when an issue occurs that causes an outage, steps can be taken to mitigate its impact. One of the best options is to split your region's authoritative service among multiple service providers. Enabling split authority is as simple as configuring two or more sets of name servers for your zone at your domain registrar, and DNS requests will be split across the list. However, you must keep records in these areas synchronized across multiple providers, and depending on the situation this can be either complex to set up or a completely manual process.

$ dig NS github.com. @a.gtld-servers.net.
...
;; QUESTION SECTION:
;github.com. IN NS
;; AUTHORITY SECTION:
github.com. 172800 IN NS ns4.p16.dynect.net.
github.com. 172800 IN NS ns-520.awsdns-01.net.
github.com. 172800 IN NS ns1.p16.dynect.net.
github.com. 172800 IN NS ns3.p16.dynect.net.
github.com. 172800 IN NS ns-421.awsdns-52.com.
github.com. 172800 IN NS ns-1283.awsdns-32.org.
github.com. 172800 IN NS ns2.p16.dynect.net.
github.com. 172800 IN NS ns-1707.awsdns-21.co.uk.
...

The above query is to ask the TLD name server for the NS record of github.com. It returns the value that we configured in the domain registrar, in this case, there are two DNS service providers with four records each. If one of the providers experiences an outage, there is hope that the others can still service the request. We sync records everywhere and can safely modify them without worrying about stale data or incorrect state.

The final part of fully configuring split authority is to add all name servers in both DNS service providers as top-level NS records to the root of the zone.

$ dig NS github.com. @ns1.p16.dynect.net.
...
;; QUESTION SECTION:
;github.com. IN NS
;; ANSWER SECTION:
github.com. 551 IN NS ns1.p16.dynect.net.
github.com. 551 IN NS ns2.p16.dynect.net.
github.com. 551 IN NS ns-520.awsdns-01.net.
github.com. 551 IN NS ns3.p16.dynect.net.
github.com. 551 IN NS ns-421.awsdns-52.com.
github.com. 551 IN NS ns4.p16.dynect.net.
github.com. 551 IN NS ns-1283.awsdns-32.org.
github.com. 551 IN NS ns-1707.awsdns-21.co.uk.

At GitHub, we have dozens of areas and thousands of records, and most of these areas are not critical enough to require redundancy, so we only have to deal with a subset. We would like to have a solution that can keep these records synchronized across multiple DNS service providers, and more generally manage all DNS records internally and externally. So today we’re announcing OctoDNS.

Configuration

OctoDNS allows us to reinvent our DNS workflow. Our regions and records are stored in configuration files in the Git repository. Use GitHub flows for changes to them and deploy them with branches just like a site. We can even make an "empty" deployment to preview which records will be modified in the change. Configuration files are yaml dictionaries, one per region, whose top-level keys are record names and key values ​​are ttl, type, and type-specific data. For example, when included in the zone file github.com.yaml, the following configuration will create an A record for octodns.github.com.

octodns:
type: A
values:
- 1.2.3.4
- 1.2.3.5

The second part of the configuration maps the sources of record data to DNS service providers. The following code snippet tells OctoDNS to load the zone github.com from the config provider and sync its results to dyn and route53.

zones:
github.com.:
sources:
- config
targets:
- dyn
- route53

Synchronize

Once our configuration is complete, OctoDNS can evaluate the current state and build a plan that outlines the set of changes that will be needed to match the target state to the source. In the example below, octodns.github.com is a new record, so the required action is to create records in both.

$ octodns-sync --config-file=./config/production.yaml
...
********************************************************************************
* github.com.
********************************************************************************
* route53 (Route53Provider)
* Create
* Summary: Creates=1, Updates=0, Deletes=0, Existing Records=0
* dyn (DynProvider)
* Create
* Summary: Creates=1, Updates=0, Deletes=0, Existing Records=0
********************************************************************************
...

By default octodns-sync is in simulated run mode, so no action will be taken. Once we have reviewed the changes and are happy with them, we can add the `--doit' flag and run the command again. OctoDNS will continue its processing flow, this time making the necessary changes in Route53 and Dynect to create the new record.

$ octodns-sync --config-file=./config/production.yaml --doit
...

At this point, we have the same data records in both DNS service providers and can easily split our DNS requests between them, knowing that they will provide accurate results. When we run the OctoDNS command above directly, our internal workflow relies on deployment scripts and chatops. You can find more information in the workflow section of the README.

Summarize

We think most websites could benefit from splitting authority, and hopefully with OctoDNS, the biggest hurdle has been removed. Even if you're not interested in splitting authority, OctoDNS is still worth a look because it brings the benefits of infrastructure as code to DNS.

Want to help the GitHub SRE team solve interesting problems? We'd love to join us. Apply here.

The above is the detailed content of How to use OctoDNS and DNS split authoritative configuration. For more information, please follow other related articles on the PHP Chinese website!