In mid-February we informed about Reducing TTL in the .cz zone by one hour. Then, at a similar hour every Wednesday, we reduced it by another hour, until on March 15, 2017 we reached the required value of 1 hour (i.e. TTL=3600).
We reduced the TTL at the following times:
February 22, 2017 at 2:50 PM,
March 1, 2017 at 3:25 PM,
March 8, 2017 at 3:50 PM,
March 15, 2017 at 3:25 PM,
As a precaution, we reduced TTL in one-week intervals, as a major change by 4 hours could bring about a more pronounced increase of queries at the time of expiration of cache records on recursive DNS resolvers. A gradual reduction enabled us to continuously monitor the impacts of the change on the operation and, where appropriate, to adapt the next steps to the observed impacts. We have been long collecting all the data about the DNS traffic for our system, thanks to which we are able to retrieve not only summary statistics, but also data on individual DNS requests.
Interesting facts:
On average, all the anycast DNS servers, both in the Czech Republic and in foreign locations, serve around one billion requests per day (as of March 2017).
The most of the DNS traffic in the .cz zone comes from O2 (an average of 150,000,000 queries/day) and Google (130,000,000 queries/day). The ranking continues with Seznam.cz ranks and its e-mail services (53,000,000 queries/day) and Microsoft (30,000,000 queries/day). These are followed by larger ISPs, hosting companies and mobile operators.
Most DNS traffic is served by the sites in the Czech Republic – 36%, followed by sites in the UK – 19%, USA – 14%, Austria – 12%, Germany – 10%, and the rest of the world (Chile, Sweden, Japan) – 9%.
Let’s take a closer look at DNS traffic. By DNS traffic we mean the DNS queries from the DNS servers to a query name (QName) along with the query type (QTYPE), and related response with the return code.
The most commonly occurring query types from our statistics for March 2017 are shown in the following chart. Just as a reminder:
A – Address Record (contains the IPv4 address assigned to the name),
AAAA – IPv6 Address Record (contains the IPv6 address assigned to the name),
DS – Delegation Signer (contains the hash of a public key for the DNSSEC),
MX – Mail Exchange Record (contains the address and priority for receiving electronic mail),
NS – Name Server Record (contains the address of the authoritative DNS server for the name)
TXT – Text Record (contains any text string),
SRV – Service Record (contains a reference to a different address and port),
CNAME – Canonical Name Record (contains an alias for the name).
Response share by query type
The most common return codes in response are:
0 – NOERROR (query successfully executed),
1 – FORMERR (error in query format),
2 – SERVFAIL (query cannot be executed, error),
3 – NXDOMAIN (query name does not exist),
4 – NOTIMP (unsupported query type),
5 – REFUSED (DNS server has refused to respond to the query),
9 – NOTZONE (query name is not located in a zone).
Response share by return codes
Every day approximately between 2:50 PM and 5 PM, we register a distinctive increase in DNS queries returning the response 3 – NXDOMAIN.
What are those non-existent domain names and types of records? Most commonly, it is the domain names that have already been deleted from the registry and from the DNS servers and which recursive DNS servers are still requesting. Some, however, are most likely fictitious or request directly the authoritative DNS servers of CZ.NIC instead of the delegated NS servers.
The impact of the TTL reducing on the operation of DNS servers
The biggest impact of the TTL reducing was a rapid surge in all DNS queries expired from the DNS resolvers’ cache.
When analyzing all DNS traffic, the number of queries per second during peak periods of each day appeared to be on average 50% higher after the TTL reducing. As is evident from the charts below, the surge subsided after a few minutes.
We also observed an increased number of NXDOMAIN responses – responses to DNS queries for non-existent or expired records. These may have been caused by DNS resolvers without using the cache.
The graph below shows surges in NXDOMAIN, lasting 15 minutes according to the configured NXDOMAIN TTL.
However, from the long-term perspective, the change of TTL had no significant effect on the DNS traffic. The short-term increase in the number of DNS queries on the servers matched our expectations; it was neither radical nor nonstandard.
By reducing the TTL, we have achieved faster changes to the delegations in the registry that will be soon notable on both authoritative DNS servers and DNS resolvers. The currently set TTL in the .cz zone can be verified using the command $ dig +multiline soa cz, or online, for example on http://www.webdnstools.com/dnstools/dns-lookup.
For comparison – TTL of national domains of the surrounding countries range from one hour to two days. Examples:
Netherlands (.nl) – 1 hour,
Hungary (.hu) – 1 hour,
Germany (.de) – 1 day,
Slovakia (.sk) – 1 day,
Poland (.pl) – 1 day,
Austria (.at) – 2 days,
France (.fr) – 2 days,
United Kingdom (.uk) – 2 days.
The interesting thing is that generic domains .com, .net, .biz and .org have TTL set up at just 15 minutes.
Using the same method, it is possible to determine the TTL value of popular domains of second and lower levels.