We are releasing dns-collector, an entry part of our pipeline for monitoring of our DNS servers and analysis of the DNS traffic. Together with advanced analysis of the collected data, we can not only monitor the DNS traffic for urgent problems, but also detect and examine long-term trends and issues (e.g. misconfiguration of other servers). We have presented this system at the IT 15.2 conference (video and slides in Czech).
The collector is an early part of the analysis pipeline, listening to the DNS traffic and creating a pre-processed form of the observed queries suitable for efficient storage, transport and further analysis. The requests are properly matched with responses according to IETF draft, the collector supports several EDNS extensions, outputs either CSV or CBOR structured data, admits BPF filtering and recorded feature selection, compression and time-based output file rotation, and of course supports PCAP-based offline processing. For the highest performance, the collector is written in plain C atop our libknot DNS library (part of KNOT DNS server) and can reliably handle over 100 000 queries per second. You can run it either on the DNS server or on a dedicated server with port-mirroring setup.
You can find the collector released under GPLv3 at our github and pre-build packages for Ubuntu, Debian and other distros at OpenBuildService.