Prototyping Unbound extensions in minutes with Python and Docker
How, theoretically, could one lower the barrier to trying out the connection testing feature of the Internet.nl application.
By Ximon Eighteen
I’m fairly new to Unbound and I’ve got a lot to learn about DNS, so when a “how can we do that?” conversation quickly turned into a proposal, I didn’t expect to be prototyping an implementation just a few hours later. If my development environment were already setup it would have been minutes. In this short article I’d like to share just how easy it is to extend Unbound and how Docker can get you started even quicker.
Essentially the question at hand was:
How, theoretically, could one lower the barrier to trying out the connection testing feature of the Internet.nl application when deploying it for yourself?
The challenge lies in the fact that:
a) the Internet.nl connection testing feature uses a custom Unbound C module which needs to run in an Unbound instance somewhere; and
b) to setup such a zone so that it can be queried via standard DNS resolution from a client computer you need the right to delegate from a parent zone.
Docker makes deployment of Internet.nl for evaluation and testing purposes quick and easy. However, Unbound and DNS zone setup on the Internet can only be automated if you already have edit rights in a parent zone, and use a DNS provider that offers an API through which edits can be automated.
Creating long lived zone edits is overkill if the sub-domain will be short lived which is often the case with dynamic just-in-time deployments for dev/test/CD purposes, or in the Internet.nl evaluation and testing use case.
Standing on the shoulders of giants
Plenty of services exist to associate an IP address with a sub-domain of a 3rd party service, but does a service exist to delegate from such a sub-domain to your own name servers? Why not do it like xip.io, suggested @wk?
What is xip.io?
xip.io is a magic domain name that provides wildcard DNS
for any IP address.
xip.io implements resolving a.b.c.d.xip.io to an a.b.c.d A record using a “PowerDNS pipe backend adapter”, i.e. it allows anyone to delegate to any name server from any sub-domain of the service without needing edit rights in a parent zone, only enables the sub-domain when needed and only for clients that need it!
If PowerDNS can be extended, could Unbound also be similarly extended and if so is it easy to do? A quick Google and a kindly posted blog and the answer appeared to be yes and yes!
The Unbound module interface
Each Unbound Python module has to implement a very simple interface:
- init_standard(id, env)
- deinit(id)
- operate(id, event, qstate, qdata)
- inform_super(id, qstate, superqstate, qdata)
init_standard() and its counterpart deinit() are useful for resource management, e.g. opening and closing connections to services whose help you will need when processing DNS queries such as databases, caches, cloud APIs, etc. init_standard() should be used in preference to the older init().
operate() and inform_super() are where the work of processing queries happens. If while processing a query another “sub” query needs to be resolved, inform_super() will be called on completion of the “sub” query. More information on when these functions are called can be found in the Unbound source code documentation.
My first Unbound module
So, forget the “theoretically” part of the question, I realized I could actually try this out!
The hardest part of this exercise was compiling Unbound with Python module support, as the stock Unbound package for Ubuntu 18.04 LTS (my go to Docker base image when testing out ideas) doesn’t have the “--with-pythonmodule” configure option enabled.
After solving that (and creating a handy Docker image for others to use, see below) I wrote a simple Python string match against the domain being queried in the DNS request, wrote a couple of lines of code to construct the response, and configured Unbound to use my Python module and the xip.io case was working:
From there a proof-of-concept of the DNS delegation case was just a trivial extension:
Note: you need a few more lines of code to make it actually work, the above snippets just show the main logic for this use case.
Seeing it in action
First, the xip.io case: a.b.c.d.xip.io to A a.b.c.d:
$ host 127.0.0.1.asa.inet_aton.ximoneighteen.com
127.0.0.1.asa.inet_aton.ximoneighteen.com has address 127.0.0.1In this example the xip.io like service was running Unbound on a virtual machine which acted as the authority for the inet_aton.ximoneighteen.com DNS zone. My local resolver recursively queries DNS servers on the Internet and/or its own cache to determine which DNS server knows the A record (IPv4 address) for 127.0.0.1.asa.inet_aton.ximoneighteen.com. Behind the scenes the query is resolved something like this:
- From root hints the local resolver finds the root servers, which direct it via an NS delegation and A glue records to name servers for .com.
- The .com name servers provide NS delegation and A glue records identifying AWS Route 53 as the authority forximoneighteen.com.
- AWS Route 53 name servers provide NS delegation and A records identifying my virtual machine running Unbound as the authority for inet_aton.ximoneighteen.com.
- Unbound invokes Python to process the DNS query. Python matches asa (as an “A” record) in the query, extracts the IPv4 address, and responds with a dynamically generated “A” record pointing domain name 127.0.0.1.asa.inet_aton.ximoneighteen.com to IPv4 address 127.0.0.1.
Second, the xipdelegate.io case: some.<base 32 a.b.c.d> to NS (+ A glue) pointing to a.b.c.d:
$ dig @ns1.ximoneighteen.com $(echo 172.31.23.137 | base32).nsdel.inet_aton.ximoneighteen.com
...
;; AUTHORITY SECTION:
MTcyLjMxLjIzLjEzNwo=.isdel.inet_aton.ximoneighteen.com. 60 IN NS 172.31.23.137.
;; ADDITIONAL SECTION:
MTcyLjMxLjIzLjEzNwo=.asa.inet_aton.ximoneighteen.com. 60 IN A 172.31.23.137
...(this example assumes that you have a Linux base32 command, like the base64 command)
When querying the default recursive resolver (instead of directly querying my Unbound name server) it will “follow” the NS record to the zone authoritative DNS server and obtain the A record for the sub-domain:
$ dig some.$(echo 172.31.23.137 | base32).nsdel.inet_aton.ximoneighteen.com
...
;; ANSWER SECTION:
some.MTcyLjMxLjIzLjEzNwo=.nsdel.inet_aton.ximoneighteen.com. 6672 IN A 127.0.0.1
...Note: This isn’t meant to be a production-ready implementation. In production you should delegate to more than one name server, implement full error handling, logging and metrics, and ideally IPv6 and DNSSEC would be supported (and would be needed for the Internet.nl use case). You might also want to re-implement the module in ‘C’ which is presumably faster and leaner.
Side note: could LDNS or OpenDNSSec be used to automate DNSSEC signing?
RTFM!
Now, what I didn’t realize until after I got this working was that Unbound comes with Python examples. If you want to write a Python module for Unbound I suggest taking a look at the examples to help you getting started.
No, really, RTFM!
I also discovered later that the Unbound Python module HTML documentation that can be built as part of the source code could be really useful as it documents the interface, the types and comes with examples!
Dockerizing for the future
To make it even easier to prototype Unbound extensions with Python in future I created a little Dockerfile that creates a working Hello World style Python module, and published it to Docker Hub.
Setting up a working Unbound with Python module support enabled where you can edit the configuration and Python code and run it in Unbound is now as simple as:
$ docker run -it nlnetlabs/pythonunbound
root@nnn:/usr/local/etc/unbound#: vi helloworld.py
root@nnn:/usr/local/etc/unbound#: unbound
root@nnn:/usr/local/etc/unbound#: dig +noall +answer @127.0.0.1
helloworld. 300 IN A 127.0.0.1Thanks for reading, I hope this helps someone and I’d love to hear back about your own experiences extending Unbound.
Credits
Thanks to George Thessalonikefs, Ralph Dolmans and Wouter Wijngaards for corrections and improvements and to @wk for the inspiration that triggered this experiment!
Useful resources
- Unbound with Python module support as a Docker image
- Unbound Python module implementing http://xip.io/ like functionality
- Unbound Python module examples
- Unbound Python module documentation