Maintaining the love for coding in the time of AI

By Alex Band

Over the last year, we have seen the profound effects Large Language Models (LLMs) have on our open-source software work at NLnet Labs. These can be grouped into two main areas: community contributions and security reports. Before we dive into this topic, let me first give you some context and history on how we've developed our software over the last 25 years.

Developing software at NLnet Labs

I want to take a step back and explain how we maintain open-source projects that are intended to run in critical infrastructure. Because when people think of open-source, they almost automatically assume that there is a community of individual developers collaborating on a project.

However, we develop software for internet standards like DNS, BGP and RPKI, which are all rather niche. Also, we build software in C and Rust, arguably making our line of work a niche of a niche. As a result, we work in small, dedicated teams, almost like a commercial company would, with a roadmap, project planning and scheduled releases. So, while everything we do is open-source and liberally licensed, in the last 25 years we haven’t received many external contributions on projects like Unbound, NSD and Routinator other than small bug and typo fixes.

There have been a couple of occasions where for example our DNS resolver Unbound received a substantial pull request. The origins of these were in most cases from organizations who had customized Unbound for their specific use case and were looking to have their changes merged upstream. The reasons were in most cases a matter of maintainability: in having their special sauce become part of the main branch maintained by NLnet Labs, they no longer had to apply their custom patches to newly released versions of Unbound.

In a perfect scenario, this dynamic benefits everyone. Our project gets additional functionality that benefits the wider community, and the organization who contributed the code now has the features they require and are guaranteed long-term maintenance by the core development team. Win-win!

Of course, reality is a little more nuanced. Our teams meticulously look after the architecture, long-term maintainability and coding style of our projects. It’s one of the reasons a project like Unbound has been able to evolve for almost 20 years without requiring a complete rewrite to live up to the requirements operators have of a DNS resolver for the internet of today.

In fact, we apply this rigorous review process internally within the teams as well. When one developer designs and builds a new feature, a team member will meticulously review their work line for line, so that they understand and are able to maintain the new functionality. Code will often go through several review cycles before it's finally merged and released.

There is a social dynamic to consider as well. Even though functionality is designed with the best intentions, developers can be particular about coding style and even line width. You know who you are 😉. But it also requires the heart to tell the author that you really don’t think that their unwieldy XML schema is the right choice.

For external contributions, in some cases the conclusion is that it’s better to let the core team at NLnet Labs rewrite the feature entirely from scratch. We have agreed to this several times in the past, like for example with Serve Stale in Unbound, which we implemented for an agreed fee. In other cases, both parties may conclude that the contribution is a better fit for the /contrib folder, where features live that are not part of the official distribution but may be helpful to someone.

Contributions: The Next Generation

This contribution dynamic has radically changed in the last 12 months with the release of several frontier AI models. As a result, there is now a league of developers who feel they have gained AI-fueled superpowers. This is fine for many use cases, but we felt we needed to approach this new reality with caution.

It has to be said: it’s a sight to behold to tell your favorite AI agent to “Implement Catalog Zones defined in RFC 9432 in Cascade by NLnet Labs. Stop only when all the tests pass and all specifications in the RFC are covered. The final result must include man pages, tests and full coverage of the RFC.” 

You can go grab some coffee and when you come back, you’ll have 4,000 lines of plausible looking code that may even work to some extent. For arguments sake, let’s gloss over all the design choices that were made for the specifics that aren’t described in the RFC. 

Your next step could be creating a pull request and feel like you’ve made a valuable contribution to not just NLnet Labs, but the entire DNS operator community. This kind of contribution to your favorite open-source project may feel to you like “free as in beer” and “free as in speech” but to us it is like being handed a “free puppy”: well intended, but accepting it has big consequences for years to come.

But before I dive into the lasting effects of AI-generated and assisted code contributions, I’d like to first address another result of the latest LLMs: the influx of security reports found with the help of AI… 

Security research, but make it an avalanche

For reasons explained earlier, our work is not open-source so that it can become a joint community project, but to allow us to be open and transparent about the way our software works. An important reason for us to develop open-source software is so that everyone can inspect our code to verify correct behavior and find potential flaws and vulnerabilities. This has made all of our projects a fertile hunting ground ranging from PhD students to seasoned security professionals. The result is a suite of incredibly robust applications that get more secure with every release.

Not even a year ago, developers in our community were lamenting AI-slop. We smiled at bug reports that made no sense at all, and were mildly annoyed they interrupted our day a couple of times a month. Because that’s the thing: you can’t ignore them, because what if there is one that is actually a vulnerability with potentially catastrophic consequences? We must look at the report and respond to the reporter, no matter what.

Fast forward to today and the AI-slop problem has certainly resolved itself. Bug reports are now extremely accurate, and come packed with steps to reproduce and potential fixes. What's more worrying is that they come in with literally hundreds at a time. It’s truly an avalanche. 

Triaging these reports takes a significant amount of time. Our teams can literally spend an entire day just figuring out what the issue that's reported actually is. Whilst it is true that there are often steps to reproduce the problem, the issue description still tends to be quite vague. The latest batch of Routinator LLM bug reports contains gems like "Round-trip infidelity", "K-file storm pins validation" and "Linear-scan burn via block sort position".

If you’re wondering why you haven’t seen any major new features in our software since the beginning of this year, it’s because by now almost all of our teams are maxed out with releasing bug fixes, in some cases in close coordination with other software vendors. There have been times that the stream seemed to dry up and we thought we reached the end of the tunnel, but then a new AI model was released and the process started over.

It’s a complicated dilemma for us. As a manager I get to experience what makes a software developer at NLnet Labs tick: solving a complicated problem with a beautiful, elegant solution. Artisanal if you will. But for months on end now, most of our teams have just been triaging bugs and coordinating releases. It has truly taken all the fun out of the job now, and it’s difficult to predict where this road will lead.

Where do we go from here?

As it stands, we see no other option than to publish the LLM policy that we have now, requiring all code and documentation contributions to be authored by a human. We do accept reports of vulnerabilities found with LLMs. Some people may think we're being unreasonable, because even when assisted by AI, it's possible that a human can be accountable for every line of code they contribute.

While it's true that a seasoned C or Rust developer who is well versed in DNS or BGP could make a contribution with the assistance of AI and take complete responsibility for their code, sadly that is not our experience. Moreover, while I have framed our reasons in this article solely around practical arguments, ethical and intellectual property considerations play an important part in our decision too.

Still, the road ahead is fraught with uncertainty. The immediate problem is one of scale. NLnet Labs has fifteen employees, twelve of which are software developers. Our foundation doesn't have the financial means to add additional headcount. Relying on LLMs for triaging ourselves would not only cost us a fortune and even if we did, what would that mean — that LLM-generated bug reports are triaged, fixed, merged and released by LLMs? If that is the solution to keep our head above water, I would say that goes against everything we believe in. Humans need to make the difference.

I firmly believe that we have a product portfolio we can be proud of and have a long and bright future. An enormous part of the internet’s core depends on our tooling, ranging from DNS resolvers to RPKI validators. We’re excited about the prospects of our new DNSSEC signer Cascade, and everything else we have in the pipeline.

We find ourselves at a crossroads, with the well-being of our team and the long-term viability of our organization at stake. We expect that people who argue that it’s irresponsible to use programming languages that don’t guarantee memory safety will perhaps soon claim that it’s irresponsible to release software that isn’t tested by LLMs during development. We’ll keep a keen eye on CI/CD tooling that facilitates LLM testing for free, open-source software, while ensuring that we remain true to our values and look after the sustainability of our foundation.

With ♥️ from the people at NLnet Labs.