tl;dr: I am closing the SIXFW project. It hurts, but having learned from it makes it easier.
A bit over a year ago, at the Chaos Communication Congress 2015, I was part of the team responsible for the NAT64 part of the congress network. We usually used a commercial appliance to do the IPv6-to-IPv4 translation. Running super-expensive carrier equipment is at the heart of our operations, but whenever possible we like to deploy open-source software or equipment developed by the hacker community. There was no NAT64 appliance that fell under this definition at that time and I think as of today there still is none. At least none that I am aware of. So I decided to do something about that and initiated the SIXFW project with a clear objective:
An easy-to-use, non-bloated firewall/NAT64 appliance that thinks IPv6 first.
In retrospect, this was a hell of a objective.
I gave a lightning talk about my idea and the day after a couple of interested people gathered providing opinions, tips and ideas. Up to this day, however, I never released anything that comes close to my goal. Here is how and why I failed:
Lesson 1: Scope!
The attentive reader may have seen the contradiction in our objective already. A contradiction I did not see back then, though. When we aim for ease of use, we can hardly address the professional market for NAT64 appliances. Commercial NAT64 appliances come with a lot of configuration options, there is a knob or switch for everything. They pose a frustratingly high barrier for network engineers who are new to NAT64 networking. Lowering that barrier was one of my goals. This meant less features, less options to chose from and less ability to adapt to the very network the appliance should serve. Eventually, turning the professional appliance into a consumer router with NAT64 capabilities. I wanted to develop an appliance for a complex technology (which has it’s nifty caveats) that could be used by the average grandmother. I did not scope my project properly. I did not have a clear definition of the target audience.
Lesson 2: Do your research!
I quickly settled for OpenBSD as the underlying operating system. Partly because I liked the stability of it, and partly because I believed that there are already enough Linux- or FreeBSD-based firewall distributions around. What I did not do was proper research. I should have asked myself which operating system really is the best for the problem, taking into account the ecosystem, packaging infrastructure, release schedules and security patching processes. If I had approached this topic more open-minded, I may or may not have chosen another operating system.
Lesson 3: Do not re-invent the wheel!
After a year of development, I began to realize that I would not have my prototype ready for the 2016 Chaos Communication Congress. Damn! Instead of panicking, I did something that often helps me to clear my head in messed-up situations: Stepping back and start dissecting the ashes.
It took me a couple of hours to clear my head and evaluate the code, the objective and the possibilities. Then I decided to try something different: Why not take an existing firewall distribution and turn it into a NAT64 appliance. Just for the fun of it, just to see what it would look like, just to learn from how else I could approach the problem. So I took a look at OpenWrt/LEDE and started adding Jool to make it run NAT64 in kernel mode (there were user space tools for that already, but they are not performing well). I then extended the existing Unbound DNS package to support DNS64 configuration options. Now there were NAT64 and DNS64 capabilities in a well-established distribution. And for the most common scenarios, they were even configurable via web interface! It was a great pleasure to see such success in such a short time after spending too much time and resources on developing a similar solution from scratch.
This happened close to the end of the year and it was about time to think about the event’s NAT64 network. Again, stepping back and trying another approach helped me clear my mind. I ended up writing a small Go program that would generate all necessary configuration files to turn an freshly installed OpenBSD into a NAT64 appliance. No web interface, to fancy bling-bling, just plain and reliable configuration files. It worked very well!
So, where will I go from here?
- I will continue to contribute to OpenWrt/LEDE. Especially in the areas of IPv6-only networking and NAT64. There is still a lot to be done in this field. I’d rather contribute to a successful project that actually helps people than swimming in my own soup alone. I see more potential for learning for all of us when IPv6-only networking is accessible to a broad range of interested people.
- Since NAT64 works surprisingly well and reliable on OpenBSD, I will keep my little Go program and generate updated configuration files every time I need to deploy an appliance-like server for NAT64. But no further energy will be invested into making it a fancy, shiny pet. It works and that is good enough in this case.
- I will be shutting down the SIXFW project, including it’s website, Twitter and GitHub accounts.
Thank your reading this and certainly thanks to all of you who supported the SIXFW project in one way or the other. I learned a lot and hope sharing my lessons learned helps someone else some day, too! What a ride!
Although I will not proceed with SIXFW but shutdown the project, I wanted to think about how I could have done it better. So I imagined myself being back in December 2015 and starting the project. This time by writing a project outline to explain the project, divide the work and conquer it.
The Internet suffers a severe sickness called IPv4 address exhaustion (see appendix for details). It took the Internet community a while to develop and apply the cure in form of a new Internet Protocol called IPv6. Today, the patient is stabilizing and IPv6 deployment rates have been growing more than linear. One problems remains, though: These two protocols are not designed to be interoperable. Once a network goes IPv6-only, connectivity to the old Internet is gone. All nodes in that network (end users, servers, everything) will be excluded from information that is not accessible via IPv6. The negative side effect of the emergency cure we just applied is the patient slowly loosing vision on one eye. As a believer in the free flow of information, I think this is a problem that needs to be solved.
The scope of this project is to build a NAT64/DNS64 translating network appliance. This involves choosing a suitable hardware platform alongside a mature, well-performing operating system. The software will use the platform efficiently. However, it will not be bound to a specific product or vendor and will provide compatibility with a large range of products from different price categories. For example, SIXFW is expected to run on a low-cost computer as well as on a high-performance enterprise server. The software will be designed using multiple components. A user interface will provide statistics and system health data as well as allow configuration of operational parameters. Using a dedicated control daemon, system tasks will be separated from the user interface to provide an additional layer of security. Proper access management and sane defaults (read: preconfigured for most applications) will ensure that initial functionality will provided with minimal administrative overhead.
The SIXFW appliance will be useful for anyone who uses or maintains a state-of-the-art network connected to the Internet. Although the vast majority of end users will not know that their information flow to the old Internet is being translated, they will benefit from the existence of an open-source ready-to-use NAT64/DNS64 appliance. More tech-savvy users may want to use SIXFW in their home networks alongside existing customer premise equipment (CPE, e.g. a Fritz!Box). Organizations may chose to prefer open-source solutions for their connectivity for security or budgetary reasons. As SIXFW matures, it may operate side by side with enterprise appliances at internet service provider level, transparently serving hundreds or thousands of people. SIXFW significantly lowers the barriers to access information that would otherwise be invisible. Everyone who regularly accesses wired or wireless networks and varying locations with different uplink capabilities is the target audience. If you use the Internet, you are part of this project’s target audience. However, I will do my best to avoid you noticing it, so that you can access the information you want without disturbance.
Reaching mile stone EVALHW depends on the availability of different hardware platforms. Supply shortage may lead to delays. This risk has been mitigated by an upfront investment in hardware. I have at least three platforms to evaluate, independent from market availability.
Mile stone ALPDEV consists the major development part of the project. Due to its highly integrated nature, single components are expected to be develop in parallel using an incremental approach. Further division of this mile stone into smaller tasks is intentionally avoided. This poses a risk on the overall project, as time estimates may be significantly off. Furthermore, unforeseen complexity issues or unexpected inter- dependencies could add additional workload.
The event, which will provide the environment for testing taking place to reach mile stone ALPTST, is in an early planing stage. I follow the progress of the planning committee closely to get early notice of relevant changes. For example, the event could be canceled due to funding issues or external threats like harsh weather conditions. I plan to mitigate this risk by choosing another test environment, e.g. an indoor conference, if necessary.
- Evaluate hardware platforms (3-4) and operating systems (2-3) for reference implementation
- Install different OS on different platforms
- Document experiences issues and caveats
- Document opportunities
- Evaluate how fit-for-purpose each platform is
- Investigate possible OS and security update workflows
- Assess NAT64/DNS64 installation routines
- Hardware platform evaluation result matrix
- Operating system evaluation result matrix
Develop reference implementation
- Install chosen operating system on chosen platform (clean install)
- Manually install and configure NAT64/DNS64 services
- Manually install and configure router services
- Develop sane default firewall rules (as a starting point for further improvements later)
- Run client connectivity tests with major mobile and workstation operating systems
- Reference implementation source code
- Full documentation on how reference implementation was build
SIXFW reference implementation field test
- Deploy and integrate reference implementation at 33C3 hacker conference network (I am member of the network operating center and will be responsible for transition technologies in the NAT64 part of the network)
- Document and fix bugs that show up during real-life deployment (~2 days)
- Document bugs, issues and collect anonymized usage data for later analysis during operational phase of the conference (~4 days)
- Short, summarizing field test report
- Updated source code of reference implementation
Develop alpha release
- Design solution architecture
- Proposed architecture contains: control daemon, user interface, statistics daemon, update functionality, user interface security and access management
- Source code of alpha release
SIXFW alpha release field test
- Deploy and integrate reference implementation at the next outdoor hacker event network (I am member of the network operating center and will be responsible for transition technologies in the NAT64 part of the network)
- Document and fix bugs that show up during real-life deployment (~2 days)
- Document bugs, issues and collect anonymized usage data for later analysis during operational phase of the event (~4 days)
- Short, summarizing field test report
- Updated source code of alpha release
Build supporting infrastructure
- Develop tools for release management
- Develop tools for building and signing of pre-configured ready-to-go releases
- Create initial documentation
- Design and update the sixfw.com website to reflect the project (current page content is misleading and not in line with the project goals)
- Updated website