It took us really long time to release Turris OS 7.0, but it is finally out now. The only change in this release is a switch to a newer OpenWrt – namely 22.03. We are not introducing any new features (although we are for the first time using “Staging Updates“). We are even sticking with iptables for now, although upstream has moved towards nftables. All that to minimize impact on our users and provide smooth update experience. That was also part of the reason why it took so long. We tested it over and over again, fixed the issues we found and started with the tests from scratch again. Sometimes we found our feature not working, sometimes we encountered some problems with state of the upstream distribution we had to help to fix. But finally we got into state where we could release it and we did exactly that.
Have you checked your router and realized that it is still running Turris OS 6? How is that possible? Didn’t I just wrote that we already released Turris OS 7.0? Well we released it, but we used a new feature that we implemented and that we call Staging Updates. And this is something I actually want to write a little bit more about.
First, let’s talk about what motivated this feature. We try to test every release as best as we can. On top of that, our nightly builds are publicly available for anybody to test. We also release public release candidates when we feel that everything is in good enough state. Our routers feature Btrfs snapshots available at your fingertips, so it is easy to give those options a try and it is risk free. At worse, you rollback and complain at our forum. And that is what we are after – customers feedback. It’s not that we are lazy to test it ourselves. But given the openness and versatility of our devices, the options are limitless. And therefore we can’t test everything. Even though we have a great community and quite some of our users are helping us test our release candidates, sometimes there is just too many changes. Staging Updates gives us another option how to deploy the release and lower the impact of any potentially overlooked bug.
How does it work? Simple. When we are releasing a new version, we decide how long will it take to roll it out to everybody. In case of Turris OS 7.0, we decided that 13 days sound like a good middle ground. How do we decide who gets the update? We use combination of routers serial number and release timestamp. We consider the seconds we release a particular release to be quite random. We also have no idea how serial numbers are distributed between our users. And based on those two numbers we divide people into groups and every day one of those groups installs the update, while others stay at the old release. One of the advantages is that this way, we are dividing routers for every release. There is no way to predict whether you get next update on the day of the release or whether you will be the last one to get it.
I wrote that we divide the routers, but the reality is actually a little trickier. We wrote the script, but the decision is done on every particular router. We don’t get a list of serial numbers and our server doesn’t decide who is getting updates first and who is second. We just provide our routers with release date and into how many days we want to spread the release. And the routers decide whether to install it or not.
As big part of routers migrated already, we can say that all the preparations paid off. Most support requests we are getting are about some deprecated packages no longer being available in newer OpenWrt. And we can now start focusing on migration to nftables and start preparing Turris OS 8.0.