WEBVTT 00:00:01.750 --> 00:00:06.690 Here's Rust in Production, a podcast about companies who use Rust to shape the 00:00:06.690 --> 00:00:07.510 future of infrastructure. 00:00:08.050 --> 00:00:12.750 My name is Matthias Endler from corrode and today we talk to John Seeger from 00:00:12.750 --> 00:00:16.030 Canonical about oxidizing Ubuntu with Rust. 00:00:19.250 --> 00:00:22.330 John, thanks so much for taking the time for the interview today. 00:00:22.670 --> 00:00:26.830 Can you quickly introduce yourself and Canonical, the company you work for? 00:00:27.690 --> 00:00:31.410 Of course. So my name is John Seeger. I'm the VP of Engineering for Ubuntu, 00:00:31.730 --> 00:00:33.230 which is a Linux operating system. 00:00:33.350 --> 00:00:37.410 And Canonical is the company which ships Ubuntu, as well as a host of other 00:00:37.410 --> 00:00:43.830 open source utilities for building data centers and cloud applications and various other tools. 00:00:44.510 --> 00:00:49.430 You took over the Ubuntu management in January, as far as I remember, 00:00:49.610 --> 00:00:52.110 right? What drew you into that position? 00:00:52.770 --> 00:00:57.210 So yeah, I did take over in January. I've actually been at Canonical for four and a half years. 00:00:57.430 --> 00:01:01.330 I'd spent a lot of time focusing on cloud orchestration, a tool called Juju, 00:01:01.510 --> 00:01:03.950 and some operators for things like databases. 00:01:04.470 --> 00:01:08.790 And at the tail end of last year, Mark Shuttleworth approached me and asked 00:01:08.790 --> 00:01:11.270 if I would think about taking over Ubuntu. 00:01:11.410 --> 00:01:13.690 We didn't have a VP who was kind of looking after Ubuntu. 00:01:14.190 --> 00:01:18.430 And he felt like there was some room for improvement in terms of the vision 00:01:18.430 --> 00:01:22.430 and what we were looking to achieve with Ubuntu and perhaps how the teams were 00:01:22.430 --> 00:01:27.150 working and what ultimately we wanted Ubuntu to be in the marketplace of Linux distributions. 00:01:27.410 --> 00:01:32.230 There's quite a spectrum from very exciting, new shiny things to very stable, 00:01:33.050 --> 00:01:35.090 less shiny things in the distribution market. 00:01:35.250 --> 00:01:39.230 And I think it was time for us to think about where we sat in that and what 00:01:39.230 --> 00:01:41.750 we wanted Ubuntu to be for the next 20 years. 00:01:41.910 --> 00:01:46.430 Last year was 20 years of Ubuntu but with that success i think it's important 00:01:46.430 --> 00:01:51.550 to have a point of introspection and decide you know where next we've had 20 00:01:51.550 --> 00:01:57.750 years of success and we've enjoyed that but what do we want ubuntu to be as we look forward yeah. 00:01:57.750 --> 00:02:03.190 I can't believe it's been 20 years already it feels like yesterday to me now, 00:02:04.080 --> 00:02:09.800 You've been making quite some headlines recently with a very bold move, 00:02:09.960 --> 00:02:15.580 at least for some people, which is that in the next version of Ubuntu, 00:02:15.820 --> 00:02:21.960 you will make a bold step towards adopting Rust in the core part of the distribution. 00:02:22.180 --> 00:02:24.300 Can you talk a little bit about that? 00:02:24.920 --> 00:02:30.380 Yes. So this was part of a set of kind of four initiatives or four themes that 00:02:30.380 --> 00:02:33.180 I started working towards with Ubuntu, one of which was modernization. 00:02:33.480 --> 00:02:37.600 And so the experiment that we're running at the moment is replacing GNU core 00:02:37.600 --> 00:02:42.800 utils with a Rust rewrite of the core utils project by an organization called 00:02:42.800 --> 00:02:46.660 UUtils and replacing sudo with a Rust rewrite called sudo-rs. 00:02:47.540 --> 00:02:52.860 And really, I think these projects are really impressive. I think they have 00:02:52.860 --> 00:02:54.200 great communities around them. 00:02:56.230 --> 00:03:00.910 I liked the level of enthusiasm I was seeing around these kind of core parts 00:03:00.910 --> 00:03:01.850 of a Linux distribution. 00:03:03.150 --> 00:03:07.430 And because of the kind of memory safety benefits that we get from Rust and 00:03:07.430 --> 00:03:10.650 the potential for there to be performance improvements, I thought they were 00:03:10.650 --> 00:03:12.050 an interesting place for us to start. 00:03:13.130 --> 00:03:17.230 And because they're so foundational to the experience that people have at the 00:03:17.230 --> 00:03:19.130 Linux command line, of course, it grabbed headlines. 00:03:19.390 --> 00:03:24.470 I think lots of people who use Linux for the first time and jump into the terminal, 00:03:24.650 --> 00:03:29.550 often the very first commands they type are from core utils, ls and rm and cp. 00:03:29.830 --> 00:03:33.610 And with sudo, my view is that's even more important because that's right at 00:03:33.610 --> 00:03:36.530 the security boundary between privileged and unprivileged users. 00:03:36.810 --> 00:03:41.690 And so for me, that one feels more critical from a security perspective, 00:03:41.690 --> 00:03:45.330 which is often what our paying customers are looking for from Ubuntu, right? 00:03:45.390 --> 00:03:49.570 It's security and resilience in the operating system that they ship for whatever their use case is. 00:03:50.210 --> 00:03:56.550 Yeah. Now a critical person might say wait but you already had a battle-tested 00:03:56.550 --> 00:04:02.230 set of tools which were developed for 30 plus years why would you rewrite them 00:04:02.230 --> 00:04:06.970 wouldn't you introduce more problems than you would solve what would you reply to that. 00:04:07.710 --> 00:04:11.590 My reply to that is, that is true of every technological advancement in the 00:04:11.590 --> 00:04:12.990 history of humankind, right? 00:04:13.110 --> 00:04:17.070 Like, why do we need internal combustion engines when horses were great? 00:04:17.630 --> 00:04:20.390 They were great. And internal combustion engines have been useful to us, 00:04:20.490 --> 00:04:22.830 and whatever comes next will be useful to us again. 00:04:23.090 --> 00:04:27.890 I think, I certainly haven't done this to throw any shade on the GNU-Core Utils 00:04:27.890 --> 00:04:30.590 project, or indeed the original sudo project, right? 00:04:30.670 --> 00:04:34.430 And one of the things that I liked about both of these projects was the attitude 00:04:34.430 --> 00:04:37.230 they had towards the original implementations. 00:04:37.650 --> 00:04:42.050 So in the process of rewriting core utils, which started out as a hobby project 00:04:42.050 --> 00:04:46.270 for somebody to experiment with Rust, they uncovered bugs in GNU core utils, 00:04:46.370 --> 00:04:49.730 which they fired issues for and worked with the maintainers on to correct. 00:04:49.990 --> 00:04:53.830 And similarly in sudo, the sudo-rs maintainers are in really regular contact 00:04:53.830 --> 00:04:55.690 with the original sudo maintainer. 00:04:55.870 --> 00:04:56.870 They have uncovered bugs in 00:04:56.870 --> 00:04:59.490 their implementation, which turned out to be present in the original too. 00:04:59.690 --> 00:05:01.870 And so it is a bit of a collaborative effort, right? 00:05:02.530 --> 00:05:08.410 They are both a sort of modern interpretation of a set of well-known tools and utilities. 00:05:08.750 --> 00:05:14.210 I think one of the bits of context for Ubuntu is part of our success has been 00:05:14.210 --> 00:05:15.430 in offering long-term support. 00:05:15.710 --> 00:05:17.930 The idea of an LTS was a thing that. 00:05:18.940 --> 00:05:21.900 Only came about because of ubuntu right and so 00:05:21.900 --> 00:05:27.300 when we ship a tool or we ship a utility we ship some code we will stand by 00:05:27.300 --> 00:05:31.360 that for 12 and in some cases 15 years and so for me it's really important that 00:05:31.360 --> 00:05:35.680 we pick tools and utilities that have got momentum and community support and 00:05:35.680 --> 00:05:40.600 are written in languages which we can continue to employ talent to maintain and look after, 00:05:41.360 --> 00:05:46.220 and i think increasingly rust is gaining adoption in these kind of system level 00:05:46.220 --> 00:05:50.020 utilities people are interested in writing Rust, they're interested in learning about it. 00:05:50.200 --> 00:05:55.520 And it's a way of us encouraging talent, but also encouraging contribution to Ubuntu. 00:05:55.740 --> 00:06:01.180 I think with open source, the way it's gone, it's been in many ways so very successful. 00:06:01.320 --> 00:06:04.780 There are so many things that one could contribute to in open source these days. 00:06:05.340 --> 00:06:09.320 And I would like to focus very much on building the next generation of contributors 00:06:09.320 --> 00:06:12.300 to Linux, not just the applications that run on top of Linux, 00:06:12.480 --> 00:06:15.180 which is maybe taken for granted by some, I suppose. 00:06:16.120 --> 00:06:23.100 I guess you mentioned a very important point, which is the maintenance of the tools we all depend on. 00:06:23.940 --> 00:06:29.440 I don't know how many installations the core utils have, but it must be in the billions. 00:06:30.000 --> 00:06:36.240 And those tools run also billions of times a day on all of the machines combined, I guess. 00:06:36.420 --> 00:06:40.340 But the main point is, this is a thing that people take for granted. 00:06:40.540 --> 00:06:47.260 And it's also, to some extent at least, a maintenance burden because of C, 00:06:47.460 --> 00:06:50.180 because of the way it was written and so on. 00:06:50.520 --> 00:06:56.260 Can you elaborate on this part as well? the part of the project that helps people 00:06:56.260 --> 00:07:01.800 transition from C to Rust and people maybe preferring to work in Rust code bases. 00:07:02.060 --> 00:07:05.120 Yes, and this has been called out a couple of times in commentary about this 00:07:05.120 --> 00:07:06.780 move in Ubuntu, where I've referred 00:07:06.780 --> 00:07:10.660 to, you know, as replacing 30-year-old code, like that's a bad thing. 00:07:10.800 --> 00:07:14.200 I have no, I have nothing against the GNU Core Utils code base. 00:07:14.280 --> 00:07:15.700 I suspect it's very good quality, right? 00:07:15.840 --> 00:07:19.000 It's been in production a long time. It's been finely tuned. 00:07:19.000 --> 00:07:22.380 But the fact remains that the 00:07:22.380 --> 00:07:25.360 single biggest vector for attack on modern 00:07:25.360 --> 00:07:28.120 systems in or certainly in the last few years 00:07:28.120 --> 00:07:31.420 has been through memory safety right it has been through where someone 00:07:31.420 --> 00:07:34.320 has missed an edge case or failed to validate something 00:07:34.320 --> 00:07:39.760 or you know missed a check for a null pointer and these whether we like it or 00:07:39.760 --> 00:07:42.720 not whether we want to admit it this is where the vast majority of security 00:07:42.720 --> 00:07:47.180 vulnerabilities come from the opportunity with something like rust core utils 00:07:47.180 --> 00:07:52.880 is to have a generation of software where that is significantly less likely. 00:07:53.080 --> 00:07:56.220 Not impossible, but significantly less likely. And as you say, 00:07:56.400 --> 00:07:58.880 billions of invocations per day, potentially, right? 00:07:58.980 --> 00:08:04.700 In scripts, as part of other software, in provisioning, in day-to-day use by end users. 00:08:04.820 --> 00:08:10.480 If all of those invocations were of utilities that fundamentally cannot suffer 00:08:10.480 --> 00:08:15.760 from the same issues and have the potential for higher performance and an engaged 00:08:15.760 --> 00:08:18.100 community that's going to look after them for the next few years, 00:08:18.540 --> 00:08:23.520 I struggle to see a disadvantage in that. It doesn't have to be. 00:08:24.660 --> 00:08:28.700 In spite of the original project, right? Like there will even be use cases where 00:08:28.700 --> 00:08:31.740 we probably still use utilities from GNU core utils for whatever reason. 00:08:31.780 --> 00:08:34.880 And I think that's okay. The two don't have to be fighting, right? They can coexist. 00:08:35.640 --> 00:08:37.140 And I think they will for a very long time. 00:08:38.640 --> 00:08:44.640 And the other misconception that a lot of people have is that the GNU core utils are basically done. 00:08:44.800 --> 00:08:47.860 They are finished. There are no more changes that need to be made. 00:08:47.980 --> 00:08:49.580 But in reality, that's also not true. 00:08:50.620 --> 00:08:54.280 Even just this week, there were multiple commits to 00:08:54.280 --> 00:08:57.720 the repository and the other 00:08:57.720 --> 00:09:00.880 part is vulnerabilities also still get discovered 00:09:00.880 --> 00:09:04.220 in those tools i just checked yesterday and 00:09:04.220 --> 00:09:07.800 there was a cve in sort which 00:09:07.800 --> 00:09:10.800 allowed you to if you controlled the input 00:09:10.800 --> 00:09:13.920 have access to 00:09:13.920 --> 00:09:17.420 memory that you shouldn't have access to because of 00:09:17.420 --> 00:09:20.420 you know i guess 00:09:20.420 --> 00:09:27.720 just a mishandling of the input or you know an exception so it's it's probably 00:09:27.720 --> 00:09:32.960 more than just language evangelism that that you propose here right it's there's 00:09:32.960 --> 00:09:37.580 a genuine technical necessity to move to safer languages it's. 00:09:37.580 --> 00:09:41.780 Actually not language evangelism at all in the sense that i have a few rust 00:09:41.780 --> 00:09:45.320 projects but it's single digit numbers of rust projects right like i'm not a, 00:09:45.800 --> 00:09:49.360 I'm not a Rust person, if you will. I know Rust. I like the language. 00:09:49.560 --> 00:09:51.960 I've used it for a few projects. I think it is interesting. 00:09:52.360 --> 00:09:57.360 What is almost more interesting to me is the set of principles that the Rust 00:09:57.360 --> 00:10:01.040 community seem to rally around when it comes to safety and performance. 00:10:01.970 --> 00:10:04.910 Now, that doesn't mean all Rust software is better than all C software. 00:10:05.150 --> 00:10:08.270 It doesn't mean that everything that was written in C in Ubuntu will get replaced 00:10:08.270 --> 00:10:09.630 with something that has been written in Rust. 00:10:09.850 --> 00:10:12.910 But where there are compelling alternatives and engaged communities, 00:10:13.390 --> 00:10:14.650 they're on the table, right? 00:10:15.570 --> 00:10:18.770 Like I say, Ubuntu has been around for 20 years, and I hope it will be around for another 20. 00:10:18.950 --> 00:10:23.310 And it probably shouldn't look exactly like it does today in 20 years. 00:10:23.450 --> 00:10:28.870 Again, that would be a failing, I think, for our open source community, for our customers. 00:10:28.970 --> 00:10:32.030 It needs to continue to evolve and i think this is just a part of that. 00:10:32.030 --> 00:10:38.910 As someone who also sees the operational sides of things of maintaining a full 00:10:38.910 --> 00:10:44.090 distribution and also maybe taking some of that risk to make sure that you can 00:10:44.090 --> 00:10:50.010 ship on time and so on was adding rust to the toolchain a huge operational burden. 00:10:50.870 --> 00:10:56.650 So when i begun this effort we already had rust the rust toolchain in ubuntu 00:10:56.650 --> 00:10:59.010 right like so you You could already get the Rust compiler. 00:10:59.230 --> 00:11:02.530 You could always already get tools like RustUp. So like we already had, 00:11:02.790 --> 00:11:07.070 because of the fact that we're downstream from Debian and us and the Debian 00:11:07.070 --> 00:11:09.650 community have been working on how to package Rust software. 00:11:10.410 --> 00:11:12.150 This was a relatively easy transition. 00:11:12.310 --> 00:11:17.110 In fact, Rust core utils and sudo-rs were already packages that existed in Ubuntu's archive. 00:11:17.330 --> 00:11:21.450 And so this is really a policy change, which says that we're going to bring 00:11:21.450 --> 00:11:22.570 that into what we call main, 00:11:22.830 --> 00:11:26.810 which is the part of the archive which has the longest and kind of the most 00:11:26.810 --> 00:11:30.810 stringent set of security promises from us, if you will, we're going to bring 00:11:30.810 --> 00:11:33.110 those packages into main and we're going to ship them by default. 00:11:33.840 --> 00:11:38.740 The old ones will continue to be available. And in fact, one of the more complicated pieces of work, 00:11:38.900 --> 00:11:43.380 one of the more genuinely complex things that we had to do here was some work 00:11:43.380 --> 00:11:47.760 on apt itself to make sure that the path to switch back to the old implementation 00:11:47.760 --> 00:11:48.940 was as simple as possible. 00:11:49.260 --> 00:11:52.940 This is not about us forcing this upon everybody at all costs. 00:11:53.080 --> 00:11:55.840 The GNU core utils will continue to be available on Ubuntu. 00:11:56.180 --> 00:12:00.960 And we have done actual engineering work in apt to make sure that going back is one command. 00:12:01.300 --> 00:12:04.120 And we'll publish that in the release notes. Same with sudo, right? sudo was 00:12:04.120 --> 00:12:06.840 a little easier the reason we had to do the work in apt is because if 00:12:06.840 --> 00:12:09.920 by the time apt got halfway through the process 00:12:09.920 --> 00:12:12.920 of swapping them it removes the new version of core utils suddenly 00:12:12.920 --> 00:12:15.780 all the utilities it needed to install the new one weren't there anymore 00:12:15.780 --> 00:12:21.180 so we had to do some work in terms of like sequencing that so yes i mean my 00:12:21.180 --> 00:12:26.900 view is it you know for 90% 99% of our users this will be a no-op they won't notice 00:12:26.900 --> 00:12:31.600 it but it makes it easier for us to maintain and it lowers the likelihood of 00:12:31.600 --> 00:12:33.620 issues like the one you just described in sort. 00:12:34.760 --> 00:12:37.000 Which don't come around often, but they do still come around. 00:12:37.120 --> 00:12:38.540 That's evidence that it does still happen. 00:12:39.360 --> 00:12:43.560 Do you get a lot of beta testers for the distributions usually? 00:12:43.980 --> 00:12:49.220 Do people test it stuff, or are you still sanding down the edges on the Rust integration? 00:12:50.320 --> 00:12:53.940 Yeah, it's difficult to know exact numbers, but we definitely do get pretty good engagement. 00:12:54.140 --> 00:12:57.740 This change actually coincided with another change I made here on to this year, 00:12:57.940 --> 00:13:00.200 which was to do monthly snapshot releases. 00:13:00.980 --> 00:13:03.840 It's a slightly tangential subject, but part of my 00:13:03.840 --> 00:13:06.780 push towards kind of modernizing ubuntu was to streamline our 00:13:06.780 --> 00:13:09.560 release process and modernize that a little bit and i 00:13:09.560 --> 00:13:12.420 challenged our release team to stop just doing it every six months 00:13:12.420 --> 00:13:15.300 when we actually wanted to release but to basically exercise that process every 00:13:15.300 --> 00:13:18.220 month and put out snapshot releases and as a side effect 00:13:18.220 --> 00:13:21.260 of that we've had ubuntu isos out that have 00:13:21.260 --> 00:13:25.300 had the rust tools for for months right like incrementally more and more of 00:13:25.300 --> 00:13:29.420 the GNU core utils were replaced with the rusty ones and so now in the beta 00:13:29.420 --> 00:13:34.000 we're looking like the vast majority will there are still a couple of incompatibilities 00:13:34.000 --> 00:13:37.660 so we you know some of the utilities will still be the new ones but it means 00:13:37.660 --> 00:13:41.220 that people have had better access to it i don't know exact numbers but we certainly see, 00:13:41.920 --> 00:13:45.680 response to discourse posts and blogs saying that people are trying it out and 00:13:45.680 --> 00:13:49.260 we've also seen a significant uptick in bugs far against the packages as people 00:13:49.260 --> 00:13:53.140 have been testing it and finding things which has been really helpful to us and. 00:13:53.140 --> 00:13:58.720 On a technical level when you want to switch tool by tool you just symlink that 00:13:58.720 --> 00:14:01.740 and you make it the default in someone's shell right. 00:14:01.740 --> 00:14:06.700 Yeah essentially it is just it is just a symlink i can actually if it's interesting 00:14:06.700 --> 00:14:08.940 for the show notes or something there's a link where you can actually see the 00:14:08.940 --> 00:14:12.140 set of links that we have right now where we're pointing to gnu tools still 00:14:12.140 --> 00:14:15.340 it's going to shrink in the next couple of days because the folks at just cut 00:14:15.340 --> 00:14:19.020 a new release which solves a bunch of issues that we had so yeah it's just a 00:14:19.020 --> 00:14:19.860 set of index essentially. 00:14:20.690 --> 00:14:26.110 Did you catch any surprises? Because in my understanding, 00:14:26.110 --> 00:14:30.890 that would probably be one of the hardest projects to work on because you have 00:14:30.890 --> 00:14:38.030 so many undocumented features of the old tools that people start to depend on in their workflows. 00:14:38.410 --> 00:14:43.870 But now you kind of change it from underneath them and you need to make it look 00:14:43.870 --> 00:14:46.810 as if it was the old thing or if it behaved exactly like the old thing, 00:14:46.930 --> 00:14:52.670 at least in the first version, right? Did you find any nasty surprises in that transition period? 00:14:53.010 --> 00:14:57.070 So I think we should talk about core utils and sudo-rs differently because I 00:14:57.070 --> 00:15:00.230 think it's kind of interesting. The two projects actually approach this quite differently. 00:15:00.470 --> 00:15:07.110 So the uutils project aims to be bug-for-bug compatible with the original core utils. 00:15:07.330 --> 00:15:11.610 If core utils does a thing, uutils aims to do it identically, essentially. 00:15:12.110 --> 00:15:13.810 That's actually not the case in sudo-rs. 00:15:14.750 --> 00:15:18.330 But if we focus on core utils still, there were a couple of things that just 00:15:18.330 --> 00:15:20.130 simply weren't implemented when we started. 00:15:20.850 --> 00:15:24.830 So an example of this was most of the tools didn't have support for the dash 00:15:24.830 --> 00:15:30.770 capital Z, capital Z for your American listeners, for essentially manipulating 00:15:30.770 --> 00:15:33.390 viewing SC Linux context information for files. 00:15:33.550 --> 00:15:36.310 That functionality just hadn't been implemented. And that was one of the things 00:15:36.310 --> 00:15:40.450 that we sponsored the utils developers to implement before we shipped it. 00:15:40.590 --> 00:15:45.770 Now, we don't actually use SELinux by default on Ubuntu, but there are people 00:15:45.770 --> 00:15:49.210 in the community and people in our customer base who presumably do use SELinux 00:15:49.210 --> 00:15:52.370 with Ubuntu, and so we wanted to make sure that was present before we shipped it. 00:15:52.830 --> 00:15:57.870 Likewise, localization, internationalization, there was no infrastructure for translations. 00:15:58.470 --> 00:16:02.310 So, sort worked great in English and not so well in German or in French. 00:16:02.650 --> 00:16:06.450 And so, part of the work that we sponsored was to essentially build infrastructure 00:16:06.450 --> 00:16:08.350 into the project to allow localizations. 00:16:09.580 --> 00:16:14.580 The story with Sudo is a little bit different because they don't aim to be 100% bug for bug compatible. 00:16:14.900 --> 00:16:19.620 They aim to be mostly compatible with a view that some of the things in the 00:16:19.620 --> 00:16:25.620 original Sudo were implemented well and in good faith, but that the ideas didn't age so well. 00:16:25.940 --> 00:16:29.120 Like the idea about what the Sudo tool should do and how it should behave, 00:16:29.400 --> 00:16:33.120 perhaps if we were starting again in 2025, we'd either do differently or we 00:16:33.120 --> 00:16:35.280 just wouldn't do in the Sudo tool at all. 00:16:35.640 --> 00:16:38.500 And so there aren't many of these features 00:16:38.500 --> 00:16:41.360 but an example would be in the original 00:16:41.360 --> 00:16:44.740 sudo you can pass it a dash capital e and 00:16:44.740 --> 00:16:47.940 that will take your entire like environment variable context 00:16:47.940 --> 00:16:50.920 from your unprivileged environment into 00:16:50.920 --> 00:16:53.620 the privileged environment and they chose not to 00:16:53.620 --> 00:16:56.380 implement that on the grounds that they feel like it is safer that 00:16:56.380 --> 00:16:59.280 people are conscious about which parts of the environment they're pushing 00:16:59.280 --> 00:17:04.960 into that privileged space and so people who have the dash capital e and their 00:17:04.960 --> 00:17:08.300 scripts will find that that doesn't work which is why you know so either they 00:17:08.300 --> 00:17:11.000 can choose to say okay yep maybe i should be a bit more careful i need to take 00:17:11.000 --> 00:17:14.360 this variable and that variable or they can on their installations they can 00:17:14.360 --> 00:17:16.420 switch it back to the old version and carry on as they were. 00:17:16.420 --> 00:17:23.320 Wow do they get some exit code which tells them that this was the issue like 00:17:23.320 --> 00:17:28.560 passing the entire environment into privileged context sounds like a massive 00:17:28.560 --> 00:17:31.780 food I'm not sure why they even added that in the first place, 00:17:31.900 --> 00:17:36.820 but do they get some error message, which tells them how to do the right thing? 00:17:37.720 --> 00:17:42.040 Well, they will get an error message from sudo-rs saying, I don't know what that switch is. 00:17:42.900 --> 00:17:47.040 So we haven't put code into sudo-rs to say, hey, we don't know what that switch 00:17:47.040 --> 00:17:49.240 is because you're using a nice shiny new version. Yeah. 00:17:50.620 --> 00:17:55.260 In that sense, it is a little bit opaque, but part of the effort has been updating 00:17:55.260 --> 00:17:58.400 our own documentation to try and make sure this information is captured. 00:17:58.520 --> 00:18:01.160 And when people search, it will come up in the Ubuntu docs to say, 00:18:01.300 --> 00:18:02.440 you know, we've made this change. 00:18:02.600 --> 00:18:05.940 This is how you can get back, or this is, you know, the alternatives. 00:18:07.180 --> 00:18:12.340 So, and, you know, unusually, mostly when Ubuntu releases a new feature, 00:18:12.440 --> 00:18:14.560 we just release a new feature and write about it in the release notes. 00:18:14.560 --> 00:18:18.840 As a as a kind of exception here we are actually going to write we've done this 00:18:18.840 --> 00:18:21.400 thing and here's how you can revert it if you if you care. 00:18:21.400 --> 00:18:25.760 Yeah um well if 00:18:25.760 --> 00:18:29.960 you have a tool so central and i'm talking about both the core utils and sudo-rs 00:18:29.960 --> 00:18:34.900 but if you have tools like this which are so central to your even the outputs 00:18:34.900 --> 00:18:41.440 become part of the api the interface yeah because people do various things with 00:18:41.440 --> 00:18:43.320 that output, which they shouldn't probably. 00:18:43.500 --> 00:18:46.020 They grab for certain keywords and so on. 00:18:46.420 --> 00:18:53.720 For example, as a German user, I know that some of these tools also have locale-dependent output. 00:18:53.940 --> 00:18:58.240 They might print something in German, whereas on a different machine they might 00:18:58.240 --> 00:18:59.580 print something in English, for example. 00:19:00.740 --> 00:19:04.200 How much attention do you have to pay to these details? 00:19:05.120 --> 00:19:12.680 And were there any things that, for example, were not solved in the Rust ecosystem specifically to, 00:19:13.810 --> 00:19:15.790 you know, handle all of these use cases. 00:19:16.270 --> 00:19:20.010 So I think internationalization was the big one there. So because the uutils 00:19:20.170 --> 00:19:23.750 project is aiming for like a bug for bug compatibility, theoretically, 00:19:23.890 --> 00:19:24.990 the output should be identical. 00:19:25.390 --> 00:19:29.130 And if it's not, it's a bug and they'll fix it. Right. So we haven't really run into that. 00:19:29.370 --> 00:19:34.690 I don't think there is, there was an issue with the date tool where it failed 00:19:34.690 --> 00:19:37.430 to parse a certain kind of date time or it failed. 00:19:37.550 --> 00:19:41.490 I forget the exact issue, but either the way it outputted it or the way it interpreted 00:19:41.490 --> 00:19:45.670 a date was different from the GNU tool, and that caused us a huge headache in 00:19:45.670 --> 00:19:46.430 our build infrastructure. 00:19:46.650 --> 00:19:49.870 And so for a while, we were using the GNU date tool still. I believe that's 00:19:49.870 --> 00:19:51.810 now fixed. So we have seen issues like that. 00:19:52.410 --> 00:19:56.490 It's also worth mentioning why now in Ubuntu. 00:19:56.850 --> 00:20:00.170 So lots of people will know about Ubuntu and they know about the LTS, 00:20:00.530 --> 00:20:02.270 but it's worth talking about our release model. 00:20:02.450 --> 00:20:05.670 So every two years in April, we release a new LTS. 00:20:06.050 --> 00:20:09.390 So 20.04, 22.04, 24.04, 26.04. 00:20:09.770 --> 00:20:14.070 And that is where a huge majority of our user base lies. 00:20:14.290 --> 00:20:17.930 Lots of people will stay on an LTS until the next LTS comes around, then an update. 00:20:18.930 --> 00:20:22.450 But then every six months, we still do interim releases, what we call interim releases. 00:20:22.670 --> 00:20:28.370 So these are 25.04, 25.10, 26.10, 27.04, et cetera. 00:20:28.870 --> 00:20:33.750 And we see dramatically less users here, and particularly in the enterprise 00:20:33.750 --> 00:20:36.650 space where people aren't generally running production workloads on these. 00:20:37.290 --> 00:20:41.270 And so these interim releases are the opportunity for us to test this out. 00:20:41.410 --> 00:20:45.130 We will get thousands, tens of thousands, maybe hundreds of thousands of users. 00:20:46.440 --> 00:20:49.780 And we still want those to be very high quality. We want them to be stable, 00:20:49.780 --> 00:20:52.240 but it is our opportunity to try new things. 00:20:52.620 --> 00:20:57.340 So an example of this is, I think, back in 17.10, I want to say, 00:20:57.480 --> 00:21:00.800 either 17.10 or 17.04, long before I joined Canonical. 00:21:00.960 --> 00:21:04.560 I seem to recall they tried switching to Wayland by default. 00:21:04.980 --> 00:21:07.960 And in the next interim release, they were like, that didn't go very well. We're going to go back. 00:21:08.240 --> 00:21:11.000 Right? I think we're now back on Wayland by default. But the point is, 00:21:11.140 --> 00:21:15.020 that was an example of where we used the interim release to try something that 00:21:15.020 --> 00:21:17.960 we thought was going to work. It didn't work out that well, so we switched it back. 00:21:18.580 --> 00:21:22.940 And I am really committed to getting this change across the line in the LTS, 00:21:23.200 --> 00:21:26.480 but not to the point of destruction, if you see what I mean. 00:21:26.760 --> 00:21:32.820 If we roll 25.10 out the door and this change doesn't go well or it causes huge 00:21:32.820 --> 00:21:34.880 issues, then we'll have to roll it back. 00:21:35.160 --> 00:21:38.100 It doesn't mean we'll roll back forever, and it might be that we roll back some 00:21:38.100 --> 00:21:39.420 and not all of the utilities, 00:21:40.680 --> 00:21:43.420 but you know this is just the balance this is the trade-off right like we have 00:21:43.420 --> 00:21:47.360 to work out whether or not the thing is ready for 26.04 which is our next lts 00:21:47.360 --> 00:21:53.700 so we still have seven months eight months to get ready for that lts and 25.10 00:21:53.700 --> 00:21:56.840 is kind of a milestone on the way to that and it gives people it gives us a 00:21:56.840 --> 00:22:02.020 way to get new ideas into the hands of lots of people to see whether it's ready for the lts now. 00:22:02.020 --> 00:22:05.180 You probably have a lot of integration tests anyway nowadays. 00:22:05.180 --> 00:22:06.000 Yeah 00:22:06.000 --> 00:22:06.600 for these tools, 00:22:08.780 --> 00:22:11.940 Certainly packages themselves are tested with a thing called auto package test 00:22:11.940 --> 00:22:16.160 which we inherit from debian so most all of the packages in main have to have 00:22:16.160 --> 00:22:18.100 auto package test tests to be in main, 00:22:18.640 --> 00:22:22.220 and actually that's where we've inflicted some pain on ourselves like where 00:22:22.220 --> 00:22:24.980 there have been differences in the utilities we've seen them in our build infrastructure 00:22:24.980 --> 00:22:28.320 because suddenly thousands of packages won't build or there'll be an error you 00:22:28.320 --> 00:22:31.400 know the test will fail and so that's been a useful tool for us for kind of 00:22:31.400 --> 00:22:32.280 identifying issues 00:22:32.280 --> 00:22:33.460 When people 00:22:33.460 --> 00:22:36.940 usually propose rust it's for 00:22:36.940 --> 00:22:40.740 a few reasons the main one is improved security 00:22:40.740 --> 00:22:44.340 but another one is better performance now 00:22:44.340 --> 00:22:47.800 in your case you're competing with c which is already quite performant 00:22:47.800 --> 00:22:50.400 you might even say that's a bit 00:22:50.400 --> 00:22:53.280 of a regression here because there are still statements 00:22:53.280 --> 00:22:58.240 out there that rust cannot be as fast as c and that very much depends on your 00:22:58.240 --> 00:23:03.660 use case but what would you say to that would you say well we take a performance 00:23:03.660 --> 00:23:09.520 hit yeah we accept it or is there even a possibility for the tools to be as 00:23:09.520 --> 00:23:11.520 fast or even faster than the older version. 00:23:11.520 --> 00:23:16.080 I think in the mid to long term there's an opportunity for those tools to be 00:23:16.080 --> 00:23:20.460 faster and for a couple of reasons there was an article that went round recently 00:23:20.460 --> 00:23:23.920 which showed that you know there were some performance shortcomings uncovered 00:23:23.920 --> 00:23:27.420 in this i think the example was the checksum tool i actually found a regression 00:23:27.420 --> 00:23:30.920 in while we were engineering sprint in the WC, 00:23:31.200 --> 00:23:33.300 the word count utility that turned out to be slower. 00:23:33.620 --> 00:23:38.480 I was attempting to do the 1 billion rows challenge with WC and I found that it was slower. 00:23:39.180 --> 00:23:42.820 But the interesting thing to me is, that's not surprising. 00:23:43.280 --> 00:23:47.620 The software is newer, it's less mature than the GNU stuff. But what is interesting 00:23:47.620 --> 00:23:51.160 to me is how quickly these issues get resolved, how quickly people step up to find ways. 00:23:51.640 --> 00:23:55.220 And when they do step up, they often exceed the performance of the original tools. 00:23:55.380 --> 00:23:59.960 So the issue that was filed about checksum being 17 times slower than the GNU 00:23:59.960 --> 00:24:02.420 one was filed by a canonical engineer who works on our, 00:24:03.400 --> 00:24:07.020 in our foundations team, who was going out looking for problems to make sure 00:24:07.020 --> 00:24:09.840 that we ship it in the best shape we possibly can. That issue is now solved. 00:24:10.180 --> 00:24:13.340 The checksum tool in new utils, I believe, now outperforms the original. 00:24:13.540 --> 00:24:17.420 And the same with the word count utility. That was, I think it was under a day. 00:24:17.540 --> 00:24:22.500 And it went from being 1.1 times slower to 1.3 times faster. 00:24:23.080 --> 00:24:27.400 And I think on your point about C versus Rust, can Rust be faster? 00:24:28.560 --> 00:24:33.060 I buy the argument that in some cases there's overhead and, you know, 00:24:33.160 --> 00:24:36.920 a very, very optimized C program and a very, very optimized Rust program, 00:24:37.640 --> 00:24:39.740 the likelihood is that you could do better with C. 00:24:39.920 --> 00:24:42.100 I buy that argument. Okay. I get it. 00:24:42.940 --> 00:24:48.140 That said, things like threading from a developer experience perspective are 00:24:48.140 --> 00:24:51.840 significantly friendlier in Rust and it's significantly more likely to be correct. 00:24:52.100 --> 00:24:56.700 So, you know, very, very talented C developers with a real eye for performance, 00:24:56.860 --> 00:25:01.020 I'm sure they can squeeze the absolute maximum performance out of C. 00:25:01.500 --> 00:25:04.840 The question is, how many engineers around the world can do it? 00:25:05.080 --> 00:25:10.260 And I think in Rust, there's a likelihood that we will see a higher degree or 00:25:10.260 --> 00:25:15.600 a larger number of faster implementations, because the language makes some of 00:25:15.600 --> 00:25:17.780 the more advanced primitives a little bit more easy to hold, 00:25:17.940 --> 00:25:20.080 a little bit more easy to reason about, a little easier to debug, 00:25:20.220 --> 00:25:23.160 a little easier to test. So it is a trade-off, ultimately. 00:25:24.160 --> 00:25:27.800 Well, two things that I wanted to say here. one 00:25:27.800 --> 00:25:31.340 is that multi-threading is one the 00:25:31.340 --> 00:25:34.480 other one especially when it comes to for example 00:25:34.480 --> 00:25:40.940 the work count thing wc is simd support right correct yeah i think simd support 00:25:40.940 --> 00:25:47.260 in rust is pretty decent by now so you could even dare to try and make that 00:25:47.260 --> 00:25:51.500 use those cpu features which i'm not 100 sure, 00:25:52.980 --> 00:25:56.700 GNU core utils do i i honestly doubt it to be honest. 00:25:56.700 --> 00:25:59.440 I yeah i don't think they do and you're absolutely right 00:25:59.440 --> 00:26:03.960 so the the speed up that we got on the workout utility i believe was assembly 00:26:03.960 --> 00:26:07.460 implementation i think it's the same for the checksum utility and so you know 00:26:07.460 --> 00:26:10.200 a counter argument there might be well what if i'm running on a processor that 00:26:10.200 --> 00:26:14.240 doesn't have those features and the answer is okay might be a bit slower um 00:26:14.240 --> 00:26:18.400 you can't literally have everything do you know 00:26:18.560 --> 00:26:27.100 We can't win on binary size and all-out performance and security and maintainability 00:26:27.100 --> 00:26:29.180 and community all at the same time, right? 00:26:29.280 --> 00:26:31.960 It's very, well, maybe there are projects that have achieved that. 00:26:32.020 --> 00:26:32.980 It's very difficult to do. 00:26:33.200 --> 00:26:36.720 And so part of this initiative is trying to raise the odds of raising the bar 00:26:36.720 --> 00:26:38.240 in each of those as much as we can. 00:26:38.880 --> 00:26:43.160 Do you compile those tools for different architectures with different CPU features? 00:26:44.280 --> 00:26:49.860 Yes. So we support, at the moment, we support AMD64, we support ARM, 00:26:50.020 --> 00:26:54.740 ARMHF, S390, PowerPC64, and RISC-V. 00:26:55.100 --> 00:26:59.760 And the deal we have in Canonical with Ubuntu is if we support an architecture, 00:26:59.820 --> 00:27:00.700 we support an architecture. 00:27:00.960 --> 00:27:04.860 We don't do cross-compilation. We buy actual hardware with those architectures. 00:27:04.940 --> 00:27:09.300 So we have actual mainframes in our data centers, like S390 machines, 00:27:09.480 --> 00:27:12.560 on which we compile packages for the archive and we test packages for the archive. 00:27:12.560 --> 00:27:15.300 We also have landed some 00:27:15.300 --> 00:27:18.060 work in launchpad and in the archive that will 00:27:18.060 --> 00:27:21.480 allow us to deliver binaries that take advantage of 00:27:21.480 --> 00:27:24.320 micro architectural variants so you know 00:27:24.320 --> 00:27:27.200 if you're on a machine that has amd64v3 support 00:27:27.200 --> 00:27:29.960 you might be able to get access to a binary that's been 00:27:29.960 --> 00:27:34.200 compiled to take advantage of amd64v3 features where that makes sense and the 00:27:34.200 --> 00:27:39.960 same would be true of amd64v5 you've got armv8 armv9 armv10 and obviously i 00:27:39.960 --> 00:27:43.900 think in the risk ecosystem we're seeing kind of an explosion of this as the 00:27:43.900 --> 00:27:48.060 architecture or the instruction set is in its infancy, people are building processes, 00:27:48.830 --> 00:27:53.190 with wildly different capabilities and so we've had to make sure that our kind 00:27:53.190 --> 00:27:57.170 of build infrastructure has the capability to deal with that that's landed relatively 00:27:57.170 --> 00:27:59.710 recently there's a blog post coming in a couple weeks about it. 00:28:00.350 --> 00:28:03.990 And not many people know that some 00:28:03.990 --> 00:28:06.850 of these tools especially the legacy ones they highly 00:28:06.850 --> 00:28:09.770 underuse modern cpus there are 00:28:09.770 --> 00:28:15.090 certain things in there which are just not really done and they could could 00:28:15.090 --> 00:28:19.330 be done but it's not an easy task and it's a bit risky like wouldn't that also 00:28:19.330 --> 00:28:25.430 be an opportunity to leverage some of those functionalities that have been in 00:28:25.430 --> 00:28:29.070 cpus for a decade now or longer right yeah. 00:28:29.070 --> 00:28:32.530 And again it's a trade-off between compatibility and performance right one of 00:28:32.530 --> 00:28:36.750 the nice things about gnu core utils is they it works the same in lots of places 00:28:36.750 --> 00:28:42.250 that perhaps will not be said of the uutils project where they have kind of 00:28:42.250 --> 00:28:45.250 conditional implementations based on CPU features, 00:28:45.430 --> 00:28:50.530 but where the vast majority of a huge data center, for example, 00:28:50.590 --> 00:28:55.650 in a cloud has AMD64v3 and has these CINDY features and stuff, 00:28:55.810 --> 00:28:58.950 surely we want to take advantage of those as much as we possibly can, 00:28:59.110 --> 00:29:01.410 not just from the performance perspective, 00:29:01.630 --> 00:29:06.670 but on the other side of the same coin is power usage and how much energy is 00:29:06.670 --> 00:29:08.190 consumed by doing a task. 00:29:08.410 --> 00:29:11.510 And often using these implementations sees a huge advantage there too. 00:29:12.090 --> 00:29:19.930 Yeah yeah my first thought was well it's a user space tool and probably not 00:29:19.930 --> 00:29:25.430 really that energy hungry but if you call it billions of times a day then it 00:29:25.430 --> 00:29:27.570 adds up very quickly right, 00:29:28.790 --> 00:29:34.870 and there's another misconception about core utils which i hear a lot which is, 00:29:36.560 --> 00:29:39.660 They are optimized already to a certain extent. 00:29:39.840 --> 00:29:43.940 And I guess that's my second point is that people think they can't really go 00:29:43.940 --> 00:29:45.440 in and improve those tools. 00:29:45.640 --> 00:29:50.760 But in reality, it's not the case. For example, I once wrote a faster version 00:29:50.760 --> 00:29:56.860 of cat just to see where the bottleneck was and if I could make it as fast as the new CoriTools one. 00:29:58.200 --> 00:30:04.780 And mine was just three times as fast. The reason was not that the code was specifically clever. 00:30:04.780 --> 00:30:07.740 It was just using a new feature in the linux kernel 00:30:07.740 --> 00:30:11.060 which is the splice feature and you can avoid 00:30:11.060 --> 00:30:17.220 one copy from kernel space to user space with that now did i dare to send in 00:30:17.220 --> 00:30:21.500 a patch for that functionality to be added to new corridors no of course not 00:30:21.500 --> 00:30:26.260 because i honestly i'm not a c developer and i kind of was afraid of the process 00:30:26.260 --> 00:30:30.380 some of these things they are not accessible to, 00:30:30.960 --> 00:30:35.880 everyday normal developers they have different processes and so on so i hope 00:30:35.880 --> 00:30:41.820 that also contributing to the uutils might be easier in the future yeah. 00:30:41.820 --> 00:30:47.320 I think so people have various opinions about code forges github microsoft licenses 00:30:47.320 --> 00:30:52.880 it's all a trade-off at the end of the day and i think you know whatever microsoft's 00:30:52.880 --> 00:30:58.140 motivations which you know so far don't seem too impure to me There's millions 00:30:58.140 --> 00:31:00.380 of developers on GitHub. That's a huge audience. 00:31:00.880 --> 00:31:05.060 And so bringing the development of something so foundational as the core utils 00:31:05.060 --> 00:31:08.820 into a space where there exist millions of developers who know the contribution 00:31:08.820 --> 00:31:12.420 process and how to open pull requests and write comments and interact with people, 00:31:13.640 --> 00:31:17.340 again, I find that hard to kind of put across as a bad thing. 00:31:18.210 --> 00:31:22.810 You said that sudo-rs has a slightly different focus than core utils. 00:31:23.170 --> 00:31:24.950 Can you elaborate on that a bit? 00:31:25.390 --> 00:31:29.610 I guess sudo was the first thing you will integrate. 00:31:30.090 --> 00:31:34.770 And why start with this? Wouldn't it be easier to start with a completely new 00:31:34.770 --> 00:31:40.090 tool instead of rewriting something so foundational in the core of the distribution? 00:31:40.830 --> 00:31:45.630 So I think it's precisely because, I mean, say, we didn't write sudo-rs, 00:31:45.630 --> 00:31:47.710 Canonical didn't rewrite sudo. 00:31:48.090 --> 00:31:54.270 This rewrite existed. It is authored by a non-profit called the Trifector Tech 00:31:54.270 --> 00:31:59.210 Foundation, and they are an organization who aims essentially to provide resilient 00:31:59.210 --> 00:32:02.450 software for public good, if you will, like infrastructure-level software. 00:32:03.590 --> 00:32:08.030 And I think, honestly, if there was going to be anywhere on the system where 00:32:08.030 --> 00:32:12.450 I would even trade performance for guarantees, it would be in the sudo tool. 00:32:13.330 --> 00:32:16.050 There are alternate approaches to privilege escalation on Linux. 00:32:16.250 --> 00:32:20.010 There's a new approach in Systemd called run0, which has been popular recently 00:32:20.010 --> 00:32:20.910 in some other distributions. 00:32:21.710 --> 00:32:25.390 I'm sure it's very good. The Systemd project has written some great software. 00:32:25.870 --> 00:32:28.070 But the sudo-RS project gives 00:32:28.070 --> 00:32:32.370 us a 90-something percent compatibility guarantee with memory safety. 00:32:32.810 --> 00:32:36.550 And to make it even more fun, they were absolutely delightful to work with. 00:32:36.650 --> 00:32:39.710 They were very excited to have their software in Ubuntu. They worked with us. 00:32:39.830 --> 00:32:40.850 They They were really professional. 00:32:41.430 --> 00:32:45.950 They worked with our security team. You know, they work with the OG kind of sudo developer. 00:32:46.230 --> 00:32:50.470 And so where you have this intersection of really motivated, 00:32:50.750 --> 00:32:55.270 really professional people and a programming language that kind of enforces 00:32:55.270 --> 00:32:59.910 certain ways of thinking and working for safer software, it just seemed like a natural fit. 00:33:00.170 --> 00:33:03.970 The security boundary is such a sensitive part of the operating system that 00:33:03.970 --> 00:33:06.230 it felt like a kind of natural place to start. 00:33:07.250 --> 00:33:15.130 Now, sudo-rs has 42,000 lines of code, and to some that might seem like very 00:33:15.130 --> 00:33:17.870 little code, to some that might seem like a lot of code. 00:33:18.490 --> 00:33:23.310 What does it do exactly that requires that many lines of code? 00:33:23.670 --> 00:33:27.950 So there's a few things, really. There is the fundamentals of kind of escalating 00:33:27.950 --> 00:33:28.910 privilege of our process. 00:33:29.030 --> 00:33:32.690 There's all the configuration, parsing, different options that have evolved over time. 00:33:34.950 --> 00:33:39.490 One of the features that we asked to be added was AppArmor support like the 00:33:39.490 --> 00:33:43.210 ability for sudo and AppArmor to interact which is the LSM that we use, 00:33:43.270 --> 00:33:44.210 the Linux security module, 00:33:45.610 --> 00:33:50.170 they need to be able to talk to PAM for example and SSSD for things like LDAP 00:33:50.170 --> 00:33:53.490 authentication so that is another difference between the original sudo which 00:33:53.490 --> 00:34:00.470 can natively speak LDAP the folks behind sudo-rs decided that that wasn't something 00:34:00.470 --> 00:34:03.890 they wanted their tool to do and they'd rather hand that off to to SSSD, essentially. 00:34:05.290 --> 00:34:09.510 And so, I mean, there's a lot of features, right? Again, localization support 00:34:09.510 --> 00:34:12.890 is something that, you know, it doesn't come for free. It comes with code overhead. 00:34:13.990 --> 00:34:19.250 So it's not only sudo-rs that you add. It's also an entire chain of dependencies 00:34:19.250 --> 00:34:22.230 in an ecosystem of Rust grades. 00:34:23.050 --> 00:34:26.070 Right, and things like sudo-edit, so you can do the kind of visudo thing, 00:34:26.210 --> 00:34:30.790 right? Edit, like, it has its kind of own secure editor for editing the permissions. 00:34:30.990 --> 00:34:35.630 It is, yeah, it is a whole ecosystem. And the folks there have been diligent 00:34:35.630 --> 00:34:38.930 in the dependencies that they pull in, and they're very careful about it for obvious reasons. 00:34:39.070 --> 00:34:42.530 But yes, it is a whole ecosystem, right? It's a fairly big project. 00:34:43.030 --> 00:34:47.790 As a Rust consultant, I'm super happy to hear that because it adds support to 00:34:47.790 --> 00:34:49.270 the entire Rust ecosystem. 00:34:49.450 --> 00:34:54.050 But from your perspective, I would wonder, am I exposing myself to a lot of 00:34:54.050 --> 00:34:58.010 additional risk because a lot of that code hasn't been... 00:34:59.280 --> 00:35:05.180 You know part of such a long-term project as ubuntu for such a long time and 00:35:05.180 --> 00:35:10.760 maybe hasn't seen a lot of production the reality of production let's say. 00:35:10.760 --> 00:35:13.580 Yeah it is a risk like you're absolutely right 00:35:13.580 --> 00:35:18.480 it is a risk in fact of all of the challenges in the rust community i would 00:35:18.480 --> 00:35:24.560 say one of the largest is in creating an ecosystem of crates that we can all 00:35:24.560 --> 00:35:30.820 have trust in it's easy it's easy for people to poke fun at NPM, you know, left pad. 00:35:31.020 --> 00:35:34.100 And there was some recent things with the kind of color libraries, 00:35:34.380 --> 00:35:37.160 you know, crates.rs isn't that much different at the end of the day. 00:35:37.260 --> 00:35:38.340 Anyone can upload code there. 00:35:38.600 --> 00:35:46.100 And so what I would look for in projects like sudo-rs is a diligence in how that ecosystem is used. 00:35:46.400 --> 00:35:49.780 Like, do you pull in an entire dependency that's maintained by somebody else that you don't know? 00:35:49.900 --> 00:35:53.380 Or do you vendor some of the code, you know, following the licensing guidelines 00:35:53.380 --> 00:35:55.440 and bring it in and maintain it yourself, right? 00:35:55.460 --> 00:35:58.380 These are all software engineering decisions at the end of the day that you 00:35:58.380 --> 00:36:01.220 wait, you trade off being able to move fast and being able to move very safely. 00:36:02.900 --> 00:36:08.900 So it is a risk, but I mean, isn't any change to the world's most deployed Linux 00:36:08.900 --> 00:36:11.140 operating system a risk at the end of the day? 00:36:11.300 --> 00:36:14.600 Like we just have to balance that out. And again, that's why it's landed in 25.10 first. 00:36:17.000 --> 00:36:19.780 I suspect there will be people out there who can't 00:36:19.780 --> 00:36:22.900 wait for this to land to see if they can go and find a security vulnerability in sudo-RS 00:36:22.900 --> 00:36:26.680 that doesn't exist in the original and i suspect they will probably succeed 00:36:26.680 --> 00:36:31.200 right like i i don't believe that it is possible to write software that is perfect 00:36:31.200 --> 00:36:37.600 all the time what i'm gambling on is the long-term probability of that thing 00:36:37.600 --> 00:36:40.420 being safer in the round yeah. 00:36:40.420 --> 00:36:45.540 And even if they find a security vulnerability my assumption would be that it 00:36:45.540 --> 00:36:52.340 would make a lot of rust code safer because unless they use unsafe blocks everywhere. 00:36:52.700 --> 00:36:55.660 And then I haven't checked how many unsafe lines of code are in there. 00:36:55.880 --> 00:36:58.220 Maybe you know, that would be interesting to know. 00:36:58.280 --> 00:36:59.040 Not at the top of my head. 00:36:59.120 --> 00:37:00.640 No. We can double check. 00:37:01.630 --> 00:37:07.210 Unless they use a lot of unsafe code, if vulnerability gets discovered, 00:37:07.530 --> 00:37:13.410 that means that's a compiler bug and that would be fixed probably immediately. 00:37:13.770 --> 00:37:19.250 And then a lot of other tools would not be exposed to that security vulnerability anymore. 00:37:19.610 --> 00:37:26.650 Yeah, I think this is part of what I see as Canonical's responsibility to the tech industry. 00:37:27.470 --> 00:37:32.910 The tech industry really rallied to Ubuntu 20 years ago, and we have enjoyed success from it. 00:37:33.270 --> 00:37:38.750 We have lots of commercial customers, we have enough revenue to fund 800 engineers 00:37:38.750 --> 00:37:41.750 on good salaries all around the world to work on open source and Linux, 00:37:41.930 --> 00:37:43.810 and that's a position of huge privilege. 00:37:44.510 --> 00:37:48.530 But I think part of our giving back to that community, in a sense, 00:37:48.730 --> 00:37:53.910 is to use the exposure that we have got to give products or projects like this 00:37:53.910 --> 00:37:58.070 a bit of exposure, to give them the benefit of all of those eyes and all that 00:37:58.070 --> 00:37:59.370 testing to progress them. 00:37:59.550 --> 00:38:02.810 I think, you know, so long as we do that in a careful and kind of respectful 00:38:02.810 --> 00:38:07.190 way, that's just us evolving the OS and using our position to... 00:38:08.400 --> 00:38:12.520 Try and put some of these projects on a platform where we think that they you 00:38:12.520 --> 00:38:15.700 know their views and their approach aligns with ours other. 00:38:15.700 --> 00:38:21.000 Than sudo and core utils what what else are you working on that is written in rust. 00:38:21.000 --> 00:38:27.940 Yeah so we considering ntpd rs which would be a time-syncing daemon written 00:38:27.940 --> 00:38:32.100 in rust as also a trifecta tech foundation what is that exactly. 00:38:32.100 --> 00:38:33.660 Can you elaborate on that a bit. 00:38:34.020 --> 00:38:40.060 So if you think NTPD or crony or systemd timesyncd, things that do time syncing 00:38:40.060 --> 00:38:44.160 for the system clock, essentially, we are looking at ntpd-rs, 00:38:44.300 --> 00:38:46.600 which is a Rust implementation of that demon, essentially. 00:38:47.600 --> 00:38:51.060 I'm hoping we can try and do that for 26.04, but at this point, 00:38:51.160 --> 00:38:52.580 it's very much a hopeful, not a given. 00:38:52.920 --> 00:38:56.420 If not, we'll be trying that out in 26.10 to see how it plays. 00:38:56.860 --> 00:39:01.020 And this goes quite nicely hand in hand with a recent move that we made, 00:39:01.160 --> 00:39:05.820 which is to enable NTS by default, which is the secure version of that protocol for time syncing. 00:39:06.040 --> 00:39:09.420 So think HTTP versus HTTPS, similar sort of transition. 00:39:10.300 --> 00:39:16.160 We're also looking at switching the OpenPGP library that Apt uses to Sequoia, 00:39:16.380 --> 00:39:18.800 which is Rust implementation of OpenPGP. 00:39:18.980 --> 00:39:21.980 So that would be for package signature verification kind of thing. 00:39:24.020 --> 00:39:28.860 Various other utilities in the distro, we're doing some thinking about certificate 00:39:28.860 --> 00:39:31.340 revocation lists and how to handle those in the distribution. 00:39:31.340 --> 00:39:33.460 And I think there'll be some Rust involved there. 00:39:34.740 --> 00:39:37.440 Our own compositor Mir has some Rust code in it. 00:39:38.120 --> 00:39:41.180 Mir isn't shipped by default in Ubuntu at the moment but it is shipped in a bunch of places. 00:39:41.540 --> 00:39:46.580 I think Fedora's LXQt spin uses Mir as a compositor and there's some Rust in there. 00:39:47.340 --> 00:39:53.440 We have some work ongoing with real-time operating systems where some Rust exists. We have a, 00:39:54.370 --> 00:39:58.890 cloud streaming platform called unbox cloud inside which there is some rust 00:39:58.890 --> 00:40:01.870 so there's a few places right like we we're using it in a bunch of different projects. 00:40:01.870 --> 00:40:04.770 It's always cool to hear that 00:40:04.770 --> 00:40:07.690 because it's not the main 00:40:07.690 --> 00:40:10.790 selling point of all of these tools and applications 00:40:10.790 --> 00:40:13.550 it's just that no one 00:40:13.550 --> 00:40:16.290 kind of forced you to use rust for these tools you 00:40:16.290 --> 00:40:23.590 could have used anything else but you saw value in using rust specifically and 00:40:23.590 --> 00:40:30.650 for a lot of companies now rust is no longer just hype thing that i want to 00:40:30.650 --> 00:40:34.850 try it's just normal part of everyday infrastructure i. 00:40:34.850 --> 00:40:38.170 Would agree with it and and for a bunch of different reasons i mean i think 00:40:38.170 --> 00:40:43.070 we were quite tentative in our adoption of rust at canonical we you know we 00:40:43.070 --> 00:40:46.050 haven't been on the bandwagon jumping up and down about this for years, 00:40:46.270 --> 00:40:50.050 we're quite deliberate about the languages that we choose. 00:40:50.210 --> 00:40:55.830 So we try not to have one team in Go, one in Erlang, one in OCaml, one in JavaScript. 00:40:56.150 --> 00:40:59.890 We don't have this huge proliferation of languages. It's generally Go, 00:41:00.190 --> 00:41:03.030 Python, C, and now we're adding Rust to that. 00:41:04.130 --> 00:41:07.710 And I just sort of believe in using the right tool for the job at the end of the day. 00:41:07.770 --> 00:41:10.470 If we were doing something that involved lots of concurrency, 00:41:10.470 --> 00:41:14.150 and not of asynchronous networking, I would probably choose Go at the moment. 00:41:14.370 --> 00:41:18.370 But if we're writing low-level systems utilities that are performance-critical 00:41:18.370 --> 00:41:20.050 or safety-critical, then I would choose Rust. 00:41:22.010 --> 00:41:26.030 What was the first language you mentioned? We don't use that here. 00:41:27.230 --> 00:41:29.090 I think you said Rust twice, right? 00:41:29.730 --> 00:41:30.150 Oh, did I? 00:41:31.650 --> 00:41:36.910 I'm just kidding. No-Go is a very fine choice for networking services. 00:41:37.150 --> 00:41:40.490 We have a lot of Go at Canonical, right? Juju is a huge Go code base. 00:41:41.350 --> 00:41:45.750 We have things like Pebble. It's creeping into Maz. We have a lot of Go. 00:41:46.010 --> 00:41:47.790 And it's served us very well, I think. 00:41:49.050 --> 00:41:53.610 I do just believe in tools for the job. There are some things in the Rust ecosystem 00:41:53.610 --> 00:41:57.490 that are still relatively early that Go really excels at. 00:41:57.730 --> 00:42:01.350 And there are some things that Go can never do well. It's a garbage-collected 00:42:01.350 --> 00:42:02.230 language at the end of the day. 00:42:02.510 --> 00:42:06.210 And so in places like Mir, for example, you don't want a garbage collector kicking 00:42:06.210 --> 00:42:07.850 in when you're in the middle of drawing frames on the screen. 00:42:08.610 --> 00:42:10.530 And so it would naturally not be a fit. 00:42:11.250 --> 00:42:16.990 Another thing we haven't talked about yet is a project called Anbox that you 00:42:16.990 --> 00:42:19.530 mentioned in passing. What's that about? 00:42:20.350 --> 00:42:25.170 So Anbox started out as a project for running Android applications on Linux. 00:42:25.330 --> 00:42:27.810 So it's like an Android emulator type thing. 00:42:27.990 --> 00:42:31.550 It was led by a chat called Simon Fels, who we hired some years ago, 00:42:31.630 --> 00:42:34.050 and he has now been building Anbox at Canonical. 00:42:34.450 --> 00:42:38.010 And with that, we have built a commercial product called Anbox Cloud, 00:42:38.350 --> 00:42:42.610 which is essentially sits on top of some of our other products like LXD, 00:42:42.810 --> 00:42:44.530 the kind of hypervisor that we maintain. 00:42:45.150 --> 00:42:48.690 And it is used for streaming android applications 00:42:48.690 --> 00:42:51.350 over the cloud so one of the biggest use cases for this 00:42:51.350 --> 00:42:54.170 is there's a major mobile device manufacturer who uses this 00:42:54.170 --> 00:42:57.390 for game streaming to their devices so where the graphical 00:42:57.390 --> 00:43:00.170 requirements of the game is much higher 00:43:00.170 --> 00:43:04.190 than you could ever get out of a mobile device at the moment we can stream the 00:43:04.190 --> 00:43:06.850 video at very low latency and very high quality to the 00:43:06.850 --> 00:43:09.590 devices and get the kind of input stream from the device in terms 00:43:09.590 --> 00:43:12.570 of controls back and you're playing a game as if natively on 00:43:12.570 --> 00:43:15.230 the device but you're actually streaming it from the cloud but it 00:43:15.230 --> 00:43:19.150 has other applications in kind of VDI infrastructure and 00:43:19.150 --> 00:43:22.630 one of the other big use cases we see is in 00:43:22.630 --> 00:43:28.370 like automotive development so you think about someone who sat at a car manufacturer 00:43:28.370 --> 00:43:33.650 working on an android auto dashboard you know like in-car entertainment system 00:43:33.650 --> 00:43:39.390 they can use anbox to run anbox and do local development against android whilst 00:43:39.390 --> 00:43:42.270 they're sat on their workstations, right? Maybe that's streamed from the cloud, maybe it's local. 00:43:43.430 --> 00:43:46.510 And so there are some parts of Anbox which are moving to Rust. 00:43:47.690 --> 00:43:51.070 Perhaps were written in C or other languages before and some combination of 00:43:51.070 --> 00:43:54.570 performance and kind of ease of development have led them down that path. 00:43:54.830 --> 00:43:59.770 Earlier you said that you don't really impose Rust on anyone so it's a bit of 00:43:59.770 --> 00:44:04.070 a team decision and there was one project specifically, 00:44:05.710 --> 00:44:12.470 where people transitioned from Rust from another language which is dqlite. 00:44:12.850 --> 00:44:18.630 Can you elaborate a little bit on what that is and what the language was that they used before. 00:44:18.630 --> 00:44:21.610 Yeah so this is this is ongoing so there isn't a 00:44:21.610 --> 00:44:24.490 rust dqlite yet in general 00:44:24.490 --> 00:44:30.330 we are encouraging heavily our teams who have large c++ or c code bases to consider 00:44:30.330 --> 00:44:35.150 looking at rust whether that's building a feature in rust you know bringing 00:44:35.150 --> 00:44:39.490 some library functionality in rust we are we are you know asking them to go 00:44:39.490 --> 00:44:42.070 and familiarize themselves with that ecosystem and think about it. 00:44:43.150 --> 00:44:46.170 dqlite is essentially distributed to sqlite. 00:44:46.450 --> 00:44:50.510 So you can think of it as a thin layer of kind of the raft consensus protocol 00:44:50.510 --> 00:44:54.450 over the top of sqlite, which allows you to do high availabilitysqlite 00:44:56.390 --> 00:45:00.710 And the bit that kind of manages the VFS layer syncing is written in C. 00:45:01.430 --> 00:45:05.150 And we have been doing some work on dqlite. We're embedding it into Juju. 00:45:05.410 --> 00:45:07.930 It's in LXD. It's in a lot of our products. 00:45:08.990 --> 00:45:11.970 And one of the considerations as we're trying to think about 00:45:11.970 --> 00:45:14.850 how to improve the performance of dqlite and the reliability of 00:45:14.850 --> 00:45:19.490 dqlite is do we consider rewriting some or all of it in Rust and so there have 00:45:19.490 --> 00:45:23.790 been some kind of early prototyping efforts around that and one again one of 00:45:23.790 --> 00:45:26.870 the kind of interesting aspects there was the threading model in Rust and whether 00:45:26.870 --> 00:45:29.890 that would allow us a slightly easier development for some of the kind of more 00:45:29.890 --> 00:45:33.950 thorny concurrency elements and so not a hundred percent, 00:45:34.880 --> 00:45:37.580 there's a reasonable chance that we would see that happening over the next year or so. 00:45:38.120 --> 00:45:43.560 Are there any parts at Canonical that you wanted to mention that use Rust that 00:45:43.560 --> 00:45:44.680 we haven't touched on yet? 00:45:45.420 --> 00:45:48.760 There are a number of projects that we haven't touched on. I know of projects 00:45:48.760 --> 00:45:52.560 in our hardware certifications team. I know of Rust products, 00:45:52.700 --> 00:45:54.660 again, in real-time operating systems projects. 00:45:55.380 --> 00:45:58.300 I don't claim to know all of the projects using Rust in Canonical. 00:45:58.420 --> 00:46:00.320 They seem to be increasing, which I'm pleased about. 00:46:00.620 --> 00:46:04.020 There's also the Rust for Linux project, right? so Rust in the Linux kernel, 00:46:04.380 --> 00:46:08.660 we employ lots of kernel engineers, tens of kernel engineers in every time zone. 00:46:09.260 --> 00:46:13.380 And so naturally, we have to make sure that that team is up to speed on what's 00:46:13.380 --> 00:46:17.520 going on in the Rust for Linux project, because ultimately, we support kernels for a really long time. 00:46:17.740 --> 00:46:21.980 And so we will be, for better or for worse, supporting kernels that have Rust 00:46:21.980 --> 00:46:26.220 kernel modules and drivers for at least the next 12 years, probably longer. 00:46:26.700 --> 00:46:30.240 So that's another area where I think it will grow over the next couple of years. 00:46:31.100 --> 00:46:39.620 Yeah. And that involves work like a Nova driver for NVIDIA GPU support and various 00:46:39.620 --> 00:46:42.680 other bits and pieces that are written in Rust now in the Linux kernel. 00:46:43.100 --> 00:46:48.580 Even the Apple Silicon video drivers, we have Ubuntu Asahi as a kind of spin 00:46:48.580 --> 00:46:51.120 of Ubuntu, which is, again, somebody who works at Canonical, 00:46:51.240 --> 00:46:55.240 and my understanding is much of the graphics drivers for that hardware is written in Rust, so... 00:46:56.820 --> 00:46:58.900 Even all of it, as far as I'm aware. 00:46:59.020 --> 00:46:59.500 Yeah, I think so. 00:47:00.140 --> 00:47:03.520 Which is a pretty crazy project in and of itself. 00:47:04.060 --> 00:47:09.240 Being able to run a fully free operating system on Apple Silicon is kind of cool. 00:47:09.480 --> 00:47:10.680 Yeah, it's a remarkable project. 00:47:11.960 --> 00:47:20.120 Now, strategically, moving away from the technical side, as someone who is on 00:47:20.120 --> 00:47:22.260 the siding end of things, 00:47:23.160 --> 00:47:29.760 and you are responsible for 12 million, ubuntu desktop users what are things 00:47:29.760 --> 00:47:35.440 that are important for rust adoption to grow in companies like canonical. 00:47:35.440 --> 00:47:38.200 Yeah so the the numbers game is always 00:47:38.200 --> 00:47:41.040 a fun one we we have various ways of kind of collecting 00:47:41.040 --> 00:47:44.160 information about our users you know some basic telemetry 00:47:44.160 --> 00:47:47.540 you can read about it it's stuff it's mostly opt-in and very easy 00:47:47.540 --> 00:47:50.960 to work around the last numbers i heard was something along 00:47:50.960 --> 00:47:54.540 the lines of like we know of 12 million daily ubuntu 00:47:54.540 --> 00:47:57.920 desktop devices we also know that there's a whole bunch that we don't know of 00:47:57.920 --> 00:48:00.700 because they're behind corporate firewalls and proxies and things and there's 00:48:00.700 --> 00:48:03.920 a bunch of server like goodness knows how many instances of that are containers 00:48:03.920 --> 00:48:07.760 you know base images for containers that are scheduled around the world on kubernetes 00:48:07.760 --> 00:48:12.620 so like we don't really know how many ubuntu instances there are but lots of, 00:48:13.270 --> 00:48:16.270 definitely tens if not hundreds of millions, I would suggest. 00:48:17.370 --> 00:48:22.650 The decision-making process is probably not as complicated as it seems in the 00:48:22.650 --> 00:48:27.970 sense that the number one thing we have to maintain is that Ubuntu stays a trusted 00:48:27.970 --> 00:48:30.490 and reliable and resilient operating system. 00:48:31.210 --> 00:48:35.450 It's become known as the Linux that everyone can use, right? 00:48:35.730 --> 00:48:39.550 Linux for human beings, that was the whole thing. And I don't want to jeopardize that. 00:48:39.730 --> 00:48:44.590 As excited as I am about things like NixOS and other fun Linux distributions, 00:48:45.030 --> 00:48:47.890 it would be a failing for us to turn Ubuntu into that. 00:48:47.990 --> 00:48:51.130 That's not what we are, right? There's a space for those more experimental and 00:48:51.130 --> 00:48:54.610 kind of exotic distributions, but we have to build Linux for human beings, 00:48:54.690 --> 00:48:55.470 Linux for everyone, right? 00:48:55.550 --> 00:48:58.370 For students, for entrepreneurs, for huge businesses. 00:48:58.830 --> 00:49:02.930 And so my view is I'm always interested, I will always entertain suggestions 00:49:02.930 --> 00:49:08.270 for software that could be replaced, so long as it goes hand in hand with our 00:49:08.270 --> 00:49:10.290 vision for providing a resilient, 00:49:10.570 --> 00:49:15.710 performant, open source Linux operating system that gives people access to what 00:49:15.710 --> 00:49:18.830 we consider to be the very best of open source at that moment. 00:49:19.790 --> 00:49:23.770 To me, that's what it's really all about. It's about bringing open source to 00:49:23.770 --> 00:49:25.110 as many people as we possibly can. 00:49:25.310 --> 00:49:28.450 And not just open source, but the best open source that we can find and can support. 00:49:30.120 --> 00:49:36.940 Do you interact with the Rust Foundation? And what about the future of sponsoring 00:49:36.940 --> 00:49:40.100 that work? You mentioned Trifecta, for example. 00:49:40.520 --> 00:49:45.140 How do you see that collaboration evolving? 00:49:45.480 --> 00:49:50.140 I hope it will continue. Honestly, we don't have an open ticket to just write 00:49:50.140 --> 00:49:52.940 sponsorship for any Rust project on the planet, right? 00:49:53.060 --> 00:49:58.980 But certainly, it's not fair for us to suddenly impose a set of expectations 00:49:58.980 --> 00:50:03.500 on a community or a maintainer without first having a discussion and think about 00:50:03.500 --> 00:50:04.420 funding and helping them. 00:50:04.600 --> 00:50:08.560 So before we announced publicly that we were going to do core utils or sudo-rs, 00:50:08.860 --> 00:50:11.240 we spoke with the maintainers and said, hey, this is an idea we've had. 00:50:11.500 --> 00:50:14.860 Do you think the project is ready for it? And we need these things to be fixed. 00:50:14.900 --> 00:50:16.000 And what would that cost us? 00:50:16.340 --> 00:50:18.800 I think that's a reasonable conversation to have. 00:50:19.420 --> 00:50:24.080 And I intend to continue that. Again, this is us, in a sense, paying it forward. 00:50:24.280 --> 00:50:27.640 We've enjoyed great success from Ubuntu. We're in a position to be able to help 00:50:27.640 --> 00:50:31.520 other projects that we see as promising to progress and gain adoption. 00:50:31.860 --> 00:50:35.860 And that can't be free, right? It can't always be free. So, of course, 00:50:35.980 --> 00:50:37.260 in some cases, we will need to pay. 00:50:37.740 --> 00:50:40.840 I'm having a conversation, which maybe you'll hear about in a few weeks, 00:50:41.000 --> 00:50:44.680 with another maintainer this week about another project that we might fund because 00:50:44.680 --> 00:50:46.180 we're very interested in what they want to achieve. 00:50:47.100 --> 00:50:49.260 So, yeah, I think we will continue. 00:50:49.840 --> 00:50:53.720 And when will you rewrite Apt in Rust? Just kidding. 00:50:54.900 --> 00:50:57.140 I don't know that that's on the cars just yet, but I mean, I, 00:50:57.140 --> 00:51:03.140 you know, I certainly, I speak frequently with the maintainer of act and, you know, it's not, 00:51:03.980 --> 00:51:07.240 it's not never going to happen. It doesn't have to never happen, if you see what I mean. 00:51:07.340 --> 00:51:09.180 Like, I wouldn't be against it, but there would need to be a reason, 00:51:09.320 --> 00:51:11.800 right? Like, perhaps if we were going to introduce some major new functionality 00:51:11.800 --> 00:51:15.120 where it would be a good fit, then perhaps we could begin introducing Rust. 00:51:16.300 --> 00:51:21.220 Yeah, yeah. And I guess a bit more seriously, where do you see Rust in the Linux 00:51:21.220 --> 00:51:22.720 ecosystem in five years? 00:51:24.040 --> 00:51:28.060 Honestly, I hope what we see is a sort of continuation of where we have started. 00:51:28.220 --> 00:51:32.440 I am excited by things like graphics drivers in Rust and safety critical code 00:51:32.440 --> 00:51:35.300 in Rust. I hope that things have settled down a little bit. 00:51:35.700 --> 00:51:38.080 Rust in Lilix has not been without its controversies. 00:51:38.640 --> 00:51:40.040 And I think that's natural. 00:51:41.640 --> 00:51:46.460 One of the observations I have about the kernel community is it doesn't seem 00:51:46.460 --> 00:51:48.620 to be attracting lots and lots of new talent. 00:51:49.260 --> 00:51:54.720 And I think rust could be an important way in which the kernel could attract new maintainers. 00:51:55.100 --> 00:52:00.640 So I hope that as many of them already do, the kernel community continues to 00:52:00.640 --> 00:52:04.880 recognize while there may be some trade-offs, people may have to learn new languages 00:52:04.880 --> 00:52:08.300 and adopt a new way of doing things. Perhaps build pipelines become more complicated. 00:52:08.480 --> 00:52:11.560 There's also a set of opportunities associated with landing Rust code in the kernel. 00:52:11.780 --> 00:52:15.240 So my guess is as that becomes more developed and more mainstream, 00:52:15.580 --> 00:52:18.180 the disconnect there will settle down a little bit. 00:52:18.260 --> 00:52:21.600 And I hope that that happens sooner rather than later so we can carry on with 00:52:21.600 --> 00:52:25.840 building great software and worry less about arguing about the logistics of 00:52:25.840 --> 00:52:28.480 building great software ultimately i. 00:52:28.480 --> 00:52:33.140 Would like to get back to a point that you mentioned earlier which keeps 00:52:33.140 --> 00:52:38.920 coming back to me which is crates.io is also just a registry and everyone can 00:52:38.920 --> 00:52:46.460 push code there and for someone who has such large requirements on long-term 00:52:46.460 --> 00:52:50.440 maintenance isn't that a huge risk of supply chain security, 00:52:51.360 --> 00:52:55.460 You end up trusting so many dependencies and sub-dependencies and so on. 00:52:55.740 --> 00:52:56.740 You build a distribution. 00:52:57.540 --> 00:53:03.460 Isn't that a lot of risk? And do you see ways on how Rust can improve there? 00:53:04.120 --> 00:53:06.900 So it is a risk. I mean, anytime you use someone else's software, 00:53:07.000 --> 00:53:08.800 it's a risk, right? Anytime you bring in a dependency. 00:53:09.820 --> 00:53:14.360 We have strategies for managing that. We have a huge security team whose sole 00:53:14.360 --> 00:53:17.960 job is essentially to scour the internet for security reports and patch things 00:53:17.960 --> 00:53:21.160 in Ubuntu, patch things in Debian, try to patch things upstream and make sure 00:53:21.160 --> 00:53:24.400 that the version of something that we are shipping is not vulnerable. 00:53:26.180 --> 00:53:30.460 The crates.io is an amazing piece of infrastructure. I use it myself. 00:53:30.660 --> 00:53:34.280 It's, you know, why would you not? But I have noticed a trend for there being 00:53:34.280 --> 00:53:40.400 lots of kind of small libraries, which is a trend that we saw in other ecosystems. 00:53:40.540 --> 00:53:41.860 I guess this is a hard problem to solve. 00:53:42.020 --> 00:53:47.420 I think one area where Rust could maybe benefit is from perhaps a slightly stronger 00:53:47.420 --> 00:53:50.940 standard library and kind of like an approved approach to solving some problems. 00:53:51.500 --> 00:53:55.580 There obviously is a standard library in Rust, and the language hasn't been 00:53:55.580 --> 00:53:57.620 around for that long, so perhaps that will come. 00:53:57.940 --> 00:54:02.000 But if I make a direct comparison between the development time I've spent in 00:54:02.000 --> 00:54:05.240 Go and the development time I've spent in Rust, I more frequently have to reach 00:54:05.240 --> 00:54:07.260 for Excel dependencies in Rust than I do in Go. 00:54:07.820 --> 00:54:10.620 And I think that has been a long-touted benefit of Go in fairness. 00:54:10.720 --> 00:54:12.180 I think it's a very strong language there. 00:54:12.620 --> 00:54:16.900 Python is also very strong in the standard library that it brings. Rust to me feels, 00:54:18.080 --> 00:54:22.300 it is less strong there. And as a result, you end up with lots of small crates 00:54:22.300 --> 00:54:25.920 of kind of unknown origin, essentially. So it makes it harder. 00:54:26.740 --> 00:54:30.520 But I think part of this will come with maturity, right? There are certain superstar 00:54:30.520 --> 00:54:34.780 libraries that everyone knows about, clap and serde and various others. 00:54:35.440 --> 00:54:39.360 But in configuration file handling, for example, I shudder to think how many 00:54:39.360 --> 00:54:44.420 implementations there are of parsing a Toml config file on crates.io, right? 00:54:44.520 --> 00:54:49.320 Which seems yeah maybe that's useful i it seems like an interesting like bifurcation 00:54:49.320 --> 00:54:52.400 of effort in a sense for something that's so common yeah. 00:54:52.400 --> 00:54:55.800 It's a double-edged sword somehow on 00:54:55.800 --> 00:54:59.860 one side you also expose yourself to a 00:54:59.860 --> 00:55:06.580 lot of you know side effects and maybe you also accumulate a lot of functionality 00:55:06.580 --> 00:55:10.940 in the standard library if you follow this batteries included approach but on 00:55:10.940 --> 00:55:16.240 the other side you also push the burden more to the users which now have to vet smaller crates. 00:55:16.240 --> 00:55:19.820 Yeah or choose not to bring a crate in and just vendor some of that code right 00:55:19.820 --> 00:55:24.620 is always an option it's an approach i would generally take myself i think you 00:55:24.620 --> 00:55:28.680 know if you often you can be at risk of bringing in thousands of lines of code 00:55:28.680 --> 00:55:32.400 from a library where you need like a 30 line function from it yeah these are 00:55:32.400 --> 00:55:36.080 all to a limited extent like this is all within a developer's control at the end of 00:55:36.140 --> 00:55:41.740 day i feel like the it's about the community's approach to it that can kind 00:55:41.740 --> 00:55:45.980 of signpost people to the right to make the right decisions more often it's. 00:55:45.980 --> 00:55:50.200 True i always wonder why this is not done more often especially for the smaller 00:55:50.200 --> 00:55:54.220 functions and libraries that you use as long as the license is compatible. 00:55:54.220 --> 00:55:55.200 Exactly. 00:55:56.500 --> 00:56:01.540 We're getting close to the end. And traditionally, the final question is, 00:56:02.160 --> 00:56:05.040 what's your message to the Rust community? 00:56:05.900 --> 00:56:11.620 I think my message to the Rust community is twofold. One is we're hiring Rust developers. 00:56:11.920 --> 00:56:14.900 So, you know, reach out if that's of interest to you. And secondly, 00:56:15.160 --> 00:56:19.200 if you've got a really interesting Rust project and you think it's a great fit 00:56:19.200 --> 00:56:22.540 for some of the work that we're doing and you want to have a conversation, then reach out. 00:56:22.980 --> 00:56:26.500 I'm open to ideas. I don't know all of the high-quality Rust projects out there. 00:56:26.620 --> 00:56:31.100 And if there are implementations of things that are very widespread that you 00:56:31.100 --> 00:56:32.960 think we should be using, then let me know. 00:56:33.980 --> 00:56:39.320 John, thanks so much for the interview. And thanks to Canonical for supporting the Rust ecosystem. 00:56:39.820 --> 00:56:40.980 Thank you very much. It's been a pleasure. 00:56:41.720 --> 00:56:45.400 Rust in Production is a podcast by corrode. It is hosted by me, 00:56:45.700 --> 00:56:48.480 Matthias Endler, and produced by Simon Brüggen. 00:56:48.620 --> 00:56:52.900 For show notes, transcripts, and to learn more about how we can help your company 00:56:52.900 --> 00:56:55.820 make the most of Rust, visit corrode.dev. 00:56:55.960 --> 00:56:58.360 Thanks for listening to Rustin Production.