Matthias: Here's Rust in Production, a podcast about companies who use Rust to shape the
Matthias: future of infrastructure.
Matthias: My name is Matthias Endler from corrode and today we talk to John Seeger from
Matthias: Canonical about oxidizing Ubuntu with Rust.
Matthias: John, thanks so much for taking the time for the interview today.
Matthias: Can you quickly introduce yourself and Canonical, the company you work for?
Jon: Of course. So my name is John Seeger. I'm the VP of Engineering for Ubuntu,
Jon: which is a Linux operating system.
Jon: And Canonical is the company which ships Ubuntu, as well as a host of other
Jon: open source utilities for building data centers and cloud applications and various other tools.
Matthias: You took over the Ubuntu management in January, as far as I remember,
Matthias: right? What drew you into that position?
Jon: So yeah, I did take over in January. I've actually been at Canonical for four and a half years.
Jon: I'd spent a lot of time focusing on cloud orchestration, a tool called Juju,
Jon: and some operators for things like databases.
Jon: And at the tail end of last year, Mark Shuttleworth approached me and asked
Jon: if I would think about taking over Ubuntu.
Jon: We didn't have a VP who was kind of looking after Ubuntu.
Jon: And he felt like there was some room for improvement in terms of the vision
Jon: and what we were looking to achieve with Ubuntu and perhaps how the teams were
Jon: working and what ultimately we wanted Ubuntu to be in the marketplace of Linux distributions.
Jon: There's quite a spectrum from very exciting, new shiny things to very stable,
Jon: less shiny things in the distribution market.
Jon: And I think it was time for us to think about where we sat in that and what
Jon: we wanted Ubuntu to be for the next 20 years.
Jon: Last year was 20 years of Ubuntu but with that success i think it's important
Jon: to have a point of introspection and decide you know where next we've had 20
Jon: years of success and we've enjoyed that but what do we want ubuntu to be as we look forward yeah.
Matthias: I can't believe it's been 20 years already it feels like yesterday to me now,
Matthias: You've been making quite some headlines recently with a very bold move,
Matthias: at least for some people, which is that in the next version of Ubuntu,
Matthias: you will make a bold step towards adopting Rust in the core part of the distribution.
Matthias: Can you talk a little bit about that?
Jon: Yes. So this was part of a set of kind of four initiatives or four themes that
Jon: I started working towards with Ubuntu, one of which was modernization.
Jon: And so the experiment that we're running at the moment is replacing GNU core
Jon: utils with a Rust rewrite of the core utils project by an organization called
Jon: UUtils and replacing sudo with a Rust rewrite called sudo-rs.
Jon: And really, I think these projects are really impressive. I think they have
Jon: great communities around them.
Jon: I liked the level of enthusiasm I was seeing around these kind of core parts
Jon: of a Linux distribution.
Jon: And because of the kind of memory safety benefits that we get from Rust and
Jon: the potential for there to be performance improvements, I thought they were
Jon: an interesting place for us to start.
Jon: And because they're so foundational to the experience that people have at the
Jon: Linux command line, of course, it grabbed headlines.
Jon: I think lots of people who use Linux for the first time and jump into the terminal,
Jon: often the very first commands they type are from core utils, ls and rm and cp.
Jon: And with sudo, my view is that's even more important because that's right at
Jon: the security boundary between privileged and unprivileged users.
Jon: And so for me, that one feels more critical from a security perspective,
Jon: which is often what our paying customers are looking for from Ubuntu, right?
Jon: It's security and resilience in the operating system that they ship for whatever their use case is.
Matthias: Yeah. Now a critical person might say wait but you already had a battle-tested
Matthias: set of tools which were developed for 30 plus years why would you rewrite them
Matthias: wouldn't you introduce more problems than you would solve what would you reply to that.
Jon: My reply to that is, that is true of every technological advancement in the
Jon: history of humankind, right?
Jon: Like, why do we need internal combustion engines when horses were great?
Jon: They were great. And internal combustion engines have been useful to us,
Jon: and whatever comes next will be useful to us again.
Jon: I think, I certainly haven't done this to throw any shade on the GNU-Core Utils
Jon: project, or indeed the original sudo project, right?
Jon: And one of the things that I liked about both of these projects was the attitude
Jon: they had towards the original implementations.
Jon: So in the process of rewriting core utils, which started out as a hobby project
Jon: for somebody to experiment with Rust, they uncovered bugs in GNU core utils,
Jon: which they fired issues for and worked with the maintainers on to correct.
Jon: And similarly in sudo, the sudo-rs maintainers are in really regular contact
Jon: with the original sudo maintainer.
Jon: They have uncovered bugs in
Jon: their implementation, which turned out to be present in the original too.
Jon: And so it is a bit of a collaborative effort, right?
Jon: They are both a sort of modern interpretation of a set of well-known tools and utilities.
Jon: I think one of the bits of context for Ubuntu is part of our success has been
Jon: in offering long-term support.
Jon: The idea of an LTS was a thing that.
Jon: Only came about because of ubuntu right and so
Jon: when we ship a tool or we ship a utility we ship some code we will stand by
Jon: that for 12 and in some cases 15 years and so for me it's really important that
Jon: we pick tools and utilities that have got momentum and community support and
Jon: are written in languages which we can continue to employ talent to maintain and look after,
Jon: and i think increasingly rust is gaining adoption in these kind of system level
Jon: utilities people are interested in writing Rust, they're interested in learning about it.
Jon: And it's a way of us encouraging talent, but also encouraging contribution to Ubuntu.
Jon: I think with open source, the way it's gone, it's been in many ways so very successful.
Jon: There are so many things that one could contribute to in open source these days.
Jon: And I would like to focus very much on building the next generation of contributors
Jon: to Linux, not just the applications that run on top of Linux,
Jon: which is maybe taken for granted by some, I suppose.
Matthias: I guess you mentioned a very important point, which is the maintenance of the tools we all depend on.
Matthias: I don't know how many installations the core utils have, but it must be in the billions.
Matthias: And those tools run also billions of times a day on all of the machines combined, I guess.
Matthias: But the main point is, this is a thing that people take for granted.
Matthias: And it's also, to some extent at least, a maintenance burden because of C,
Matthias: because of the way it was written and so on.
Matthias: Can you elaborate on this part as well? the part of the project that helps people
Matthias: transition from C to Rust and people maybe preferring to work in Rust code bases.
Jon: Yes, and this has been called out a couple of times in commentary about this
Jon: move in Ubuntu, where I've referred
Jon: to, you know, as replacing 30-year-old code, like that's a bad thing.
Jon: I have no, I have nothing against the GNU Core Utils code base.
Jon: I suspect it's very good quality, right?
Jon: It's been in production a long time. It's been finely tuned.
Jon: But the fact remains that the
Jon: single biggest vector for attack on modern
Jon: systems in or certainly in the last few years
Jon: has been through memory safety right it has been through where someone
Jon: has missed an edge case or failed to validate something
Jon: or you know missed a check for a null pointer and these whether we like it or
Jon: not whether we want to admit it this is where the vast majority of security
Jon: vulnerabilities come from the opportunity with something like rust core utils
Jon: is to have a generation of software where that is significantly less likely.
Jon: Not impossible, but significantly less likely. And as you say,
Jon: billions of invocations per day, potentially, right?
Jon: In scripts, as part of other software, in provisioning, in day-to-day use by end users.
Jon: If all of those invocations were of utilities that fundamentally cannot suffer
Jon: from the same issues and have the potential for higher performance and an engaged
Jon: community that's going to look after them for the next few years,
Jon: I struggle to see a disadvantage in that. It doesn't have to be.
Jon: In spite of the original project, right? Like there will even be use cases where
Jon: we probably still use utilities from GNU core utils for whatever reason.
Jon: And I think that's okay. The two don't have to be fighting, right? They can coexist.
Jon: And I think they will for a very long time.
Matthias: And the other misconception that a lot of people have is that the GNU core utils are basically done.
Matthias: They are finished. There are no more changes that need to be made.
Matthias: But in reality, that's also not true.
Matthias: Even just this week, there were multiple commits to
Matthias: the repository and the other
Matthias: part is vulnerabilities also still get discovered
Matthias: in those tools i just checked yesterday and
Matthias: there was a cve in sort which
Matthias: allowed you to if you controlled the input
Matthias: have access to
Matthias: memory that you shouldn't have access to because of
Matthias: you know i guess
Matthias: just a mishandling of the input or you know an exception so it's it's probably
Matthias: more than just language evangelism that that you propose here right it's there's
Matthias: a genuine technical necessity to move to safer languages it's.
Jon: Actually not language evangelism at all in the sense that i have a few rust
Jon: projects but it's single digit numbers of rust projects right like i'm not a,
Jon: I'm not a Rust person, if you will. I know Rust. I like the language.
Jon: I've used it for a few projects. I think it is interesting.
Jon: What is almost more interesting to me is the set of principles that the Rust
Jon: community seem to rally around when it comes to safety and performance.
Jon: Now, that doesn't mean all Rust software is better than all C software.
Jon: It doesn't mean that everything that was written in C in Ubuntu will get replaced
Jon: with something that has been written in Rust.
Jon: But where there are compelling alternatives and engaged communities,
Jon: they're on the table, right?
Jon: Like I say, Ubuntu has been around for 20 years, and I hope it will be around for another 20.
Jon: And it probably shouldn't look exactly like it does today in 20 years.
Jon: Again, that would be a failing, I think, for our open source community, for our customers.
Jon: It needs to continue to evolve and i think this is just a part of that.
Matthias: As someone who also sees the operational sides of things of maintaining a full
Matthias: distribution and also maybe taking some of that risk to make sure that you can
Matthias: ship on time and so on was adding rust to the toolchain a huge operational burden.
Jon: So when i begun this effort we already had rust the rust toolchain in ubuntu
Jon: right like so you You could already get the Rust compiler.
Jon: You could always already get tools like RustUp. So like we already had,
Jon: because of the fact that we're downstream from Debian and us and the Debian
Jon: community have been working on how to package Rust software.
Jon: This was a relatively easy transition.
Jon: In fact, Rust core utils and sudo-rs were already packages that existed in Ubuntu's archive.
Jon: And so this is really a policy change, which says that we're going to bring
Jon: that into what we call main,
Jon: which is the part of the archive which has the longest and kind of the most
Jon: stringent set of security promises from us, if you will, we're going to bring
Jon: those packages into main and we're going to ship them by default.
Jon: The old ones will continue to be available. And in fact, one of the more complicated pieces of work,
Jon: one of the more genuinely complex things that we had to do here was some work
Jon: on apt itself to make sure that the path to switch back to the old implementation
Jon: was as simple as possible.
Jon: This is not about us forcing this upon everybody at all costs.
Jon: The GNU core utils will continue to be available on Ubuntu.
Jon: And we have done actual engineering work in apt to make sure that going back is one command.
Jon: And we'll publish that in the release notes. Same with sudo, right? sudo was
Jon: a little easier the reason we had to do the work in apt is because if
Jon: by the time apt got halfway through the process
Jon: of swapping them it removes the new version of core utils suddenly
Jon: all the utilities it needed to install the new one weren't there anymore
Jon: so we had to do some work in terms of like sequencing that so yes i mean my
Jon: view is it you know for 90% 99% of our users this will be a no-op they won't notice
Jon: it but it makes it easier for us to maintain and it lowers the likelihood of
Jon: issues like the one you just described in sort.
Jon: Which don't come around often, but they do still come around.
Jon: That's evidence that it does still happen.
Matthias: Do you get a lot of beta testers for the distributions usually?
Matthias: Do people test it stuff, or are you still sanding down the edges on the Rust integration?
Jon: Yeah, it's difficult to know exact numbers, but we definitely do get pretty good engagement.
Jon: This change actually coincided with another change I made here on to this year,
Jon: which was to do monthly snapshot releases.
Jon: It's a slightly tangential subject, but part of my
Jon: push towards kind of modernizing ubuntu was to streamline our
Jon: release process and modernize that a little bit and i
Jon: challenged our release team to stop just doing it every six months
Jon: when we actually wanted to release but to basically exercise that process every
Jon: month and put out snapshot releases and as a side effect
Jon: of that we've had ubuntu isos out that have
Jon: had the rust tools for for months right like incrementally more and more of
Jon: the GNU core utils were replaced with the rusty ones and so now in the beta
Jon: we're looking like the vast majority will there are still a couple of incompatibilities
Jon: so we you know some of the utilities will still be the new ones but it means
Jon: that people have had better access to it i don't know exact numbers but we certainly see,
Jon: response to discourse posts and blogs saying that people are trying it out and
Jon: we've also seen a significant uptick in bugs far against the packages as people
Jon: have been testing it and finding things which has been really helpful to us and.
Matthias: On a technical level when you want to switch tool by tool you just symlink that
Matthias: and you make it the default in someone's shell right.
Jon: Yeah essentially it is just it is just a symlink i can actually if it's interesting
Jon: for the show notes or something there's a link where you can actually see the
Jon: set of links that we have right now where we're pointing to gnu tools still
Jon: it's going to shrink in the next couple of days because the folks at just cut
Jon: a new release which solves a bunch of issues that we had so yeah it's just a
Jon: set of index essentially.
Matthias: Did you catch any surprises? Because in my understanding,
Matthias: that would probably be one of the hardest projects to work on because you have
Matthias: so many undocumented features of the old tools that people start to depend on in their workflows.
Matthias: But now you kind of change it from underneath them and you need to make it look
Matthias: as if it was the old thing or if it behaved exactly like the old thing,
Matthias: at least in the first version, right? Did you find any nasty surprises in that transition period?
Jon: So I think we should talk about core utils and sudo-rs differently because I
Jon: think it's kind of interesting. The two projects actually approach this quite differently.
Jon: So the uutils project aims to be bug-for-bug compatible with the original core utils.
Jon: If core utils does a thing, uutils aims to do it identically, essentially.
Jon: That's actually not the case in sudo-rs.
Jon: But if we focus on core utils still, there were a couple of things that just
Jon: simply weren't implemented when we started.
Jon: So an example of this was most of the tools didn't have support for the dash
Jon: capital Z, capital Z for your American listeners, for essentially manipulating
Jon: viewing SC Linux context information for files.
Jon: That functionality just hadn't been implemented. And that was one of the things
Jon: that we sponsored the utils developers to implement before we shipped it.
Jon: Now, we don't actually use SELinux by default on Ubuntu, but there are people
Jon: in the community and people in our customer base who presumably do use SELinux
Jon: with Ubuntu, and so we wanted to make sure that was present before we shipped it.
Jon: Likewise, localization, internationalization, there was no infrastructure for translations.
Jon: So, sort worked great in English and not so well in German or in French.
Jon: And so, part of the work that we sponsored was to essentially build infrastructure
Jon: into the project to allow localizations.
Jon: The story with Sudo is a little bit different because they don't aim to be 100% bug for bug compatible.
Jon: They aim to be mostly compatible with a view that some of the things in the
Jon: original Sudo were implemented well and in good faith, but that the ideas didn't age so well.
Jon: Like the idea about what the Sudo tool should do and how it should behave,
Jon: perhaps if we were starting again in 2025, we'd either do differently or we
Jon: just wouldn't do in the Sudo tool at all.
Jon: And so there aren't many of these features
Jon: but an example would be in the original
Jon: sudo you can pass it a dash capital e and
Jon: that will take your entire like environment variable context
Jon: from your unprivileged environment into
Jon: the privileged environment and they chose not to
Jon: implement that on the grounds that they feel like it is safer that
Jon: people are conscious about which parts of the environment they're pushing
Jon: into that privileged space and so people who have the dash capital e and their
Jon: scripts will find that that doesn't work which is why you know so either they
Jon: can choose to say okay yep maybe i should be a bit more careful i need to take
Jon: this variable and that variable or they can on their installations they can
Jon: switch it back to the old version and carry on as they were.
Matthias: Wow do they get some exit code which tells them that this was the issue like
Matthias: passing the entire environment into privileged context sounds like a massive
Matthias: food I'm not sure why they even added that in the first place,
Matthias: but do they get some error message, which tells them how to do the right thing?
Jon: Well, they will get an error message from sudo-rs saying, I don't know what that switch is.
Jon: So we haven't put code into sudo-rs to say, hey, we don't know what that switch
Jon: is because you're using a nice shiny new version. Yeah.
Jon: In that sense, it is a little bit opaque, but part of the effort has been updating
Jon: our own documentation to try and make sure this information is captured.
Jon: And when people search, it will come up in the Ubuntu docs to say,
Jon: you know, we've made this change.
Jon: This is how you can get back, or this is, you know, the alternatives.
Jon: So, and, you know, unusually, mostly when Ubuntu releases a new feature,
Jon: we just release a new feature and write about it in the release notes.
Jon: As a as a kind of exception here we are actually going to write we've done this
Jon: thing and here's how you can revert it if you if you care.
Matthias: Yeah um well if
Matthias: you have a tool so central and i'm talking about both the core utils and sudo-rs
Matthias: but if you have tools like this which are so central to your even the outputs
Matthias: become part of the api the interface yeah because people do various things with
Matthias: that output, which they shouldn't probably.
Matthias: They grab for certain keywords and so on.
Matthias: For example, as a German user, I know that some of these tools also have locale-dependent output.
Matthias: They might print something in German, whereas on a different machine they might
Matthias: print something in English, for example.
Matthias: How much attention do you have to pay to these details?
Matthias: And were there any things that, for example, were not solved in the Rust ecosystem specifically to,
Matthias: you know, handle all of these use cases.
Jon: So I think internationalization was the big one there. So because the uutils
Jon: project is aiming for like a bug for bug compatibility, theoretically,
Jon: the output should be identical.
Jon: And if it's not, it's a bug and they'll fix it. Right. So we haven't really run into that.
Jon: I don't think there is, there was an issue with the date tool where it failed
Jon: to parse a certain kind of date time or it failed.
Jon: I forget the exact issue, but either the way it outputted it or the way it interpreted
Jon: a date was different from the GNU tool, and that caused us a huge headache in
Jon: our build infrastructure.
Jon: And so for a while, we were using the GNU date tool still. I believe that's
Jon: now fixed. So we have seen issues like that.
Jon: It's also worth mentioning why now in Ubuntu.
Jon: So lots of people will know about Ubuntu and they know about the LTS,
Jon: but it's worth talking about our release model.
Jon: So every two years in April, we release a new LTS.
Jon: So 20.04, 22.04, 24.04, 26.04.
Jon: And that is where a huge majority of our user base lies.
Jon: Lots of people will stay on an LTS until the next LTS comes around, then an update.
Jon: But then every six months, we still do interim releases, what we call interim releases.
Jon: So these are 25.04, 25.10, 26.10, 27.04, et cetera.
Jon: And we see dramatically less users here, and particularly in the enterprise
Jon: space where people aren't generally running production workloads on these.
Jon: And so these interim releases are the opportunity for us to test this out.
Jon: We will get thousands, tens of thousands, maybe hundreds of thousands of users.
Jon: And we still want those to be very high quality. We want them to be stable,
Jon: but it is our opportunity to try new things.
Jon: So an example of this is, I think, back in 17.10, I want to say,
Jon: either 17.10 or 17.04, long before I joined Canonical.
Jon: I seem to recall they tried switching to Wayland by default.
Jon: And in the next interim release, they were like, that didn't go very well. We're going to go back.
Jon: Right? I think we're now back on Wayland by default. But the point is,
Jon: that was an example of where we used the interim release to try something that
Jon: we thought was going to work. It didn't work out that well, so we switched it back.
Jon: And I am really committed to getting this change across the line in the LTS,
Jon: but not to the point of destruction, if you see what I mean.
Jon: If we roll 25.10 out the door and this change doesn't go well or it causes huge
Jon: issues, then we'll have to roll it back.
Jon: It doesn't mean we'll roll back forever, and it might be that we roll back some
Jon: and not all of the utilities,
Jon: but you know this is just the balance this is the trade-off right like we have
Jon: to work out whether or not the thing is ready for 26.04 which is our next lts
Jon: so we still have seven months eight months to get ready for that lts and 25.10
Jon: is kind of a milestone on the way to that and it gives people it gives us a
Jon: way to get new ideas into the hands of lots of people to see whether it's ready for the lts now.
Matthias: You probably have a lot of integration tests anyway nowadays.
Jon: Yeah
Matthias: for these tools,
Jon: Certainly packages themselves are tested with a thing called auto package test
Jon: which we inherit from debian so most all of the packages in main have to have
Jon: auto package test tests to be in main,
Jon: and actually that's where we've inflicted some pain on ourselves like where
Jon: there have been differences in the utilities we've seen them in our build infrastructure
Jon: because suddenly thousands of packages won't build or there'll be an error you
Jon: know the test will fail and so that's been a useful tool for us for kind of
Jon: identifying issues
Matthias: When people
Matthias: usually propose rust it's for
Matthias: a few reasons the main one is improved security
Matthias: but another one is better performance now
Matthias: in your case you're competing with c which is already quite performant
Matthias: you might even say that's a bit
Matthias: of a regression here because there are still statements
Matthias: out there that rust cannot be as fast as c and that very much depends on your
Matthias: use case but what would you say to that would you say well we take a performance
Matthias: hit yeah we accept it or is there even a possibility for the tools to be as
Matthias: fast or even faster than the older version.
Jon: I think in the mid to long term there's an opportunity for those tools to be
Jon: faster and for a couple of reasons there was an article that went round recently
Jon: which showed that you know there were some performance shortcomings uncovered
Jon: in this i think the example was the checksum tool i actually found a regression
Jon: in while we were engineering sprint in the WC,
Jon: the word count utility that turned out to be slower.
Jon: I was attempting to do the 1 billion rows challenge with WC and I found that it was slower.
Jon: But the interesting thing to me is, that's not surprising.
Jon: The software is newer, it's less mature than the GNU stuff. But what is interesting
Jon: to me is how quickly these issues get resolved, how quickly people step up to find ways.
Jon: And when they do step up, they often exceed the performance of the original tools.
Jon: So the issue that was filed about checksum being 17 times slower than the GNU
Jon: one was filed by a canonical engineer who works on our,
Jon: in our foundations team, who was going out looking for problems to make sure
Jon: that we ship it in the best shape we possibly can. That issue is now solved.
Jon: The checksum tool in new utils, I believe, now outperforms the original.
Jon: And the same with the word count utility. That was, I think it was under a day.
Jon: And it went from being 1.1 times slower to 1.3 times faster.
Jon: And I think on your point about C versus Rust, can Rust be faster?
Jon: I buy the argument that in some cases there's overhead and, you know,
Jon: a very, very optimized C program and a very, very optimized Rust program,
Jon: the likelihood is that you could do better with C.
Jon: I buy that argument. Okay. I get it.
Jon: That said, things like threading from a developer experience perspective are
Jon: significantly friendlier in Rust and it's significantly more likely to be correct.
Jon: So, you know, very, very talented C developers with a real eye for performance,
Jon: I'm sure they can squeeze the absolute maximum performance out of C.
Jon: The question is, how many engineers around the world can do it?
Jon: And I think in Rust, there's a likelihood that we will see a higher degree or
Jon: a larger number of faster implementations, because the language makes some of
Jon: the more advanced primitives a little bit more easy to hold,
Jon: a little bit more easy to reason about, a little easier to debug,
Jon: a little easier to test. So it is a trade-off, ultimately.
Matthias: Well, two things that I wanted to say here. one
Matthias: is that multi-threading is one the
Matthias: other one especially when it comes to for example
Matthias: the work count thing wc is simd support right correct yeah i think simd support
Matthias: in rust is pretty decent by now so you could even dare to try and make that
Matthias: use those cpu features which i'm not 100 sure,
Matthias: GNU core utils do i i honestly doubt it to be honest.
Jon: I yeah i don't think they do and you're absolutely right
Jon: so the the speed up that we got on the workout utility i believe was assembly
Jon: implementation i think it's the same for the checksum utility and so you know
Jon: a counter argument there might be well what if i'm running on a processor that
Jon: doesn't have those features and the answer is okay might be a bit slower um
Jon: you can't literally have everything do you know
Jon: We can't win on binary size and all-out performance and security and maintainability
Jon: and community all at the same time, right?
Jon: It's very, well, maybe there are projects that have achieved that.
Jon: It's very difficult to do.
Jon: And so part of this initiative is trying to raise the odds of raising the bar
Jon: in each of those as much as we can.
Matthias: Do you compile those tools for different architectures with different CPU features?
Jon: Yes. So we support, at the moment, we support AMD64, we support ARM,
Jon: ARMHF, S390, PowerPC64, and RISC-V.
Jon: And the deal we have in Canonical with Ubuntu is if we support an architecture,
Jon: we support an architecture.
Jon: We don't do cross-compilation. We buy actual hardware with those architectures.
Jon: So we have actual mainframes in our data centers, like S390 machines,
Jon: on which we compile packages for the archive and we test packages for the archive.
Jon: We also have landed some
Jon: work in launchpad and in the archive that will
Jon: allow us to deliver binaries that take advantage of
Jon: micro architectural variants so you know
Jon: if you're on a machine that has amd64v3 support
Jon: you might be able to get access to a binary that's been
Jon: compiled to take advantage of amd64v3 features where that makes sense and the
Jon: same would be true of amd64v5 you've got armv8 armv9 armv10 and obviously i
Jon: think in the risk ecosystem we're seeing kind of an explosion of this as the
Jon: architecture or the instruction set is in its infancy, people are building processes,
Jon: with wildly different capabilities and so we've had to make sure that our kind
Jon: of build infrastructure has the capability to deal with that that's landed relatively
Jon: recently there's a blog post coming in a couple weeks about it.
Matthias: And not many people know that some
Matthias: of these tools especially the legacy ones they highly
Matthias: underuse modern cpus there are
Matthias: certain things in there which are just not really done and they could could
Matthias: be done but it's not an easy task and it's a bit risky like wouldn't that also
Matthias: be an opportunity to leverage some of those functionalities that have been in
Matthias: cpus for a decade now or longer right yeah.
Jon: And again it's a trade-off between compatibility and performance right one of
Jon: the nice things about gnu core utils is they it works the same in lots of places
Jon: that perhaps will not be said of the uutils project where they have kind of
Jon: conditional implementations based on CPU features,
Jon: but where the vast majority of a huge data center, for example,
Jon: in a cloud has AMD64v3 and has these CINDY features and stuff,
Jon: surely we want to take advantage of those as much as we possibly can,
Jon: not just from the performance perspective,
Jon: but on the other side of the same coin is power usage and how much energy is
Jon: consumed by doing a task.
Jon: And often using these implementations sees a huge advantage there too.
Matthias: Yeah yeah my first thought was well it's a user space tool and probably not
Matthias: really that energy hungry but if you call it billions of times a day then it
Matthias: adds up very quickly right,
Matthias: and there's another misconception about core utils which i hear a lot which is,
Matthias: They are optimized already to a certain extent.
Matthias: And I guess that's my second point is that people think they can't really go
Matthias: in and improve those tools.
Matthias: But in reality, it's not the case. For example, I once wrote a faster version
Matthias: of cat just to see where the bottleneck was and if I could make it as fast as the new CoriTools one.
Matthias: And mine was just three times as fast. The reason was not that the code was specifically clever.
Matthias: It was just using a new feature in the linux kernel
Matthias: which is the splice feature and you can avoid
Matthias: one copy from kernel space to user space with that now did i dare to send in
Matthias: a patch for that functionality to be added to new corridors no of course not
Matthias: because i honestly i'm not a c developer and i kind of was afraid of the process
Matthias: some of these things they are not accessible to,
Matthias: everyday normal developers they have different processes and so on so i hope
Matthias: that also contributing to the uutils might be easier in the future yeah.
Jon: I think so people have various opinions about code forges github microsoft licenses
Jon: it's all a trade-off at the end of the day and i think you know whatever microsoft's
Jon: motivations which you know so far don't seem too impure to me There's millions
Jon: of developers on GitHub. That's a huge audience.
Jon: And so bringing the development of something so foundational as the core utils
Jon: into a space where there exist millions of developers who know the contribution
Jon: process and how to open pull requests and write comments and interact with people,
Jon: again, I find that hard to kind of put across as a bad thing.
Matthias: You said that sudo-rs has a slightly different focus than core utils.
Matthias: Can you elaborate on that a bit?
Matthias: I guess sudo was the first thing you will integrate.
Matthias: And why start with this? Wouldn't it be easier to start with a completely new
Matthias: tool instead of rewriting something so foundational in the core of the distribution?
Jon: So I think it's precisely because, I mean, say, we didn't write sudo-rs,
Jon: Canonical didn't rewrite sudo.
Jon: This rewrite existed. It is authored by a non-profit called the Trifector Tech
Jon: Foundation, and they are an organization who aims essentially to provide resilient
Jon: software for public good, if you will, like infrastructure-level software.
Jon: And I think, honestly, if there was going to be anywhere on the system where
Jon: I would even trade performance for guarantees, it would be in the sudo tool.
Jon: There are alternate approaches to privilege escalation on Linux.
Jon: There's a new approach in Systemd called run0, which has been popular recently
Jon: in some other distributions.
Jon: I'm sure it's very good. The Systemd project has written some great software.
Jon: But the sudo-RS project gives
Jon: us a 90-something percent compatibility guarantee with memory safety.
Jon: And to make it even more fun, they were absolutely delightful to work with.
Jon: They were very excited to have their software in Ubuntu. They worked with us.
Jon: They They were really professional.
Jon: They worked with our security team. You know, they work with the OG kind of sudo developer.
Jon: And so where you have this intersection of really motivated,
Jon: really professional people and a programming language that kind of enforces
Jon: certain ways of thinking and working for safer software, it just seemed like a natural fit.
Jon: The security boundary is such a sensitive part of the operating system that
Jon: it felt like a kind of natural place to start.
Matthias: Now, sudo-rs has 42,000 lines of code, and to some that might seem like very
Matthias: little code, to some that might seem like a lot of code.
Matthias: What does it do exactly that requires that many lines of code?
Jon: So there's a few things, really. There is the fundamentals of kind of escalating
Jon: privilege of our process.
Jon: There's all the configuration, parsing, different options that have evolved over time.
Jon: One of the features that we asked to be added was AppArmor support like the
Jon: ability for sudo and AppArmor to interact which is the LSM that we use,
Jon: the Linux security module,
Jon: they need to be able to talk to PAM for example and SSSD for things like LDAP
Jon: authentication so that is another difference between the original sudo which
Jon: can natively speak LDAP the folks behind sudo-rs decided that that wasn't something
Jon: they wanted their tool to do and they'd rather hand that off to to SSSD, essentially.
Jon: And so, I mean, there's a lot of features, right? Again, localization support
Jon: is something that, you know, it doesn't come for free. It comes with code overhead.
Matthias: So it's not only sudo-rs that you add. It's also an entire chain of dependencies
Matthias: in an ecosystem of Rust grades.
Jon: Right, and things like sudo-edit, so you can do the kind of visudo thing,
Jon: right? Edit, like, it has its kind of own secure editor for editing the permissions.
Jon: It is, yeah, it is a whole ecosystem. And the folks there have been diligent
Jon: in the dependencies that they pull in, and they're very careful about it for obvious reasons.
Jon: But yes, it is a whole ecosystem, right? It's a fairly big project.
Matthias: As a Rust consultant, I'm super happy to hear that because it adds support to
Matthias: the entire Rust ecosystem.
Matthias: But from your perspective, I would wonder, am I exposing myself to a lot of
Matthias: additional risk because a lot of that code hasn't been...
Matthias: You know part of such a long-term project as ubuntu for such a long time and
Matthias: maybe hasn't seen a lot of production the reality of production let's say.
Jon: Yeah it is a risk like you're absolutely right
Jon: it is a risk in fact of all of the challenges in the rust community i would
Jon: say one of the largest is in creating an ecosystem of crates that we can all
Jon: have trust in it's easy it's easy for people to poke fun at NPM, you know, left pad.
Jon: And there was some recent things with the kind of color libraries,
Jon: you know, crates.rs isn't that much different at the end of the day.
Jon: Anyone can upload code there.
Jon: And so what I would look for in projects like sudo-rs is a diligence in how that ecosystem is used.
Jon: Like, do you pull in an entire dependency that's maintained by somebody else that you don't know?
Jon: Or do you vendor some of the code, you know, following the licensing guidelines
Jon: and bring it in and maintain it yourself, right?
Jon: These are all software engineering decisions at the end of the day that you
Jon: wait, you trade off being able to move fast and being able to move very safely.
Jon: So it is a risk, but I mean, isn't any change to the world's most deployed Linux
Jon: operating system a risk at the end of the day?
Jon: Like we just have to balance that out. And again, that's why it's landed in 25.10 first.
Jon: I suspect there will be people out there who can't
Jon: wait for this to land to see if they can go and find a security vulnerability in sudo-RS
Jon: that doesn't exist in the original and i suspect they will probably succeed
Jon: right like i i don't believe that it is possible to write software that is perfect
Jon: all the time what i'm gambling on is the long-term probability of that thing
Jon: being safer in the round yeah.
Matthias: And even if they find a security vulnerability my assumption would be that it
Matthias: would make a lot of rust code safer because unless they use unsafe blocks everywhere.
Matthias: And then I haven't checked how many unsafe lines of code are in there.
Matthias: Maybe you know, that would be interesting to know.
Jon: Not at the top of my head.
Matthias: No. We can double check.
Matthias: Unless they use a lot of unsafe code, if vulnerability gets discovered,
Matthias: that means that's a compiler bug and that would be fixed probably immediately.
Matthias: And then a lot of other tools would not be exposed to that security vulnerability anymore.
Jon: Yeah, I think this is part of what I see as Canonical's responsibility to the tech industry.
Jon: The tech industry really rallied to Ubuntu 20 years ago, and we have enjoyed success from it.
Jon: We have lots of commercial customers, we have enough revenue to fund 800 engineers
Jon: on good salaries all around the world to work on open source and Linux,
Jon: and that's a position of huge privilege.
Jon: But I think part of our giving back to that community, in a sense,
Jon: is to use the exposure that we have got to give products or projects like this
Jon: a bit of exposure, to give them the benefit of all of those eyes and all that
Jon: testing to progress them.
Jon: I think, you know, so long as we do that in a careful and kind of respectful
Jon: way, that's just us evolving the OS and using our position to...
Jon: Try and put some of these projects on a platform where we think that they you
Jon: know their views and their approach aligns with ours other.
Matthias: Than sudo and core utils what what else are you working on that is written in rust.
Jon: Yeah so we considering ntpd rs which would be a time-syncing daemon written
Jon: in rust as also a trifecta tech foundation what is that exactly.
Matthias: Can you elaborate on that a bit.
Jon: So if you think NTPD or crony or systemd timesyncd, things that do time syncing
Jon: for the system clock, essentially, we are looking at ntpd-rs,
Jon: which is a Rust implementation of that demon, essentially.
Jon: I'm hoping we can try and do that for 26.04, but at this point,
Jon: it's very much a hopeful, not a given.
Jon: If not, we'll be trying that out in 26.10 to see how it plays.
Jon: And this goes quite nicely hand in hand with a recent move that we made,
Jon: which is to enable NTS by default, which is the secure version of that protocol for time syncing.
Jon: So think HTTP versus HTTPS, similar sort of transition.
Jon: We're also looking at switching the OpenPGP library that Apt uses to Sequoia,
Jon: which is Rust implementation of OpenPGP.
Jon: So that would be for package signature verification kind of thing.
Jon: Various other utilities in the distro, we're doing some thinking about certificate
Jon: revocation lists and how to handle those in the distribution.
Jon: And I think there'll be some Rust involved there.
Jon: Our own compositor Mir has some Rust code in it.
Jon: Mir isn't shipped by default in Ubuntu at the moment but it is shipped in a bunch of places.
Jon: I think Fedora's LXQt spin uses Mir as a compositor and there's some Rust in there.
Jon: We have some work ongoing with real-time operating systems where some Rust exists. We have a,
Jon: cloud streaming platform called unbox cloud inside which there is some rust
Jon: so there's a few places right like we we're using it in a bunch of different projects.
Matthias: It's always cool to hear that
Matthias: because it's not the main
Matthias: selling point of all of these tools and applications
Matthias: it's just that no one
Matthias: kind of forced you to use rust for these tools you
Matthias: could have used anything else but you saw value in using rust specifically and
Matthias: for a lot of companies now rust is no longer just hype thing that i want to
Matthias: try it's just normal part of everyday infrastructure i.
Jon: Would agree with it and and for a bunch of different reasons i mean i think
Jon: we were quite tentative in our adoption of rust at canonical we you know we
Jon: haven't been on the bandwagon jumping up and down about this for years,
Jon: we're quite deliberate about the languages that we choose.
Jon: So we try not to have one team in Go, one in Erlang, one in OCaml, one in JavaScript.
Jon: We don't have this huge proliferation of languages. It's generally Go,
Jon: Python, C, and now we're adding Rust to that.
Jon: And I just sort of believe in using the right tool for the job at the end of the day.
Jon: If we were doing something that involved lots of concurrency,
Jon: and not of asynchronous networking, I would probably choose Go at the moment.
Jon: But if we're writing low-level systems utilities that are performance-critical
Jon: or safety-critical, then I would choose Rust.
Matthias: What was the first language you mentioned? We don't use that here.
Matthias: I think you said Rust twice, right?
Jon: Oh, did I?
Matthias: I'm just kidding. No-Go is a very fine choice for networking services.
Jon: We have a lot of Go at Canonical, right? Juju is a huge Go code base.
Jon: We have things like Pebble. It's creeping into Maz. We have a lot of Go.
Jon: And it's served us very well, I think.
Jon: I do just believe in tools for the job. There are some things in the Rust ecosystem
Jon: that are still relatively early that Go really excels at.
Jon: And there are some things that Go can never do well. It's a garbage-collected
Jon: language at the end of the day.
Jon: And so in places like Mir, for example, you don't want a garbage collector kicking
Jon: in when you're in the middle of drawing frames on the screen.
Jon: And so it would naturally not be a fit.
Matthias: Another thing we haven't talked about yet is a project called Anbox that you
Matthias: mentioned in passing. What's that about?
Jon: So Anbox started out as a project for running Android applications on Linux.
Jon: So it's like an Android emulator type thing.
Jon: It was led by a chat called Simon Fels, who we hired some years ago,
Jon: and he has now been building Anbox at Canonical.
Jon: And with that, we have built a commercial product called Anbox Cloud,
Jon: which is essentially sits on top of some of our other products like LXD,
Jon: the kind of hypervisor that we maintain.
Jon: And it is used for streaming android applications
Jon: over the cloud so one of the biggest use cases for this
Jon: is there's a major mobile device manufacturer who uses this
Jon: for game streaming to their devices so where the graphical
Jon: requirements of the game is much higher
Jon: than you could ever get out of a mobile device at the moment we can stream the
Jon: video at very low latency and very high quality to the
Jon: devices and get the kind of input stream from the device in terms
Jon: of controls back and you're playing a game as if natively on
Jon: the device but you're actually streaming it from the cloud but it
Jon: has other applications in kind of VDI infrastructure and
Jon: one of the other big use cases we see is in
Jon: like automotive development so you think about someone who sat at a car manufacturer
Jon: working on an android auto dashboard you know like in-car entertainment system
Jon: they can use anbox to run anbox and do local development against android whilst
Jon: they're sat on their workstations, right? Maybe that's streamed from the cloud, maybe it's local.
Jon: And so there are some parts of Anbox which are moving to Rust.
Jon: Perhaps were written in C or other languages before and some combination of
Jon: performance and kind of ease of development have led them down that path.
Matthias: Earlier you said that you don't really impose Rust on anyone so it's a bit of
Matthias: a team decision and there was one project specifically,
Matthias: where people transitioned from Rust from another language which is dqlite.
Matthias: Can you elaborate a little bit on what that is and what the language was that they used before.
Jon: Yeah so this is this is ongoing so there isn't a
Jon: rust dqlite yet in general
Jon: we are encouraging heavily our teams who have large c++ or c code bases to consider
Jon: looking at rust whether that's building a feature in rust you know bringing
Jon: some library functionality in rust we are we are you know asking them to go
Jon: and familiarize themselves with that ecosystem and think about it.
Jon: dqlite is essentially distributed to sqlite.
Jon: So you can think of it as a thin layer of kind of the raft consensus protocol
Jon: over the top of sqlite, which allows you to do high availabilitysqlite
Jon: And the bit that kind of manages the VFS layer syncing is written in C.
Jon: And we have been doing some work on dqlite. We're embedding it into Juju.
Jon: It's in LXD. It's in a lot of our products.
Jon: And one of the considerations as we're trying to think about
Jon: how to improve the performance of dqlite and the reliability of
Jon: dqlite is do we consider rewriting some or all of it in Rust and so there have
Jon: been some kind of early prototyping efforts around that and one again one of
Jon: the kind of interesting aspects there was the threading model in Rust and whether
Jon: that would allow us a slightly easier development for some of the kind of more
Jon: thorny concurrency elements and so not a hundred percent,
Jon: there's a reasonable chance that we would see that happening over the next year or so.
Matthias: Are there any parts at Canonical that you wanted to mention that use Rust that
Matthias: we haven't touched on yet?
Jon: There are a number of projects that we haven't touched on. I know of projects
Jon: in our hardware certifications team. I know of Rust products,
Jon: again, in real-time operating systems projects.
Jon: I don't claim to know all of the projects using Rust in Canonical.
Jon: They seem to be increasing, which I'm pleased about.
Jon: There's also the Rust for Linux project, right? so Rust in the Linux kernel,
Jon: we employ lots of kernel engineers, tens of kernel engineers in every time zone.
Jon: And so naturally, we have to make sure that that team is up to speed on what's
Jon: going on in the Rust for Linux project, because ultimately, we support kernels for a really long time.
Jon: And so we will be, for better or for worse, supporting kernels that have Rust
Jon: kernel modules and drivers for at least the next 12 years, probably longer.
Jon: So that's another area where I think it will grow over the next couple of years.
Matthias: Yeah. And that involves work like a Nova driver for NVIDIA GPU support and various
Matthias: other bits and pieces that are written in Rust now in the Linux kernel.
Jon: Even the Apple Silicon video drivers, we have Ubuntu Asahi as a kind of spin
Jon: of Ubuntu, which is, again, somebody who works at Canonical,
Jon: and my understanding is much of the graphics drivers for that hardware is written in Rust, so...
Matthias: Even all of it, as far as I'm aware.
Jon: Yeah, I think so.
Matthias: Which is a pretty crazy project in and of itself.
Matthias: Being able to run a fully free operating system on Apple Silicon is kind of cool.
Jon: Yeah, it's a remarkable project.
Matthias: Now, strategically, moving away from the technical side, as someone who is on
Matthias: the siding end of things,
Matthias: and you are responsible for 12 million, ubuntu desktop users what are things
Matthias: that are important for rust adoption to grow in companies like canonical.
Jon: Yeah so the the numbers game is always
Jon: a fun one we we have various ways of kind of collecting
Jon: information about our users you know some basic telemetry
Jon: you can read about it it's stuff it's mostly opt-in and very easy
Jon: to work around the last numbers i heard was something along
Jon: the lines of like we know of 12 million daily ubuntu
Jon: desktop devices we also know that there's a whole bunch that we don't know of
Jon: because they're behind corporate firewalls and proxies and things and there's
Jon: a bunch of server like goodness knows how many instances of that are containers
Jon: you know base images for containers that are scheduled around the world on kubernetes
Jon: so like we don't really know how many ubuntu instances there are but lots of,
Jon: definitely tens if not hundreds of millions, I would suggest.
Jon: The decision-making process is probably not as complicated as it seems in the
Jon: sense that the number one thing we have to maintain is that Ubuntu stays a trusted
Jon: and reliable and resilient operating system.
Jon: It's become known as the Linux that everyone can use, right?
Jon: Linux for human beings, that was the whole thing. And I don't want to jeopardize that.
Jon: As excited as I am about things like NixOS and other fun Linux distributions,
Jon: it would be a failing for us to turn Ubuntu into that.
Jon: That's not what we are, right? There's a space for those more experimental and
Jon: kind of exotic distributions, but we have to build Linux for human beings,
Jon: Linux for everyone, right?
Jon: For students, for entrepreneurs, for huge businesses.
Jon: And so my view is I'm always interested, I will always entertain suggestions
Jon: for software that could be replaced, so long as it goes hand in hand with our
Jon: vision for providing a resilient,
Jon: performant, open source Linux operating system that gives people access to what
Jon: we consider to be the very best of open source at that moment.
Jon: To me, that's what it's really all about. It's about bringing open source to
Jon: as many people as we possibly can.
Jon: And not just open source, but the best open source that we can find and can support.
Matthias: Do you interact with the Rust Foundation? And what about the future of sponsoring
Matthias: that work? You mentioned Trifecta, for example.
Matthias: How do you see that collaboration evolving?
Jon: I hope it will continue. Honestly, we don't have an open ticket to just write
Jon: sponsorship for any Rust project on the planet, right?
Jon: But certainly, it's not fair for us to suddenly impose a set of expectations
Jon: on a community or a maintainer without first having a discussion and think about
Jon: funding and helping them.
Jon: So before we announced publicly that we were going to do core utils or sudo-rs,
Jon: we spoke with the maintainers and said, hey, this is an idea we've had.
Jon: Do you think the project is ready for it? And we need these things to be fixed.
Jon: And what would that cost us?
Jon: I think that's a reasonable conversation to have.
Jon: And I intend to continue that. Again, this is us, in a sense, paying it forward.
Jon: We've enjoyed great success from Ubuntu. We're in a position to be able to help
Jon: other projects that we see as promising to progress and gain adoption.
Jon: And that can't be free, right? It can't always be free. So, of course,
Jon: in some cases, we will need to pay.
Jon: I'm having a conversation, which maybe you'll hear about in a few weeks,
Jon: with another maintainer this week about another project that we might fund because
Jon: we're very interested in what they want to achieve.
Jon: So, yeah, I think we will continue.
Matthias: And when will you rewrite Apt in Rust? Just kidding.
Jon: I don't know that that's on the cars just yet, but I mean, I,
Jon: you know, I certainly, I speak frequently with the maintainer of act and, you know, it's not,
Jon: it's not never going to happen. It doesn't have to never happen, if you see what I mean.
Jon: Like, I wouldn't be against it, but there would need to be a reason,
Jon: right? Like, perhaps if we were going to introduce some major new functionality
Jon: where it would be a good fit, then perhaps we could begin introducing Rust.
Matthias: Yeah, yeah. And I guess a bit more seriously, where do you see Rust in the Linux
Matthias: ecosystem in five years?
Jon: Honestly, I hope what we see is a sort of continuation of where we have started.
Jon: I am excited by things like graphics drivers in Rust and safety critical code
Jon: in Rust. I hope that things have settled down a little bit.
Jon: Rust in Lilix has not been without its controversies.
Jon: And I think that's natural.
Jon: One of the observations I have about the kernel community is it doesn't seem
Jon: to be attracting lots and lots of new talent.
Jon: And I think rust could be an important way in which the kernel could attract new maintainers.
Jon: So I hope that as many of them already do, the kernel community continues to
Jon: recognize while there may be some trade-offs, people may have to learn new languages
Jon: and adopt a new way of doing things. Perhaps build pipelines become more complicated.
Jon: There's also a set of opportunities associated with landing Rust code in the kernel.
Jon: So my guess is as that becomes more developed and more mainstream,
Jon: the disconnect there will settle down a little bit.
Jon: And I hope that that happens sooner rather than later so we can carry on with
Jon: building great software and worry less about arguing about the logistics of
Jon: building great software ultimately i.
Matthias: Would like to get back to a point that you mentioned earlier which keeps
Matthias: coming back to me which is crates.io is also just a registry and everyone can
Matthias: push code there and for someone who has such large requirements on long-term
Matthias: maintenance isn't that a huge risk of supply chain security,
Matthias: You end up trusting so many dependencies and sub-dependencies and so on.
Matthias: You build a distribution.
Matthias: Isn't that a lot of risk? And do you see ways on how Rust can improve there?
Jon: So it is a risk. I mean, anytime you use someone else's software,
Jon: it's a risk, right? Anytime you bring in a dependency.
Jon: We have strategies for managing that. We have a huge security team whose sole
Jon: job is essentially to scour the internet for security reports and patch things
Jon: in Ubuntu, patch things in Debian, try to patch things upstream and make sure
Jon: that the version of something that we are shipping is not vulnerable.
Jon: The crates.io is an amazing piece of infrastructure. I use it myself.
Jon: It's, you know, why would you not? But I have noticed a trend for there being
Jon: lots of kind of small libraries, which is a trend that we saw in other ecosystems.
Jon: I guess this is a hard problem to solve.
Jon: I think one area where Rust could maybe benefit is from perhaps a slightly stronger
Jon: standard library and kind of like an approved approach to solving some problems.
Jon: There obviously is a standard library in Rust, and the language hasn't been
Jon: around for that long, so perhaps that will come.
Jon: But if I make a direct comparison between the development time I've spent in
Jon: Go and the development time I've spent in Rust, I more frequently have to reach
Jon: for Excel dependencies in Rust than I do in Go.
Jon: And I think that has been a long-touted benefit of Go in fairness.
Jon: I think it's a very strong language there.
Jon: Python is also very strong in the standard library that it brings. Rust to me feels,
Jon: it is less strong there. And as a result, you end up with lots of small crates
Jon: of kind of unknown origin, essentially. So it makes it harder.
Jon: But I think part of this will come with maturity, right? There are certain superstar
Jon: libraries that everyone knows about, clap and serde and various others.
Jon: But in configuration file handling, for example, I shudder to think how many
Jon: implementations there are of parsing a Toml config file on crates.io, right?
Jon: Which seems yeah maybe that's useful i it seems like an interesting like bifurcation
Jon: of effort in a sense for something that's so common yeah.
Matthias: It's a double-edged sword somehow on
Matthias: one side you also expose yourself to a
Matthias: lot of you know side effects and maybe you also accumulate a lot of functionality
Matthias: in the standard library if you follow this batteries included approach but on
Matthias: the other side you also push the burden more to the users which now have to vet smaller crates.
Jon: Yeah or choose not to bring a crate in and just vendor some of that code right
Jon: is always an option it's an approach i would generally take myself i think you
Jon: know if you often you can be at risk of bringing in thousands of lines of code
Jon: from a library where you need like a 30 line function from it yeah these are
Jon: all to a limited extent like this is all within a developer's control at the end of
Jon: day i feel like the it's about the community's approach to it that can kind
Jon: of signpost people to the right to make the right decisions more often it's.
Matthias: True i always wonder why this is not done more often especially for the smaller
Matthias: functions and libraries that you use as long as the license is compatible.
Jon: Exactly.
Matthias: We're getting close to the end. And traditionally, the final question is,
Matthias: what's your message to the Rust community?
Jon: I think my message to the Rust community is twofold. One is we're hiring Rust developers.
Jon: So, you know, reach out if that's of interest to you. And secondly,
Jon: if you've got a really interesting Rust project and you think it's a great fit
Jon: for some of the work that we're doing and you want to have a conversation, then reach out.
Jon: I'm open to ideas. I don't know all of the high-quality Rust projects out there.
Jon: And if there are implementations of things that are very widespread that you
Jon: think we should be using, then let me know.
Matthias: John, thanks so much for the interview. And thanks to Canonical for supporting the Rust ecosystem.
Jon: Thank you very much. It's been a pleasure.
Matthias: Rust in Production is a podcast by corrode. It is hosted by me,
Matthias: Matthias Endler, and produced by Simon Brüggen.
Matthias: For show notes, transcripts, and to learn more about how we can help your company
Matthias: make the most of Rust, visit corrode.dev.
Matthias: Thanks for listening to Rustin Production.