Matthias: It's time for another episode of Rust in Production, a podcast about companies
Matthias: who use Rust to shape the future of infrastructure.
Matthias: I'm Matthias Endler from corrode and today's guest is Charlie Marsh from Astral.
Matthias: We talk about improving the Python ecosystem with Rust.
Matthias: Charlie, thanks for being a guest. Can you say a few words about yourself and
Matthias: about Astral, the company you work for?
Charlie: Yeah, of course. Thanks so much for having me. first of all.
Charlie: My name is Charlie Marsh.
Charlie: I run a company called Astral. We build high-performance developer tools for the Python ecosystem.
Charlie: So we're best known for two tools, RUF, which is a sort of combined linter formatter
Charlie: code transformation tool.
Charlie: You could think of it like kind of Rust format and Clippy together,
Charlie: which is again for Python code, but written in Rust.
Charlie: And then UV, which is our project manager, Python package manager,
Charlie: Python tool chain manager. It's, it's, it's, you could think of it a little
Charlie: bit like cargo and Rust up.
Charlie: So it tries to bootstrap or will bootstrap Python for you and then help you
Charlie: manage your dependencies, install things, lock them into lock files and reproducible
Charlie: versions and all that kind of stuff.
Charlie: So yeah, everything we've built so far is open source and written in Rust.
Charlie: We're a team of about 15 people spread from the Pacific, like Pacific time in the U.S.
Charlie: We have people in pacific time central time eastern time
Charlie: and we have one person in the uk we have four people in cet or like germany
Charlie: switzerland the netherlands and then one person in india so not only are we
Charlie: like remote we're like very distributed kind of like a lot of open source and
Charlie: we spend all our time basically writing rust to try and make python better i.
Matthias: Heard about astral the first time when you published ruff but it took off with uv i would say like.
Charlie: There was.
Matthias: Definitely a huge sympathy for what you do in the packaging space as well did
Matthias: you see that as well from a company perspective that there was a huge growth.
Charlie: I think so. Yeah. I mean, I think, you know, we started with Ruff.
Charlie: I started working on Ruff before the company existed.
Charlie: And it was just something I was building.
Charlie: Well, for a lot of reasons, but largely because I saw these were problems that
Charlie: I had experienced in my own projects.
Charlie: And I was like, what would it be like if I wrote, you know, Python tooling and Rust instead?
Charlie: And that was sort of the genesis of Ruff. and ruff
Charlie: grew extremely quickly like we had you know
Charlie: decades old projects like scipy and
Charlie: stuff like adopting it while it was still like what i would consider to be very
Charlie: unstable so it was clear that people wanted you know something in this in this
Charlie: arena of faster python tooling so ruff grew very fast and you know we were
Charlie: we were seeing what was happening with ruff and we were like well.
Charlie: As a company and in terms of like the problems we're trying to
Charlie: solve like we're not just we're not trying to
Charlie: build just that like we want to build a python
Charlie: tool chain effectively like we want to solve the hard
Charlie: problems in python and for me like packaging was
Charlie: kind of like if you want to be like the python tooling
Charlie: company which aspirational i guess we'd like
Charlie: to be then like you have to work on packaging i think
Charlie: because it's it's the thing that everyone has
Charlie: trouble with and the thing that has been so many people have tried to
Charlie: solve and and done good work on but but i
Charlie: don't think anyone would have would consider it solved and so
Charlie: you know i i kind of knew from early on that we wanted to do something in packaging
Charlie: and before we released it i definitely felt nervous in that ruff had grown
Charlie: well and people really liked it and it was it was kind of felt like it was going
Charlie: to be a tough act to follow like we don't want to be a one-hit wonder you know
Charlie: with the tools that we built.
Charlie: Like I wanted to build something that hopefully would be as exciting and grow
Charlie: as well and have as much of an impact as Ruff.
Charlie: And I think UV has actually really surpassed that in a lot of ways.
Charlie: It's like, I think the impact it's had on Python has probably been more significant.
Charlie: And I think like the, not that I don't love Ruff.
Charlie: I mean, it was my first big project and we still like half the people on the
Charlie: team are still working, are working on Ruff and the static analysis tooling.
Charlie: It's like a big focus area for us but i think i've been really amazed by how
Charlie: quickly uv has had the impact that it's had like we released it in february
Charlie: of last year so it's just it just is one turned one year old and in that time it's grown to like.
Charlie: I think last I checked, it's like a little over 12.5% of all requests to the
Charlie: Python index come from UV, which is like over 200 million requests a day, which is wild, right?
Charlie: And I spend a lot of my time talking to companies. There are tons of like 1
Charlie: billion, 10 billion, even $100 billion companies that are using this thing in
Charlie: production and have been for a long time.
Charlie: So just like the way it grew i think really surpassed
Charlie: my my expectations and i feel
Charlie: like we now have a a platform to like keep hopefully making python better but
Charlie: but i think you're right that like uv i think we shifted into a little bit of
Charlie: another gear we spread beyond just being like the company was just ruff before
Charlie: and now we are like python tooling and we try to solve like a much wider surface area of problems now.
Matthias: You said that ruff was your first major rust project
Matthias: maybe the first actual project that you tried after
Matthias: like learning rust i'm not sure about this but it certainly
Matthias: was big milestone for you in your rust journey but then
Matthias: you compared it with uv and did you already learn a few things that you wanted
Matthias: to avoid with uv that you did with ruff and also did you set up the uv code
Matthias: base early on for such growth.
Charlie: Yeah so So I would say I really learned Rust in the process of writing Rust.
Charlie: I had done some Rust at my previous job, but someone else on the team introduced
Charlie: Rust, really great engineer.
Charlie: And when I was contributing to that code base, I was mostly trying to get in
Charlie: and out as quickly as possible because I didn't really know Rust.
Charlie: I was like, I need to fix a bug. How do I get this to compile?
Charlie: I wasn't really taking the time to learn it. I was just using it when I had
Charlie: to. So I had some Rust exposure.
Charlie: But part of why I worked on ruff in the first place was I felt like I wanted
Charlie: to really learn rust. And I thought that in order to do that,
Charlie: I had to build something from scratch. And I had to like, it's just kind of
Charlie: the way that I like to learn. I was like, I need to build something.
Charlie: And like, even if that requires like, wasting a ton of time trying to understand
Charlie: like lifetimes, like maybe I'll burn like two days trying to get this to compile.
Charlie: But like, that's kind of how I learn is just like, I have to build things and
Charlie: like fight through the problems. And I wanted to learn the ecosystem and just
Charlie: the tooling and everything.
Charlie: So I did really learn Rust in the process of writing ruff.
Charlie: I think that showed, like there was probably still shows a little bit today,
Charlie: like, you know, especially early on, there were a lot of things I was doing
Charlie: that were just that I would now, I mean, this is, this is part of personal growth, right?
Charlie: There's lots of things I was doing that I would now look at and laugh about
Charlie: and say, that's obviously not the right way to do it.
Charlie: Whether it was like silly performance things or the way I was structuring the
Charlie: code or or I don't know, anything.
Charlie: By the time that we started working on UV, I think I felt a lot more comfortable in Rust.
Charlie: And I'd also learned more from looking at other projects too,
Charlie: I think. And we had kind of evolved Ruff a little bit over time.
Charlie: So for example, like we have a really, a fairly wide like crate structure.
Charlie: Like we just create a lot of crates.
Charlie: Like if you go and open up Ruff or UV, the crate subdirectory,
Charlie: there's like in UV, there's probably.
Charlie: There's at least 20 crates, maybe like 30. So we just create like lots of crates.
Charlie: And that was something that I started to do in ruff because our,
Charlie: our build and compile times got really bad.
Charlie: And it became like, if I just put everything in one crate, it was like our build
Charlie: and compile times got worse and worse. And the development loop was painful.
Charlie: And, you know, at the time I was reading a
Charlie: lot about how sort of like crates are like the atomic unit for like
Charlie: Rust-C and like you can parallelize across crates and you need
Charlie: to be thinking about like how your crate graph looks so
Charlie: that there's a lot of like fan out and stuff and i was like okay we're going
Charlie: to start creating like lots more smaller crates and we're going to sort of change
Charlie: the atomic unit of like what the code base is from like the module to the crate
Charlie: and so we started carving out like lots of little crates and we did basically
Charlie: the same thing and for example in ruff like you know.
Charlie: The core linter crate like doesn't
Charlie: depend on clap for example like it doesn't have it doesn't depend on
Charlie: any of the cli stuff the cli stuff is in a separate crate
Charlie: that depends on the linter crate and you just try and get this structure in this
Charlie: organization or like the parser and the ast those are
Charlie: all their own crates so we can if you just need to test the
Charlie: parser right it's really fast to compile and build or if people
Charlie: if other people even just need to pull in the parser as a library which
Charlie: some people do it's much easier so by the time we got
Charlie: to uv i think we had ironed out a
Charlie: lot of things that made it so we put we just
Charlie: put a lot of stuff in place from the start like we just had a better separation
Charlie: in terms of what went into what crates i think
Charlie: we'd ironed out a lot of our workflows for example like what's our clippy configuration
Charlie: how do we do how do we pin the rust toolchain version like how do we install
Charlie: rust in ci like there were just all these little things that we were able to
Charlie: just copy over from ruff and i think that made things a lot easier.
Charlie: At the same time, there were still a lot of design decisions in UV architecturally
Charlie: that were pretty different because the design of a package manager and the design
Charlie: of the linter are just the things they need to do are very different.
Charlie: So, for example, UV has a whole networking stack, right?
Charlie: It needs to make lots of HTTP requests and the linter doesn't need to do any of that.
Charlie: So suddenly we had to think about requests or basically the whole HTTP stack.
Charlie: We had to think about like open ssl we had to think about all these system dependencies
Charlie: like git that was another two like we support git dependencies so suddenly you
Charlie: have to think about how you depend on git i think there was just a lot of complexity
Charlie: that came with building a package manager that didn't we didn't necessarily
Charlie: have to encounter when we were working on the linter,
Charlie: which is good because i kind of like encountered those problems over time when.
Matthias: I understand you correctly to summarize you didn't have a rough start with uv because.
Matthias: You could copy over.
Matthias: A lot of the let's say templates or a lot of the best practices from the previous project.
Charlie: At least what we thought were best practices yeah and i guess to some degree
Charlie: we still think our best practices but when.
Matthias: You used uv when you started with uv did you already compartmentalize some of
Matthias: the functionality into smaller crates somewhat subconsciously did you say oh
Matthias: yeah this definitely goes into a separate crate now or did you still start with a single large crate.
Charlie: No i think we were like compartmentalizing stuff basically from the start and
Charlie: for example like you know we had one crate for like parsing python versions
Charlie: there's a whole spec around versioning that requires,
Charlie: a lot of there's a lot of intricacies to it so that's its own crate we had one crate for like,
Charlie: parsing requirements txt files that's another thing that's common in
Charlie: python that's there's actually no spec for that that's like implementation defined
Charlie: and so we have like one crate for that you know we have a crate for
Charlie: creating virtual environments we had a crate for the cli we
Charlie: had a crate for the resolver so yeah we broke things down i think pretty early
Charlie: on and that i mean that's generally served us very well so i'm a big fan of
Charlie: that structure i think it's like much the trade-off is really different when
Charlie: you're publishing a library like we don't publish any of our we don't,
Charlie: we don't publish any of anything in rougher uv as
Charlie: a library like we don't publish to crates io any of
Charlie: the stuff that we build right now like the public api for our stuff tends to
Charlie: be the command line so the cli the command line interface so like if we had
Charlie: to publish things i think we would have to think a lot more carefully well there's
Charlie: all sorts of considerations that come with publishing that you don't have to
Charlie: think about if you aren't publishing.
Charlie: But one is, if we had a really granular create structure, I think it becomes
Charlie: a bit harder for people to use your... There's a lot more of a maintenance burden,
Charlie: and it's also harder for people to use and compose your things.
Matthias: But you don't go and...
Matthias: Introduce a crate for types specifically because that's one thing that i see some,
Matthias: some people do some companies do that they have
Matthias: a types crate where they put everything that is related to
Matthias: you know their basic types into one thing
Matthias: and then at the end they don't
Matthias: really get a lot of benefits from using a workspace because you need that types
Matthias: crate anyway in all of your other crates and so it kind of goes against to what
Matthias: a workspace is about in my opinion but i want to hear it from you like what
Matthias: are some anti-patterns for building workspaces.
Charlie: Yeah so like i guess
Charlie: yeah we try not to do things like like
Charlie: that i don't know what we would call that pattern but like
Charlie: create a crate that's just kind of like used everywhere there's
Charlie: maybe some of that but we try to generally avoid it
Charlie: i think the other anti-patterns are like you can
Charlie: make your crate structure like too granular like if
Charlie: you find yourself creating a crate for that's like
Charlie: one function and sometimes
Charlie: you'll find yourself doing that because you have like circular dependencies
Charlie: or something in the dependency chain that's requiring you to do this
Charlie: like i have two crates that need this functionality and they can't depend on
Charlie: one can't depend on the other or something and then you find yourself putting
Charlie: a function into like like it's just a crate that's like a function at that point
Charlie: you've probably gone too granular and you kind of need to rethink like the organization
Charlie: because there is some overhead to having all these crates and like,
Charlie: you know it's not always like super fun to maintain i think
Charlie: a couple things that we do though that are kind of nice for this
Charlie: is like we prefix all the crates
Charlie: in the workspace with uv or with ruff so like it's easy to differentiate what's
Charlie: us create in the workspace versus what's a third party crate like all of our
Charlie: crates are like uv virtual env or like uv resolver or like whatever else that's
Charlie: pretty nice and then the other thing is we.
Charlie: We declare them all in the workspace root so that every crate that depends on
Charlie: other crates in the workspace can just use workspace equals true.
Charlie: This is a little bit nuanced and maybe hard to visualize, but it basically means
Charlie: that in the cargo.toml for all the crates that are in the workspace,
Charlie: it's very obvious what else is in the workspace.
Charlie: Another thing that we do is we actually try
Charlie: to use workspace equals true for all basically
Charlie: all dependencies in the workspace so we
Charlie: put everything that we depend on in the root cargo toml
Charlie: like the workspace root and then we use workspace equals true
Charlie: everywhere and that tends to simplify
Charlie: things a lot it basically means we have like one dependency specifier for
Charlie: request for example and a
Charlie: common set of like default extras or no default extras or
Charlie: sorry not extras features extras is the python version is
Charlie: the python analog to to features but like no default features
Charlie: or you know or whatever else we have like one dependency declaration for
Charlie: everything that we need and then all the workspace members their cargo terminals
Charlie: are just very straightforward it's like the things they require with workspace
Charlie: equals true so we don't have to think about oh do we have request dependency
Charlie: specifiers across like 10 or 15 different crates we just have one definition
Charlie: for it so like that's also i think been a really handy thing right.
Matthias: Workspaces seem to be one of these things that i keep missing in other ecosystems
Matthias: and i can't remember if there's even another language that has such a feature from the top.
Charlie: Of my head yeah think about one i mean we've thought about
Charlie: it a fair amount in python so like uv
Charlie: has a has workspace functionality that's like very
Charlie: much modeled after cargo um so
Charlie: you have a root it defines the members
Charlie: we use very similar actually like members and excludes syntax
Charlie: and there's nice things like that you know
Charlie: we you can do like uv run dash p just like you can do
Charlie: cargo run right so like we've we've just like
Charlie: copied a lot of things from cargo because it's great
Charlie: and we're like we want to have those things in python
Charlie: and for workspaces i think that's
Charlie: worked really well it's been very interesting to introduce them because for
Charlie: a lot of people it's fairly new concept like when they come to python right
Charlie: like like most people who use uv i've never used Rust right they've never
Charlie: heard of a cargo workspace before and like why should they
Charlie: you know if they're just working in python so it's been interesting to try and
Charlie: like i communicate that and like help people understand like what it's for and
Charlie: like why you might use it and like what what an example workspace might look
Charlie: like in practice there are some things that we miss that are hard to get a little
Charlie: bit hard to get like without standards like for example.
Charlie: We don't support, I just spent all this time talking about this incredibly boring
Charlie: workspace equals true thing that all your listeners are probably like,
Charlie: why is he so much time talking about that?
Charlie: But like, we can't really do that in UV because like there isn't really a way
Charlie: to express that in the standards.
Charlie: And so, you know, there are some things that we have to kind of either can't
Charlie: support or have to get creative about how we support.
Charlie: But I think the workspace concept is excellent. And like, I'm really glad that
Charlie: we made it such a first class thing in UV.
Matthias: Now, I'm pretty sure that we could
Matthias: talk about workspaces for hours because there's a lot of nuance to it.
Matthias: And I think a lot of people that haven't tried it themselves,
Matthias: they just don't know what the fuss is all about.
Matthias: But I do believe that there's more to it than just the term.
Matthias: Now, one other thing that you mentioned, though, which is very close to my heart
Matthias: is parsing stuff, especially the requirements txt, which is unspecified, I heard.
Matthias: Now, isn't it super simple?
Charlie: Yeah isn't.
Matthias: It super simple like someone might hear this and say oh yeah i just opened my
Matthias: editor i string split every line on the equal sign and that's my requirements
Matthias: txt parser why is that not the case.
Charlie: Yeah so for requirements txt it's like
Charlie: this is basically this is a file format that
Charlie: exists for pip really pip is
Charlie: like the i guess sort of how should
Charlie: i describe it like the reference implementation for a lot of things
Charlie: it's really been like the python package installer for a long
Charlie: time yeah um and requirements txt
Charlie: is a file format that exists for pip and you know
Charlie: kind of the way that you can think about it which
Charlie: maybe people don't think about it this way is like it's
Charlie: basically like each line is a command line argument because you
Charlie: can not only put requirements in there you can also put command line arguments
Charlie: and settings yeah which is interesting so like on pip and pip you can pass an
Charlie: index url on the command line which is like what registry should i use for fetching
Charlie: packages you can actually put that in requirements txt too you can just do dash dash index url.
Charlie: You can also like embed like you can also nest them so like within a requirements
Charlie: txt you can do dash r and point to a different requirements txt and then it's
Charlie: sort of like it gets inlined roughly so there's a lot of complexity to the stuff
Charlie: that's in requirements txt and that people don't really think about.
Charlie: And there are also lots of very subtle behaviors where, especially for us, we often have to decide,
Charlie: well, what do we want to do? Do we want to be like bug for bug compatible with
Charlie: pip or do we want to do things like slightly different?
Charlie: And a lot of these are edge cases, but there's just a lot of nuance to it.
Charlie: Like in a requirements.txt file, you can have, I guess what I would call like a named dependency.
Charlie: Like you could say like flask and then the version that you want,
Charlie: or you could do flask at, and then the URL that you should fetch it from or
Charlie: like a Git repository or something like that.
Charlie: But you can also just do the URL or just do the Git repository.
Charlie: And it turns out that in PIP, there's slightly different behaviors around how
Charlie: those things are parsed, like the URL, if it's just the URL versus if it's after a name dependency.
Charlie: And how white space is handled, like error recovery, all this stuff is just a little bit different.
Charlie: And so over time, we've had to add, basically, the only way to know if we're
Charlie: doing the right thing is we just see what PIP does. and then we try to mimic that to some degree.
Charlie: Or if we think we can do things that are unambiguously clearer, then we'll do that.
Charlie: But yeah, that one's especially hard. It's a little bit easier for other things
Charlie: like Python version specifiers where there's a clear standard on how these things should be parsed.
Charlie: And if we see something that's not clear, we can actually ask about it through
Charlie: the standards process and be like, how should this be handled?
Charlie: Like blah, blah, blah, blah, blah.
Matthias: In essence, you make the standards stronger for everyone
Matthias: which is great what kind of piqued
Matthias: my interest there was did you have to
Matthias: go and read the pip source code to be able to understand what's going on because
Matthias: this is a standard that sort of developed over 30 years i think python is 30
Matthias: years by now it's from 1994 as far as i remember so and did you really have
Matthias: to read the source code yes.
Charlie: Sometimes we definitely have to go read source code,
Charlie: and which is fine like i don't i don't
Charlie: really mind that like as long as we can figure out like why a certain behavior exists
Charlie: in a certain way and like what it's motivated by but yes we often have to go
Charlie: read the source code to understand like how does this tool handle this case
Charlie: and like and and also why right like a lot of the time you're trying to understand
Charlie: why people made certain decisions not just how it works that.
Matthias: Means you don't even you don't only have to read the source code you also have to read the history the.
Charlie: Yes version history yes but do.
Matthias: They have proper tests at least where you can go through and maybe implement
Matthias: these tests in in your implementation.
Charlie: Yeah they do have tests yeah it's not always like super
Charlie: straightforward to take their tests and move them
Charlie: over to uv like a lot of the times i don't
Charlie: know just the way that you write tests in like python and rust often tends to
Charlie: be quite different like or at least the way that we write tests
Charlie: tends to be quite different like you know like in python like
Charlie: mocking is very popular for example and like we
Charlie: don't really have we have like almost no walking in uv like
Charlie: almost all of our tests i think
Charlie: are what you would probably consider like integration tests
Charlie: like the whole way that we test uv is we
Charlie: actually run real scenarios against
Charlie: a real package registry and it's almost all based on snapshot testing so like
Charlie: we snapshot the cli output a lot of the time that's like how we detect if things
Charlie: are working correctly or not is we snapshot the cli output or we snapshot the
Charlie: lock file or we snapshot things like that but like the vast majority of the
Charlie: way that you get tested is.
Charlie: Integration tests of running real
Charlie: commands on the cli and snapshotting the output and that's like we have,
Charlie: a lot of tests and they all run through that.
Charlie: So it's a little different sometimes because if I go and look,
Charlie: we do have obviously more traditional unit tests for requirements.txt parser.
Charlie: That's something that's very testable, right? And we do have tests for that.
Charlie: But often if it's more complex scenarios around how PIP behaves versus how we
Charlie: behave, it's a little bit harder to shoo it into what we have.
Matthias: Do you always run the full test harness?
Charlie: In CI?
Matthias: Yeah, or maybe locally as well, but I wonder that's probably too much.
Charlie: Yeah, we do always run, we almost always run the full test harness.
Charlie: So like we just, we skip it for changes that are purely documentation.
Charlie: Which we just detect with some file filters in GitHub Actions,
Charlie: but otherwise we run the full test suite.
Charlie: Every change we run it on Linux, macOS, and Windows.
Charlie: And that's because we do like
Charlie: that that that's not as critical for ruff but
Charlie: it's very critical for uv because with the
Charlie: package manager just like more things tend to differ across
Charlie: platform and more behaviors differ across platform and
Charlie: so we always test everything on linux mac os and windows
Charlie: i guess one interesting piece is like we we
Charlie: build uv for like a lot of different targets so you
Charlie: know when we publish to py pi or publish anywhere
Charlie: or publish to github releases like we're
Charlie: building like 15 probably somewhere between like
Charlie: 15 and 20 different builds or maybe is it 15 somewhere between 10 and 20 let's
Charlie: just say different build variants and so that's like linux x86 you know dynamically
Charlie: linked against glibc linux x86 sorry linux x86 musl linux arch GWC, Linux ArchMuzzle.
Charlie: We build for some more obscure platforms that are supported in Python,
Charlie: like PowerPC and S390X and stuff like that.
Charlie: We build for ARM and x86 Windows, ARM and x86 Mac OS.
Charlie: So we just have a lot of different builds. And...
Charlie: I mean setting first of all setting those up to actually
Charlie: build is fairly complex and so if you ever find yourself needing
Charlie: to do that you should look at our ci because we've
Charlie: you know we've just figured out how to build a rust project across all
Charlie: those different like a lot of those are cross compiled right and
Charlie: we've just figured out how to like build those across lots of different machines what
Charlie: is our ci our ci your ci
Charlie: oh yeah yeah just go look at our.github folder and like
Charlie: take stuff from it um but the
Charlie: other piece is we actually like we don't we
Charlie: rerun all of those builds whenever certain files
Charlie: change so like if we add a new dependency we rebuild across
Charlie: all those machines which is fairly because
Charlie: we've otherwise we've run into situations where we add
Charlie: a new dependency and then we go to release and then i
Charlie: don't know the arm musl build fails for some reason that we don't or the windows
Charlie: arm build fails for something some reason that we don't fully understand and
Charlie: so now we run those that i guess the only nuance is we we do rebuild everything
Charlie: on every platform if we for example change our dependencies yeah.
Matthias: I i think this is kind of where the rust build system also has its limits at
Matthias: this point in time because especially when you talk about like feature flags
Matthias: and maybe optional dependencies dev dependencies there's a lot of,
Matthias: loose ends and sometimes you
Matthias: can't really specifically and exactly say which dependencies you want to enable
Matthias: for which platform and you know on top of it you have various system dependencies
Matthias: and the buildrs files to go with it and sometimes you don't want to have a system
Matthias: dependency on a certain target and so on.
Matthias: And it's very hard to work around these limitations.
Charlie: Yeah. I mean, a lot of the complexity has come from things like accelerators,
Charlie: sorry, not accelerators, allocators.
Charlie: Like, you know, like we use, I don't even know how to pronounce these things, jemallo, right?
Charlie: Like on most platforms, but like it just like doesn't work on like,
Charlie: it like doesn't compile on like some platform.
Charlie: So we have to have like a bunch of build configuration around that.
Charlie: The other one that caused a lot of trouble is.
Charlie: Was our like zlib implementation like we
Charlie: wanted to use which is like for decompression like we
Charlie: wanted to use so like
Charlie: by default you get this pure rust minis oxide implementation
Charlie: if you just use like like flate
Charlie: and like reqwest like if you just use like the reqwest crate to
Charlie: like decompress stuff you get like this pure rust implementation by default i
Charlie: think but there's like this uh zlib-ng
Charlie: version that you can use which is a lot
Charlie: faster but it requires like it adds
Charlie: like a CMake dependency and it needs to be built it's like
Charlie: very hard to build and so we could never get that to build on certain platforms and
Charlie: so we had to have configuration around that around like where
Charlie: do we enable it and where do we not we actually totally tore
Charlie: that out recently and moved over to zlib-rs
Charlie: like the pure rust implementation of a lot
Charlie: of these zlib optimizations which in our benchmarking at least
Charlie: is actually both faster and it's like way easier to build so
Charlie: when you get something that's pure rust like it's just
Charlie: like it was it's simplified things dramatically we just like tore out this
Charlie: like CMake dependency and like all this stuff and
Charlie: sudden and got rid of all this configuration because now we can just use
Charlie: like the faster easier to build more portable like rust
Charlie: version on all platforms so a lot
Charlie: of it comes from you know if you're trying to do things that are more like customized
Charlie: or like bleeding edge like that like you use really fast system dependencies
Charlie: or like allocators like you end up running into these kind of configuration.
Charlie: Problems but we've we've we've tried to make that simpler over time must.
Matthias: Have been such a relief to pull out that cmake.
Charlie: Oh it's amazing it's amazing i actually tried to do it like
Charlie: anyway sort of like sorry i tried to do it like a few months ago and
Charlie: then we realized that they were using compile time feature
Charlie: detection for a lot of the optimizations which isn't
Charlie: great for us because it meant that like on x86 things
Charlie: were actually a lot slower because we have to build for
Charlie: like a common cpu target like and they
Charlie: recently re-released with like runtime detection and
Charlie: i was like okay cool we're doing it again so i like i i redid the change like
Charlie: twice and and like redid all the benchmarking on like my windows machine and
Charlie: on like on mac os for arm and all that kind of stuff so yeah but like what yeah
Charlie: when you can get rid of stuff like that it's like immensely helpful i realized whenever i.
Matthias: Read one of your posts or whenever i see you talk
Matthias: i realized that you care a
Matthias: lot about performance and you benchmark
Matthias: a lot and meticulously and do you have any any tips for people that want to
Matthias: make their rust code even faster from like doing it so often there must have
Matthias: been patterns that evolved over time what is a.
Charlie: Good benchmark.
Matthias: What's a bad benchmark what are some good libraries out there in general what
Matthias: are best practices what don't you measure for example that sort.
Charlie: Of yeah there's like so much to share i mean yeah,
Charlie: It's like a whole, actually like learning the tooling too around benchmarking is like itself a skill.
Charlie: And like, we have people like, there are certain things that I only know how
Charlie: to do on Mac OS and I don't really know how to do on like Linux or windows.
Charlie: And like, there's other people on my team who would like, would like know how
Charlie: to do these kinds of, like, I know how to use instruments, for example,
Charlie: on Mac OS. And that's like its own set of things.
Charlie: You know, I think for us, there's like a bunch of different forms of benchmarking.
Charlie: One is like micro benchmarking which is i guess
Charlie: like generally like the kind of thing that you could do with criterion or
Charlie: a similar crate where you're like running typically fairly
Charlie: well isolated code on test inputs
Charlie: and running like thousands and thousands and thousands or millions
Charlie: of iterations of it and trying to detect very small changes that's
Charlie: that is an incredibly useful thing if
Charlie: you can isolate your code in that way so like if i'm trying
Charlie: to figure out for example if like there's two different ways
Charlie: that i could write this function that like parses this very simple string
Charlie: in the simple way and i'm like i want to know which one's
Charlie: the faster way then like i'll do something like that i'll be like i'll try
Charlie: and just isolate like two methods and then use like
Charlie: criterion or something similar we also do micro
Charlie: benchmarking on ci so we use a tool called cod speed which does like continuous
Charlie: benchmarking continuous profiling that again is really good when you have things
Charlie: that and they run it on valgrind so like basically like if you just try and
Charlie: do this kind of benchmarking on like a GitHub Actions machine,
Charlie: it tends to be extremely unreliable.
Charlie: You have to have like a very high error tolerance. Like when we very early on
Charlie: in Ruff, when we started adding continuous benchmarking, we did on GitHub Actions
Charlie: and we were just regularly see like five plus five to 10% fluctuations,
Charlie: even for no op changes, because those machines are very noisy.
Charlie: So CodSpeed kind of solves this problem because they run all your stuff under Valgrind.
Charlie: So they get like a much cleaner, basically a much cleaner snapshot of like what's
Charlie: actually changing in in terms of performance there's some nuance to it but like
Charlie: basically if you have code io can make things like really messy with benchmarking
Charlie: in general but if you have if you can write benchmarks that don't have any io
Charlie: that are very pure like pure cpu,
Charlie: you can fit them into that kind of micro benchmarking framework it's like incredibly
Charlie: useful especially in ruff where we have a lot of that we run i mean we just
Charlie: run that in ci it catches things all the time real positive changes show up
Charlie: in you know get flagged as positive changes like it's it's it's a fairly high
Charlie: signal um so that's like if you can do that that's great,
Charlie: We can't always do that in UV. We often do more like, I just sort of make these
Charlie: words up, like macro benchmarking.
Charlie: I don't know if that's a word, but that's like, I use Hyperfine usually and
Charlie: I will compile to UV release builds and I will run some operation over and over
Charlie: and I will try to see if I can detect a difference.
Charlie: And I'll typically try and minimize IO as much as I can, but maybe it has to
Charlie: read from disk for the on-disk cache.
Charlie: And like there, it's like, you're trying to, you know, you're trying to find much smaller.
Charlie: It either needs to be really obvious or very
Charlie: consistent in order for you to catch anything this is like
Charlie: one form of benchmarking is oh it made everything 10
Charlie: faster then it's like very obvious that it made
Charlie: things faster and you can catch it in that kind of benchmark the other.
Charlie: Pieces that you can look at which aren't so much are a
Charlie: little bit harder to do direct comparisons but we will
Charlie: do a fair amount of like there's a great tool called sampling
Charlie: which we use for it's a sample sampling based
Charlie: profiler so like you can just run a uv command you
Charlie: basically you just prefix it with sampling and then it opens up
Charlie: a flame graph in your browser that has
Charlie: you know all the like all the flame graphs
Charlie: of everything that got called and where you spent time and so
Charlie: often that's often a way that it's a
Charlie: little bit harder you can't always tell
Charlie: just from a flame graph if you made your whole program faster right
Charlie: you can tell if you you ran the
Charlie: flame graph you found something that was taking up a lot of time and then you make
Charlie: changes and then it's gone like that's that's good
Charlie: that means like you got rid of that time but like that alone
Charlie: is not enough to tell that your program got faster right that's
Charlie: just that you removed that thing that you were spending time
Charlie: on but maybe it went somewhere else like blah blah so we'll often
Charlie: do that as a way to diagnose issues and if something's slow then we'll be like
Charlie: let's run it under sampling and like see where we're spending time for example
Charlie: and so that's there's like certain tools you want to to diagnose issues and
Charlie: then certain tools that we often use to try and like confirm findings or understand
Charlie: if there are regressions or anything like that yeah and.
Matthias: In my mind when you said sampling profiling and flame graphs i was sort of hoping
Matthias: that the tool would kind of update the chart as the program runs i'm not sure
Matthias: if that's the behavior of that program.
Charlie: I don't think sampling does that i think there might be a way to like compare to
Charlie: flame graphs and like cod speed tries to do this they'll
Charlie: try to diff the profiles it works okay sometimes it's hard for them to i mean
Charlie: it's like inherently that seems like a very hard problem like they have to try
Charlie: and align like what two function calls are like the same like it's not necessarily
Charlie: always trivial because code is changing but like there's some stuff like that.
Matthias: Just for context, maybe someone might not know, a sampling profiler.
Matthias: How would you describe that in a sentence or two?
Charlie: I mean, my understanding is it basically watches your program execute and it samples at some rate.
Charlie: So it takes a bunch of samples as your program is running to try and get a representation
Charlie: of where you're spending time.
Charlie: So maybe it takes, just for stupid numbers, maybe it takes a thousand samples.
Charlie: And like in or let's say 100 samples and
Charlie: in 10 of those you're in this function then you know it would say hey
Charlie: i found this we were running this function 10 of the time that i sampled
Charlie: and here is where it was being called from and like here's how you're spending
Charlie: time so it basically watches your program execute and tries to figure out based
Charlie: on probabilistically by sampling the behavior where you're spending your time
Charlie: that's my understanding i never built one of these so it actually might work
Charlie: totally differently than that but like but like that's my understanding that's.
Matthias: Also my understanding.
Charlie: Which is a little different than like in Valgrind for example like Rust,
Charlie: I know Rust C I think does this too like you can do profiling based on instruction
Charlie: counting so like you actually look,
Charlie: like the instructions that are generated or executed or whatever.
Charlie: And you, you try and do something that's, you don't look at the behavior of
Charlie: the program. You don't like run the program.
Charlie: You try to look at like, or sorry, you run the program, but you look at a different
Charlie: thing. You're not, you're not focused on time, like wall time.
Charlie: You're looking at like what instructions are being executed,
Charlie: how many instructions are being executed.
Charlie: So it gives you a little bit more of like a quantitative look i guess at like
Charlie: where you might be spending time and i believe rust c does has continuous benchmarks
Charlie: that look at i actually i could be totally wrong there's something that i've
Charlie: been shown before in the rust ecosystem that does continuous benchmarking based
Charlie: on instructions instruction counting i.
Matthias: I was kind of thinking that cargo flame graph would do that in comparison to
Matthias: sampling i never used sampling before but my go-to tool for flame graphs is
Matthias: always cargo flame graph i'm not sure if you've used that one.
Charlie: I think i have but it's been a while yeah and you find simply.
Matthias: To be more ergonomic.
Charlie: I found it to be easier yeah yeah it's just
Charlie: like what we tend to use on the team but like you know i'm sure
Charlie: that there are lots of good options but it's just like
Charlie: it just works seamlessly across mac os and linux which is really nice so like
Charlie: a lot of these tools will be like it's maybe hard to get them to work on one
Charlie: or the other for example not always i don't know if that's true of cargo flame
Charlie: graph i just mean like of things we've used in the past sometimes it's like
Charlie: oh this works great on linux but not really on mac os and i think samply tends
Charlie: to work really well on both cargo.
Matthias: Flame graph gives you an svg file that you can open in your browser and it's sort of.
Charlie: Interactive yeah yeah yeah pretty basic as well.
Matthias: I don't know if there's probably support for the chrome profiler which also
Matthias: supports flame graphs but i'm not sure.
Charlie: Yeah yeah the thing so the thing that samply gives you in the browser you can
Charlie: like click into the flame graph you can filter by name you can filter traces
Charlie: by name and stuff it's like it's pretty nice but but again like some things
Charlie: are just really hard to understand in a flame graph like you know if you have a lot of like,
Charlie: like UV is very async and like we have lots, you know, we have like different
Charlie: stuff going on. And so sometimes it's hard to tell like where time is actually being spent.
Charlie: Like it's really things that are spending lots of CPU time are always nice because
Charlie: they're obvious and you can find them and fix them.
Charlie: But things where it's like, oh, the scheduling is like slightly off or like
Charlie: we're waiting here, but like blah, blah, blah.
Charlie: Like those are tend to be harder, like more pernicious bugs to find. by.
Matthias: That definition isn't it true that profiling or benchmarking ruff is easier
Matthias: than uv because ruff is inherently cpu bound you do a lot of computations i'm assuming.
Charlie: Yeah i think it's i think it's much easier yeah like ruff does
Charlie: have a fair amount of uh it does have io right in
Charlie: the sense that we're reading files from disk to analyze
Charlie: them but that io
Charlie: is i i would say like it's a
Charlie: lot more stable and more minimal than
Charlie: in uv and also it's kind of like sort of
Charlie: like happens up front like we read the files and then we analyze them
Charlie: whereas in uv there's kind of just like constantly io
Charlie: happening whether it's we're reading
Charlie: stuff from disk from the user's project or
Charlie: reading stuff from our on disk cache or we're making small
Charlie: http requests or large http requests we're downloading
Charlie: and unzipping files like there's just constantly io
Charlie: happening so it tends to be a bit
Charlie: harder to benchmark often yeah like
Charlie: often we'll bench you know often we'll benchmark by
Charlie: we'll actually run uv with the cache because at
Charlie: least then it's a lot more stable it's like
Charlie: we're doing the work but we're like reading from the cache every time as opposed
Charlie: to if you want to benchmark anything that requires like network io really hard
Charlie: it's very hard because the amount of variants that you'll get is you will usually
Charlie: dwarf like anything that you would see in cpu code changes that.
Matthias: Is true but at the same time you have to be careful not to change your targets you need to be sure.
Charlie: That you're benchmarking.
Matthias: The right thing.
Charlie: Yes absolutely yes and yeah we do make a different distinction between that
Charlie: we think of it as like warm performance versus cold performance it's like performance
Charlie: when you have stuff in the cache versus when you don't and we do look at both
Charlie: There's some things you can do, like you can set up a network link conditioner.
Charlie: That's what it's called on macOS at least. So you can intentionally throttle
Charlie: your own network connection to try and get it to be a bit more consistent,
Charlie: like bring it down to something that would hopefully be a bit more consistent.
Charlie: But again, that's also different because, well, it measures something,
Charlie: but it measures something slightly different than what you would get on your machine typically.
Charlie: Like I have a very high speed internet connection. So like, you know,
Charlie: the bottlenecks that I experience are different when I throttle versus when
Charlie: I don't, because when I throttle, the network is slower.
Charlie: And so if we need to do things at the same time, it's easier.
Charlie: But when my network connection is really fast, like
Charlie: actually operations on the CPU can actually become
Charlie: or like local sorry on disk io can become blocking because
Charlie: like maybe i'm like streaming it faster really fast
Charlie: and trying to unzip a file and write it to disk like the bottlenecks you're
Charlie: just a little bit different the other thing we'll do sometimes is
Charlie: like if we do need to look at anything related to like
Charlie: the http stack we'll just like run a local server
Charlie: so we're still again it's different but it's
Charlie: gives you some information some particular ability yeah
Charlie: yeah yeah it's like if you're trying to like at least because at
Charlie: least then you get consistency in you know
Charlie: to i mean to to a greater degree at least around like what your what your measurements
Charlie: look like yeah the really hard thing with network io is just like it can just
Charlie: be all over the place and if you're trying to measure like a one to five percent
Charlie: performance change it's very hard to do it in the presence of making lots of network requests and.
Matthias: In that context i was surprised to learn that you're using a single-threaded tokio runtime.
Matthias: Isn't that what you're supposed to do when you want super amazing high-performance I.O. in Rust?
Matthias: Do you always use multiple threads in tokio? Like, why did you decide against that?
Charlie: Yeah, I mean, I guess like the... Sorry, the simplest answer is just that,
Charlie: we benchmarked it and we looked at it a lot. And then we found out that like,
Charlie: basically we use a single tokio thread for IO is sort of the way that I would put it.
Charlie: And we like, we found at least the theory, I guess, is that it reduces synchronization
Charlie: costs and that we don't perform like enough IO for multiple threads to be worth it.
Charlie: So like we're often, you know, what's like a lot of IO, right?
Charlie: If you think about a web server that's trying to do or
Charlie: something that's like super high throughput and they're trying to do like thousands
Charlie: of requests per second that's like pretty different than what we're doing because
Charlie: like for us maybe we're downloading 20 packages at the same time right that's
Charlie: 20 different requests that are happening it's like pretty different it's a pretty
Charlie: different amount of throughput so like,
Charlie: we found that using a single runtime for a sorry a single thread for io and
Charlie: then being really careful,
Charlie: about compute work so trying to
Charlie: make sure that we run compute work like off that main thread or off the main
Charlie: thread is also really important so for example like we have a solver in uv we
Charlie: have to solve for dependencies right like we get a big graph of the things that
Charlie: people depend on and then we look at the things that those packages depend on
Charlie: and we have to solve this like big.
Charlie: Constraint satisfaction problem that solver runs
Charlie: on its own thread so like we we move
Charlie: that compute off to like a different thread so we have to be like a little
Charlie: bit careful about like what we do on what on which thread
Charlie: and like how we orchestrate it but we did we did.
Charlie: Like actually find in practice that it was faster to just use
Charlie: like the single threaded runtime there's some
Charlie: nice other things to it too like we can
Charlie: use like rc instead of arc in like some places you know
Charlie: there's like some minor like quality of life things like that but yeah
Charlie: we just make like a lot of small network requests and
Charlie: we found that like using the
Charlie: single threaded runtime empirically was like faster for our
Charlie: program there was also a long conversation when we started uv
Charlie: about whether we should use async rust at all which is
Charlie: sort of another another topic like the thinking
Charlie: there was like do we actually have enough
Charlie: io to demand async rust
Charlie: and or some kind of multi-threaded runtime because like that's maybe what we
Charlie: would see as like the main benefit is like okay we get all this like thread coordination
Charlie: and stuff and like i was pretty into it at the time
Charlie: like other people on the team sort of tried to talk me out of it and were
Charlie: more like okay why don't we just like manage our threads for
Charlie: doing this kind of io i felt like
Charlie: i i
Charlie: felt like doing it with asyncrust was like would be more like doing
Charlie: it right and maybe moving slightly more in the direction of like the arc of
Charlie: the what the ecosystem wants i don't actually know if that's proven to be true
Charlie: like i'm sort of like i think we could have built uv without using asyncrust
Charlie: and it would have been find maybe i'll just put it that way at the same time
Charlie: i actually find working with async rust to be.
Charlie: And i think it's actually improved fairly dramatically
Charlie: even since we started the project like since we started the project like
Charlie: the rust team just like keeps shipping you know like there's like things
Charlie: just keep getting like async that now is like
Charlie: async closures async async async
Charlie: in traits didn't exist when we started the project
Charlie: we had to use like the async trait macro for
Charlie: example like there's just a lot of things in async rust
Charlie: that have actually been like stabilized and improved in the last year and
Charlie: i i really don't find myself having
Charlie: to work around async or fight async quote unquote very much
Charlie: for whatever reason i'm not saying that it's worth it
Charlie: for every project but like i actually don't think it has been a big
Charlie: challenge for us i think the only challenge i think the main challenge has been
Charlie: this stuff around scheduling and like trying to understand like how do we schedule
Charlie: really efficiently and maybe feeling like we have slightly less control than
Charlie: if we were hand-rolling our own approach to threading.
Matthias: I mean, in this situation, there's sort of a middle ground as well,
Matthias: which is instead of having a global tokio runtime,
Matthias: let's say by annotating your main function with tokio main, you could also use
Matthias: structure currency where the core of it is sync.
Matthias: And when you use a lot of IO, when you need a lot of IO, you can start your
Matthias: own little runtime even within a function say or within a struct did you consider that approach as well.
Charlie: Yeah i think that's probably roughly what we would have done if we didn't do
Charlie: this and again i think that could have been totally fine but this also worked
Charlie: and so it's like i don't know it's been good i i think i there are challenges
Charlie: with async right like the things that i mentioned before are becoming less of
Charlie: a pain, but there's still a lot of a pain,
Charlie: like async closures, right? Things have to be sent in sync.
Charlie: It's sort of,
Charlie: it's slightly infectious, right? Whereby like if one thing, if we need to call
Charlie: one async function from another function, suddenly like the async propagates upward.
Charlie: And there were some challenges like that. Like we, just as a random example,
Charlie: like our Git implementation, I originally basically vendored from Cargo.
Charlie: Not exactly, but like I looked at Cargo's Git implementation and I was like,
Charlie: okay, Cargo is good at Git.
Charlie: How do we deal with Git? and i like i sort of
Charlie: like started with what they had and then we we changed it pretty significantly
Charlie: over time but like that was an async right
Charlie: and and so like and we were like calling into
Charlie: it from an async you know from async runtime that ended up actually causing
Charlie: like kind of a lot of problems that were like pretty annoying to debug because
Charlie: for example they were using they need to make network requests and it was using
Charlie: i think it was using using curl or something and i was like okay but we use reqwest.
Charlie: I only want to use one networking stack. So let's replace it with reqwest.
Charlie: But okay, if we want to replace it with reqwest, and it's going to be sync,
Charlie: then we have to use reqwest blocking.
Charlie: And then it was actually for a period of time, I can't remember how we fixed
Charlie: this, it was actually impossible, because then reqwest blocking actually starts,
Charlie: I believe it uses async internally.
Charlie: And so it was like we were starting a tokio runtime within a tokio runtime.
Charlie: And like that would just error, right?
Charlie: And so, so it was like, we were trying to take some code that wasn't async and
Charlie: use it in an async context.
Charlie: And it was just kind of the kind of like buttheads a little bit so you do run
Charlie: into stuff like that by buying into async but but in general i think the ergonomics
Charlie: of it are actually quite good and like i i don't think we pay much of a high
Charlie: cost for using it and hopefully we'll get more and more out of it over time
Charlie: is sort of is a little bit of how i think about it yeah.
Matthias: Exactly so i would assume that the majority of the problems with async cross
Matthias: that you mentioned they don't really,
Matthias: affect you because you don't think
Matthias: libraries first your user facing interface is the cli but if you were.
Charlie: To build.
Matthias: A library and it used tokio as a sort of public interface so it was async to
Matthias: begin with then that can cause some headaches with integration.
Charlie: Because 100% Yeah, sorry. This is actually a great point. That's actually maybe
Charlie: the thing that's most painful about using async is like there are all these
Charlie: crates that we depend on that we have to depend on basically like the async
Charlie: versions of those crates or the async interfaces.
Charlie: And if I look at all those crates, they have to actually maintain like a tokio
Charlie: interface and a sync interface and maybe like an async standard interface like sometimes.
Charlie: And for example, like we have...
Charlie: Tar rs okay really popular common
Charlie: crate for creating and untarring
Charlie: creating tar balls and untarring them and like we we needed like we wanted like
Charlie: an async tar implementation so like it turns out there's something called i'm
Charlie: going to get the exact names of these things wrong because they're all so similar
Charlie: but there's something called async tar but that which is an async.
Charlie: Standard port of tar rs it's meant to be they took
Charlie: tar rs and made it async with async standard well we can't
Charlie: really use that because we're using tokio and not async standard and we
Charlie: don't want to have these two huge dependencies on different
Charlie: async runtimes okay so that it turns out
Charlie: that got forked to something called tokio tar
Charlie: so it went from tar rs to async
Charlie: tar to tokio tar and then that crate
Charlie: actually got forked like two more times um just
Charlie: by different people because it wasn't really maintained and then eventually
Charlie: we forked it ourselves to fix
Charlie: some bugs and now we like fully maintain that like
Charlie: we just like maintain it it's a that's actually a public crate that we published outside
Charlie: of uv where we've like we upstreamed or
Charlie: we downstreamed i guess a lot of things from tar rs and
Charlie: we like fixed some bad bugs that users were hitting
Charlie: and so the whole like the ecosystem problem
Charlie: i actually have no idea how to solve that which is like it
Charlie: seems like a huge pain for crates to have to maintain all
Charlie: these different interfaces and then for us it's like
Charlie: we have two different zip crates we we use zip
Charlie: rs and then we also use async zip because
Charlie: we have like slightly different contexts in which they run and like that's maybe
Charlie: like an us problem but like it it is that is the i think probably the most challenging
Charlie: piece is just the touch points with the rest of the ecosystem when you want
Charlie: to pull in async versions of things or people have to maintain async versions
Charlie: of things like like okay we're going to maintain like async.
Charlie: This async tar crate forever because like the tar rest crate is sync like that's
Charlie: like a that's kind of a bad outcome but i don't have no idea how to solve it
Charlie: i don't know if that's what you're getting at you're probably talking about
Charlie: it more from the library maintaining perspective of having to expose tokio interfaces
Charlie: like a lot of crates will have a tokio feature right that like pulls that stuff
Charlie: in but yeah it's like it's a little painful i.
Matthias: Don't know what's the way out of it here because in reality
Matthias: well the ecosystem is still evolving async std is sort of that so that's off
Matthias: the table but i do believe that there's merit in having a sync interface and
Matthias: then an async interface on top of that which is a separate crate and the as
Matthias: the crate ecosystem allows for it but yeah.
Charlie: Yeah yeah yeah i actually did once maybe just to illustrate how confusing this
Charlie: stuff is and how little I know about it.
Charlie: Like early on, I did actually pull in a crate that used async as standard.
Charlie: And I wasn't really thinking about it very hard. And I just saw I had like an
Charlie: async interface and I needed to call like .compat on like a couple of things
Charlie: or something. And I was like, okay, cool, like this works.
Charlie: And then like, obviously like massively bloated our dependency graph and everything.
Charlie: And people on my team were like, did you do this intentionally?
Charlie: And I was like, I don't even really know what the difference,
Charlie: like at that point in time, I was like, I don't really know like what the difference
Charlie: is between these things.
Charlie: They're just async, right? Blah, blah, blah. So like, it's super confusing.
Charlie: It is helpful that things have centralized more on tokio, I think.
Charlie: But it is it's a very easy like mistake to make and it's like not all clear
Charlie: how these things relate to one another honestly.
Matthias: Yeah yeah it had a ripple effect on the ecosystem which we still deal with today
Matthias: but it's a step into the right direction that the futures trade is now in the
Matthias: rust prelude so it feels like as you say things are progressing Right.
Matthias: Now, another thing that I wanted to touch on, which also kind of is interesting
Matthias: because you diverge from the norm a bit, is parser generators.
Charlie: Oh, yeah.
Matthias: I think you decided to switch to a handwritten parser in ruff.
Matthias: And I wonder, first off, what was the decision process there?
Matthias: And second, how do you handle the complexity that comes with it?
Charlie: Yeah, great question. So like, originally, the parser in ruff came from a project
Charlie: called Rust Python, which is, I guess, in some ways, like an even more ambitious
Charlie: project, because it's a whole Python interpreter.
Charlie: So they're actually trying to build like in Rust, they're trying to build a
Charlie: run, you know, a whole runtime.
Charlie: So like, as part of that, though, it has to parse code. And so we took that
Charlie: I took that parser, and we depended on it as a library.
Charlie: And that parser was based on a parser generator called Lollerpop or L-A-L-R-P-O-P.
Charlie: Again, I don't really know how to pronounce anything because I spend all my
Charlie: time just on the internet.
Charlie: Same. But it's basically like you create a Lollerpop file.
Charlie: It's a DSL, but it also can include Rust code verbatim at different points.
Charlie: So you have something that kind of looks like the grammar.
Charlie: But the thing that we were finding, when we started using that,
Charlie: the first, I guess, sign of trouble was that Rust
Charlie: Python didn't support several syntax
Charlie: features in python new feature newish newish features couple last couple years
Charlie: like match statements python has support for pattern matching and that was added
Charlie: in 3.10 i think and it didn't support it and i was like okay and it turns out
Charlie: that that's because in that in python 3.
Charlie: Oh my god i'm gonna mess this up i guess i think in 3.9 maybe they switched
Charlie: their parser so they moved from an lr1 this is not important if you know what
Charlie: these things are but they moved from an LR1 parser to a PEG parser, a PEG parser.
Charlie: And basically it meant that the grammar got more flexible.
Charlie: So they were able to have things that would be called soft keywords.
Charlie: So for example, match is a valid variable name.
Charlie: You could do match equals one, but it's also a keyword because you can do match
Charlie: object, colon, and then patterns, right? So it's both a valid variable name and a keyword.
Charlie: And whether it's a keyword depends on the context around it.
Charlie: So it depends, like the parser has to be able to support both those use cases.
Charlie: I'm just thinking, I think async, now I can't remember.
Charlie: Async might be a soft keyword as well, but largely it's for things that were
Charlie: added where they don't want to actually,
Charlie: they don't necessarily want to make changes that are backwards incompatible
Charlie: in the sense that there's a lot of existing Python code that needs to run on
Charlie: new versions of Python that might use match as a variable. And so that code
Charlie: should continue to work.
Charlie: So the grammar got more complicated and they added a more powerful parser to support that.
Charlie: Now our parser that we were using based on lollipop couldn't support that there were like,
Charlie: ambiguities in the grammar that were very hard to represent in lollipop
Charlie: and we had to get increasingly good at lollipop in order to do it so like in
Charlie: terms of like the precedence of the statements the way that you do like like
Charlie: basically we were just like learning this tool really deeply and having to put
Charlie: more and more work into actually supporting the Python grammar.
Charlie: So what it felt like at the time was there were a few properties that we wanted,
Charlie: that we thought we could get out of a new parser.
Charlie: One was we thought we could make it a lot faster to start just out of the box.
Charlie: Two, we thought it would be much easier for us to optimize further because it's
Charlie: very hard to optimize like a parser generator.
Charlie: Like the code is generated, right? As it sounds. And so like,
Charlie: you can't like, I mean, there's only so much you can do to like optimize the
Charlie: generated code, right? It's like kind of out of your control.
Charlie: And the third was error recovery. We wanted like much better control over error
Charlie: recovery so that, especially because we're building tooling that's designed to work in an editor.
Charlie: So like if you type a syntax error, like if you type.
Charlie: Def space and you're like starting to type a function we still
Charlie: want to be able to parse as much of the file as we
Charlie: can even though it's not syntactically valid anymore and
Charlie: that's that's like pretty hard like there are
Charlie: probably parser generators that support that to different degrees but
Charlie: like a lot of that is requires fairly bespoke
Charlie: handling of like what happens when you hit an error and
Charlie: like what the different fallback cases could be so there
Charlie: were things we wanted out of a parser generator which were like performance
Charlie: and error recovery and then we also found that we were spending a lot of time
Charlie: just trying to get the part the parser generous was a save you time but ultimately
Charlie: we were spending a lot of time trying to get it to work for the grammar in the
Charlie: first place so we were like we want to rewrite the parser we were like pretty sure about that,
Charlie: and actually a contributor came to us and said that they were working on they
Charlie: wanted to do write like a handwritten parser if i recall correctly i think it
Charlie: was actually for like a master's project.
Charlie: And they were like would you ever like use it in ruff if I can get it to work
Charlie: with the ruff AST like I'm building around the ruff AST sorry they were building
Charlie: around the ruff AST and they were like if I can get it to pass the ruff test
Charlie: suite would you want to use it in ruff and we were like yeah absolutely and
Charlie: we'll pay you for it and so we like,
Charlie: there was a fair amount of so this contributor like brought us this parser and
Charlie: there's a fair amount of kind of like last mile work of making sure that.
Charlie: You know, because we're being used by all these big projects,
Charlie: we need to make sure shipping a parser is like a huge change.
Charlie: So we did end up investing a lot of time in like the final 10% of like making
Charlie: sure that it works in all cases and that it won't panic for people and that
Charlie: it will work exactly as we expect.
Charlie: It's the kind of thing that it's pretty easy to test at very large scale because
Charlie: we just, we can compare our basic, for example, the diagnostics that we get on very large projects.
Charlie: There are arbitrary large projects and we can run ruff before and
Charlie: after on those projects and make sure the diagnostics are completely unchanged
Charlie: for example so we did a lot of large-scale testing and it
Charlie: was actually like an incredibly smooth release like we
Charlie: didn't i don't think we got like we got like maybe one bug report which
Charlie: is amazing i didn't even work on this so i can brag about it a lot
Charlie: but like it's amazing that that happened and ultimately
Charlie: it means we now have our own parser it's totally it's completely handwritten and
Charlie: it's been way easier for us to modify it over time because
Charlie: like it it is work for sure like
Charlie: the grammar gets extended but it we have complete control
Charlie: over like how the parser works and like the idea
Charlie: of adding new syntax to that grammar is so much
Charlie: less daunting than adding it to like the parser generator which
Charlie: isn't it's really not meant to be a knock on the parser generator like
Charlie: i think parse i think like that's a great project i think parser generators
Charlie: can definitely be great it just depends on what you're doing but for
Charlie: us like python there's just a lot in the grammar there are lots of ambiguities
Charlie: and it's evolving and it's something that we have to change so we felt more
Charlie: comfortable doing it ourselves and it was also like way faster like it sped
Charlie: up all of ruff by like 20 to 40 percent or something that's.
Matthias: Pretty impressive and it sort of paid off for you to take ownership of this
Matthias: entire parsing part because it's such an integral part of what you're building
Matthias: at astral in general you can probably use that for uv and for ruff i'm assuming i don't know.
Charlie: We don't use it in uv today the python parser we don't use it in uv today but
Charlie: we could we have other parsers in uv thankfully like the version parser the
Charlie: version specifier parser the requirements txt parser yeah a lot of the work that you do is.
Matthias: Parsing right i.
Charlie: Guess so yeah i mean often like we need to implement standards things
Charlie: that have been specified in python but only have really have python implementations
Charlie: so like versions would be a good example like sounds like a simple thing like
Charlie: 1.0.0 right but like they get actually fairly complicated and so like it's not
Charlie: like that parser is incredibly complicated but,
Charlie: It does run, I don't know, it probably, we're probably analyzing,
Charlie: I have no idea what the number would be.
Charlie: We're probably analyzing billions and billions of versions a day,
Charlie: right? Think about how many times that version parser is parsing versions, right?
Charlie: Like, so, you know, we think a lot about how do we make that fast and how do
Charlie: we also, how do we make sure that it's fully standard compliant?
Charlie: So yeah, we do build a lot of parsers.
Matthias: To me that was probably one
Matthias: of the highlights of your rust last year where
Matthias: you shared that story about version
Matthias: parsing i had such a good laugh so if someone
Matthias: hasn't seen it yet we will link it in the show notes it's that's
Matthias: a really fun example really great yeah it was such a fun
Matthias: exercise and i think you could make an entire course
Matthias: around that but anyhow probably a phd but yeah that's
Matthias: another topic still it's different
Matthias: when you build that as a fun hobby side
Matthias: project or when you do that as a business with multiple employees with a larger
Matthias: code base with multiple crates and so on so i would just wanted to take the
Matthias: opportunity to talk about day two with rust as it stands today what's the verdict i.
Charlie: Love it and i think it was i think it's been such a good choice for our projects
Charlie: so we get to we can build extremely stable extremely fast software,
Charlie: that also has the benefit of being memory safe for like you know thousands and
Charlie: thousands or millions whatever of like python users and the day-to-day is excellent
Charlie: like rust i've always thought that rust kind of like,
Charlie: I don't know. It's not that much of a secret, but like the secret behind Rust
Charlie: is I've always thought is like the tooling.
Charlie: Like I, for me at least, and I've never really written any C++ and people can
Charlie: also just think I'm a moron, which is fine.
Charlie: But like, whenever I look at a C++ project, it's like, it's super intimidating
Charlie: to figure out how do I even get this thing to build or run or like, what am I doing?
Charlie: And I don't know that I ever would have become a quote unquote systems programmer
Charlie: through c++ or i think it would have been a lot harder for me that's my prediction
Charlie: maybe i'm wrong i don't know but like rust it's just such a high confidence experience it's like.
Charlie: You install this thing with rust up you run cargo run cargo
Charlie: build and like that's how you build a project like i can clone any
Charlie: rust project and like feel fairly confident that i know
Charlie: how to run it and how to test it and how to build it and
Charlie: how to understand it and so for me like the tooling
Charlie: story is excellent i i think the only
Charlie: things i really have complaints about and these are just like things that people are
Charlie: always going to that i'm always going to complain about no matter what is like
Charlie: obviously i'd like compile times to be faster that would be
Charlie: nice because as you build a bigger and bigger project it becomes
Charlie: more and more of an issue and it just
Charlie: it's just like a tax on development but you know
Charlie: even if they were faster i'd probably be saying the same thing and saying
Charlie: i just want them to be faster like um that's that's
Charlie: like the main thing for me i think the thing one thing i'm really impressed by in.
Charlie: Rust too is just the rate of
Charlie: the rate at which the language and the tooling keeps improving
Charlie: like every rust release has something
Charlie: i want and something i've been waiting for which is
Charlie: kind of crazy like just even looking at and i'm
Charlie: not even i'm not one of the like i talked to there are people on
Charlie: our team who have been doing rust since before 1.0 right and i've been doing
Charlie: rust for a long time and like i started doing rust in like
Charlie: 2022 is when i started writing
Charlie: rust i think wow yeah i've like i haven't been writing rust for
Charlie: that long but like now i'm like every rust release like
Charlie: there's just like there's just so much progress and it's
Charlie: like awesome to feel like you're part of this community that's like building and
Charlie: growing all the time so it's both it feels so stable and like so mature but
Charlie: it's also the rate of change i think is like great and the rate of progress
Charlie: and the rate at which things are being addressed my only complaint is compile
Charlie: times but i you know i i'll take what i can get really same.
Matthias: Let's let's make it faster definitely.
Charlie: Yeah we.
Matthias: Need more crates in the workspaces but when you when you refactor across,
Matthias: crates or maybe even within crates how does it feel to you like what's that
Matthias: experience like like do you refactor with confidence is it something that you
Matthias: look forward to is it a choice refactor is it more of a you know dread.
Charlie: No i like refactoring and rest a lot i think it's not like,
Charlie: Maybe it's the nature of what we're building. It's, you know,
Charlie: like, I guess the thing that people often say about like very well typed languages
Charlie: or even like functional languages is like, if it compiles, you know,
Charlie: it, you know, it will work.
Charlie: I don't know if I believe that about Rust.
Charlie: There's still ways that your pro code
Charlie: can be wrong, but I do feel like I'm constantly guided by the compiler.
Charlie: And actually more and more, I think the way I write code is that I try and make
Charlie: it such that I will be guided by the compiler in the future.
Charlie: So like for example i think about this
Charlie: with like okay let's say i'm passing like a
Charlie: struct into a function and like i need to do
Charlie: something with every field i want
Charlie: to make sure that if i add a new field to that to that struct i'm reminded
Charlie: that i need to handle that field in that function and so
Charlie: the thing i will often destructure it because that makes
Charlie: sure that if i add it as opposed to referencing all the fields in line like
Charlie: object dot a object dot b i will destructure it at the top of the function it's
Charlie: a very small thing but it means that the compiler will tell you will remind
Charlie: you that you need to look at it if you add a field so like more and more i find
Charlie: myself trying to find ways to be guided by the compiler because it's such a powerful thing,
Charlie: and it just makes it it just makes progress i just think it makes like working
Charlie: on large complex products like so much easier the.
Matthias: Destructuring part i never heard in such detail because yeah.
Charlie: That's kind of a silly little thing but do you know you don't understand what i'm saying right no.
Matthias: I totally like it i will totally steal that idea i like i.
Charlie: Mean hopefully you don't have structs that are that complicated that need that
Charlie: that much but like you know it is i just like more and more try to think about
Charlie: like how will the compiler make sure that i do this correctly in the future yeah.
Matthias: I know about that tip in a serialization context when for example you you want
Matthias: to ensure that all the fields get serialized and deserialized properly then i.
Charlie: Know some.
Matthias: People to structure that but i it never crossed my mind that you can do that
Matthias: in just you know normal function code you could even destructure it right there
Matthias: in your arguments you don't.
Charlie: Even have.
Matthias: To do it in the first line of the body you can do it in the.
Charlie: Arguments because it's just a you.
Matthias: Know pattern match essentially or like it's a destructuring pattern yes.
Charlie: Yeah do.
Matthias: You have more such tips like where can people learn more about ideomatic rust
Matthias: and best practices where did you learn it.
Charlie: I mean a lot of it i learned from having great teammates
Charlie: which is not that's sort of a bad answer because like not ever it just depends
Charlie: on your life situation like you don't have that much control over that you have
Charlie: some control but like but it is a real thing which is i started working on ruff
Charlie: on my own and then as we grew the team i ended up hiring thankfully people who
Charlie: knew a lot more about rust than me and like,
Charlie: Like Mika, our team, who was the second employee to join the company,
Charlie: he just taught me so much about Rust.
Charlie: And then later we hired Andrew Gallant, BurntSushi, who I will often just send
Charlie: him random Rust questions rather than Googling them.
Charlie: I mean, not in a way that is exploitative of that relationship or overly burdensome
Charlie: on me, but he loves being the elder statesman at the company that can help people
Charlie: with hard Rust questions or problems.
Charlie: And so finding great people to learn from is maybe the slightly higher level
Charlie: lesson, but like I know that's not always easy.
Charlie: The other thing that I did is I read a lot of code like.
Charlie: ChatGPT and LLMs are great, but like you should also remember that like GitHub
Charlie: CodeSearch exists and like all these amazing code bases are open source and so for example,
Charlie: something I'll often do now is like if I look at a crate and I want to use it
Charlie: and it doesn't have, maybe it has great examples.
Charlie: Okay, great. if it doesn't have great examples and I don't feel the need to,
Charlie: I don't want to read all the documentation myself, I will actually just go into
Charlie: GitHub code search and I will just search for the struct name or the function name.
Charlie: And I will go find real examples in one second of real projects using that crate.
Charlie: And so like, you can just go read code, you know, like, like reading code is
Charlie: like, you know, all this stuff is accessible to some degree.
Charlie: And so that's, I don't know, that's how i've tried to pick up
Charlie: things and and at least learn like i looked at
Charlie: cargo a lot when we were building uv and tried to understand like how do
Charlie: they do certain things like i don't know how to implement git like let me go
Charlie: look at what cargo does and then let me let me read about their design decisions
Charlie: because it's all documented in their prs and you can understand the trade-offs
Charlie: and you can understand like why they did things a certain way so you know you
Charlie: can also go hunt people down and talk to them about this stuff but there's plenty
Charlie: that you can find without doing that.
Matthias: Fully agree before there were llms there was BurntSushi but unfortunately you hired
Matthias: him so there's just one of one of them but you can still read their open source
Matthias: code so ripgrep whenever someone tells me what is an idiomatic rust crate that
Matthias: i should read i always point them to ripgrep because.
Charlie: Oh yeah i liked that a lot too when we were also when we were figuring out how
Charlie: to structure our crates and how to manage workspaces and our release pipeline
Charlie: and all that stuff there's just so much good there's just so much good code
Charlie: out there and so you know go read it.
Matthias: Yeah I always cry
Matthias: when I open the ripgrep code out of joy of course I.
Matthias: Really like reading this it's amazing.
Matthias: Yeah, unfortunately, we have to come to an end. But I wonder if you have any
Matthias: final statement to the Rust community.
Charlie: Yeah, I mean, I think for me, like I, how do I, how do I say this correctly?
Charlie: Like I, it's kind of amazing, I think that I only started writing Rust like a few years ago.
Charlie: And now we've shipped, I mean, along with a great team, like we've shipped two
Charlie: of these tools that are having,
Charlie: I think at least a huge impact on Python, which
Charlie: is like the most popular or the second most popular programming ecosystem
Charlie: on earth so if you think about it like rust is
Charlie: kind of in a lot of ways rust is kind of like powering python
Charlie: at least to you know if i have a say about it at least
Charlie: rust is powering is powering python and so
Charlie: i don't know i've always just felt i again i never considered myself to be like
Charlie: a systems programmer quote unquote in most of my career i was writing typescript
Charlie: python i mean i did some java professionally but like i had never except for
Charlie: like a course in college done any C. I really hadn't done any C++.
Charlie: And like in the span of a few years, I like learned to build this kind of software.
Charlie: So I don't know. I've had just like great experiences with the community and
Charlie: being welcomed into it and learning the language.
Charlie: And I think that should continue to be a very important part of Rust is like
Charlie: welcoming people in and helping them learn.
Charlie: Because the impact that we can
Charlie: have by building this kind of stuff is just huge, even outside of Rust.
Matthias: Perfect i couldn't have said it better both languages are really close to my
Matthias: heart and i really like to see that synergy jolly your presence was much appreciated
Matthias: i thank you so much for taking the time today yeah.
Charlie: Thank you so much for having me and for for all the great questions i it was
Charlie: it was uh it was really fun.
Matthias: Rust in Production is a podcast by corrode it is hosted by me Matthias Endler
Matthias: and produced by Simon Brüggen for show notes transcripts and to To learn more
Matthias: about how we can help your company make the most of Rust, visit corrode.dev.
Matthias: Thanks for listening to Rust in Production.