Rust in Production Cover

Rust in Production

Matthias Endler
Since 12/2023 22 episodes

Season 1 Finale

Matthias reflects on the first season of "Rust in Production", the benefits of Rust for companies, and strategies for adopting it.

2024-03-07 36 min

Description & Show Notes

In this episode of Rust in Production, we reflect on the inaugural season, sharing insights from companies pioneering with Rust to shape the future of infrastructure. Starting with Rust's early days before the stable compiler, we chart the language's evolution and its growing adoption in production by leading companies. The episode explores why companies choose Rust, highlighting its unparalleled safety features, concurrency management, and performance that not only drives down costs but significantly enhances user experience. The narrative then shifts to the practicalities of adopting Rust, including learning resources, the importance of community support, and integrating Rust into existing codebases. We discuss the surprisingly positive experience of hiring Rust developers, thanks to the language's appealing prospects. Moreover, the episode delves into Rust's explicit approach to handling complexity, offering a steeper yet rewarding learning curve. We also address the misconception of Rust's limited adoption, revealing its widespread use across various industries and the consensus around certain Rust patterns and libraries. Despite the challenges in showcasing Rust's adoption due to the reticence of some companies to share their stories, the positive feedback from the podcast's audience underscores a keen interest in Rust's real-world applications. Concluding the season, we extend a heartfelt thanks to all guests and listeners, emphasizing the value of sharing Rust experiences and the potential for further growth in the Rust ecosystem. We share a wishlist for the Rust community, including expanded learning resources, improved tooling for production environments, and more public discussions of Rust's benefits. As the show gears up (see what we did there?) for its second season, listeners are encouraged to engage and suggest topics or guests, which could broaden our common understanding of Rust in production.

In the season finale of 'Rust in Production', we take a cold, hard look at our goals and achievements during the first season.

Did we manage to answer the tough questions about Rust's usage in the industry? Were we able to provide a balanced view of the challenges and benefits of using Rust? Or did we end up merely preaching to the choir?

As it turns out, the answers align nicely with the main themes from the Rust 2023 survey: the hiring market, the learning curve, the reasons for choosing Rust, the costs of adoption, and the complexity of the language. Either way, there were definitely some surprises and unexpected turns along the way!

Links from the Episode (in order of appearance):

https://blog.rust-lang.org/2015/05/15/Rust-1.0.html
https://blog.rust-lang.org/2016/11/10/Rust-1.13.html#the--operator
https://blog.rust-lang.org/2019/11/07/Async-await-stable.html
https://www.youtube.com/@HelloRust
https://corrode.dev/podcast/s01e01-influxdata/?t=55%3A40
https://corrode.dev/podcast/s01e05-tweede-golf?t=16%3A29
https://corrode.dev/podcast/s01e02-pubnub?t=16%3A24
https://corrode.dev/podcast/s01e03-apollo?t=7%3A24
https://corrode.dev/podcast/s01e04-arroyo?t=17%3A40
https://corrode.dev/podcast/s01e06-sentry?t=54%3A03
https://corrode.dev/podcast/s01e02-pubnub?t=17%3A25
https://corrode.dev/podcast/s01e05-tweede-golf?t=18%3A57
https://corrode.dev/podcast/s01e03-apollo?t=35%3A32
https://corrode.dev/podcast/s01e01-influxdata/?t=25%3A25
https://corrode.dev/podcast/s01e01-influxdata/?t=49%3A47
https://www.integer32.com/
https://corrode.dev/podcast/s01e06-sentry?t=57%3A16
https://corrode.dev/podcast/s01e01-influxdata/?t=46%3A49
https://corrode.dev/podcast/s01e04-arroyo?t=29%3A42
https://corrode.dev/podcast/s01e04-arroyo?t=29%3A20
https://corrode.dev/podcast/s01e05-tweede-golf?t=13%3A18
https://corrode.dev/podcast/s01e05-tweede-golf?t=15%3A13
https://github.com/rust-lang/rust/tree/master/library
https://crates.io/crates?sort=downloads
https://corrode.dev/podcast/s01e02-pubnub?t=37%3A24
https://corrode.dev/podcast/s01e03-apollo?t=24%3A09
https://crates.io/crates/tokio
https://crates.io/crates/axum
https://crates.io/crates/sqlx
https://newrustacean.com/
https://oxide.computer/podcasts/on-the-metal
https://github.com/M3t0r
https://hachyderm.io/@m3t0r
https://mastodon.social/@mre
https://corrode.dev/about
https://podcasts.apple.com/us/podcast/rust-in-production/id1720771330
https://open.spotify.com/show/0Hf6gWrzpSzXp1X0cebbsT

Additional Links:

https://blog.rust-lang.org/2023/12/20/Rust-2023-Survey-Results.html
https://www.premiumbeat.com/royalty-free-tracks/cassiopeia

All Episodes of Season 1:

https://corrode.dev/podcast/s01e01-influxdata
https://corrode.dev/podcast/s01e02-pubnub
https://corrode.dev/podcast/s01e03-apollo
https://corrode.dev/podcast/s01e04-arroyo
https://corrode.dev/podcast/s01e05-tweede-golf
https://corrode.dev/podcast/s01e06-sentry

Credits:
We would like to thank the guests of the first season for their time and insights. We would also like to thank the listeners for their support and feedback. We are looking forward to the second season and hope you will join us again!

Audio editing and mastering by Simon Brüggen (M3t0r).

About corrode:
"Rust in Production" is a podcast by corrode, a company that helps teams adopt Rust. We offer training, consulting, and development services to help you succeed with Rust. If you want to learn more about how we can help you, please get in touch.

https://corrode.dev/about

Transcript

This is Rust in Production, a podcast about companies who use Rust to shape the future of infrastructure. My name is Matthias Endler from corrode, and today I'm excited to reflect on the insights from the first season paving the way for the next chapter of the show. When I started using Rust in 2015, there was no stable compiler or ecosystem. It was before Rust 1.0 and most people used nightly Rust for their projects. Things would break regularly. There was no question mark operator, no incremental compilation and no async await. Fast forward to 2018 when I started to create video content on my YouTube channel called Hello Rust. I wanted to share my excitement for Rust and help others learn it. The ecosystem system had matured a lot since 2015 making it a great time to start exploring the language. During that period an increasing number of companies began to use Rust in production. We were inspired by the success stories of companies like Dropbox, Cloudflare and Mozilla using Rust for critical parts of their infrastructure. I always found these stories exciting. I was curious to hear about the challenges that companies were solving with Rust. However, this knowledge was hard to come by. Unless you attended conferences or stumbled upon the occasional blog post, learning about companies deploying Rust in production was difficult, and to this day some of these stories remain untold. Later, as a Rust consultant, I had the privilege of speaking with many great companies about their real motivations for choosing Rust. These benefits go far beyond the usual Rust is fast and safe narrative. While there is a lot of great content about Rust these days, I had been waiting for someone to launch a format specifically focusing on Rust in production, because I felt that these stories needed to be told. Eventually, I decided to take the initiative and create the podcast myself. This episode concludes the first season of Rust in Production. In it, I reflect on the season and share insights gained from the interviews. I'll also discuss my plans for season 2 and the road ahead. Chapter 1. Why Rust? Throughout the interviews, a recurring theme emerged. As companies scale, the complexity of their systems increases. This complexity leads to need for ways to make systems more reliable and easier to maintain. This is especially true in today's world of microservices and parallel computing, where systems are more distributed and complex than ever. Robustness and reliability are key concerns for companies. For example, here's what InfluxData's CTO Paul Dix had to say about why they chose Rust.
Paul
00:02:55
Yeah, so I mentioned this before, I think like the fearless concurrency, right? Having, you know, eliminating data races, essentially, which we had before, like these really gnarly bugs in version one of Influx due to stuff like that, which we just wouldn't have in Rust.
Matthias
00:03:15
This confidence in Rust's ability to make concurrency safe and frictionless is shared by many companies. Rust's type system and ownership model make it easier to reason about concurrency and write code that is both secure and reliable. Folkert de Vries from Tweede Golf shared a similar sentiment.
Folkert
00:03:35
I also think in Rust, it'll be really clear when you need a mutex, right? Whereas I I think in C, it might be much harder to know like, okay, I'm using this value over here, but is some other thread also touching this right now? Like you sort of have no idea. Whereas in Rust, a type checker will tell you, right? Like, oh, you're using this. Or, well, first of all, you have an immutable reference, so you can't mutate that thing. Ever. If you have a mutable reference, then you know it isn't shared between multiple threads, and if you do want something that is both shared and mutable right now, you need to wrap it in a mutex.
Matthias
00:04:13
As software gets more complex, companies are concerned about the maintainability of their codebases. For instance, PubNub CTO Stephen Blum mentioned that maintainability was a key reason for choosing using Rust, when they migrated some of their services from C. In their case, their C code was stable, but they were scared to touch it because of the fear of breaking existing functionality.
Stephen
00:04:38
And that was not great either, right? So we've got this thing that's over here, we want to read. And we actually the question you would ask is like, why would we choose to attempt to rewrite something that we have that's robust and C and it's stable, rewrite it in Go and the answer is maintainability because we're scared to touch it it's working right now and if we make any adjustments you know that's the danger zone.
Matthias
00:05:00
Another aspect of Rust that companies appreciate is its strong stability guarantees, Rust helps companies avoid making mistakes and catch bugs early, This is especially important for companies that are dealing with a lot of traffic where outages can be very costly here is Apollo's Nicolas Moutschen.
Nicolas
00:05:20
Most of outages come from application-level changes. Right. Not necessarily the underlying infrastructure. So changes are risky, and having, making sure, having, asking people to trust us, with all that traffic being the API, you know, the front-end API that's going to ingest all the traffic and then route it to their microservices is a huge responsibility. Safety and reliability is extremely important for us for that reason. To create a cloud router for a customer, we need to provision a bunch of different resources. And we need to avoid creating the same thing twice. We need to keep track of where everything is. And so when we are making changes on our own code base, we are making all these changes there. We need to make sure that we are not accidentally breaking things. So we put a lot of emphasis on the write safety and making sure that the state of our database is the state of what we know exists in terms of customer resources. Was always correct. And so we've encapsulated a lot of write safety into the struct and variants that we use in Rust. So we actually get, by using the strong typing of Rust and the way we structure that, we actually get a lot of safety guarantee from that perspective, which is not something that Rust enforces, but Rust made us think about.
Matthias
00:06:44
Rust is renowned for its outstanding performance, But one often overlooked aspect of that is that better performance can lead to cost reductions. This is a great benefit for startups that are looking for better ways to achieve optimal resource utilization. Arroyo's CTO Micah Wylde shared this sentiment.
Micah
00:07:03
At a high level, a system like Arroyo doesn't really need a complex scheduler like Tokio. As I mentioned, each of these operators essentially acts as a single thread. It receives one event, it does all the processing for that event, and then it sends it on to its next destination. And all this has to happen in order to uphold the correctness guarantees of the system. And because of that, the first version of Arroyo actually was built around threads and thread processing. At some point, it migrated to Tokio and AsyncRust, actually pretty early on. And the core reason for that was that so much of the ecosystem is in AsyncRust at this point, that if you want to use common network libraries or database drivers or almost anything from kind of network programming ecosystem, ecosystem, you do have to deal with async at some point. And at some point, it's easier just to move your whole system over to async. And that was definitely a challenging migration. Actually, for me, I had never worked with AsyncRust before. So it involves a lot of learning, a lot of time on the Tokio Discord channel, which is extremely helpful. But in the end, actually, the surprise was that it ended up being a lot faster. Just purely doing that migration made the system like 30% faster, which was not my expectation at all. But it turns out that the Tokio scheduler is really, really effective at this class of problems, where even though it looks at a high level, like all this processing is single-threaded, there's a lot more going on under the hood, a lot more work that has to be coordinated. Between like, you actually have threads, like in our case, like talking to S3 or talking to other systems. We have a lot of queues involved. So even though we have only a smallish number of actual processing threads, there's a lot of network exchange happening on other threads, talking to... Coordination system over gRPC. And Tokio is really good at, at, you know, organizing all of this work efficiently and really maxing out the use of your core. I think that the most surprising thing for us is that we're able to run the system at extremely high utilization, like above 95% CPU utilization, and everything remains responsive and reactive and is able to, to really work effectively at that extremely high level of CPU thrash, which has never been my experience with the systems written in other paradigms.
Matthias
00:09:45
Startups can gain a competitive advantage by using Rust. But even for established companies, Rust's performance can unlock new possibilities for products that would be harder with other languages. Here is what Sentry's Arpad Borsos had to say about how Rust's performance is enabling them to think about services which need to scale to handle large amounts of traffic.
Arpad
00:10:07
And maybe I've hinted on this previously that Sentry is also working on a metrics product. And metrics especially wouldn't be possible without having Rust code in between to handle the amounts of traffic. And to give you some numbers as well, there's been a couple of initiatives internally to rewrite some parts of the whole pipeline. The whole pipeline is enormous. It has lots of moving parts. And there's tons of Python code still there. And we try to move little bits of this to Rust, things that are especially critical or things that are costing a lot in terms of cloud costs. And here, one team has rewritten one critical part of the infrastructure in Rust, and they have observed 10 to 20x improvement in throughput and eventually in cloud cost as well. Because you can then just scale down the number of horizontally scaled workers and the CPU and memory allocations they get, and that has got 10 to 20x improvement here.
Matthias
00:11:25
And what is the amount of traffic that this new service gets?
Arpad
00:11:30
Depending on the workloads, there is one specific use case where it has more than 100k Kafka messages in this case per second.
Matthias
00:11:41
And PupNub had a similar experience.
Stephen
00:11:43
All of our new services are typically elected to be written in Rust. Everything going forward will be Rust. This is due to our scale. And, you know, we've seen fantastic results from it. We actually have some real world numbers. as we have replaced some of our services with Rust. And it has been fantastic.
Matthias
00:12:04
Can you share some of these numbers? I would love to hear them.
Stephen
00:12:07
Yeah, yeah. Okay, so some of our services are written in Python. It has some performance considerations in general with Python. We have gotten past some of those using something called PyPy, which is more about memory routing and pathing, but at runtime.
Matthias
00:12:30
Yeah, like a just-in-time compiler for Python.
Stephen
00:12:34
Exactly, yeah. And there's a trade-off though because it takes up a lot more memory okay like a lot like it can go gigabytes per process but you know the CPU is you know five times faster which is in CPUs are expensive right you pay around four pennies per hour for CPU on Amazon for CPU and then memory is what it's like half that price right based on so it's like a good trade-off in terms of cost. When we replaced that service with Rust, we saw another 5x boost, not only in memory, but in performance as well, on top of it.
Matthias
00:13:12
That means you use one-fifth of the memory in Rust compared to PyPy.
Stephen
00:13:18
Well, actually, way less. It's more like 30 megabytes or something like that in Rust versus a gigabyte.
Matthias
00:13:25
Lastly, as yet another angle, Rust's performance can be seen as a feature, which translates to a better overall user experience.
Folkert
00:13:34
So I think you can ultimately distill this into value for users. A lot of this low-level stuff is really about user experience, if you want to think about it that way. Performance is ultimately user experience. And I think we don't really appreciate that enough, but there's a lot of research from Google about if a page load is this many milliseconds longer, then this percentage of people will just lose interest. This is very well established. In practice, a lot of software is really slow, and only people excited about this low-level stuff can fix that. And then imagine the sort of like value that that actually has, the sort of multiplier of this one person going really deep down into that rabbit hole, coming up with a more efficient implementation. But then all users of that tooling, and of course, that tooling, like its usage is spreading very quickly because 1000x improvement is ridiculous. You know, the sort of the amount of time saved, the amount of extra productivity gained is really something that is almost hard to even capture, right? So like that is ultimately the power of this low level stuff. Like, yes, it is very nerdy. Yes, it is very. It seems to be very distanced from sort of the real world sometimes, but it has real outputs.
Matthias
00:14:58
In conclusion, companies choose Rust not only for its performance, but also for the secondary benefits that come with it, such as improved hardware utilization, cost reductions and a better overall user experience. In combination with Rust's other advantages, such as safety, reliability and maintainability, these benefits make Rust a compelling choice for companies looking to build robust and reliable systems that scale to handle large amounts of requests. Chapter 2. Requirements for adopting Rust I wanted to hear first-hand from companies that have successfully adopted Rust in production about what they think are the requirements for considering Rust within a company. Which resources and expertise do you need to have in place to be successful with Rust? And what non-obvious challenges might you face when adopting the language? A recurring pattern I noticed is that engineers started learning Rust on their own and then brought it to their companies. They experimented with Rust for smaller projects and gained confidence with the language before using it for anything serious. Here's how Apollo's Nicolas Moutschen learned Rust.
Nicolas
00:16:07
In my case, I'm an experiential learner. I'll learn best by actually just doing and writing code myself and doing experiments. And the first thing I'd like to say on that, by the way, when you're starting your Rust journey and you're starting to write code, it's okay. I mentioned the tier one stuff where you do a lot of clone, where the broad checker is still confusing, and sometimes you get a lot of compilation error that you don't necessarily understand. It's fine. It's totally fine to get there. By the way, even if you clone excessively resources, you might still be way better in terms of performance as what you would have if you're using JavaScript or Python. I mean, coming from Python, Python does a lot of cloning of values under the hood that you don't realize. Rust just makes things explicit, right? So some people also learn better via books, via tutorials and so on. And there's so much great resources all over the board. It's really hard for me to kind of point to one. In my case, I use the official resources that you find on a Rust lang website. So there is the Rust book. I think there's the Rust from Examples, I think it's called. I don't really exactly remember now.
Matthias
00:17:19
Rust by Example, yeah.
Nicolas
00:17:20
Yeah, yeah, yeah. These ones already helped me so much. And then I was just writing, starting with CLIs, writing some libraries and so on. I had a mentor, which when I started using Rust before at AWS and I had a mentor there, it was very helpful. We also have, because now we have a sizable number of Rust engineers, We have a Rust Slack channel where everybody can ask questions, but there's also the official Rust Discord. People are very friendly in the Rust community, so don't hesitate to reach out and ask questions to people. They will usually point you in the right direction. Then as you're getting more experience, I would highly recommend the book Rust for Rustacean. If you feel like you're getting more comfortable, you don't, you know, you rarely get compiler error nowadays or something like this, right? Go with the Rust for Rustacean. It's a great book to really level up your Rust knowledge. Yeah. So that's kind of my, yeah, like six to 12-month recommendation journey, if you're looking at learning Rust or where you might feel yourself to be in terms of Rust experience. But to be honest, like a lot of people on our team were able to make PRs in Rust after a few weeks, right? Back when I was at AWS, I was a solution architect. My job was to talk with a lot of companies, and I had people reaching out to me saying, hey, that company wants to start using Rust, but they're a bit afraid to be honest they're a bit afraid of starting of not finding the talents and so on. Well that's fine you can train people to to learn Rust and they can start to be productive, in a few weeks.
Matthias
00:18:55
Or here's the story of Paul Dix.
Paul
00:18:57
Was just like you know what I haven't given this language a fair chance. I've been kind of judgmental based on no actual knowledge so I'm going to like commit to actually really trying to learn it. And then in, I think it was in the fall of 2019, the async await stuff finally landed, right? And obviously that stuff had been in the works for a while, but it landed in a way that you could actually use it for real. At least I thought. People who were in Rust probably had already been doing it for a while. But fall of 2019 is when I remember saying like, oh, now that that's landed, Rust is going to become, basically like maybe the language of first choice for building high performance server side software, right? System software of all kinds. And we had some big things we had to do with InfluxDB. And I thought, well, maybe I should try writing this next, at least this core of the database piece in Rust.
Matthias
00:19:58
Once you start to consider using Rust for critical parts of your system, consulting can help hit the ground running by speeding up the learning process and helping make the code more idiomatic.
Paul
00:20:09
So we ended up bringing in Integer32, which is a consulting company, Jake and Carol, to help us out. One with like code, writing code and doing code reviews and like helping us out there. But we've also had them, they run like a multi-day class training session and we've run had them run people through that a few times. L ike different groups of engineers within our company through that so that I think that's really really helpful like what the experience I didn't have in the first like six months I was personally writing Rust was I had nobody I could ask really basic dumb questions to which I think would have helped my learning process a lot more, but we always do code reviews, right? So pull requests with code reviews to change stuff. And we brought Carol and Jake in pretty early to do that specifically because, I knew that we didn't have the expertise in-house, and I was like, I want to make sure that what we're creating isn't a complete mess.
Matthias
00:21:15
Chapter 3. Integration with existing codebases. No project exists in a vacuum. Companies typically run a mix of different languages and technologies. Fortunately, Rust can be brought in for performance-critical parts of the system by keeping the rest of the codebase untouched, which allows for gradual migration. This is made possible by Rust's excellent support for interoperability with other languages, either through FFI or other mechanisms like WebAssembly. For example, Arpad from Sentry mentioned that Python integration was pretty straightforward.
Arpad
00:21:50
Just recently, one of the things I did specifically was to pioneer PyO3. So we created a new Python package written using PyO3 and Maturin. And I'm actually really, really satisfied with how this turned out to look eventually. There were some really arcane ways of how to write Python extension modules previously where you had to use bindgen to generate C headers and then from there use, I don't know what the Python module is that actually consumes C headers and CFFI bindings and then just have tons more Python glue code on top of it. And PyO3 just does all of this for you. And that's really liberating. And I believe doing this one project now and actually these last couple of weeks, we've demonstrated that this actually works and it works really well and it's productive. And you can use this to write even more code in Rust that's been used from Python directly.
Matthias
00:23:03
You don't have to rewrite everything in Rust to benefit from it. Chapter 4. Hiring Rust Engineers. Going in, one major concern of companies is that it might be hard to find Rust developers. In fact, the interviewees have found that hiring Rust developers is relatively easy, because many developers are excited to use Rust for work.
Paul
00:23:24
I announced that we're working on this new core of the database in November of 2020 in a talk I did. And I said we were hiring and basically like we got a bunch of inbound interest because of the fact that it was written in Rust.
Matthias
00:23:39
Arroyo had a similar experience.
Micah
00:23:41
Because as a small company, you can attract people because they want to work in Rust. And that's a big incentive to work for you. And I think that's working in like maybe a slightly obscure language. You get those people who are really excited about it. And that can be a big boon to you. But for big companies, they kind of just see the risk side of that equation.
Matthias
00:24:06
Micah Wylde touches on a critical point here. The size of the company matters. For smaller companies, using Rust can be a competitive advantage as it can attract better talent who might otherwise go to larger companies. For larger companies, though, the risk of not finding enough talent could be a concern.
Micah
00:24:24
And if you're Google and you need to hire 10,000 engineers, I think you should be rightly concerned about hiring 10,000 Rust engineers. I doubt there are that many Rust engineers in the world. But for a smaller company, that's not an issue at all, right? Hiring three Rust engineers is pretty easy. And I think especially for a small company, it's an advantage in a way that it maybe isn't for a big company to be using Rust.
Matthias
00:24:51
That also means that bigger companies might have to invest more in training their own engineers and have a long-term mindset for investing in Rust. Chapter five. On Rust's complexity and learning curve. One aspect developers must adapt to when learning Rust is its approach to complexity. R ust doesn't conceal complexity, rather it makes it explicit, compelling you to confront it directly. This explicitness in handling complexity can be both unfamiliar and challenging but at least for me this transparency was the very reason for learning Rust, to gain a deeper understanding of lower-level concepts and have more control over hardware. However, this means Rust might present a steeper learning curve compared to other languages. There is another side to this coin though. By making complexity explicit, Rust not only demystifies the inner workings below the surface but also enriches your understanding of programming fundamentals without obscuring important details. Here's Folkert De Vries' perspective on the matter.
Folkert
00:25:56
I think with regards to how things work, I think it's also something that is sort of valued in the Rust community, that this sort of low-level detail is sort of appreciated. Like in a lot of other languages, people will say, oh, don't worry about that. The runtime will take care of it. Perhaps even the compiler will take care of it. But of course, in Rust, we know if you actually build the compiler or are quite close to it, you will know exactly what limitations it has. And also, you will know how fun it is to actually put a bunch of bytes in the right order and have that constitute a program, right? The sort of like hacking around with and really playing with computers, really commanding the machine, I think is something that you can certainly find in the Rust community. ou know, a couple of other communities as well at this point, like, it's something you don't see all that often. But that excitement about really commanding your hardware is that is really cool.
Matthias
00:26:58
One often neglected advantage of Rust is that you can start to learn about systems programming concepts by reading the standard library and the source code of popular crates.
Folkert
00:27:08
And in Rust, you can actually just go to source and like it's right there. It's a it's a bunch of like more primitive values and you actually can understand how it works like most other languages don't have this like even in C or C++ if you go to the source code of like the libc or the standard library it is incredibly arcane it looks very different from sort of standard C source code in in my view anyway as not a very experienced C programmer. Like that stuff just looks really weird. And in Rust, you can just find something, find an implementation that makes sense.
Matthias
00:27:47
This ability to understand the implementation of the language and its libraries can be very empowering. It can help you understand how things work under the hood and to make better decisions when writing your own code. However, it's crucial to keep an eye on the complexity that Rust can introduce. Especially in a team setting, understanding the learning curve for newcomers and steering clear of making things more complicated than they need to be is key.
Stephen
00:28:12
Rust comes with all sorts of fancy bells and whistles, doesn't it? What are these generics and traits and all these fun things? And it's really easy to be attracted to those because they look appealing. And they do bring some benefits. They add a huge amount of overhead for bringing new people into the team. And as we're training on the job, that level of complexity just adds a huge barrier to entry. And Rust is massive for that. That's probably the biggest downside. Can I have like an easy Rust? I want like a simple Rust. That's what I want.
Matthias
00:28:47
On a related note, I was surprised to learn that AsyncRust isn't as hard to use as it seems, particularly for end users who just utilize these abstractions, the complexity of AsyncRust appears to be overstated.
Nicolas
00:29:00
So on that end, the AsyncRust part simplifies a lot of things. I will actually go back a step and talk about what I've seen in terms of Rust developer experience, and that will be quite important for this, I promise. So what I've found, if you look at a lot of Rust developers' journey, there's kind of three main phases. There's the first phase where you're starting out, you're still a bit confused by the borrow checker and other things from the compiler, and you clone a lot of things and so on. Then there's a point when you start to be comfortable with a lot of these systems and so on, and you might start using AsyncRust and you're able to be quite productive. And then there's a third phase when you start to have mastery over some of the more intricate systems, like in-depth knowledge of how Tokio works, for example, and how some of the more more complex async stuff work. And so if you're in the second category, if you are people that are able to be proficient in Rust, but you don't necessarily know all the details of the libraries and so on, AsyncRust is already a great tool there, because you're just able to make something that works, it's going to be highly optimized, it's going to be able to handle a lot of traffic without you having to put much thought into making it work nicely. Even in the first category, even if you clone a lot of data, You know, it's still working great. There's so many great libraries nowadays that are just available, have a great API. So you can get started in AsyncRust and just already just by using AsyncRust, you need to have systems that are going to be very performant when you think about like microservices that need to handle potentially thousands of RPS of traffic or something like this. But then when you get to do the third tier and you start to dive really deep into AsyncRust, then you can start to do some pretty clever optimizations, some pretty clever thinking in terms of how to optimize your asynchronous communication, to avoid cross-system race conditions, distributed race conditions, to make sure that you actually fetch multiple data systems, multiple pieces of data at the same time, and so on, all in one go. So there's kind of this journey of growth into the kind of Rust async ecosystem, where you already get a lot of benefit just being able to make async calls and having libraries like axum and so on to just write a server that's already going to be able to handle a lot of traffic without you having to think too hard about it.
Matthias
00:31:41
Chapter 6. A Broader Look at Rust's Adoption. A common concern I frequently encounter is the belief that Rust isn't widely adopted in the tech industry. Yet, based on my conversations with numerous companies, it's evident that this perception is outdated. Rust is actively being used in production across a broad spectrum of industries, including databases, cloud infrastructure, IoT, web services, security, and finance, to address real-world challenges. What's also remarkable is the convergence on certain patterns and libraries among these companies. Many have embraced AsyncRust alongside libraries like Tokio, axum, SQLx and others. This consensus indicates Rust maturity as a language, offering a robust suite of libraries and patterns that effectively tackle issues across various sectors. The reason behind the misconception of Rust's limited use might be the lack of public discourse about its adoption. While Rust continues to advance in the tech industry, not all companies are vocal about the use of the language, contributing to a skewed perception of its usage. They say produce the content you want to consume, and that's what I tried to do with this podcast. Inspired by The New Rustacean and On the Metal, I wanted to contribute my tiny part to make Rust more visible and advocate for its use in the tech industry. It took me by surprise to see how much positive feedback I got from the listeners. Seems that many people are curious about Rust in production and are looking for more information about it. With regard to the companies I reached out to, many agreed to talk about their use of Rust. It leads me to believe that there are many more success stories out there that are yet to be told. However, I don't want to paint a too rosy picture. Getting companies to commit to an interview is hard. It's difficult to get the approval from the right people and to schedule interviews, because people are generally busy and have long lead times. That's why I'm grateful for the guests who have been on the show and took the time out of their busy schedules to share their experiences with Rust. On that note, it's important to handle the trust of guests who do agree to an interview with care. I want to make sure that every episode is a good representation of the guest's experience with Rust, and that it is presented in a professional way. This is why I'm grateful for the work of Simon Brüggen, who has been editing the show from the beginning to make it sound great. Thank you for all your effort, Simon! One tradition of the show is to ask our guests what message they would like to send to the Rust community. As we conclude this season, I'd like to share my own message with the Rust community, highlighting a few items on my wishlist for Rust. Firstly, I'm eager to see an expansion of learning resources covering intermediate Rust topics. While it's relatively straightforward to get started with Rust, there's a noticeable gap in materials for those looking to advance beyond the basics and deepen their understanding of the language. Secondly, while Rust's tooling is exemplary in many respects, I believe there is room for improvement, especially in features that would benefit companies in a production environment. For instance, enhanced support for private crate registries would be highly beneficial. Thirdly, and perhaps most importantly, I'd like to encourage more companies to share their experiences with Rust publicly. Doing so not only increases Rust's visibility, but also draws more people to the language. It fosters stronger connections between companies and the community, thereby enriching the ecosystem. If you're using Rust in production, or know someone who would be an excellent guest for the show, I'd be curious to hear from you. With that, I hope you enjoyed the first season of Rust in production. We'll return for season two in eight weeks, and we we already have some fantastic guests lined up. Now is the perfect opportunity to catch up on any episodes you might have missed and to subscribe to the show so you don't miss out on future episodes. If you'd like to support Rust in Production, consider leaving a review on Apple Podcasts or Spotify. It really helps spread the word about the show. I'm also keen to hear from you directly. Feel free to reach out on Mastodon or via email. For more content on Rust in Production visit our website at corrode.dev to all the amazing guests who have shared their insights on our show and every listener for tuning in. Your continued support is the driving force behind the show. Thank you for listening to Rust in Production!

2024 - Matthias Endler

Hosted by LetsCast.fm
Create your own podcast