Rust in Production

Matthias Endler

Thunderbird with Brendan Abolivier

Brandon Abolivier discusses integrating Rust into Thunderbird with Matthias Endler, highlighting benefits, challenges with C++ and JavaScript, and the need for funding to support continued development.

2024-05-30 63 min

Description & Show Notes

There are probably only a handful of open-source projects that had a bigger impact on the world than Mozilla Thunderbird. The email client has been around for over two decades and has been a staple for many users (me included). Dealing with a legacy codebase that servers millions of users is no easy feat. The team at MZLA, a subsidiary of Mozilla, has been working hard to modernize the core of Thunderbird by writing new parts in Rust. In this episode, I talk to Brendan Abolivier, a software engineer at MZLA, about the challenges of working on a legacy codebase, the new Rust-based Exchange protocol support, which is the first new protocol in Thunderbird in over a decade, and the future of Thunderbird.

About MZLA

The MZLA Technologies Corporation is a fully-owned subsidiary of Mozilla that develops and maintains the Thunderbird email client. The project can be supported at give.thunderbird.net.

About Brendan Abolivier

Brendan joined the Thunderbird team in 2023 and has been working on the Rust-based Exchange protocol support. Previously, he worked on the Matrix protocol and the Element chat client. He is a strong advocate for open-source software and maintains projects written in Rust, Python, and Go.

Links From The Show
Official Links

Transcript

This is Rust in Production, a podcast about companies who use Rust to shape the future of infrastructure. My name is Matthias Endler from corrode, and today we talk to Brendan Abolivier from MZLA about integrating Rust into Thunderbird. Brendan, welcome to the show. Can you quickly say a few words about yourself and about Thunderbird, the project that you work on?
Brendan
00:00:28
Sure. Sure, thanks for having me. So my name is Brendan Abolivier. I'm a software engineer at Mozilla working on Thunderbird, mostly kind of on the backend side of the desktop application. So everything around storage, protocol implementation, this kind of stuff.
Matthias
00:00:45
Amazing. And I learned that you know multiple languages. You don't only know Rust. What other languages do you commonly use?
Brendan
00:00:54
Yeah, I mean, throughout my career so far, So I've come across a few. I've done some projects in Go. Python was the main language I used at a previous job. I've done some, obviously, some C++ and JavaScript around Thunderbird. And now I'm kind of playing around with Rust a bit.
Matthias
00:01:17
Before you started working on Thunderbird, you were at Element, I assume, and you worked on the matrix.org backend. Can you just briefly describe your work there?
Brendan
00:01:30
Yeah, so I was part of the backend team at Elements, so working on backend implementations for Matrix. So that involves Synapse, which is currently the main Matrix home server implementation out there written in Python. Also some other bits, identity server, push notification gateways, and also did some commercial projects in Python and Go within Element as well.
Matthias
00:02:01
Was there any rust?
Brendan
00:02:03
There was a bit of rust. There was one commercial project that we started within the last few months from my time at Element. But it was less hands-on and more prototype-y kind of stuff. Not really as... Yes advanced and and yeah heads-on as as the stuff that i'm currently working on now.
Matthias
00:02:30
Yeah and now you work on thunderbird when did you join the team.
Brendan
00:02:35
So i joined in june of 2023 yeah the story is actually interesting i i went to so there's a an open source conference here in london where i live which is called state of open that started last year had its first edition last year it's organized by open uk which is a big collective base in the uk doing organizing meetups and actions around open source in the country, and they they had a thing in place where if you were unemployed you could reach out to them for a free ticket to the conference and at the at that time i had been laid off by elements i think a a month or two before. And so I went there and they had a job board and one of the jobs listed there was to work at Mozilla on Thunderbirds. And so I applied and a few months down the line, after a bit of waiting and a few interviews, I got it.
Matthias
00:03:39
Amazing. I assume you knew Thunderbird before and probably you used it as well. What got you interested in working on Thunderbird?
Brendan
00:03:48
Itself yeah i mean i i've been like that's kind of one of the exciting parts of the job i've been using thunderbird for as long as i can remember i think probably 10 15 years which is which also means in a kind of scary way that i've been using thunderbird for close to half of my life which is unsettling and so yeah i mean mozilla has been for for a long time company and the community and a project that I, looked up to and you know thunderbird is something that i use on a daily basis and i have used on a daily basis for years and so so when i saw the the the listing you know i thought might as well might as well apply and see where it leads the.
Matthias
00:04:35
Thunderbird project itself underwent some restructuring and there were some rough years for sure can you just briefly walk us through the history of the project?
Brendan
00:04:47
Yeah, so around 2012, 2013, Mozilla announced that Thunderbird was being put on maintenance modes, which meant that although some aspects of the project still lived at Mozilla, the actual development and maintenance of the app was handed over to the community. And so that lasted from 2012 to a bit after, I think, 2017. Around 2017, the Mozilla Foundation, working together with the community, the Thunderbird community, started trying to infuse some life into that project. They started hiring a few people and helping the community through their fundraising work. Work and and that's bad experiments paid off in 2020 the kind of ship had picked up wind in and so mozilla started a brand new subsidiary called mzla which which is dedicated to thunderbirds to developing and maintaining thunderbirds and also and you know potentially creating other projects and products around email and calendaring. And that's kind of where we are now. Nowadays, Thunderbird employs a bit over 30 people full-time. We're currently fully... Well, the project is still funded entirely by donations. We raised $6.8 million in 2022. 2022 i'm not sure i can i can speak on on the the actual number for 2023 but it's more than that so yeah the project is going really well which is good to see you know as a user i was a bit, sad to see that thunderbird was going through a rough time and it's good to it's good to see it going through this revival and also being part of being part of that i've.
Matthias
00:07:00
Been a thunderbird user for an equally long time as you can still remember that 10-15 years for sure 10-15 years ago I used Thunderbird and I used it for many years on different operating systems it always worked surprisingly well, After a while, I kind of moved on to Apple Mail. And that was mostly because of the internal turmoil. And there were some issues with, you know, the overall user experience. But two years ago, I picked up Thunderbird again because I saw a shift in momentum. And it was really transformative. And I downloaded application and I have never looked back ever since. I kind of tossed Apple Mail into the trash, and now I'm a full-time Thunderbird user, and I couldn't be more happier. But I wonder, for me, it always worked fine, at least the last couple years. And even before that, I didn't really run into a lot of bugs. And moving a big legacy application from C++ to Rust can be a risky endeavor. What do you think about that?
Brendan
00:08:16
So the first thing I want to clarify, we're currently not planning to rewrite the entirety of Thunderbird into Rust. We currently... So Rust in Thunderbird is still very early stages. We started... Actually, the first patch in Thunderbird to even enable building Rust, I think landed towards the end of December. And the first actual Rust codes landed just a few weeks ago. So it's still very early stages. And for now, we're focusing on using Rust for new components. Maybe as the time goes, we'll need to... To rewrite some big parts of our current code architecture. And in this case, we might use Rust for that.
Matthias
00:09:16
Yeah, nonetheless, migrating to a new language is always a bit risky, at least. And how does Rust fit into Thunderbird's long-term strategy?
Brendan
00:09:27
So the reason Rust came into the picture is because we're currently working on implementing support for Microsoft Exchange in Thunderbirds. That's the first time that we're implementing a new protocol in Thunderbirds since its lifetime, really, since it grew out of Netscape Communicator 20, 25 plus years ago. And so when we started these projects, we realized that we didn't know how to implement the protocol. And obviously, because all that previous work had been done so long ago, we didn't have all that institutional knowledge around anymore. And so we started by looking through the codes, surveying a bit what's already there. And we realized that the way the protocols are implemented currently, It's fine, it works, but it's not how you would implement a new protocol right now. It's how you would implement a protocol 20, 25 years ago. Better tools and platforms to implement something as critical and sensitive as, protocol support for an email for for for email because you know email is very sensitive you don't want to you don't want to mess with the actual data that's that's transmitted you don't want want to mess with. The structure of that data misrepresent anything. Rust provides us with some very nice guarantees in terms of memory safety, in terms of typing, in terms of, you know, all those kind of things that we need that weren't around 20 years ago. So that's kind of how it fits into our current strategy. And also another obvious point of why we're using Rust is because Thunderbird is built on top of Firefox at the moment. And so Firefox has Rust codes in its code base. And so a lot of the necessary infrastructure in the build system, in the CI system, etc. et cetera, already supports Rust. So even though we did have to do some work to be able to have our own Rust code in Thunderbirds, it also meant that it wasn't something that we had to start from scratch. And so it was easier than just going with another platform. Platform so in terms of the in terms of the long-term strategy it's not entirely clear yet you know what will be written in rust what won't be written in rust it's clear that rust will be will still be used to to develop new new things that are new components that that needs the kind of guarantees that Rust provides. We might, as I mentioned earlier, we might use it to rewrite existing components if we feel like there's value to it. It's not that clear-cut yet what will end up being Rust what we want.
Matthias
00:13:10
Can you imagine that some new components will be written in C++?
Brendan
00:13:16
At the moment, possibly, there's, well, at least with some C++ in it, there's the way things currently work in terms of something called XP-COME in Thunderbird and Firefox, which is kind of that interface layer between languages. It stands for cross-platform component object model. The idea is basically that you define some interfaces in a platform-neutral language called XPIDL for cross-platform interface definition language, I think. And you can then implement and use implementations of those interfaces in JavaScript, C++, Rust. Rust, the bindings for xpcom in Rust are still fairly, I don't want to say immature, but they lack that extra, they still feel a bit rough to use at the moment. And there are still some features around memory management, around pointers, around references and so on, that aren't supported or aren't supported well. And so for that, we might need to shift a bit more into the land of C++. That situation might be improved in the future. We hope so. And we'll probably work with the XP-CAM team to try to make things better. But it's likely that at least at first we will still have some C++ in our new components even though the majority will be written in Rust.
Matthias
00:15:07
As far as I remember, you said somewhat like 30 developers work on Rust more or less regularly. How many of them work with Rust code or will deal with Rust on a regular basis in the future?
Brendan
00:15:24
So at the moment, we're about three people dealing with Rust because that's the size of the amount of developers in the team working on implementing Exchange. The current backend team at the moment, I think is about five or six people. I might be wrong on that, but I think it's about that scale. And those people might have to deal with Rust in terms of writing code for it.
Matthias
00:16:02
And earlier you mentioned tooling was one of the big advantages of Rust. A lot of people mention stability and robustness and maybe performance, but you mentioned tooling, which I find interesting. First of all, why do you think this is the biggest benefit of Rust? And second, maybe can you talk about the tooling that you mean specifically?
Brendan
00:16:26
I mean, so, yeah, when I say tooling, I mean the tooling that already exists in Firefox, meaning that there's currently tools and infrastructure in place in Firefox to build Rust codes. But when you build Firefox and by extension Thunderbird, there's already code in place to generate bindings for XPCOM interfaces in Rust. And to interact with those bindings, to define components in Rust. There's already some amount of documentations on the Firefox site on how to use Rust to interact with the rest of Firefox. So, yeah, when I mentioned tooling, I meant more tooling specific for Firefox, which is helpful to us because we're based on, well, Thunderbird is based on Firefox. So if Firefox already has some infrastructure for building and using Rust, it's something that we can use as well.
Matthias
00:17:38
It probably also helps that Rust has a similar memory layout, C++, and also uses a similar build system, or at least the binaries that it produces uses are similar with respect to, you know, name mangling and all of these details that you have to deal with on an FFI basis.
Brendan
00:17:59
Yeah, definitely. I mean, I wouldn't be able to give you, you know, more details, you know, that Mozilla has, you know, a dedicated team to take care of the build system in Firefox. So I wouldn't, I don't have all the, you know, nitty gritty details. I do know that the way the build system works is by linking all Rust symbols on a more generic library. And so having that kind of compatibility is definitely helpful.
Matthias
00:18:36
When you said that you are using somewhat of the same code base as firefox my immediate thought was do you also use the rust code in firefox to some extent i know that they worked on css and layout engines and at least in thunderbird you can show html emails do you share some parts of that code or is it completely separate.
Brendan
00:18:59
I think i think we do use some of the some of the Rust code from firefox kind of indirectly because we use some of the, some of the components from firefox in thunderbirds the entire you know rendering engine front-end engine all that stuff is directly lifted from firefox the networking engine is the same as as firefox so i'm not entirely you know obviously familiar with the firefox code base not as well Well, I'm somewhat familiar with some parts of it, but definitely not as much as a Firefox developer would be. But I don't know which parts of the components that we use are in Rust. I know that there's definitely some C++ there. There's probably some Rust there. I wouldn't be able to give you an exact kind of split of what is Rust or what is C++. But there's definitely a mix there.
Matthias
00:20:01
And maybe looking at it from the other perspective, How often do you have to deal with the C++ intricacies on a daily basis?
Brendan
00:20:10
It very much varies depending on what I'm currently working on. At the moment, I'm mostly working on Microsoft Exchange. So when we need to work through implementations and stuff like that, it's going to be a majority of Rust. When it's the initial kind of survey work that I mentioned, it's going to be maturity of C++ because the existing code that we need to survey and understand is written in C++. We probably will have to write some amount of C++ for Exchange. So far, I haven't been working on a part of that project that involves doing that. I know that one of my co-workers, Sean, has had to look into that, but I don't necessarily have the details on that. But outside of that main project that I'm currently working on, when I investigate some bugs or crashes or things like that, which is part of the responsibilities of any developers, there is, at the moment, not really a lot of rust involved because, as I mentioned, we only started having Rust code in the code base a few weeks ago, and they're currently building blocks for a feature that isn't currently fully implemented and therefore isn't fully used in Thunderbirds. But I have to deal with a bit of JavaScript and a bit of C++, depending on the area of the code base that I'm working through.
Matthias
00:21:59
At least you need to understand basic building blocks and how those things work together to work on Exchange support.
Brendan
00:22:06
Yeah, definitely. I mean, one of the first steps in that Exchange project has been basically trying to figure out, like, you know, we don't know how to build a protocol in Thunderbird, how to build support for a new protocol in Thunderbird. Let's have a look at what already exists. And so, yeah, I think it was back in September, I spent two, three weeks kind of going through the implementation of IMAP in Thunderbird, which is written entirely in C++. So that was a lot of, you know, diving into C++ code, figuring out what it does, figuring out how every block fits together.
Matthias
00:22:46
Maybe before we dive into Exchange support, let's give our listeners some context. How big is the Thunderbird code base? Do you have any numbers, any statistics around that? And also, what protocols are supported?
Brendan
00:23:00
Yeah, so at the moment, in terms of support for mail protocols, we have your standards IMAP, POP3, SMTP. There's also an NTP for newsgroups slash usenets, which also exists. We also have, Thunderbird also has a whole instant messaging module, but that's kind of a separated part of the code base. In terms of lines of code, at the moment, if you accumulate the C++, the JavaScript, and the rest, you come to about 585,000 lines of code. If you remove everything but mail and newsgroups and things like that, you come down to 475,000, thereabouts. So, you know, a bit of a drop, but still a lot. A significant amount of that is C++. A big amount as well is JavaScript. One of the reasons is the front-end, currently the UI of Thunderbird, is mostly made of, you know, web technologies. So, JavaScript, CSS, HTML, or XUL in the oldest parts of the application. So, and currently we don't have a good separation between front-end and back-end in Thunderbird. That's kind of one of the things that Rust might help us with in the future. But we currently, so it's difficult at the moment to figure out, you know, which of these JavaScript is doing more front-ended bits, what is doing more kind of intermediate slash back-ended bits. And in those almost 500,000 lines of code, there's about a bit over 500 lines which are Rust at the moment. So demonstrating that we're definitely very early stages at the moment.
Matthias
00:25:15
Okay, so we set the stage for integrating Exchange support. First off, I wonder why is Exchange so important for the Thunderbird users? Is it an often requested feature?
Brendan
00:25:31
Yeah, it's actually one of the most requested features on Thunderbirds. The thing that you often find with Exchange is the people who are using it are usually not the people who made the decision that they had to use it. Because usually you find exchange in universities, in companies, in the workplace. And most users don't have the authorization or power to just change a critical part of their workplace infrastructure. There are compatibility layers that exist and that are provided by Microsoft with IMAP, SMTP, and so on. But very often, at least in a lot of workplaces, they're disabled. I'm not entirely sure about that, but I think Microsoft is usually pushing for those to be disabled automatically. Under some sort of, I think, security argument. I can't remember the specifics. And so that means that a lot of users have no real possibility of using more open standards and protocols to consume their work emails.
Matthias
00:27:12
And I guess the alternative to using Thunderbird in this case would be to use Office 365.
Brendan
00:27:19
Yeah, Outlook, etc.
Matthias
00:27:21
Outlook, yeah.
Brendan
00:27:22
And it was actually a bit controversial at first. There were some discussions about, and a bit of pushback, not a lot of it, but on whether Thunderbird should implement Exchange at all. Because Thunderbird is and has been a big advocate for open standards. And so, you know, it feels a bit counterproductive and a bit, you know, going against that to suddenly implement Microsoft's proprietary protocol right into the app. But the reality of the situation is Thunderbird doesn't have enough influence, market share, etc., to be able to influence Microsoft, obviously. Obviously, in, you know, we can't rock up to Microsoft Surface and say, you know, we, all users aren't happy, you have to enable IMAP for every account. That's just not something that's realistic at the moment. And so with that in mind, if we keep, you know, pushing back again, If we keep arguing about keeping that status of... Like restricting protocols to only open standards the user is the one that's being punished, in the end and is the one that you know would like to be to use thunderbird but just just can't because thunderbird doesn't want to support them and their and their workplace doesn't want to, to to to support thunderbirds and and i would say that i would personally say that being in this kind of situations, adding support for something proprietary, such as Microsoft Exchange, although it would be obviously better if that wasn't something that was so needed. It's actually helping, I think, Thunderbirds advocate for open standards because then you enable users to use Thunderbird to have their work email, but also their personal email, their emails that they might have, mailboxes that they might have with other organizations, etc., to have them all in one app, which is Thunderbird. And those other mailboxes are likely based on open standards in terms of communications and with a server and all this.
Matthias
00:30:11
Is that the first open implementation of the Exchange protocol that you know of?
Brendan
00:30:16
No, so there's currently at least one other open source email client that implements support for Microsoft Exchange, which is Evolution. And we've actually, in the early stages of the project, looked a lot at what Evolution does and worked a bit as well with some of the maintainers of the EWS add-on for, well, EWS being the specific protocol for Exchange that we're implementing that add-on for Evolution.
Matthias
00:30:55
Evolution and walk us through a typical day do you sit down and then you look at the spec you look at the evolution code and then you try to model what you see in rust do you work on the domain model right now and and how do you deal with with making this as idiomatic as possible in rust, So.
Brendan
00:31:19
At the moment, it's not really something that I'm working a lot with. My focus for the past couple of months has been more oriented towards code infrastructure. So when we started looking at what we needed to be able to implement Rust in Thunderbirds, we realized, you know, currently XPCOM, which is the only way we can interact with something like the networking component of Firefox slash Thunderbirds, is not very idiomatic for Rust because it uses, you know, obviously that component is written in C++ and obviously C++ doesn't have, nice stuff like, you know, asynchronous and, well, async await syntax and that kind of niceties. So one of my first areas of focus has been, to implement some kind of translation layer to first enable asynchronous codes. Well, asynchronous code to be run with the native Async 08 syntax in Rust. And then to build on top of that to kind of expose a nicer, at least, and more idiomatic HTTP client in Rust. Which is kind of based on, well, inspired by the request crates, but isn't obviously a one-to-one mapping. There's some demos and links in the page for a FOSDEM talk that we did a few weeks ago on specifically Rust in Thunderbirds. Which is called "Thunderbird: How to Exchange Rot for Rust", which is a fantastic pun that Aiki, one of my coworkers, workers came up with kind of on the spots when we were brainstorming names and there's there's a there's a couple demos on the during the talk and in the in the same page there's a few links to the the source code of of those of those crates and and so at the moment i'm taking a bit of very very short and temporary kind of step back from from rust to kind of explore how to hook how to hook exchange into the account setup feature of thunderbirds so i i'm assuming that once i'm out of you know exploration phase and i start prototyping and actually implementing i'll come back to Rust, but at the moment it's more exploratory.
Matthias
00:34:15
What was the decision not to use requests directly and instead write something else?
Brendan
00:34:24
So, I mean, it's true that we could have used requests directly. However, because, you know, Thunderbird already has a fairly, you know, a filter completes networking stack that it's borrowing from Firefox, which is called Necko and has existed for the past 20, 25 years as well. And is hooked into all the nice things like DevTools and the network inspector and things like that, it didn't feel... I mean, it felt a bit sad to just ignore that and use a completely different HTTP stack. And it also felt like it wouldn't lead to the most maintainable codes. Let's imagine you're tracking a bug in that implementation and suddenly you can't use all of the nice network debugging tools that the whole rest of the code base uses. And so, yeah, we decided to, instead of using something that already exists, like requests, to instead try to build an interface that resembles requests but under the hood uses that networking component.
Matthias
00:35:52
Did you already catch any bugs in the c++ code ever since you started integrating rust.
Brendan
00:36:00
I don't know if i if i would call them bugs i have definitely noticed a few, kind of weird behaviors that can be identified such as when you're building your HTTP request, you need to set the methods after the... Sometimes you need to set the methods after you set the body of the request because the components under the hood, like the part of NECO under the hood that's going to build your request might try to infer what method to use depending on the content of that body, because it has some backwards compatibility with some previous versions of the interface that it uses. Which is something that I don't know if it's, I would say it's more, you know, complexity that is inherent with maintaining a complete complex 20 plus years old code base than actually bugs. But it is definitely something that felt a bit unexpected, but wasn't necessarily difficult to work around.
Matthias
00:37:23
Are there any specific libraries or crates from the Rust ecosystem that you could leverage and use for Thunderbird? Or did you literally have to build everything from scratch?
Brendan
00:37:34
No, there are a few things that we're using. I know we're, you know, especially around the... So EWS, which is the protocol for Exchange that we're currently implementing.
Matthias
00:37:55
That's for Exchange Web Service?
Brendan
00:37:58
Yeah. And so that protocol is XML over HTTP. The HTTP parts, because it's network-based, we kind of decided to do our own thing. The xml parts we started by looking at the ecosystem because that's something that we can because there's a lot there that we can just reuse ultimately we decided that for the, serialization side of things because of because of. Some strictness around SADA's data model. We, I mean, it doesn't work well enough with XML, with the kind of structures that you find in XML for what we want to do. And so for serialization, we did end up implementing our own derived macros and so on. For deserialization we we are using if i remember correctly we're using a set implementation based on quick xml and and also that serialization crate is also based on quick xml so there's you know a bunch of the there's a bunch of the of existing crates that's where that we're using and we also hope to contribute to that ecosystem through the work that we're doing currently. So all the XML sterilization that we're working on and even at a higher level, the exchange support that we're working on, we're trying to design those so that we can ultimately have them as separate crates that people can just reuse.
Matthias
00:40:00
Does that mean you use two different crates for XML serialization and deserialization? You mentioned QuickXML for deserialization, I guess, and a custom version for serialization then?
Brendan
00:40:15
Yeah, we start initially, and ideally we would just use everything, we would just use the same crate for both. The issue with serialization is, The current crates and set is support of those crates, well, versions of those crates, have a few issues, which is that in XML, you have namespaces on your XML tags, and those tags can also have attributes which are different from the values and things like that. Right. In order to generate XML that has the right namespaces and attributes. We realized that we needed our own thing because the existing tools and crates within the ecosystem either didn't support correctly, for example, setting a namespace on a tag, for example, or would support setting namespaces and attributes, but with a lot of boilerplates, which, you know, for a small XML API wouldn't necessarily be that much of an issue. But the AWS protocol has, I think from memory, about 100 different operations, generations, 1600s or 1700s, different data structures. And so the smallest amount of boilerplate that we can eliminate is a huge win for the overall maintainability of the whole thing. And so that's why we've kind of looked at having our own thing for serialization. Because we do need to have our requests being formatted in a very specific way. For the responses, we don't necessarily need to be that strict. Because ultimately, in the response, there might be some data that we don't need. or that we can be a bit more flexible about. For deserialization, we can just reuse a lot of what already exists in the ecosystem.
Matthias
00:42:48
Yeah, I can imagine that it must be a nightmare to run integration tests against the Exchange protocol. Do you plan to have some sort of end-to-end test that you can run locally? How do you track regressions and and how do you make sure that you implement a protocol correctly.
Brendan
00:43:11
Yeah that's a very good question and one that's you know is very much on our mind but we don't really have a good answer to just yet the i think ultimately we would like to have like a automated end-to-end tests for now all of our end-to-end testing is done kind of manually because nowadays you can just use a hotmail.com email address to send AWS requests to. So that's not too difficult, but also obviously not that future-proof. So it is something that we'll need to figure out in the coming month. But yeah, that's going to be possibly a tricky bit.
Matthias
00:43:57
Yeah, there are tools for this which allow you to record HTTP requests and you can replay them over and over again until you get it right. But at the same time, it doesn't really help you that much because the things that you record are the things that you know about and the question comes after.
Brendan
00:44:15
I mean, ideally we would have, you know, a full-fledged Exchange server running in CI and we could just send requests to it and see, you know, if the actual implementation does the thing that we expect it to do. It's not clear yet whether that's something that we'll be able to do or whether we'll need to go another route. We'll see.
Matthias
00:44:39
Maybe you eventually need to build a mock exchange server. Fake change. I don't know.
Brendan
00:44:46
That's a good name idea. I'll keep it in mind if we need to go there. I know there's also a couple open source EWS implementations out there, server implementations are there. So maybe that could be another, that might be an alternative, yeah well it's it's something that's yeah we'll definitely need to look into.
Matthias
00:45:10
I asked a few engineering friends what would be the things that thunderbird could use rust for in thunderbird and it was surprising because people almost anonymously said for search and for plugins for search because sometimes apparently search doesn't work or at least there were some edge cases and for plugins because so far you can only write plugins in javascript i guess and it would be nice to have web assembly support and write plugins in rust what do you think about that i.
Brendan
00:45:50
Think it's an interesting prospect to consider at the moment i'm not entirely sure whether that's the route we want to go, because on the search side of things, rewriting search entirely in Rust would be a huge undertaking. And it's not even clear yet what that would look like in the future, because we're also doing a lot of work around reworking the storage layer of Thunderbirds and so on. It might be the case that in a few years, we decide to rewrite part or the entirety of the feature in Rust, but that's not really on the horizon just yet. In terms of add-ons slash extensions, I mean, I'll preface by saying that I'm definitely not the best person to talk about add-ons. We have one of my co-workers, John, is actually working full-time on taking care of the add-on ecosystem and implementing new APIs and things like that. I know that, I think I've heard at first then that, obviously, I mean, don't take my word for it, that a recent version of Firefox has added support for embedding some Rust into add-ons. That might be something that we'll want to also support in Thunderbird. I don't know the specific details. If anyone's interested in getting more information or contributing to that effort, feel free to go on Thunderbird.net, find our community channels and reach out to the Thunderbirds add-ons community. Which would probably be happy to help in that area.
Matthias
00:47:58
When we started the conversation, we established that you have a background in many languages, including Go and Rust. Can you briefly contrast the development experience in both languages?
Brendan
00:48:12
Yeah. I mean, I haven't done any Go in almost a year, Yeah, so it's kind of starting to fade a bit. I would say that there are some things that I think Rust, where Rust is nicer than Go, obviously memory management is one of them. And I think also in the area of error handling, like not having to just return error as every single place you could ever get an error. Whereas Rust has a result, and if you don't really care about doing your error handling at a specific part of the code, you just question mark it, and you're done with it. I think one area where Go is a bit nicer than Rust is with its standard library. And I'm not saying that Rust standard library is not good or is designed poorly or whatever, I don't think that's the case. I just think that Rust is still a fairly young platform. I mean, it's about, I think, 12 years old now, but it's only started booming and getting used more widely maybe five, six years ago. And that's kind of ramping up. I think I remember reading not too long ago go that you know microsoft was starting to invest into rewriting some parts of windows in in Rust which is super cool to see but the but the the standards uh library in Rust still feels a bit young i think especially like i'll i'll use a couple examples you know we we looked at the the current kind of state of, you know. Sending HTTP traffic in Go and how we can make our current work interoperable. Like, you know, having a Rust crate that is specific to Exchange, that's, you know, an external crate and you give it like an HTTP client that's either the one from Thunderbird or Request or something else that, I don't know, Thunderbird for Android when it comes around might want to use. And at the moment, there's not really a solution to that. There are third-party crates that implement some types for HTTP. I think the most widely known and used is the HTTP crate that's maintained by the hyperproject. But the way those crates are built, A, it means that any interoperability that you would get that is, you know, only if the crates, only if those, if your two clients were using those exact same times, for example, for the same time for a request, the same time for a response. And, you know, I could be wrong, but I know that request uses some of the types from that crate, but like, you know, methods and status codes. But, you know, request response types it's there on types. And the way those traits, the way those types are defined, they're defined as tracts, not really as traits, which means that you can't really have your own logic in the specific implementation of each method of that type. And I'm kind of circling back to the standard library, because obviously the HTTP crate is not part of the standard library. But it would be nice, I think, and potentially necessary in the future to have the, you know, to kind of standardize, to have a more standardized approach to those kind of operations like sending an HTTP request or, you know, even serialization and deserialization. Like Serdi is one of the major players in the Rust ecosystem to the point where I think it's even mentioned in the official API guidelines for Rust. But it's still technically a third-party crate, and it's only a de facto standard. And I think there's, you know, room for more, for better and more flexible standardization within the standard library that other platforms like, I think, Node or, you know, Go might have that I think would benefit Rust.
Matthias
00:53:38
On the other side, one could argue that having all of these crates outside of the standard library helped make them more mature. We can look at error handling, for example, if we decided on a standard like error chain before, that's pretty much what we would be stuck with right now. But this error and anyhow might arguably be better or have better abstractions because they came later in rust's you know life cycle and they used better abstractions like derives for example.
Brendan
00:54:14
yeah i mean i'm not i'm not saying i'm not saying that that you know the the the maintainers of the standard library should just take the the you know top five most used crates in in the ecosystem and just integrate them. And I'm not saying that it might make more sense, that it might make the most sense for Rust to standardize all of those aspects now. I think my point was more that it's one of the downsides of picking an ecosystem that is still maturing and maybe hasn't reached the same stage as other languages are, that, you know, you might be lacking in some, well, some operations might be a bit more difficult to achieve because there's less support for it in the standard library. Hopefully that will come in the future. And I think, you know, the future for Rust is quite bright. Yeah, that's kind of a point of comparison between Rust and other platforms.
Matthias
00:55:26
Since you started working on Thunderbird, what have you learned about rust maybe not about rusted language more about rusted ecosystem or things that were not as clear before as they are now that you use it in production as driver for a quite big project i.
Brendan
00:55:47
Mean me personally i definitely you know using rust in like this this This, the Exchange implementation project was my first time using Rust on the kind of bigger scale than just, you know, slowly prototyping and getting frustrated at the compiler. And being in a context where I could, you know, experiment a bit more with the platform and do, you know, cool things with it, really made me appreciate like basically understand why so many people like rust because, sure it's it's sure it's frustrating to when you start to start working with rust because you know even the it feels like even the simplest program takes hours to be to to build, which and and so that's kind of uh worsens the learning curve of of rust which i was talking about i was talking about that with someone from the Rust Foundation last week who told me that he told me that apparently they they even have a a term for it which is the rust hump. And but as you as you progress in your you know getting used to to rust and getting used to all all those new concepts that come with it, you come to really appreciate the fact that, once your code builds, you're sure that you have a certain standard in terms of safety, in terms of quality, and that you have a nice and more maintainable and less error-prone code. Outputs from from that and it's it also kind of makes makes you reconsider your relation with the compiler itself because you know in other compiled languages the compiler is you know just something that turns code into into binary in rust the compiler is more is that but also kind of a a bit sort of a of an advisor in you know if you do this you're going to if i if i let you do this you're going to run into those kind of issues in the future. So yeah, I think it's definitely things that I didn't necessarily think about before when I thought about Rust.
Matthias
00:58:27
Based on the experience you have now, what advice would you give to other projects considering Rust for bigger systems, maybe systems that were written in C++ plus and have existed for a while what are the sort of requirements for success and what is necessary to kick off such a project.
Brendan
00:58:51
I would say that definitely use Rust when it makes sense. I think there are a lot of use cases where Rust makes sense when you need that memory safety, when you need those performances and so on. There are also cases where you might make more sense to use a different language. You know, programming language is a tool and, you know, the programming language ecosystem is a toolbox and you choose what fits most with, you know, with the tool that fits best with the way that you need to do. So definitely consider that, you know, ask yourself those kind of questions first. Invest in in you know training your engineers to be good Rust developers because that might take more time than let's say go for example if Rust is the good fit for your for your project definitely invest in in training the people who are going to work on it because and it's it's going to it's going to pay off in the in the future but that initial investment is necessary i would say and yeah i think i think those are the the two the the two main aspects i i think i would i would say and.
Matthias
01:00:22
Lastly it has become somewhat of a tradition around here to ask a very specific final question.
Brendan
01:00:30
And that is.
Matthias
01:00:31
What would be your message to the Rust community as a whole? Anything you have in mind could be literally whatever comes to mind. Do you have a message to the Rust community?
Brendan
01:00:43
I would say that Rust is a very cool project, a very cool language. It has a lot of features and aspects that are very unique to Rust. Yeah. And yeah, I'm looking forward to, you know, write more Rust code in the future. I'm looking forward to learn more about Rust and its ecosystem. And I'm looking forward to see what improvements and changes Rust gets in the future. Because I think the future is very bright for Rust.
Matthias
01:01:21
Brendan, it has been an honor to talk to you today.
Brendan
01:01:25
Likewise. thank you thank you for having me.
Matthias
01:01:29
Yes! Thanks a lot and i will keep using thunderbird and keep on doing the great work that you do and also for all the listeners that use thunderbird maybe consider donating.
Brendan
01:01:40
Absolutely yeah as i mentioned thunderbird at the moment is entirely donations funded we're exploring some some areas to kind of diversify that revenues that all sources of revenue but it's donations are still going to be an important part in you know funding the the work that goes into developing maintaining improving thunderbirds so we're, infinitely grateful for everyone who uses thunderbirds and everyone who donates to keep the project alive
Matthias
01:02:12
Awesome thanks Brendan!
Brendan
01:02:15
Thank you!
Matthias
01:02:18
Rust in Production is a podcast by corrode and hosted by me, Matthias Endler. For show notes, transcripts, and to learn more about how I can help your company make the most of Rust, visit corrode.dev. Thanks for listening to Rust in Production.