Matthias: This is Rust in Production, a podcast about companies who use Rust to shape
Matthias: the future of infrastructure.
Matthias: My name is Matthias Endler from corrode, and today we talk to Brendan Abolivier
Matthias: from MZLA about integrating Rust into Thunderbird.
Matthias: Brendan, welcome to the show. Can you quickly say a few words about yourself
Matthias: and about Thunderbird, the project that you work on?
Brendan: Sure. Sure, thanks for having me. So my name is Brendan Abolivier.
Brendan: I'm a software engineer at Mozilla working on Thunderbird, mostly kind of on
Brendan: the backend side of the desktop application.
Brendan: So everything around storage, protocol implementation, this kind of stuff.
Matthias: Amazing. And I learned that you know multiple languages. You don't only know Rust.
Matthias: What other languages do you commonly use?
Brendan: Yeah, I mean, throughout my career so far, So I've come across a few.
Brendan: I've done some projects in Go. Python was the main language I used at a previous job.
Brendan: I've done some, obviously, some C++ and JavaScript around Thunderbird.
Brendan: And now I'm kind of playing around with Rust a bit.
Matthias: Before you started working on Thunderbird, you were at Element,
Matthias: I assume, and you worked on the matrix.org backend. Can you just briefly describe your work there?
Brendan: Yeah, so I was part of the backend team at Elements, so working on backend implementations for Matrix.
Brendan: So that involves Synapse, which is currently the main Matrix home server implementation
Brendan: out there written in Python.
Brendan: Also some other bits, identity server, push notification gateways,
Brendan: and also did some commercial projects in Python and Go within Element as well.
Matthias: Was there any rust?
Brendan: There was a bit of rust. There was one commercial project that we started within
Brendan: the last few months from my time at Element.
Brendan: But it was less hands-on and more prototype-y kind of stuff. Not really as...
Brendan: Yes advanced and and yeah heads-on as as the stuff that i'm currently working on now.
Matthias: Yeah and now you work on thunderbird when did you join the team.
Brendan: So i joined in june
Brendan: of 2023 yeah the story is
Brendan: actually interesting i i went to so there's
Brendan: a an open source conference here in london where i live which is called state
Brendan: of open that started last year had its first edition last year it's organized
Brendan: by open uk which is a big collective base in the uk doing organizing meetups
Brendan: and actions around open source in the country,
Brendan: and they they had a thing in place where if you were unemployed you could reach
Brendan: out to them for a free ticket to the conference and at the at that time i had
Brendan: been laid off by elements i think a a month or two before.
Brendan: And so I went there and they had a job board and one of the jobs listed there
Brendan: was to work at Mozilla on Thunderbirds.
Brendan: And so I applied and a few months down the line, after a bit of waiting and
Brendan: a few interviews, I got it.
Matthias: Amazing. I assume you knew Thunderbird before and probably you used it as well.
Matthias: What got you interested in working on Thunderbird?
Brendan: Itself yeah i mean i i've been like
Brendan: that's kind of one of the exciting parts of
Brendan: the job i've been using thunderbird for as long
Brendan: as i can remember i think probably 10 15
Brendan: years which is which also means in a kind of scary way that i've been using
Brendan: thunderbird for close to half of my life which is unsettling and so yeah i mean
Brendan: mozilla has been for for a long time company and the community and a project that I,
Brendan: looked up to and you know thunderbird is something that i use on a daily basis
Brendan: and i have used on a daily basis for years and so so when i saw the the the
Brendan: listing you know i thought might as well might as well apply and see where it leads the.
Matthias: Thunderbird project itself underwent some restructuring and there were some
Matthias: rough years for sure can you just briefly walk us through the history of the project?
Brendan: Yeah, so around 2012, 2013,
Brendan: Mozilla announced that Thunderbird was being put on maintenance modes,
Brendan: which meant that although some aspects of the project still lived at Mozilla,
Brendan: the actual development and maintenance of the app was handed over to the community.
Brendan: And so that lasted from 2012 to a bit after, I think, 2017.
Brendan: Around 2017, the Mozilla Foundation, working together with the community,
Brendan: the Thunderbird community, started trying to infuse some life into that project.
Brendan: They started hiring a few people and helping the community through their fundraising work.
Brendan: Work and and that's bad
Brendan: experiments paid off in 2020 the
Brendan: kind of ship had picked up wind in and so mozilla started a brand new subsidiary
Brendan: called mzla which which is dedicated to thunderbirds to developing and maintaining
Brendan: thunderbirds and also and you know potentially creating other
Brendan: projects and products around email and calendaring.
Brendan: And that's kind of where we are now. Nowadays, Thunderbird employs a bit over 30 people full-time.
Brendan: We're currently fully... Well, the project is still funded entirely by donations.
Brendan: We raised $6.8 million in 2022.
Brendan: 2022 i'm not sure i can i can speak on on the the actual number for 2023 but
Brendan: it's more than that so yeah the project is going really well which is good to
Brendan: see you know as a user i was a bit,
Brendan: sad to see that thunderbird was going through a rough time and it's good to
Brendan: it's good to see it going through this revival and also being part of being part of that i've.
Matthias: Been a thunderbird user for an equally long time as you can still remember that
Matthias: 10-15 years for sure 10-15 years ago I used Thunderbird and I used it for many
Matthias: years on different operating systems it always worked surprisingly well,
Matthias: After a while, I kind of moved on to Apple Mail.
Matthias: And that was mostly because of the internal turmoil.
Matthias: And there were some issues with, you know, the overall user experience.
Matthias: But two years ago, I picked up Thunderbird again because I saw a shift in momentum.
Matthias: And it was really transformative.
Matthias: And I downloaded application and I have never looked back ever since.
Matthias: I kind of tossed Apple Mail into the trash, and now I'm a full-time Thunderbird
Matthias: user, and I couldn't be more happier.
Matthias: But I wonder, for me, it always worked fine, at least the last couple years.
Matthias: And even before that, I didn't really run into a lot of bugs.
Matthias: And moving a big legacy application from C++ to Rust can be a risky endeavor.
Matthias: What do you think about that?
Brendan: So the first thing I want to clarify, we're currently not planning to rewrite
Brendan: the entirety of Thunderbird into Rust.
Brendan: We currently... So Rust in Thunderbird is still very early stages. We started...
Brendan: Actually, the first patch in Thunderbird to even enable building Rust,
Brendan: I think landed towards the end of December.
Brendan: And the first actual Rust codes landed just a few weeks ago.
Brendan: So it's still very early stages. And for now, we're focusing on using Rust for new components.
Brendan: Maybe as the time goes, we'll need to...
Brendan: To rewrite some big parts of our current code architecture.
Brendan: And in this case, we might use Rust for that.
Matthias: Yeah, nonetheless, migrating to a new language is always a bit risky, at least.
Matthias: And how does Rust fit into Thunderbird's long-term strategy?
Brendan: So the reason Rust came into the picture is because we're currently working
Brendan: on implementing support for Microsoft Exchange in Thunderbirds.
Brendan: That's the first time that we're implementing a new protocol in Thunderbirds
Brendan: since its lifetime, really, since it grew out of Netscape Communicator 20, 25 plus years ago.
Brendan: And so when we started these projects, we realized that we didn't know how to
Brendan: implement the protocol.
Brendan: And obviously, because all that previous work had been done so long ago,
Brendan: we didn't have all that institutional knowledge around anymore.
Brendan: And so we started by looking through the codes, surveying a bit what's already there.
Brendan: And we realized that the way the protocols are implemented currently,
Brendan: It's fine, it works, but it's not how you would implement a new protocol right now.
Brendan: It's how you would implement a protocol 20, 25 years ago.
Brendan: Better tools and platforms to implement something as critical and sensitive as,
Brendan: protocol support for an email for for for email because you know email is very
Brendan: sensitive you don't want to you don't want to mess with the actual data that's
Brendan: that's transmitted you don't want want to mess with.
Brendan: The structure of that data misrepresent anything.
Brendan: Rust provides us with some very nice guarantees in terms of memory safety,
Brendan: in terms of typing, in terms of, you know,
Brendan: all those kind of things that we need that weren't around 20 years ago.
Brendan: So that's kind of how it fits into our current strategy.
Brendan: And also another obvious point of why we're using Rust is because Thunderbird
Brendan: is built on top of Firefox at the moment.
Brendan: And so Firefox has Rust codes in its code base.
Brendan: And so a lot of the necessary infrastructure in the build system,
Brendan: in the CI system, etc. et cetera, already supports Rust.
Brendan: So even though we did have to do some work to be able to have our own Rust code
Brendan: in Thunderbirds, it also meant that it wasn't something that we had to start from scratch.
Brendan: And so it was easier than just going with another platform.
Brendan: Platform so in terms
Brendan: of the in terms of the long-term
Brendan: strategy it's not entirely clear yet you
Brendan: know what will be written in rust what won't be written in rust it's clear that
Brendan: rust will be will still be used to to develop new new things that are new components
Brendan: that that needs the kind of guarantees that Rust provides.
Brendan: We might, as I mentioned earlier, we might use it to rewrite existing components
Brendan: if we feel like there's value to it.
Brendan: It's not that clear-cut yet what will end up being Rust what we want.
Matthias: Can you imagine that some new components will be written in C++?
Brendan: At the moment, possibly, there's, well, at least with some C++ in it,
Brendan: there's the way things currently work in terms of something called XP-COME in
Brendan: Thunderbird and Firefox, which is kind of that interface layer between languages.
Brendan: It stands for cross-platform component object model.
Brendan: The idea is basically that you define some interfaces in a platform-neutral
Brendan: language called XPIDL for cross-platform interface definition language, I think.
Brendan: And you can then implement and use implementations of those interfaces in JavaScript, C++, Rust.
Brendan: Rust, the bindings for xpcom in Rust are still fairly, I don't want to say immature,
Brendan: but they lack that extra,
Brendan: they still feel a bit rough to use at the moment.
Brendan: And there are still some features around memory management, around pointers,
Brendan: around references and so on, that aren't supported or aren't supported well.
Brendan: And so for that, we might need to shift a bit more into the land of C++.
Brendan: That situation might be improved in the future.
Brendan: We hope so. And we'll probably work with the XP-CAM team to try to make things better.
Brendan: But it's likely that at least at first we will still have some C++ in our new
Brendan: components even though the majority will be written in Rust.
Matthias: As far as I remember, you said somewhat like 30 developers work on Rust more or less regularly.
Matthias: How many of them work with Rust code or will deal with Rust on a regular basis in the future?
Brendan: So at the moment, we're about three people dealing with Rust because that's
Brendan: the size of the amount of developers in the team working on implementing Exchange.
Brendan: The current backend team at the moment, I think is about five or six people.
Brendan: I might be wrong on that, but I think it's about that scale.
Brendan: And those people might have to deal with Rust in terms of writing code for it.
Matthias: And earlier you mentioned tooling was one of the big advantages of Rust.
Matthias: A lot of people mention stability and robustness and maybe performance,
Matthias: but you mentioned tooling, which I find interesting.
Matthias: First of all, why do you think this is the biggest benefit of Rust?
Matthias: And second, maybe can you talk about the tooling that you mean specifically?
Brendan: I mean, so, yeah, when I say tooling,
Brendan: I mean the tooling that already exists in Firefox, meaning that there's currently
Brendan: tools and infrastructure in place in Firefox to build Rust codes.
Brendan: But when you build Firefox and by extension Thunderbird, there's already code
Brendan: in place to generate bindings for XPCOM interfaces in Rust.
Brendan: And to interact with those bindings, to define components in Rust.
Brendan: There's already some amount of documentations on the Firefox site on how to
Brendan: use Rust to interact with the rest of Firefox.
Brendan: So, yeah, when I mentioned tooling, I meant more tooling specific for Firefox,
Brendan: which is helpful to us because we're based on, well, Thunderbird is based on Firefox.
Brendan: So if Firefox already has some infrastructure for building and using Rust,
Brendan: it's something that we can use as well.
Matthias: It probably also helps that Rust has a similar memory layout,
Matthias: C++, and also uses a similar build system,
Matthias: or at least the binaries that it produces uses are similar with respect to,
Matthias: you know, name mangling and all of these details that you have to deal with on an FFI basis.
Brendan: Yeah, definitely. I mean, I wouldn't be able to give you, you know,
Brendan: more details, you know, that Mozilla has, you know, a dedicated team to take
Brendan: care of the build system in Firefox.
Brendan: So I wouldn't, I don't have all the, you know, nitty gritty details.
Brendan: I do know that the way the build system works is by linking all Rust symbols
Brendan: on a more generic library.
Brendan: And so having that kind of compatibility is definitely helpful.
Matthias: When you said that you are using somewhat
Matthias: of the same code base as firefox my immediate thought was do you also use the
Matthias: rust code in firefox to some extent i know that they worked on css and layout
Matthias: engines and at least in thunderbird you can show html emails do you share some
Matthias: parts of that code or is it completely separate.
Brendan: I think i think we do use some of the some of the Rust code from firefox kind
Brendan: of indirectly because we use some of the,
Brendan: some of the components from firefox in thunderbirds the
Brendan: entire you know rendering engine front-end engine all that stuff is directly
Brendan: lifted from firefox the networking engine is the same as as firefox so i'm not
Brendan: entirely you know obviously familiar with the firefox code base not as well Well,
Brendan: I'm somewhat familiar with some parts of it,
Brendan: but definitely not as much as a Firefox developer would be.
Brendan: But I don't know which parts of the components that we use are in Rust.
Brendan: I know that there's definitely some C++ there. There's probably some Rust there.
Brendan: I wouldn't be able to give you an exact kind of split of what is Rust or what
Brendan: is C++. But there's definitely a mix there.
Matthias: And maybe looking at it from the other perspective, How often do you have to
Matthias: deal with the C++ intricacies on a daily basis?
Brendan: It very much varies depending on what I'm currently working on.
Brendan: At the moment, I'm mostly working on Microsoft Exchange.
Brendan: So when we need to work through implementations and stuff like that,
Brendan: it's going to be a majority of Rust.
Brendan: When it's the initial kind of survey work that I mentioned, it's going to be
Brendan: maturity of C++ because the existing code that we need to survey and understand is written in C++.
Brendan: We probably will have to write some amount of C++ for Exchange.
Brendan: So far, I haven't been working on a part of that project that involves doing
Brendan: that. I know that one of my co-workers, Sean, has had to look into that,
Brendan: but I don't necessarily have the details on that.
Brendan: But outside of that main project that I'm currently working on,
Brendan: when I investigate some bugs or crashes or things like that,
Brendan: which is part of the responsibilities of any developers,
Brendan: there is, at the moment, not really a lot of rust involved because, as I mentioned,
Brendan: we only started having Rust code in the code base a few weeks ago,
Brendan: and they're currently building blocks for a feature that isn't currently fully
Brendan: implemented and therefore isn't fully used in Thunderbirds.
Brendan: But I have to deal with a bit of JavaScript and a bit of C++,
Brendan: depending on the area of the code base that I'm working through.
Matthias: At least you need to understand basic building blocks and how those things work
Matthias: together to work on Exchange support.
Brendan: Yeah, definitely. I mean, one of the first steps in that Exchange project has
Brendan: been basically trying to figure out, like, you know, we don't know how to build
Brendan: a protocol in Thunderbird, how to build support for a new protocol in Thunderbird.
Brendan: Let's have a look at what already exists. And so, yeah,
Brendan: I think it was back in September, I spent two, three weeks kind of going through
Brendan: the implementation of IMAP in Thunderbird, which is written entirely in C++.
Brendan: So that was a lot of, you know, diving into C++ code, figuring out what it does,
Brendan: figuring out how every block fits together.
Matthias: Maybe before we dive into Exchange support, let's give our listeners some context.
Matthias: How big is the Thunderbird code base? Do you have any numbers,
Matthias: any statistics around that? And also, what protocols are supported?
Brendan: Yeah, so at the moment, in terms of support for mail protocols,
Brendan: we have your standards IMAP, POP3, SMTP.
Brendan: There's also an NTP for newsgroups slash usenets, which also exists.
Brendan: We also have, Thunderbird also has a whole instant messaging module,
Brendan: but that's kind of a separated part of the code base.
Brendan: In terms of lines of code, at the moment, if you accumulate the C++,
Brendan: the JavaScript, and the rest, you come to about 585,000 lines of code.
Brendan: If you remove everything but mail and newsgroups and things like that,
Brendan: you come down to 475,000, thereabouts.
Brendan: So, you know, a bit of a drop, but still a lot. A significant amount of that is C++.
Brendan: A big amount as well is JavaScript.
Brendan: One of the reasons is the front-end, currently the UI of Thunderbird,
Brendan: is mostly made of, you know, web technologies.
Brendan: So, JavaScript, CSS, HTML, or XUL in the oldest parts of the application.
Brendan: So, and currently we don't have a good separation between front-end and back-end in Thunderbird.
Brendan: That's kind of one of the things that Rust might help us with in the future.
Brendan: But we currently, so it's difficult at the moment to figure out,
Brendan: you know, which of these JavaScript is doing more front-ended bits,
Brendan: what is doing more kind of intermediate slash back-ended bits.
Brendan: And in those almost 500,000 lines of code, there's about a bit over 500 lines
Brendan: which are Rust at the moment.
Brendan: So demonstrating that we're definitely very early stages at the moment.
Matthias: Okay, so we set the stage for integrating Exchange support.
Matthias: First off, I wonder why is Exchange so important for the Thunderbird users?
Matthias: Is it an often requested feature?
Brendan: Yeah, it's actually one of the most requested features on Thunderbirds.
Brendan: The thing that you often find with Exchange is the people who are using it are
Brendan: usually not the people who made the decision that they had to use it.
Brendan: Because usually you find exchange in universities, in companies, in the workplace.
Brendan: And most users don't have the authorization or power to just change a critical
Brendan: part of their workplace infrastructure.
Brendan: There are compatibility layers that exist and that are provided by Microsoft
Brendan: with IMAP, SMTP, and so on.
Brendan: But very often, at least in a lot of workplaces, they're disabled.
Brendan: I'm not entirely sure about that, but I think Microsoft is usually pushing for
Brendan: those to be disabled automatically.
Brendan: Under some sort of, I think, security argument. I can't remember the specifics.
Brendan: And so that means that a lot of users have no real possibility of using more
Brendan: open standards and protocols to consume their work emails.
Matthias: And I guess the alternative to using Thunderbird in this case would be to use Office 365.
Brendan: Yeah, Outlook, etc.
Matthias: Outlook, yeah.
Brendan: And it was actually a bit controversial at first.
Brendan: There were some discussions about, and a bit of pushback, not a lot of it,
Brendan: but on whether Thunderbird should implement Exchange at all.
Brendan: Because Thunderbird is and has been a big advocate for open standards.
Brendan: And so, you know, it feels a bit counterproductive and a bit,
Brendan: you know, going against that to suddenly implement Microsoft's proprietary protocol right into the app.
Brendan: But the reality of the situation is Thunderbird doesn't have enough influence,
Brendan: market share, etc., to be able to influence Microsoft, obviously.
Brendan: Obviously, in, you know, we can't rock up to Microsoft Surface and say,
Brendan: you know, we, all users aren't happy, you have to enable IMAP for every account.
Brendan: That's just not something that's realistic at the moment.
Brendan: And so with that in mind, if we keep, you know, pushing back again,
Brendan: If we keep arguing about keeping that status of...
Brendan: Like restricting protocols to only open standards the user is the one that's being punished,
Brendan: in the end and is the one that you know would like to be to use thunderbird
Brendan: but just just can't because thunderbird doesn't want to support them and their
Brendan: and their workplace doesn't want to,
Brendan: to to to support thunderbirds and and i would say that i would personally say
Brendan: that being in this kind of situations,
Brendan: adding support for something proprietary, such as Microsoft Exchange,
Brendan: although it would be obviously better if that wasn't something that was so needed.
Brendan: It's actually helping, I think, Thunderbirds advocate for open standards because
Brendan: then you enable users to use Thunderbird to have their work email,
Brendan: but also their personal email,
Brendan: their emails that they might have,
Brendan: mailboxes that they might have with other organizations, etc.,
Brendan: to have them all in one app, which is Thunderbird.
Brendan: And those other mailboxes are likely based on open standards in terms of communications
Brendan: and with a server and all this.
Matthias: Is that the first open implementation of the Exchange protocol that you know of?
Brendan: No, so there's currently at least one other open source email client that implements
Brendan: support for Microsoft Exchange, which is Evolution.
Brendan: And we've actually, in the early stages of the project,
Brendan: looked a lot at what Evolution does and worked a bit as well with some of the
Brendan: maintainers of the EWS add-on for,
Brendan: well, EWS being the specific protocol for Exchange that we're implementing that add-on for Evolution.
Matthias: Evolution and walk us
Matthias: through a typical day do you sit down and then you look at the spec you look
Matthias: at the evolution code and then you try to model what you see in rust do you
Matthias: work on the domain model right now and and how do you deal with with making
Matthias: this as idiomatic as possible in rust,
Matthias: So.
Brendan: At the moment, it's not really something that I'm working a lot with.
Brendan: My focus for the past couple of months has been more oriented towards code infrastructure.
Brendan: So when we started looking at what we needed to be able to implement Rust in
Brendan: Thunderbirds, we realized,
Brendan: you know, currently XPCOM, which is the only way we can interact with something
Brendan: like the networking component of Firefox slash Thunderbirds,
Brendan: is not very idiomatic for Rust because it uses, you know, obviously that component
Brendan: is written in C++ and obviously C++ doesn't have,
Brendan: nice stuff like, you know, asynchronous and, well, async await syntax and that kind of niceties.
Brendan: So one of my first areas of focus has been,
Brendan: to implement some kind of translation layer to first enable asynchronous codes.
Brendan: Well, asynchronous code to be run with the native Async 08 syntax in Rust.
Brendan: And then to build on top of that to kind of expose a nicer, at least,
Brendan: and more idiomatic HTTP client in Rust.
Brendan: Which is kind of based on, well, inspired by the request crates,
Brendan: but isn't obviously a one-to-one mapping.
Brendan: There's some demos and links in the page for a FOSDEM talk that we did a
Brendan: few weeks ago on specifically Rust in Thunderbirds.
Brendan: Which is called "Thunderbird: How to Exchange Rot for Rust", which is a fantastic pun that Aiki,
Brendan: one of my coworkers, workers came up with kind of
Brendan: on the spots when we were brainstorming names and there's
Brendan: there's a there's a couple demos on the
Brendan: during the talk and in the in the same page there's a
Brendan: few links to the the source code
Brendan: of of those of those crates and
Brendan: and so at the moment i'm
Brendan: taking a bit of very very short
Brendan: and temporary kind of step back from from rust
Brendan: to kind of explore how to hook how to hook exchange into the account setup feature
Brendan: of thunderbirds so i i'm assuming that once i'm out of you know exploration
Brendan: phase and i start prototyping and actually implementing i'll come back to Rust,
Brendan: but at the moment it's more exploratory.
Matthias: What was the decision not to use requests directly and instead write something else?
Brendan: So, I mean, it's true that we could have used requests directly.
Brendan: However, because, you know, Thunderbird already has a fairly,
Brendan: you know, a filter completes networking stack that it's borrowing from Firefox,
Brendan: which is called Necko and has existed for the past 20, 25 years as well.
Brendan: And is hooked into all the nice things like DevTools and the network inspector
Brendan: and things like that, it didn't feel...
Brendan: I mean, it felt a bit sad to just ignore that and use a completely different HTTP stack.
Brendan: And it also felt like it wouldn't lead to the most maintainable codes.
Brendan: Let's imagine you're tracking a bug in that implementation and suddenly you
Brendan: can't use all of the nice network debugging tools that the whole rest of the code base uses.
Brendan: And so, yeah, we decided to, instead of using something that already exists, like requests,
Brendan: to instead try to build an interface that resembles requests but under the hood
Brendan: uses that networking component.
Matthias: Did you already catch any bugs in the c++ code ever since you started integrating rust.
Brendan: I don't know if i if i would call them bugs i have definitely noticed a few,
Brendan: kind of weird behaviors that can be identified such as when you're building
Brendan: your HTTP request, you need to set the methods after the...
Brendan: Sometimes you need to set the methods after you set the body of the request
Brendan: because the components under the hood,
Brendan: like the part of NECO under the hood that's going to build your request might
Brendan: try to infer what method to use
Brendan: depending on the content of that body,
Brendan: because it has some backwards compatibility with some previous versions of the interface that it uses.
Brendan: Which is something that I don't know if it's, I would say it's more, you know,
Brendan: complexity that is inherent with maintaining a complete complex 20 plus years
Brendan: old code base than actually bugs.
Brendan: But it is definitely something that felt a bit unexpected, but wasn't necessarily
Brendan: difficult to work around.
Matthias: Are there any specific libraries or crates from the Rust ecosystem that you
Matthias: could leverage and use for Thunderbird?
Matthias: Or did you literally have to build everything from scratch?
Brendan: No, there are a few things that we're using.
Brendan: I know we're, you know, especially around the... So EWS, which is the protocol
Brendan: for Exchange that we're currently implementing.
Matthias: That's for Exchange Web Service?
Brendan: Yeah. And so that protocol is XML over HTTP.
Brendan: The HTTP parts, because it's network-based, we kind of decided to do our own thing.
Brendan: The xml parts we started by looking at the ecosystem because that's something
Brendan: that we can because there's a lot there that we can just reuse ultimately we decided that for the,
Brendan: serialization side of things because of because of.
Brendan: Some strictness around SADA's data model.
Brendan: We, I mean, it doesn't work well enough with XML, with the kind of structures
Brendan: that you find in XML for what we want to do.
Brendan: And so for serialization, we did end up implementing our own derived macros and so on.
Brendan: For deserialization we
Brendan: we are using if i
Brendan: remember correctly we're using a set implementation based
Brendan: on quick xml and and also
Brendan: that serialization crate is also based on quick xml so there's you know a bunch
Brendan: of the there's a bunch of the of existing crates that's where that we're using
Brendan: and we also hope to contribute to that ecosystem through the work that we're doing currently.
Brendan: So all the XML sterilization that we're working on and even at a higher level,
Brendan: the exchange support that we're working on,
Brendan: we're trying to design those so that we can ultimately have them as separate
Brendan: crates that people can just reuse.
Matthias: Does that mean you use two different crates for XML serialization and deserialization?
Matthias: You mentioned QuickXML for deserialization, I guess, and a custom version for serialization then?
Brendan: Yeah, we start initially, and ideally we would just use everything,
Brendan: we would just use the same crate for both.
Brendan: The issue with serialization is,
Brendan: The current crates and set is support of those crates, well,
Brendan: versions of those crates,
Brendan: have a few issues, which is that in XML, you have namespaces on your XML tags,
Brendan: and those tags can also have attributes which are different from the values and things like that.
Brendan: Right. In order to generate XML that has the right namespaces and attributes.
Brendan: We realized that we needed our own thing because the existing tools and crates
Brendan: within the ecosystem either didn't support correctly,
Brendan: for example, setting a namespace on a tag, for example,
Brendan: or would support setting namespaces and attributes, but with a lot of boilerplates,
Brendan: which, you know, for a small XML API wouldn't necessarily be that much of an issue.
Brendan: But the AWS protocol has, I think from memory, about 100 different operations,
Brendan: generations, 1600s or 1700s, different data structures.
Brendan: And so the smallest amount of boilerplate that we can eliminate is a huge win
Brendan: for the overall maintainability of the whole thing.
Brendan: And so that's why we've kind of looked at having our own thing for serialization.
Brendan: Because we do need to have our requests being formatted in a very specific way.
Brendan: For the responses, we don't necessarily need to be that strict.
Brendan: Because ultimately, in the response, there might be some data that we don't
Brendan: need. or that we can be a bit more flexible about.
Brendan: For deserialization, we can just reuse a lot of what already exists in the ecosystem.
Matthias: Yeah, I can imagine that it must
Matthias: be a nightmare to run integration tests against the Exchange protocol.
Matthias: Do you plan to have some sort of end-to-end test that you can run locally?
Matthias: How do you track regressions and and how do you make sure that you implement a protocol correctly.
Brendan: Yeah that's a very good question and one that's
Brendan: you know is very much on our mind but
Brendan: we don't really have a good answer to just yet the
Brendan: i think ultimately we would like to have like a automated end-to-end tests for
Brendan: now all of our end-to-end testing is done kind of manually because nowadays
Brendan: you can just use a hotmail.com email address to send AWS requests to.
Brendan: So that's not too difficult, but also obviously not that future-proof.
Brendan: So it is something that we'll need to figure out in the coming month.
Brendan: But yeah, that's going to be possibly a tricky bit.
Matthias: Yeah, there are tools for this which allow you to record HTTP requests and you
Matthias: can replay them over and over again until you get it right.
Matthias: But at the same time, it doesn't really help you that much because the things
Matthias: that you record are the things that you know about and the question comes after.
Brendan: I mean, ideally we would have, you know, a full-fledged Exchange server running
Brendan: in CI and we could just send requests to it and see, you know,
Brendan: if the actual implementation does the thing that we expect it to do.
Brendan: It's not clear yet whether that's something that we'll be able to do or whether
Brendan: we'll need to go another route. We'll see.
Matthias: Maybe you eventually need to build a mock exchange server. Fake change. I don't know.
Brendan: That's a good name idea. I'll keep it in mind if we need to go there.
Brendan: I know there's also a couple open source EWS implementations out there,
Brendan: server implementations are there.
Brendan: So maybe that could be another, that might be an alternative,
Brendan: yeah well it's it's something that's yeah we'll definitely need to look into.
Matthias: I asked a few engineering friends what would be the things that thunderbird
Matthias: could use rust for in thunderbird and it was surprising because people almost
Matthias: anonymously said for search and for plugins
Matthias: for search because sometimes apparently search
Matthias: doesn't work or at least there were some edge cases and for plugins because
Matthias: so far you can only write plugins in javascript i guess and it would be nice
Matthias: to have web assembly support and write plugins in rust what do you think about that i.
Brendan: Think it's an interesting prospect to consider at the moment i'm not entirely
Brendan: sure whether that's the route we want to go, because on the search side of things,
Brendan: rewriting search entirely in Rust would be a huge undertaking.
Brendan: And it's not even clear yet what that would look like in the future,
Brendan: because we're also doing a
Brendan: lot of work around reworking the storage layer of Thunderbirds and so on.
Brendan: It might be the case that in a few years, we decide to rewrite part or the entirety
Brendan: of the feature in Rust, but that's not really on the horizon just yet.
Brendan: In terms of add-ons slash extensions, I mean, I'll preface by saying that I'm
Brendan: definitely not the best person to talk about add-ons.
Brendan: We have one of my co-workers, John, is actually working full-time on taking
Brendan: care of the add-on ecosystem and implementing new APIs and things like that.
Brendan: I know that, I think I've heard at first then that, obviously,
Brendan: I mean, don't take my word for it, that a recent version of Firefox has added
Brendan: support for embedding some Rust into add-ons.
Brendan: That might be something that we'll want to also support in Thunderbird.
Brendan: I don't know the specific details. If anyone's interested in getting more information
Brendan: or contributing to that effort,
Brendan: feel free to go on Thunderbird.net, find our community channels and reach out
Brendan: to the Thunderbirds add-ons community.
Brendan: Which would probably be happy to help in that area.
Matthias: When we started the conversation, we established that you have a background
Matthias: in many languages, including Go and Rust.
Matthias: Can you briefly contrast the development experience in both languages?
Brendan: Yeah. I mean, I haven't done any Go in almost a year, Yeah, so it's kind of starting to fade a bit.
Brendan: I would say that there are some things that I think Rust, where Rust is nicer
Brendan: than Go, obviously memory management is one of them.
Brendan: And I think also in the area of error handling, like not having to just return
Brendan: error as every single place you could ever get an error.
Brendan: Whereas Rust has a result, and if you don't really care about doing your error
Brendan: handling at a specific part of the code, you just question mark it, and you're done with it.
Brendan: I think one area where Go is a bit nicer than Rust is with its standard library.
Brendan: And I'm not saying that Rust standard library is not good or is designed poorly
Brendan: or whatever, I don't think that's the case.
Brendan: I just think that Rust is still a fairly young platform.
Brendan: I mean, it's about, I think, 12 years old now, but it's only started booming
Brendan: and getting used more widely maybe five, six years ago.
Brendan: And that's kind of ramping up.
Brendan: I think I remember reading not too
Brendan: long ago go that you know microsoft was starting to
Brendan: invest into rewriting some parts of windows in in Rust which
Brendan: is super cool to see but the but the the standards uh library in Rust still
Brendan: feels a bit young i think especially like i'll i'll use a couple examples you
Brendan: know we we looked at the the current kind of state of,
Brendan: you know.
Brendan: Sending HTTP traffic in Go and how we can make our current work interoperable.
Brendan: Like, you know, having a Rust crate that is specific to Exchange,
Brendan: that's, you know, an external crate and you give it like an HTTP client that's
Brendan: either the one from Thunderbird or Request or something else that, I don't know,
Brendan: Thunderbird for Android when it comes around might want to use.
Brendan: And at the moment, there's not really a solution to that.
Brendan: There are third-party crates that implement some types for HTTP.
Brendan: I think the most widely known and used is the HTTP crate that's maintained by the hyperproject.
Brendan: But the way those crates are built, A,
Brendan: it means that any interoperability that you would get that is,
Brendan: you know, only if the crates, only if those,
Brendan: if your two clients were using those exact same times, for example,
Brendan: for the same time for a request, the same time for a response.
Brendan: And, you know, I could be wrong, but I know that request uses some of the types
Brendan: from that crate, but like, you know, methods and status codes.
Brendan: But, you know, request response types it's there on types.
Brendan: And the way those traits, the way those types are defined, they're defined as
Brendan: tracts, not really as traits,
Brendan: which means that you can't really have your own logic in the specific implementation
Brendan: of each method of that type.
Brendan: And I'm kind of circling back to the standard library, because obviously the
Brendan: HTTP crate is not part of the standard library.
Brendan: But it would be nice, I think, and potentially necessary in the future to have the,
Brendan: you know, to kind of standardize, to have a more standardized approach to those
Brendan: kind of operations like sending an HTTP request or, you know,
Brendan: even serialization and deserialization.
Brendan: Like Serdi is one of the major players in the Rust ecosystem to the point where
Brendan: I think it's even mentioned in the official API guidelines for Rust.
Brendan: But it's still technically a third-party crate, and it's only a de facto standard.
Brendan: And I think there's, you know, room for more,
Brendan: for better and more flexible standardization within the standard library that
Brendan: other platforms like, I think, Node or,
Brendan: you know, Go might have that I think would benefit Rust.
Matthias: On the other side, one could argue that having all of these crates outside of
Matthias: the standard library helped make them more mature.
Matthias: We can look at error handling, for example, if we decided on a standard like
Matthias: error chain before, that's pretty much what we would be stuck with right now.
Matthias: But this error and anyhow might arguably be better or have better abstractions
Matthias: because they came later in rust's you know life cycle and they used better abstractions like derives for example.
Brendan: yeah i mean i'm not i'm not saying i'm not saying that that
Brendan: you know the the the maintainers of the standard library should just take the
Brendan: the you know top five most used crates in in the ecosystem and just integrate them.
Brendan: And I'm not saying that it might make more sense, that it might make the most
Brendan: sense for Rust to standardize all of those aspects now.
Brendan: I think my point was more that it's one of the downsides of picking an ecosystem
Brendan: that is still maturing and maybe hasn't reached the same stage as other languages are,
Brendan: that, you know, you might be lacking in some, well, some operations might be
Brendan: a bit more difficult to achieve because there's less support for it in the standard library.
Brendan: Hopefully that will come in the future. And I think, you know,
Brendan: the future for Rust is quite bright.
Brendan: Yeah, that's kind of a point of comparison between Rust and other platforms.
Matthias: Since you started working on Thunderbird, what
Matthias: have you learned about rust maybe not about rusted language more about rusted
Matthias: ecosystem or things that were not as clear before as they are now that you use
Matthias: it in production as driver for a quite big project i.
Brendan: Mean me personally i definitely you know using rust in like this this This,
Brendan: the Exchange implementation project was my first time using Rust on the kind
Brendan: of bigger scale than just,
Brendan: you know, slowly prototyping and getting frustrated at the compiler.
Brendan: And being in a context where I could, you know, experiment a bit more with the
Brendan: platform and do, you know, cool things with it,
Brendan: really made me appreciate like basically understand why so many people like rust because,
Brendan: sure it's it's sure it's frustrating to when you start to start working with
Brendan: rust because you know even the it feels like even the simplest program takes hours to be to to build,
Brendan: which and and so that's kind of uh worsens the learning curve of of rust which
Brendan: i was talking about i was talking about that with someone from the Rust Foundation
Brendan: last week who told me that he told me that apparently they they even have a
Brendan: a term for it which is the rust hump.
Brendan: And but as you as you progress in your you know getting used to to rust and
Brendan: getting used to all all those new concepts that come with it,
Brendan: you come to really appreciate the fact that,
Brendan: once your code builds, you're sure that you have a certain standard in terms
Brendan: of safety, in terms of quality,
Brendan: and that you have a nice and more maintainable and less error-prone code.
Brendan: Outputs from from that and it's it also
Brendan: kind of makes makes you reconsider your relation with
Brendan: the compiler itself because you know in other compiled
Brendan: languages the compiler is you know just something
Brendan: that turns code into into binary in
Brendan: rust the compiler is more is that but also kind of a a bit sort of a of an advisor
Brendan: in you know if you do this you're going to if i if i let you do this you're
Brendan: going to run into those kind of issues in the future.
Brendan: So yeah, I think it's definitely things that I didn't necessarily think about
Brendan: before when I thought about Rust.
Matthias: Based on the experience you have now, what advice would you give to other projects
Matthias: considering Rust for bigger systems,
Matthias: maybe systems that were written in C++ plus and have existed for a while what
Matthias: are the sort of requirements for success and what is necessary to kick off such a project.
Brendan: I would say that definitely use Rust when it makes sense.
Brendan: I think there are a lot of use cases where Rust makes sense when you need that
Brendan: memory safety, when you need those performances and so on.
Brendan: There are also cases where you might make more sense to use a different language.
Brendan: You know, programming language is a tool and, you know, the programming language
Brendan: ecosystem is a toolbox and you choose what fits most with, you know,
Brendan: with the tool that fits best with the way that you need to do.
Brendan: So definitely consider that, you know, ask yourself those kind of questions first.
Brendan: Invest in in you
Brendan: know training your engineers to be
Brendan: good Rust developers because that might take more
Brendan: time than let's say go for example if Rust
Brendan: is the good fit for your for your
Brendan: project definitely invest in in training the
Brendan: people who are going to work on it because and
Brendan: it's it's going to it's going to pay off in the in the future but that initial
Brendan: investment is necessary i would say and yeah i think i think those are the the
Brendan: two the the two main aspects i i think i would i would say and.
Matthias: Lastly it has become somewhat of a tradition around here to ask a very specific final question.
Brendan: And that is.
Matthias: What would be your message to the Rust community as a whole?
Matthias: Anything you have in mind could be literally whatever comes to mind.
Matthias: Do you have a message to the Rust community?
Brendan: I would say that Rust is a very cool project, a very cool language.
Brendan: It has a lot of features and aspects that are very unique to Rust. Yeah.
Brendan: And yeah, I'm looking forward to, you know, write more Rust code in the future.
Brendan: I'm looking forward to learn more about Rust and its ecosystem.
Brendan: And I'm looking forward to see what improvements and changes Rust gets in the future.
Brendan: Because I think the future is very bright for Rust.
Matthias: Brendan, it has been an honor to talk to you today.
Brendan: Likewise. thank you thank you for having me.
Matthias: Yes! Thanks a lot and i will keep using thunderbird and keep on doing the great work
Matthias: that you do and also for all the listeners that use thunderbird maybe consider donating.
Brendan: Absolutely yeah as i mentioned thunderbird at
Brendan: the moment is entirely donations funded we're exploring
Brendan: some some areas to kind of diversify that revenues that all sources of revenue
Brendan: but it's donations are still going to be an important part in you know funding
Brendan: the the work that goes into developing maintaining improving thunderbirds so we're,
Brendan: infinitely grateful for everyone who uses thunderbirds and everyone who donates
Brendan: to keep the project alive
Matthias: Awesome thanks Brendan!
Brendan: Thank you!
Matthias: Rust in Production is a podcast by corrode and hosted by me, Matthias Endler. For
Matthias: show notes, transcripts, and to learn more about how I can help your company
Matthias: make the most of Rust, visit corrode.dev. Thanks for listening to Rust in Production.