Rust in Production

Matthias Endler

GitButler with Scott Chacon and Kiril Videlov

In this episode, Matthias Endler speaks with GitHub's Scott Chacon and GitButler's Kiril Videlov about using Rust to innovate version control, the challenges faced, and the collaborative spirit of the Rust community.

2024-11-28 81 min

Description & Show Notes

Version control is a critical part of any modern software project and git is the most popular tool for the job. But it can be complex and confusing, especially for beginners.

The team behind GitButler believes there is a better way.

They are building a modern Git client that streamlines the process of managing branches, backing up your work, and more. We hear from co-founders Scott Chacon and Kiril Videlov about how they're making Git easier for everyone -- all without sacrificing the power and flexibility that makes Git so popular in the first place.

About GitButler

Git Branching, Refined — that's the promise of GitButler. GitButler is a Git client that helps you manage your Git branches more effectively. GitButler records and backups up your work, helps with your code and much more. Their focus is everything after writing code in your editor and before sharing it on GitHub. The client is written in Rust, by the way.

About Scott Chacon

Scott Chacon is a co-founder of GitButler. Previously, he was a co-founder at GitHub. Scott is the author of the famous Pro Git book (which taught me Git, thank you very much) and an investor in a number of companies through his incubator, SCNE. He somehow also found the time to get a B2 in German.

About Kiril Videlov

Kiril Videlov is a co-founder and CTO of GitButler. He has a background in software engineering, especially in the fields of fintech and code analysis. He is a YC alum and has a Master's degree in Computer Science.

Links From The Episode (In Chronological Order) 

Official Links


About corrode

"Rust in Production" is a podcast by corrode, a company that helps teams adopt Rust. We offer training, consulting, and development services to help you succeed with Rust. If you want to learn more about how we can help you, please get in touch.

Transcript

This is Rust in Production, a podcast about companies who use Rust to shape the future of infrastructure. My name is Matthias Endler from corrode, and I welcome you to this very special episode of the podcast. I had the pleasure to travel to Berlin and meet with Scott Chacon and Kiril Videlov from GitButler. We talk about GitHub, Git Internals, and how the team at GitButler is refining Git branching with Rust. Scott, Kiril, thanks so much for having me and if you have a few minutes to introduce yourself that would be amazing.
Scott
00:00:37
Yeah, thanks for inviting us. So my name is Scott Chacon, I was one of the co-founders of GitHub way back in the day, I'm very old, and today Kiril and I and Anne have started a new company called Git Butler and so we're spending all of our time on that which is partially written in Rust which is I think why we're on the show. Yeah, that's me.
Kiril
00:00:57
All right. Thanks for having me on the show as well. My name is Kiril. I'm a software engineer, co-founder of GitButtler. I've been working with version control stuff for a number of years now. So I'm kind of really, really excited about what we're building at GitButtler.
Matthias
00:01:15
So a couple of weeks ago, I was at the We Are Developers conference, and this is where I met you guys. But in fact, I know one of you at least for a very, very long time, because, Scott, you wrote a book called ProGit. And yeah, you taught me Git back in the day. It's kind of crazy. And it was amazing. So I know that you are amazing when it comes to communication, when it comes to developer products. I guess you care a lot about developers. In general and what i find interesting about git butler is that i think it's an iteration on what you did before so it's maybe you can express it in your own words how would you describe git butler to someone who maybe is aware of github and git already.
Scott
00:02:05
Yeah i mean i think i think kind of the impetus for this was was a product that Kiril had done before which was called sturdy and it was a different take on version control that that was kind of asking questions like you know why is version control not more like google docs you know where you're not committing stuff but it is doing version control right like there's other ways of doing version control than than how software developers do it and i found that really compelling of just rethinking sort of the client i think i think github did a relatively good job of making sort of how people are using mailing lists type type things approachable but there was no we never really owned the client right like when we never really tried to make the client experience better. We just kind of relied on Git and all of the existing Git clients, most of which wrapped the Git CLI and those commands. And nobody really stepped back and said, if we did this from scratch, how would we do this, right? If the GitHub guys sort of created the client in the first place, what might it look like today? And I find that fascinating. So I think that's kind of where it started. And, and it's, it's a, an interestingly difficult thing to do, right. To, to kind of, to take, to go to first principles on something that you've been using for a really long time. Like it's been around for, I mean, like you said that I wrote ProGit the first version of that in 2009, right. So that was 15 years ago. And so there's, there's a whole generation of people that have never used anything other than git. And so to take a step back and say, let's just throw everything away and figure out what can we imagine would be an ideal workflow, like not sort of leaning on anything that came before, what might that look like? And that's exciting to me every day like it's always fun to work on it's always fun to reimagine kind of review processes or, how to share work or how to talk about work and how to get work integrated into some sort of production mainstream you know line and so that's where it started it's actually kind of funny because I feel like you mentioned ProGit like I think. One of the reasons that I've been successful in these things that I'm doing is that I'm not very smart so I think You know, if Linus or something creates a CLI, you know, version control system, it's really hard to use because he's very smart and he just assumes that everybody else is going to take the time to learn it. And I had I really struggled with it. And so when I finally figured out kind of how to learn it and use it, I started writing about it because I was like this took me way too long to do. And so I want to do and it's kind of the same with the CLI, right? It took me a really long time kind of to learn how to be good at Git. And so I want to make something that makes it makes you be able to do the same things where you're not worrying about that and spending your time and your effort on on how do I rebase, you know, stuff properly, but but to give you the same tools, but make it much more approachable and much more easy.
Matthias
00:04:59
A while ago, you wrote a blog post that kind of became a phenomenon. It was on Hacker News. I saw it all around the web. It was titled Why GitHub Won. And in that article, you also compare Git with things that came before Git, like SVN. And sometimes it helps to learn from the past. Are there things out there that you got inspired by when starting Git Butler? Any new ideas that have not been integrated into any version control system right now?
Scott
00:05:31
Yeah, there's lots of stuff. I mean, there's lots of stuff in, like I said, in Kiril's product Sturdy, where it would kind of be live streaming the changes that you're doing. I think one of the main things that we looked at was, I feel like the git commit is overloaded. And so you use it to save your work. So people do work in progress commits, right? When that's not really committing, it's not really documenting a change. It's just saving your work, which I think is kind of stupid. Like with Kiril's, you didn't have to do that because it was constantly saving and it was constantly pushing in the background. And Git Brothers has this timeline thing that kind of does a version of that that I think is really nice. It also, Git commit is used to share your change. You can't push anything over the wire unless you commit it. So people will also do commits just to be able to push it somewhere, right? So one is saving, one is sharing. And then the third is actually documenting, which is what people think it's used for, right? But it's actually used for all of these other things and it's kind of overloaded to do these things because it's the only way you can do them. And I think being able to separate that in a client is one of the main important things that we want to try to hit first, right, is we're backing up your stuff whether you commit or not. We're able to save and push that and share that with people, whether you commit or not, right? Like even work in progress stuff, you can get some review on, right? And then you can use your commits to actually have a nice documentation of what you're doing. And so, yeah, there's a lot of, I think, tools that came before, even like, you know, I feel like we were trying to solve one very specific problem with distributed version control systems, but we lost other strengths of centralized version control systems. Like they're not all horrible, right? Like being able to say, hey, I'm working on this file. Like we all got frustrated. Those of us that are old enough to have used centralized version control systems. Like you get frustrated that a file is locked and you can't edit it or you can't, you know, if somebody locks it in per force or something, right? Like it's very frustrating, but you know that they're working on that file. And so, you know, or if it doesn't necessarily mean like we need to get to the point where I can't edit it, but it would be nice to say, hey, did you know that Kiril is working on that too, right? Like you guys are probably going to have merge conflicts at some point. You might want to look at that before it becomes too bad, right? The way that people tend to do it with sort of GitHub-based workflows now is, you know, you work on something, somebody else works on something. Whoever merges first has 0% of the work and whoever merges second has 100% of the merge conflict work, right? And so like, I think there's a middle, there's some middle grounds in some of these things where we can look at older systems and say, what were they actually good at that Git and GitHub is not good at and, you know, what. You know, what's interesting from Bitkeeper, what's interesting from from mailing list patch workflows, right? Like, there are there are interesting things that they did that they do better than the workflows that we have today with sort of GitHub pull requests. And I think trying to learn from that and pull that into how can we productize that in a way that GitHub might have, right, like that makes it accessible and easy to use. I think that's a something only a slightly stupid person can do because I you know, you have to you have to get frustrated by it, right?
Kiril
00:08:33
I mean, one thing to add to your answer, like one thing that I think I did wrong in my previous project in version control was trying to reinvent everything, right? Like trying to reinvent version control entirely from scratch or like inventing the snapshotting entirely from zero. And that, well, that is very difficult. It turns out that Git is really, really good at that part. But it's also really difficult to have people adopt something that is not compatible with Git because Git is not just a tool. It's a format. It's a protocol. It's a protocol for distributing the software from your machine or from CI to a production environment. So I think that's a big one. So I think what's really interesting that we're doing at the moment is we are thinking about the workflow and we're thinking about the user interaction. But at the same time always producing compatible data structures like git data structures so at the end of the day what what the two outputs is is just git git trees that become branches and you can you can have that consumed by other tools that are in the ecosystem.
Matthias
00:09:44
I interacted with Git mostly from the command line. So I typed in my commands and then it would do something magically. But behind that, there's this .git folder, which a lot of people might know of. And there's an entire structure of objects and hashes and so on. Is that ill-specified or is it in effect really well-specified so that you can reimplement parts of it in Rust?
Scott
00:10:09
It depends on what you're trying to abuse, right? So like we're, there's, there's a couple of, of libraries in Rust that are very good. LibGit2 is, or Git2RS is built on LibGit2. So it's, it's actually C, but it has the Rust bindings. And we worked on that at GitHub, you know, years ago. And now Git Oxide, and we, we work with Sebastian on, on trying to see which is sort of faster implementations of these. But, but the, the actual data structure in Git is relatively straightforward, I think most from, you know, what we're trying to do. The, but you know, we, we try to do some other stuff. We're learning some stuff from jujutsu from some other systems. And, you know, we want to inject things into the header and it took us a while to figure out there's no like RFC on sort of, you know, git like formats or protocols or anything. Right. Even, even like some of the server protocols are still documented by me, like just like learning it, but like, no, no, it's not written down in lots of places. Right. And so it's, it's, you have to kind of guess what's going to break because there's lots of different implementation there was no the one of the problems with git is that there was no like standard linkable library for for a very long time which is why libgit2 which is why it's called libgit2 right because originally it was libgita libgit.a but it was non-reentrant so just anytime you hit an error condition it would just exit and so you can't link anything to it because it'll just kill you know a gui or whatever and so they had this like unix philosophy of just shell out and run these commands and fork exec. And, and, and so, yeah, now it's, it's difficult to know what did JGit do the Java implementation, right? Or what did libgit2 do, or what does core git do? If it sees a null byte here, is it going to explode? If you add a header in the wrong place, like which of these implementations is going to have a problem? Will GitOxide, you know, it re-implemented a lot of this as well. Will it have a problem? And so since they didn't have a linkable library, there was just five, 15, you know, re-implementations of this that different people use. And so we have to be a little careful of like, or try it out in lots of things. If we're going to do something that's very non-standard, something that core Git wouldn't do and be like, is this, I mean, I know it's not core Git, but like, is it acceptable? Like, well, everything else, ignore it so that we can, we can use it. Right. I mean, so that's always, that's always kind of fun.
Matthias
00:12:23
Did those implementations diverge over time?
Scott
00:12:26
They've tried. I mean, diverge in that, in that, you know, nothing does everything that libgit A does, right? Like that the core Git library does. So even libgit 2 only does, you know, maybe 80% of stuff. Like there's even you know stuff that was merged maybe a year ago ref table stuff that that isn't that it doesn't work at all right and libgit2 or and gitoxide or in probably jgit even even though sean came up with the spec 10 years ago but yeah it's it's so it depends on what you're trying to do i think most of the core stuff is you know the old pack files and loose objects and all that stuff is is very standardized sub modules like once you start getting into things that the 10% of people use or 5% of people, then, then it gets a little bit, a little bit more iffy. A lot of the stuff in Git in the last, you know, eight years or so has been done by GitHub and Microsoft and GitLab and stuff like that, that are mostly server side anyways. Like it's trying to scale large server sort of implementation. So most of us don't run into it when we're doing it on client stuff. And that's a lot of the stuff that isn't implemented everywhere because it's just not worth it for most, most implementations. But, but yeah, it's, it's definitely, it's not standard, right? Like there's lots of different degrees of percentage of implementation from these different libraries.
Matthias
00:13:40
It makes it pretty hard, or at least in my mind as an engineer, I wouldn't even know where to start. So I wonder, Kiril, when you started working on the predecessor of GitButler, Sturdy, how did you get started with that? And did you just literally pop up .git folder and then start implementing?
Kiril
00:13:59
The error in my initial approach was trying to implement a brand new incompatible implementation of a version control system. So that was not even attempting to be compatible. So I think I'm ill-equipped to answer this in a good way. We tried something that was – so we had a file system watcher that would observe what happened on disk and then pick those changes up and attempt to create a snapshot and attempt to record that snapshot and then do that continuously and then handle the various corner cases that you can encounter accordingly. And that particular bit is really, really hard. So it was not long until we kind of, at the time, gave up on this approach and kind of shifted towards using Git as an engine, as a database for what we're trying to do.
Matthias
00:14:50
And did you shell out to the command line to use Git?
Kiril
00:14:53
So for applications that are trying to do something with Git, there are a number of ways of trying to do it. One is shelling out fork exec to the Git binary, perform the command and pick up the output. And that kind of can be limiting, and it can also be slow if you do it in a kind of tight loop. So for a lot of things, we would use libgit, libgit2 rather, in order to actually bind a proper library that would interface with Git structures. And for GitButler, this is actually our approach that we take in GitButler is we use O3. We have so we have libgit through a rust crate so we're interfacing with libgit libgit2 rather and we do a lot of tree manipulation or you know modifying git data structures that way and then for certain operations like fetching and pushing it is difficult to do that with libgit because of you know differences in operating systems libgit2 has a its own transport for. SSH, you know, fetch and push. And that is difficult. So specifically for those operations, we fork exec. And now since recently, we have been playing around with GitOxide. And GitOxide is attempting to be, you know, a Git library, a Git compatible library. And that allows you to do the same things that you could do with LibGit2, but ideally in a faster way, ideally in a more flexible way. And it is faster. It is faster for a lot of use cases.
Scott
00:16:41
Yeah, it's interesting. In the early days of GitHub, this was one of the first things that I worked on when I started at GitHub was we were fork-execing everything. But like fork-exec has a pretty big overhead if you're doing it thousands of times, right? And you're trying to do it in one web request or something like that. And so I would go in and look at what was the slowest and try to reimplement those things in Ruby, like just in pure Ruby, because that's what we were using on the back end. And we're kind of doing the same thing, right? It's like, if we can take a fork exec overhead, it's almost always faster and definitely compliant, right? If we can do it in libgit2, then there's a nice interface for that in Rust, right? And it runs in C and it's fairly cross-platform. And then if we can do it as fast as possible, like if we can do it in git oxide, then it's nice because it's purely in Rust, right? And so like now it's not even doing the C bindings or anything. And so, yeah, I mean, it's fun. It makes for a really interesting code base.
Kiril
00:17:33
I have not shared this with you, Scott, yet.
Scott
00:17:36
Oh, no.
Kiril
00:17:37
So recently we migrated the way we fetched the branches, the branches data, or listing just branches in a repository. We're now using GitOxite for that. And it appears that at the minute we have the fastest client at fetching all branches. So the test repository had 20,000 branches, which I think is a ludicrous amount of branches. I don't know who needs that many. But in that scenario, our app is currently faster than the CLI in answering the question, what are my 20,000 branches?
Matthias
00:18:12
How do you measure that? Do you have some sort of benchmarking tool for this? Or how do you do that?
Kiril
00:18:18
A very primitive timestamp-based approach.
Matthias
00:18:21
And you get some granularity by having a lot of branches and working with big repository, I guess.
Kiril
00:18:29
So the particular repository is the GitLab repo. And that one has a lot of branches. I don't know why. And just listing them, all of them in the CLI is... A little bit slow and we do it faster.
Matthias
00:18:43
Maybe it sounds almost tautological but at some point you needed to decide what language to use to write git butler in and of course you could have picked c but you could have picked any other language but you chose Rust to do so so.
Kiril
00:19:01
For our use case i think it's not necessarily the language characteristics that were a driving driving force, but rather the ecosystem or the libraries and the tools that are available. In particular, in terms of what we're trying to achieve with the application, so we're building something that manipulates Git data structures on your computer. So we need to be able to interface with Git and with the file system in a good way. We need to be able to create custom user interfaces. We need to have that set up in a secure way, in a performant way. We need to be able to cross-compile for different platforms. And it just so happens that on that point of being able to create custom interfaces, there is this toolkit called Tauri. So Tauri is a framework or a toolkit that allows you to create desktop apps and use web technologies for the user interface. In a way, it's similar to Electron, but what's different from Electron is they use the native operating system WebView, so they don't need to package the Chromium with the binary. And also, they have a very strict and specific way of communicating between the user interface code and the core of the application with the positive impact for security. right now so yeah.
Scott
00:20:31
No i was gonna say i i just from a i i was trying to remember us talking about this like when we first approached it because we did his so sturdy was was an electron app right right right so he had some electron experience i obviously you know i came from from github so we did Atom and extracted electron out of it and so i i've i even wrote some documentation for electron or for Atom actually back in back in the day and so i was familiar with that but like when we were looking at how do we build the desktop app, that is interesting, right? I think those are really the only two options that came up that are, that could, you know, we could easily put it out on all the platforms that we wanted to. It worked more or less the same. We could do the user interface in HTML and JavaScript. And then really the question was the difference of the backend. Is it going to be a node or is it going to be in Rust?
Kiril
00:21:18
And there was a node backend for a couple of days.
Scott
00:21:21
Yeah, I mean, I mean, the, you know, when it comes down to it, I hate JavaScript, but I don't think that's actually what the deciding factor was. I think the deciding factor was, was A, we assumed that it would probably be faster and B, we want to do a CLI, I think, version of it at some point. And so it was kind of a nice concept to me, at least, that we could have the one user interface be in HTML and JavaScript, the backend, sort of the core crates and stuff be entirely in Rust, purely in Rust with nice APIs. And then we could write a CLI interface or TUI interface or something that would have all of the important stuff already done, right? And it would kind of work the same everywhere. And CLIs and Rust are much nicer to distribute than CLIs in Node, right? And so all of those things, I think, were kind of what tipped us over. I mean, it is nicer that it has smaller binaries because of using the operating system's sort of default web view, but it also causes some issues because we don't control it, right? And so there are problems with you know, some Linux distro using some ridiculous, you know, web view that we have to debug and be like, why isn't it working on this thing? Right. So I think, I feel like that's, that's pluses and minuses, but the, the, the, the idea of Rust being sort of the core library that we can throw whatever user interface on to do the things that we're, we're, you know, hopefully is battle tested and good at and not have to rewrite it in Swift and rewrite it, you know, do it, do it in a bunch of different languages or something per platform. Like all of those, I think made it fairly clear that Tauri would be a good idea.
Matthias
00:23:00
It's surprising because I don't hear that a lot, that people come to Rust also because of the nice UI libraries. And it feels like that was at least some factor. I don't know if Rust would have been a non-starter, if Tauri didn't exist. I don't think so. Maybe you would have split it up and maybe kept the back end written in Rust and the front end written in something else.
Kiril
00:23:20
It was certainly the gateway drug, right? We never.
Scott
00:23:24
I don't think we ever would have chosen Rust.
Kiril
00:23:26
Nobody on the team had Rust experience. So we started working on this with zero Rust background.
Scott
00:23:34
Yeah. It was a struggle for me. I've done systems programming in such a long time. Like I'm a Ruby guy, right? And like I'm coming in, I'm like, why can't I just fucking...
Kiril
00:23:45
I've seen the light. I think it's fantastic.
Scott
00:23:48
I used ChatGPT a lot in the first days. And then I sat down and read the Rust book, thank God. Helped a lot that was the first time i'd even thought about the stack in the heap and like like since i was from college it was it's been such a long time we.
Matthias
00:24:04
Had Steve Klabnik on the podcast just a couple episodes ago so shout out to him.
Scott
00:24:10
Yeah he was he's in the jj discord and and we we sort of said hi to each other there not not too long ago as well he's he actually wrote like speaking of other version control systems if you've done something on jujutsu yet, But Steve wrote like the tutorial book for that. So if you do want to learn Jujutsu, you have to kind of find his tutorial book that's online somewhere and read through that because it's a great sort of intro to that system.
Matthias
00:24:37
Yeah, he also mentioned that on the podcast. So yeah, definitely if you want, you should definitely check it out.
Scott
00:24:43
I've only sent one patch via Jujutsu, like one actual thing. And it was to correct some stuff in that tutorial book. So I'm also a co-author. I'm kidding. There's like, it's like a change of two lines or something. But I used that to learn how to use that system, actually. It was pretty cool. Thanks, Steve.
Matthias
00:25:05
Kiril, you had some previous Rust experience before? No, no.
Kiril
00:25:08
None of us. Nobody in the team had any Rust experience.
Matthias
00:25:11
Oh, wow. Okay. And how long did it take the both of you to pick up the language? It's not a competition. I'm just curious. Is it days, weeks, months, years?
Kiril
00:25:20
There are really different levels of competence. Like you can get started with messing things up very quickly, right? But to be truly productive and to truly know what you're doing, maybe like a month or something.
Matthias
00:25:35
What's your background?
Kiril
00:25:36
I mean, I'm a software engineer. So I've done a number of languages. I have enjoyed Go, I've done a bunch of Java, I've done some C. I had a phase of really being into Clojure. Like I really, really liked the Lisp style of programming.
Matthias
00:25:53
But essentially you have a different background than Scott.
Scott
00:25:57
I mean, you're much, he's much more hardcore programmer than I am.
Kiril
00:26:01
I mean, computers are cool.
Scott
00:26:03
I've, I've just done, I'm very, I'm a very practical programmer. Like I, I like whatever gets the job done easiest. And so that's usually been Ruby for me because I usually do websites. And so I, I, before that it was PHP, but I've been doing Ruby for, for 20 years now, at least. And I'm very comfortable with it. And anytime I try to do something, it usually comes down to that. Unless if I'm trying to do mobile apps, I'll do like react native or something because, you know, exposed like a nice, I'll just do, I'll do whatever's easiest to see if an idea works, I think. And so for, for Rust, I kind of still learned the minimal amount to get something to work. Like I'll still, I'll throw a patch in every once in a while. And then, you know, somebody will come in and be like, Jesus, Scott, and you know, make it, they'll fix it, but like, I'll get it to work and make sure that it's even a good idea. Right. And so that's, that's, you know, I'm old enough now that I don't have to, I don't have to learn new things. but.
Matthias
00:26:52
If you can maybe remember one of the things that you learned from rust which turned out to be surprisingly good what would it be.
Scott
00:26:59
I mean i like yeah again i from the ruby world it's it's it couldn't be a more different language right and so i i do like the the compiler stuff like like it's it is nice to be like if it compile like you know most of it is dealing with compiler issues rather than runtime issues so so if it compiles and it actually runs something then it probably runs the way that I meant it to. Whereas in Ruby, you don't know until it runs into an issue and there's no good debugger and things like that in the same world. But like, it's very different doing systems programming and doing sort of Ruby, you know, almost scripting type programming, right?
Kiril
00:27:36
The error messages are so good. So good from the compiler. I think I'm still impressed. You know, like just having the compiler tell you, all right, you move this over here on this line, this part, this is super useful. I think you don't get this in C++ to my knowledge.
Matthias
00:27:55
So that's also ergonomics.
Kiril
00:27:58
Exactly. The getting started experience for me was fairly pleasant. There is some mountains to climb, but it's obvious that there is some thought put into getting people up to speed with Rust.
Scott
00:28:14
I've also found that there's not a lot of boilerplate. like like i feel like if when i write you know like i don't know c c code i have to do a bunch of header files that kind of you know redeclares the same stuff like you're not you're not doing a lot of or java java has a lot of that right like just random bullshit that you're that you're typing out because you have to not because it actually tells the compiler something that's valuable right or or new like i i feel like rust is relatively fast to write in i i don't always, have the entire memory model. Like I suck at lifetimes and things like that. And so I tend not to deal with them as little as possible until somebody's like, this is not how you do this. But I feel like it's still a fairly succinct language for a systems language, right? I was kind of constantly impressed by that.
Kiril
00:29:03
It was one thing that I found very difficult while getting started, which is the approach challenge for error handling or the way you want to structure your application error-wise. Now, I understand that there is the anyhow camp, and then there is the camp of using typed errors and just finding the balance or understanding which way one would use for, let's say, building a library versus building an application like ours, the trade-offs. Yeah, it is more challenging for... I think there's no right answer, right? And therefore, the ecosystem has not standardized on one thing. Yeah.
Matthias
00:29:44
Any tips on how to navigate that ecosystem?
Kiril
00:29:46
Well, I think understand what you're trying to achieve with errors. I think if you... I'm now repeating, I think, advice that I've read. But if you are creating a library, you have one set of... You're trying to set the expectations that other people have. In our case, we are building an application and creating. It so happens that for us creating types for all the different subsystems and functionalities in our application does not necessarily add the value to warrant the additional type creation and type conversion. So that was creating a boilerplate for us. And we've recognized that, okay, we care about what, So our application works on like an API. So you have commands that git processed and we do care what happened. We do care if there is an error at the end such that we can present it in a meaningful way to the user. But we seldomly have the application flow switch on the error types. And for that reason, because we have this user interface that is interacting with a core of Git Butler, we're using Anyhow, we're using Anyhow, and we're adding context to errors where appropriate, and we're adding error codes as context in such a way that that gets passed on to the front end, and then the front end knows what to do with the error codes.
Scott
00:31:19
It was pretty funny having a team coming in that all had sort of different, like we all read books and we all copied from different code, and so we had three different ways of doing error reporting, like throughout the, throughout the application. Right. And then kind of had to go and clean all, had to choose one, right. Like figure out like what, what would be a good way? Is it anyhow, like, is it typed or like how, like, what do we choose? And now can we go through and make everything the same? Right. And it was like waves of, I mean, Kiril did a lot of this, right. Like going through and changing all of the error sort of handling stuff to try to be somewhat consistent. Right. But like, this is another thing of having a bunch of people start not knowing, like not really having a lot of previous background in this and just doing whatever seemed right. And not knowing, like nobody really had any particular taste at the beginning, but it's definitely gotten better as the longer that we've worked with it.
Kiril
00:32:07
Yeah, I think that error handling and also structuring an application that is growing and evolving. It's a non-obvious thing. How do you structure it? Do you have one crate or do you have multiple crates?
Scott
00:32:20
Yeah, we went from like one crate to 50 crates, and now we're like at 20 crates or something.
Kiril
00:32:26
Actually, it's 31.
Matthias
00:32:28
How do you strike the balance there? So when to crate, crate, and when not to?
Kiril
00:32:36
So we we started off from a place of having very few and we had code depending on other code that should not have so i think when it comes to software in general i think a lot about this idea of optimizing for deletion what does it mean to optimize for deletion it means to be able to say okay i don't like this feature i want to just delete it and if that feature is intertwined in 20 different places it's hard to delete but what i just said is just a fancy way of saying make modular software right it's it's not a new concept and so we recognized that this this was an issue for us we were adding things and old things were difficult to clean up so at some point we we just bit the bullet and and started splitting things off now i had i come from a background of using build systems like Bazel. And there you are really encouraged to make build targets or compilation targets that are small, kind of modular. And this also forces the dependency graph to be unidirectional and very explicit. And I think this explicitness. It forces you to think about the boundaries, forces you to consider, okay, do I want this to depend on these other things? Or you recognize that, oh, there's going to be a cycle here. It's impossible for me to make it a separate crate. Why is that? Maybe that should belong to this other functionality. So we mostly take package by feature approach or crate by feature. Or we strive to put different aspects of the client in separate crates.
Matthias
00:34:17
That means if I delete a crate, then I make a couple modifications and it mostly just works again.
Kiril
00:34:24
This is the cool thing about Rust. We've made such dramatic changes to the structure of not just where the code lives, but the interfaces and such. And it all just works because you have a statically typed language with a compiler that helps. So if you can get it to compile, it will work. That's the beauty of the setup that we've got.
Scott
00:34:49
Yeah, I doubled down on that as well. Like you were asking what I liked about Rust and refactoring stuff is so nice in Rust compared to Ruby or any other language that I've done because you change the one thing to be kind of what do you want this call to look like now? And then it's like, here's the 47 other places that you're going to have a problem with. And you're like, I'll just start going through the list, right? Like it's, it's, if, you know, if you don't have some tool kind of helping you do this type of stuff, it's very clear that you've done your job or not. Right. And like 99% of the time you fix all those things and then you run it and it's like, yep, everything works. That's great. I've never, that's not in my, my, you know, history of working with software before. So it's, it's very nice.
Kiril
00:35:29
There is one thing on this subject matter that I've been thinking about, and I don't know the exact answer that is good. So now that we have our code base relatively modular, we have 31 crates. Well, some crates have modules, and in some cases we export the entire modules. And it is more difficult to discover what the API of a crate is when you have like a more nested or like more hierarchical surface area that is exported and I've recently been toying around with this idea of putting the or exporting the functions that are from inner modules of a crate just putting them in the root for easier discoverability and making it more obvious what the API is and what one can do with something. I do not know if that's a good idea. I would love to hear from somebody. Do you know if this is a good idea?
Matthias
00:36:28
So there is a blog post by Matt Clad, the creator of Rust Analyzer, and he describes that he likes to keep his hierarchy extremely flat, mostly for pragmatic reasons because he has a hard time coming up with names for things. And then if you put things into the wrong directory or into the wrong crate and sub-crate and module and whatever, then it becomes harder to move that out again. So it tries to keep it as flat as possible. So it goes along the lines of what you just described.
Kiril
00:36:56
So not totally wrong.
Matthias
00:36:59
It's a bit like Git branching. You can have branches upon branches, but at some point I do like to keep my hierarchy flat at least because I need to keep track of it in the end. Do you use something like preludes or a glob import?
Kiril
00:37:16
We avoid that. I don't think we do that anywhere.
Matthias
00:37:21
Yeah. I agree because I literally also wrote a blog post about it that maybe you shouldn't have a glob import or a preload for that. Does it mean you have something like an application and then on top of it you have a couple of library craze that technically you could use for any other reason? For example, interacting with Git itself.
Kiril
00:37:43
Well so we we have 31 crates and most of the if you if you just read what the titles are they're like different features from from the app and one of those crates is the to gitbutler-tauri that is the binary crate that bootstraps everything so we mentioned earlier that we yeah we're using Tauri but we are not really the core of the app is not necessarily dependent on There is one crate that combines the APIs, sets everything up, and exposes. But then we also have a CLI, a GitButler-CLI crate, which is mostly useful testing. But we have multiple binaries. So nothing in there is necessarily library-grade to be used or to be consumed by another application necessarily. But we have multiple things that can be built and composed. we can just as easily create a web server if that's what we needed to do out of the API.
Matthias
00:38:45
Nowadays, there's also Leptos and Dioxus and a couple of other UI libraries like Slint. Are you still content and happy with Tauri or did you run into any issues so far?
Scott
00:38:57
Well, I mean, the user interface is Svelte, right? So it's basically just, here's a WebKit thing. It gives us a way to invoke calls to Rust. And so I think we're relatively happy with that because, you know, we have a bunch of JavaScript, programmers who are good at, I mean, we like CSS, we like, you know, we like, we like being able to, to build user interface with the tooling that we know from, from web, from building on the web. And, and Svelte gives us a lot of stuff that I think it would be difficult to go. I, I, I find it would be really difficult, I think, for what we're trying to do, which is, which is very doable in, in HTML and JavaScript to, to go down like the, the Zed route, right. Of, of being like, let's, let's do a completely new sort of user interface stuff. Cause that's now you're, you're reinventing all the wheels and it can be much faster, right. Or it can be, it can be great for what you're trying to do. But, but right now I think we're not a hundred percent sure of exactly where we want to land. And it would be, it's easier to kind of use the wheels that are there to, to get us real close to that before we go down some, some larger, road that's that difficult.
Kiril
00:40:07
Zed builds their own UI library, GPU accelerated and everything, and it's fascinating, but it would be so much more difficult to iterate and try things with such an approach.
Matthias
00:40:20
We had them on the show. So if you want to check it out, listen to the episode we've said where they described that UI approach. I think it's nice to compare those two approaches because I looked at GitButler and what I found was it looks nice. It's clean, it's snappy, it almost feels native. And I don't know how you'd pull it off, but definitely you did a great job there.
Scott
00:40:44
Yeah, I mean, it's an interesting question, right? Like going back to what do we want to build? Like we are still very much experimenting with stuff, right? Like we, we, we, we're, all we know is we want to throw everything off the table and then figure out, you know, what do we want to be on the table a year from now? And we're still constantly kind of arguing about stuff and, and we try stuff out and we see what works and we use it ourselves and we dog food it. And we're like, okay, this feels good. This needs to be changed. This is going in the right direction. Let's rethink this, blah, blah, blah. I think with the Zed guys, you know, I mean, I've known Nathan for, for since GitHub days. So, but like, I feel like they knew what they wanted to build, right? They want to build an editor. Like they know what editors look like. They know what, you know, VS Code looks like. They know what Atom, he worked on Atom before that. And so he had a good idea of like, really, where do we want this to go down? They had a lot of taste and they knew where they wanted to go. And so I think they could be, it was much, it made more sense to say, here's what we need a UI library to do because we want to do kind of this, but much faster. Right. And, and so that, that makes it, whereas we, we haven't, I feel like we have no idea where we're going to land, you know, two years from now, what this is really going to look like and how it's going to feel. We just, we know that we're getting closer to what makes us happy, sort of doing, doing these, these code management things every day. And, and so, yeah, it really depends on what you're trying to build. Like, do you really know what it's going to be and, and making it faster or doing it a rust or, you know, whatever will definitely help you get there. Or are you inventing and if you're inventing then then it's much easier to have a general toolkit.
Matthias
00:42:16
And is it the same experience when you move things around on the front end side in the ui do you have the same feeling like when you do something in the rust core where refactoring is easy and the language helps you move things around and you can make changes with confidence or do you have this i would say chasm from the front end to the back end a typical thing where you change something and then you throw things over the fence yeah.
Scott
00:42:43
I mean that's really a question of like what's easier or harder than that like javascript or or rust like i prefer to do stuff in rust because because i feel more confident that i'm not screwing something up than i do in typescript but yeah you work more in both of those than i do i.
Kiril
00:42:57
Work in both but my preference would be yeah refactoring the back end we have the api or in in the Tauri language we have those commands that are, the kind of the thing that needs to be stable or when we change it, we change it consciously. But yeah, change to me, change in Rust is much, much easier.
Matthias
00:43:17
The other thing I noticed when I played around with Git Butler was that certain operations were snappy, even though they were, I guess, in the background, quite complex. And I wonder how you pulled this off. Do you have a lot of async Rust code in the core? Or how do you handle that part? Because is Tauri essentially async as well?
Kiril
00:43:38
So, yeah, it's specific. So if you look at our functions, the whole chain, the whole stack, there's no async mentioned. However, on the top level where we define the Tauri command, there is a macro where we annotate it or we declare it as an async function. So my understanding of the framework is that it will certainly run this in its own thread, the entire command. But within that scope, it is all synchronous functions. And this is actually, at some point, we did have async functions in those calls. And as you may know, mixing sync and async code is very, very, or can be painful. There is a constraint for our use case where certain structures that come from git2 or the libgit2 bindings in Rust, Because there is this C operation, those structures are not sent sync. And because of that, yeah, they need to be in this sync context. And that made it quite difficult to do certain things. We had to do a spawn thread manual and do certain things. So I think we are in a much better place now that everything within an API code or within a command is sync.
Matthias
00:45:07
Yeah. I personally also believe that the core of an application should be sync ideally because then you can use it in other contexts. As you say, mixing sync and async is not easy. And if you have a library that cannot use async, then you're kind of bound to building a weird bridge between the two. For example, you could technically run that thing on an embedded system, which maybe is not fit to run async code. It could be with embassy, but it's not necessarily the case. So keeping that separate I think was a good decision and this is how you keep the UI snappy too because at the end of the day this macro that you described is that a part of Tauri? Is that something that is built into the yes.
Kiril
00:45:48
This is a Tauri.
Matthias
00:45:49
Macro okay and they dispatch it in the back and then it comes back with the result this.
Kiril
00:45:57
Is my understanding that they do the dispatching themselves.
Matthias
00:46:01
So all the git operations they complete at some point and then you update the user interface.
Kiril
00:46:09
So if you look at our API, we have commands, which are things that you trigger, and we have subscriptions or one subscription. Which is for for the state of of of the workspace so there is things that happen let's say on the file system or things that you've triggered in the user interface and doesn't matter on which thread that happened or it's just things that can happen, that is that is being propagated to the interface through through um yeah a subscription almost.
Matthias
00:46:45
Like an event system.
Kiril
00:46:46
Yeah i think from the java so we we use the we use the the tooling uh by by Tauri and to me that looks very much like a web socket subscription on the javascript side and yeah, on the rust side we have a uh how to say like a file system watching that that is observing the file system if something occurs it gets picked up it gets processed and then then you know we have a channel in which uh that that change is propagated and then ends up in the user user interface or in the javascript code.
Matthias
00:47:25
So that's a nice separation of concerns.
Kiril
00:47:28
Right so there's no need to do polling for example in the front end.
Matthias
00:47:31
How many people write rust at git butler.
Kiril
00:47:35
You count me probably not i Actually.
Scott
00:47:38
Give me a 0.5.
Kiril
00:47:40
Well, if we can't you. Okay, so we actually, I want everybody to be writing Rust. So we encourage everybody to, including folks that have, I have to say, primarily front-end JavaScript background. We encourage everybody to get their hands dirty or rusty.
Scott
00:47:58
Let's say. You're waiting the whole episode for that one.
Kiril
00:48:04
We should do the joke part. No, so we encourage everybody to get into this because A, everybody's asking for it. Like it's interesting. It's interesting to try to expand one's horizon. And B, actually it makes sense to work full stack. With that said, the people who work regularly in the Rust side of the code is probably three, four people out of say seven.
Scott
00:48:32
Yeah, engineering-wise, yeah.
Matthias
00:48:34
How does the onboarding work when you tell someone to write some Rust code? Do you just throw them into the cold water? Do you have them go through the Rust book or some tutorial? Or do you sit with them for a while?
Kiril
00:48:47
They just jump straight into it. I think this is what we did. I mean, there is a pattern that one can follow, right? You can see how, just let's imagine any front-end developer that has never touched Rust. So the part where the front-end or the JavaScript code talks to the Tauri part, that's obvious. You can see how other commands or requests work. So you can copy-paste that. And then you can see, you can search for the string in the code base. And you can see, oh, this is where we declare those. Okay, this is the name of the thing. I can see what we're doing. And you can just, you know, copy paste yourself into something.
Scott
00:49:29
That's how I do it.
Kiril
00:49:30
That's how we all do it, you know. And then hopefully you're following a good pattern. And, you know, you should be brave. You should try things. I think this is what makes us also happy as developers, you know, just trying things and seeing what works and what doesn't.
Matthias
00:49:48
And still, I also do the same. I'm with you on this one. Still, sometimes I wish for some coding guidelines when I start with a new project or start with a new company. I'd like to be a good citizen of the code base.
Kiril
00:50:01
Right?
Matthias
00:50:01
Do you have something like that? Any internal guidelines?
Kiril
00:50:04
So we use Clippy. So we have rules. We also, I mean, the way I do it for me is I look at other projects. I look at how other projects structure their application logic and how they approach abstracting things, to what extent they use traits and how they use traits, how they, I mean, you know, extension traits for certain things, but not for certain other things, or just multiple implementation blocks, just getting into the dramatic rust process. I sometimes would write some code and I would reach out to somebody more experienced on the team and be like, is this idiomatic? Is this how you do it? And then I would learn that you can actually express it in a more beautiful way or in a more succinct way. One phrase that I've heard within the Rust ecosystem score is this idea or this sentence of make invalid state unrepresentable. So you can express the states of your application or of that particular piece of logic in a way that you can't have the wrong state using the some types, enums and such. Which is really cool to think about just your programming as putting a puzzle together and just creating a structure that ultimately represents the kind of your application truth or the thing that it's supposed to do and it's not supposed to do, like thinking in terms of invariance.
Matthias
00:51:36
I come from Python, and it's a bit similar to Ruby in a sense that sometimes I had a larger change and I would run the project for maybe 20 or 30 minutes or so. And then it would crash, crash way down in the stack with some silly mistake that I made because the type system wasn't saving me. It wasn't a compiled language. And I think at GitHub, you had some sort of code style guide. You had something like that. I guess there was something. I can remember you wrote a bit about it on the blog, on a GitHub blog, and at least the code itself was written by expert Ruby programmers. Some of that do you miss it in rust sometimes where there are not that those clean guidelines for example when to write functional code when to write object-oriented code and and stuff like that.
Scott
00:52:27
I don't i mean i don't miss it like i like i said i'm fairly practical i i i do there's a lot more now i think especially than in the ruby world there's a lot of stuff like clippy and prettier and, and with like, there's, there's, you know, I save the file in VS code and it, it fixes sort of idiomatically, you know, what, like, like what everything kind of should look like from a formatting perspective or tells me when I'm doing something wrong in JavaScript or something. So, you know, I have, I have at least a little bit of a hint beforehand that I'm, that I'm formatting something wrong or, or that it's, it's going to look wrong or something like there's no, there's no spaces tab sort of like arguments and, you know, anymore, right? Like, like, I think the whole industry has kind of moved on from that. But from a – man, I don't know. It's like from this type of thing, I think –. I've been around enough of these where it changes over time. And I've seen a lot of this in the Ruby community as well, right? Where people are saying, here's how you should, you know, do you have, you know, fat model, skinny controllers or blah, blah, blah. And they'll come up with something and then everybody will try to do that. And then five years later, it'll kind of change a little bit. And then you'll like refactor code and the new kids will come in and be like, this is all bullshit, right? And so like, like, I think as long as the code works and customers are happy, like they're not looking at your code, right? Like a lot of it is going to be what does the next person that comes on, can they understand your code? Can they contribute to your code? I don't like clever code, right? Like I like dumb code that is easy to understand and easy to modify. And so I always kind of go away from super clever stuff because even though it makes you feel smart, it may not. And that's why it makes you feel smart is because somebody else has a harder time understanding it or figuring it out. Like I don't like magic. Anyways, yeah, I don't know. I would move towards as simple and as understandable as it possibly can be. But you have to make the next person that comes on feel like they can own the code too, right?
Kiril
00:54:25
And to your point, if you optimize for deletion, if you optimize for making the code base modular, you can always take bits out, replace, reimplement, and swap parts out. I think that's what makes it easier to move faster because that's one of the things that are really important for us, to be able to iterate on our ideas. As Scott said, we have no idea what we're doing.
Scott
00:54:51
That may not be the message that we want to put across. We're slowly getting a better idea of what we're doing.
Kiril
00:54:59
I mean, this is tongue-in-cheek, but we iterate, we get feedback, and we always need to incorporate those learnings. And sometimes our approach for a particular solution is not ideal. So we take it out or we want to be able to take it out and replace it with something better. Yeah.
Matthias
00:55:17
How much of the code base is functional and how much is object-oriented?
Kiril
00:55:22
Well, object-oriented, I don't think we necessarily have any object-oriented. I mean, not everything is purely functional. We would have mute, if that's what you mean. We would have things that we mutate. But in terms of state, you start up GitButler as an application, and there is very little to no state that the application holds, when you get a command or a request, something happens and now there is more state. But that state goes out of scope when that command is over or has been completed. The only kind of state that is, the ultimate piece of state is, okay, which repository are we working on? Like which is the repository? And that's just the path. So a path is the core piece of state. And then, because if you consider that our application is dealing with, well, Git structures, files on disk and the Git state, which the source of truth is on disk, not in our application's memory while it's running. And for that reason, after each operation, there is no more state that is relevant. We completed the operation, we return, and that's that. Which is, I think, beautiful. It's nice to not have to maintain controllers that...
Matthias
00:56:52
That means you don't have any long-running mutexes, any synchronization that you need to take care of.
Kiril
00:56:59
Correct.
Matthias
00:57:00
Mostly depend on the file system, and then you do your next operation.
Kiril
00:57:04
Correct. So we initially had those simply because of our background, and we assumed that we did need that. And then through some help and guidance, we've come to understand that we don't need it, and we've been able to take it out. And by taking it out a lot of things have become simpler things that used to get a control project controller or a user controller now get nothing the only thing that you need to know is okay what which which path am i dealing with and then everything can be derived at any point in time.
Matthias
00:57:40
It's very cool because what you said was you optimize for deletion and that makes a lot of sense in my opinion and what you said is that you also have a very pragmatic approach and you want the next person to have a good time do i hear that you also want to have simple rust you want to have a code base that is understandable is simplicity a core value.
Kiril
00:58:01
Yes my very soft yes no actually i want to say i want to share a piece for for the audience early in my career i watched a video by Rich Hickey called Simple Made Easy. And he makes the distinction between simple and easy. And he talks about the value of simplicity in software. So I watched that talk and it really changed my perspective on software engineering profoundly. And I have since rewatched that talk together with colleagues at any given time with different set of colleagues. But I've seen this talk, I would say more than 10 times and I think it's really valuable. So simplicity in software is a core value for us.
Matthias
00:58:49
And what's the difference between simple and easy?
Kiril
00:58:53
Well, I would not do it justice. I think people should watch the talk, but in a nutshell, simple relates to having not interleaving concepts. If you have something that is dealing with let's say. Okay, firstly, you don't want to mix up state and functionality. So there is that separation. And secondly, you don't want to have or you want to avoid this endangling of concerns. Like you have a struct that is responsible for this and that. Can it be separate? Is it appropriate for it to be separate? And the difference with easy is so rich makes a very eloquent point about how easy it's more relates more to if if a if something is at hand or familiar so it's it's a distinction in okay familiarity so this is resembles something i've done before and and that familiarity is good it's great if something can be familiar but simple is has a has a more more pure and and this this talk he he talks about actually simple comes from from from the latin or simplex and that means one braid you're not yeah you're not complecting it i'll not do it i will not do it in the podcast, the talk is called simple made easy by rich hickey this is the creator of the closure programming language and i had a closure phase for lasted many years yeah.
Matthias
01:00:23
And in that talk he also refers to that and says just because something is familiar doesn't mean that it's simple because maybe you got used to it in some weird way maybe we got used to svn at some point or We got used to the complexity, maybe not complexity, but with the familiarity of Git. And maybe that might not have been the perfect abstraction. We just got used to it, to branching and naming things and rebasing and so on. But those things are easy in the sense that you said is familiar, but maybe not simple. Yeah. Not intertwined with other ideas.
Kiril
01:00:59
Right yeah and.
Matthias
01:01:01
When you when you talk about deletion i guess you enforce it somehow because if you can only delete things that are isolated.
Kiril
01:01:07
Correct yes yes so packaging or creating by by feature as like a core tenant like that way preventing the the complecting the word is complect common word in english.
Scott
01:01:21
I thought you said it was latin so So I've never heard the word complect before, so.
Kiril
01:01:27
Okay.
Scott
01:01:28
If it is, I'm going to look it up now.
Kiril
01:01:30
Yeah, I think that word spiked in popularity in 2011 after that talk.
Matthias
01:01:37
Was that a similar principle that you lived up to at GitHub where you said, make it easy to, I'm sorry, make it simple, make it isolated? Or was it a large code base and it kind of grew organically?
Scott
01:01:56
That's a great question. I mean, it really depended. I feel like for our user interface, for user-facing stuff, it was different, right? Like we really wanted, there was a lot of thinking about and understanding what do we present our users and this concept of everything added dilutes everything else, right? So we didn't want to be Jira or something. Like we didn't want to just add features because people said that they wanted features or said that they would pay for features. Like we really wanted to keep it. I mean, we wanted to keep it simple, right? Like we wanted to keep it that there were only so many things you could do and that we felt that these were the important things to do because everything else that you add makes it more complex by definition, right? From a backend standpoint, from a code standpoint, from an infrastructure standpoint, I think because it wasn't customer facing, you know, we could experiment more. We had a lot of people coming on at a relatively fast rate at some point. And so like a lot of it, it just depended on what system you were in and what that particular team cared about, right? Like, did it care about performance? Did it care about maintenance? Did it care about observability, right? Like there were different things that different teams cared about and would optimize for. So I don't know that we, I would say that, you know, every team had like some, like GitHub had some overarching sort of concept. I think it really depends. It was very, it was very, I don't know, tribal, right? Like, like there were groups and they had their things and it wasn't like, here's the GitHub philosophy.
Matthias
01:03:32
When GitHub started, Ruby was also pretty new, I guess, wasn't it?
Scott
01:03:36
I mean, boy, that's a good question. I mean, Rails was new-ish, I think, you know, five years or four years or something. Ruby was around longer, but I think Rails really kind of picked up Ruby.
Kiril
01:03:49
It was the prime time of Rails.
Scott
01:03:51
Yeah. I mean, it was when, I mean, there's still, you know, Shopify and Twitter and, you know, like all of these.
Kiril
01:03:59
Airbnb.
Scott
01:03:59
Airbnb, like all of these companies. And they were all in San Francisco and we knew all these guys, right? And so, like, I think it was a really nice culture of entrepreneurship and building interesting things, right? And having customer focus. Like it was a great environment for, for all of that. I think the Ruby community and, and, you know, like I may have said at the beginning, I think it really helped make Git the, the thing that won, right. Is, is because, you know, we started part of the reason I think is we started GitHub and we were giving everybody, we obviously preferred Git and everybody in these, all of these communities started using Git to, to some degree because of, of GitHub or because of us personally, just, we knew a lot of these, these people. I'm really curious if GitHub hadn't come along, if Mercurial would be what everybody's using now, right, or something else. But, yeah, I mean, like you were saying before, like, we're all familiar with Git. There's kind of this local maxima problem. Like, it's not, there's nothing better than that. It's where everybody's kind of entrenched to some degree. So we do, we are familiar with it. It does have this easy, you know, even though it's not easy, like, it's easier than learning something else.
Kiril
01:05:06
So I- It is not easy, but it's simple. Yeah.
Scott
01:05:10
Is it?
Kiril
01:05:12
Okay, maybe not.
Scott
01:05:13
Maybe. It works.
Matthias
01:05:17
The reason why I was asking was that back in the day when you started a new application, Ruby, was it a positive signal to the outsiders, maybe investors? I know that GitHub never took any investments as far as I remember, but would it have raised any eyebrows? Or did investors not care at all?
Scott
01:05:36
You know, I, so we, we bootstrapped until we were four years in, we had almost, almost a hundred employees from, from, you know, such a sort of a bootstrap background. So when we talked to investors, it was, they didn't care what we were building it on, right? Like we, we had millions of users at that point and everybody, we had taken no VC. So, so everybody wanted in on that. So I didn't have normal VC conversations at that time. I, I don't, even now I don't, I do some angel investing or I have, I've talked to lots of investors about this. Nobody's cared about the tech stack so far. I don't remember seeing it in lots of decks or something like, I don't know that, that most investors care so much about how you're building it as much as what you're building. Like, like that is important. Like, is there conceivably a market, right? Like doing it in Rust versus Ruby versus whatever, like whatever you're familiar with and whatever you think you can move fast on or get to product market fit quickly. I'm not sure that there's that many investors. I mean, I haven't run into a lot of investors who have really cared.
Matthias
01:06:43
If you talk to someone else, a friend from another company and they started building up their tech stack and they were considering Rust and maybe other languages like Go or Zig, what would you tell them, what advice would you give?
Scott
01:06:57
Yeah, I mean, I would say work with whatever the founders or the technical founders, enjoy working in right can can move fast in because you can always i mean you know twitter's not ruby anymore i think they they moved to Scala or something at some point like once you get once you once you find product market fit like once you start getting users and you start getting traction like and you make it you can rewrite it in anything you want like it's it's actually kind of interesting that that shopify and github and you know some of these big big companies are still running on ruby stacks because they they still like using it i think for the most part i'm sure there are people in those companies that are like we should rewrite this and go or whatever. But, but yeah, in the beginning, I think honestly, from a VC or from a, from a startup standpoint, I, I would, I would heavily say it does not matter, right? Like we could have chosen Electron. I think we'd be roughly as far as, as we are today that I, I think that Rust and Tauri were a better choice for, for, you know, future reasons and things like that. But like, you know, From trying to find out, is what I'm building valuable, I honestly, I have a hard time saying you can't work your way around. Everything has problems in some way and everything has strengths in some way. And so, you know, like it just depends. I think it's all shakes out. Like you'll work around the problems if you make it.
Matthias
01:08:16
But that's also a very positive thing because maybe someone is considering to use Rust. Maybe they are considering that it would be harder to get investments. And you as an angel investor, you know how these conversations work. So they shouldn't be concerned about that.
Scott
01:08:32
Yeah, no, I don't think any, from a VC, from a raising money standpoint, I don't think anybody really cares. I think they'd much rather that you do what you're comfortable with and you feel like you don't have to have a big learning curve, you know, so that you can experiment and figure it out. But it's the same in any, like it's like, you know, what JavaScript framework do I use if I like job? Like who cares, right? Use Angular, use Svelte, use React, who cares? There's pluses and minuses to all these things. It's just, you know, what can we move fast with at first and then we'll figure it out.
Matthias
01:09:03
Do you like to work on the code base every day, Kiril? How does it feel to come to work and touch the Rust code? Do you feel you know, do you feel, Like you're making progress? Do you feel like there's a lot of velocity? Like it's maintainable? What's your feeling on the current code base right now?
Kiril
01:09:25
I mean, it's very exciting. I think it's a little bit like a puzzle that's coming together more and more. It's coming together in terms of performance or fixing issues that people are having. So that's very exciting to me. Like knowing that you fix an issue that somebody has reported gives me a huge kick. Actually, just this morning, we made a release. And I know that in that release comes some fixes that I made yesterday and people have reported those problems and that feels really good. So that's the part that gives me the biggest satisfaction. But just from a technical standpoint, like working with Rust, it is satisfying. It is interesting because I feel like I'm over the initial hump where, you know, I struggle with, you know, the systems that are different in Rust. And now I can just focus on what we're building rather than what we're building with. So from that perspective, it's cool. I'm very happy with Rust as a choice. I feel, so to add to what Scott said, iteration speed is everything. The ability to say, all right, I have an idea, I wanna try it and see if it works and then iterate from there. That's critical for somebody who is making something new and trying to understand if other people want it. So... With that said, I mean, Rust can be slower to prototype in if one is not at all familiar with Rust or if it's something that is... I actually honestly think the user interface stuff would be quite difficult to do in Rust just because of the nature of user interaction. But for something that feels very functional, like I described earlier with a command that gets executed, that fits very nicely with the Rust model. Like you don't need a lot of state you don't have to think about state very much you're you're transforming something on disk you you it's something that you can test in a good way and then it's a very good fit so depending on the use case depending on the kind of context yeah it can be just the right to to to for the job you.
Matthias
01:11:34
Say you're happy with the code base but one thing that people always criticize are the compile times the rust compile times do you have any issues with that in your code pace i.
Scott
01:11:43
Wish it was faster i wish it was in in developer mode it takes about if it's if it's an iterative build and i've made a change to one crate for us it takes about, two seconds or three seconds to to kind of make the new target and run it again of course i wish it was faster and i wish it can do it as fast as go go is really fast if if i need to build everything from scratch it takes about a minute with that said it's not a huge issue.
Matthias
01:12:25
I've seen way worse.
Scott
01:12:26
Yeah, I haven't. I was, I've never really, like when I'm working on the Rust code, I haven't really noticed it being horrible. I mean, the big thing for us is that we're dogfooding it, right? So we're working on the version control system that we're using to try to version control the version control system that we're working on. And so, like, we have, we can't have our main code base in the development build of the thing because then you change some Rust code and it dies, right? Or you do something and it corrupts, like, your database, right? Like i've run into that several times so like i have to have the nightly now that we have a nightly it's much nicer because i do the nightly for the for the gip other code and then and then other repositories with the development build but like yeah that that type of thing where it just kill it and then start it back up again was but i mean yeah it only takes a couple seconds even even for this this larger sort of gui type program it's it's not bad at all.
Matthias
01:13:15
We need to keep an eye on the time because you need to go
Kiril
01:13:18
the jokes we. Haven't done the jokes can we do the jokes so.
Matthias
01:13:23
Yes but be careful what you say in the podcast. So whatever you say we'll be able.
Kiril
01:13:27
Okay i have a good one so firstly let me set this up well we at gate butler have a dad joke culture we like puns we like funny dad jokes to the extent where during our friday all hands meeting we do an event where everybody goes on a stage, tells their dad joke of the week. And then whoever gets the biggest laughs wins a little trophy. And then we pass the trophy around. So we got it out, that, that joke off and yeah. So today we are having an old hands and I've prepared a joke.
Scott
01:14:06
I want to hear your joke.
Kiril
01:14:07
So what do you call a dog that can make magic tricks? You call it the labracadabrador.
Scott
01:14:21
That's a good one. I like that one.
Kiril
01:14:23
Make sure you laugh just as much later when the jury's out.
Scott
01:14:30
So last night I was just about to fall asleep and the ghost of Gloria Gaynor came into my room. First I was afraid. I was petrified. It's a song. That one didn't land as well.
Kiril
01:14:44
I didn't know the song.
Matthias
01:14:45
I liked it.
Scott
01:14:48
Oh, first I was afraid. I was petrified. Is that better? Should I do it that way?
Kiril
01:14:54
Yes.
Scott
01:14:54
Yeah, I got to sing it.
Kiril
01:14:55
I'm so sorry. You should sing it for the job. Oh, this is good. You should sing it later.
Scott
01:15:05
He's rethinking all of his decisions, inviting us on this podcast.
Matthias
01:15:09
A friend of mine is trying to date a woman from the south of France. And he was unsure if he could approach her and talk to her. And I said Corsican.
Scott
01:15:23
That's a good one.
Matthias
01:15:24
Oh, my God. That's a good one. So you should make this a thing on your podcast.
Scott
01:15:31
Too. You should make all of your guests at least share one to start it out with.
Kiril
01:15:36
We should do it on our YouTube channel.
Scott
01:15:37
Yeah, we should. Yeah.
Matthias
01:15:39
Okay. And it has become a bit of a tradition in this show. That's the actual tradition. To have one last statement about the Rust community. So if you had the chance to address the Rust community as a whole, anything you could say, what would you say? Could be positive or negative, could be about the community ecosystem, compile times, whatever.
Scott
01:16:06
I mean, I love the crates system and the crates ecosystem actually like coming from, you know, sort of Ruby gems and, you know, sort of the, I don't know, crating the gem sort of system there. I feel like it's, it's, or like, you know, NPM or, or whatever, like I, I actually really, really appreciate how nice it is and how nice it has been to work with and to find stuff and to get documentation. So, you know, from my perspective, I've, I've been very, very surprised by, by how good that is. And then, you know, and then from a personal standpoint, I would say it's funny. Steve has been great. Like, like he's my, he's like the biggest introduction to the Rust community that I know. And I found him in the JJ chat rooms. And he was like, hey, Scott, thanks for the pro git book. And I was like, hey, Steve, thanks for the Rust book. Fantastic. And then everybody's like, oh, these guys patting each other on the back.
Kiril
01:17:04
I love the language. Thanks for Rust. Thanks for the ecosystem things. I mean, the libraries have been great.
Scott
01:17:10
Yeah, I mean, it's interesting. Like if we I think if we think back, like we didn't really know at the beginning of this, like, is, is Electron the right call? Is, is Tauri the right call? I think, you know, after having worked it in, in, in two years, we would, I believe we would both probably recommend that somebody uses that use, use Tauri, learn Rust. Like, I think we've, if we had to do it again, we would do the same thing again, which is actually very nice. Like, like, as opposed to being like, man, I wish we'd gone down the Electron route or something. So So we really do like using the system to build our company on, which is not a not scary thing to do.
Kiril
01:17:48
Actually, I have something useful here. And now that I've thought about this a little bit longer, I like that the Rust community is generally very thoughtful for making things in the ecosystem that work and that are relatively well thought through. It's not not just tossing tossing crates out in the open and it's like okay, that doesn't matter there is there is some some consideration for for okay producing high quality software i think there there are i was thinking about the dependencies that we use uh we have. As we discussed git 2 which is bindings for libgit 2 which works really well it's uh it's documented and it functions very really well we we have used a library called notify for file system watching and that works slightly differently under the hood for different platforms but it works for us in a transparent way which is brilliant. You have Tauri which has a very healthy attitude towards security like the way they treat isolating let's say the JavaScript code from the Rust which has access to the file system and just handling things in a secure way and not least Git Oxide which is a piece of software that is well thought through as well in terms of security, in terms of performance. And it feels like the tooling in the ecosystem produced by the community is of high quality. So I think this is probably the highest praise I can give. Just the median quality of a library or crate that you can find it's greater than the medium quality of other ecosystems. And I have been exploring languages for many years now, and I genuinely feel this way.
Matthias
01:19:49
Amazing clothing words. Where can people find out more about Git Butler?
Scott
01:19:53
Yeah, I mean, the easiest thing is to go to gitbutler.com, and we have links to our source code on GitHub and our Discord, which is kind of interesting. We just recently kind of did this experiment where our entire corporate chat is on the same Discord that is public. So we have some private rooms for just us. But for the most part, a lot of what we do is build in public and it's on Discord. So you can find our blog there, which you mentioned, I think, at the beginning of this as well. But yeah, gitbutler.com is also a really pretty website. Actually, Pavel, our designer, approached it in a really interesting, awesome way that I think is very unique. So it's not hard on the eyes.
Matthias
01:20:34
Find all the links in the show notes. And I guess that sums it up. Scott, Kiril, thanks so much for taking the time.
Scott
01:20:43
Yeah, thanks for having us.
Kiril
01:20:43
Thank you very much.
Matthias
01:20:45
Rust in Production is a podcast by corrode. It is hosted by me, Matthias Endler, and produced by Simon Brüggen. For show notes, transcripts, and to learn more about how we can help your company make the most of Rust, visit corrode.dev Thanks for listening to Rust in Production.