WEBVTT 00:00:01.650 --> 00:00:06.710 Welcome to Season 5 of Rust in Production, a podcast about companies who use 00:00:06.710 --> 00:00:08.590 Rust to shape the future of infrastructure. 00:00:08.930 --> 00:00:12.610 My name is Matthias Endler from corrode, and today we talk to Alexandru Ene, 00:00:12.790 --> 00:00:17.090 Principal Engineer at Prime Video, about building modern user interfaces in Rust. 00:00:19.250 --> 00:00:25.350 Alex, thanks so much for taking the time. Can you introduce yourself and the company you work for? 00:00:25.350 --> 00:00:29.070 So, hi, thanks for inviting me to do this. Super fun. 00:00:29.790 --> 00:00:31.970 I'm Alex. I'm a principal engineer 00:00:31.970 --> 00:00:35.490 for Prime Video London. I've been with them for about eight years. 00:00:36.590 --> 00:00:41.430 Actually, almost nine. And before that, I used to do video games, 00:00:41.590 --> 00:00:43.450 game engines and such things at Ubisoft. 00:00:43.710 --> 00:00:48.410 And right now, as I said, I work with Prime Video on the Prime Video app that 00:00:48.410 --> 00:00:50.790 you have on your living room devices. 00:00:51.130 --> 00:00:57.810 And you do a lot of rust there. so that's why i'm yeah and. 00:00:57.810 --> 00:01:03.490 And when you say you did video games does that mean you also used rust for that 00:01:03.490 --> 00:01:05.230 or was it in some other language. 00:01:05.230 --> 00:01:08.050 No that was c++ and c sharp 00:01:08.050 --> 00:01:11.010 for unity but mainly c++ at 00:01:11.010 --> 00:01:14.190 ubisoft i don't know what they're using now i left 00:01:14.190 --> 00:01:17.170 in 2016 and i learned about rust 00:01:17.170 --> 00:01:21.170 in 2017 18 i think 2017 so 00:01:21.170 --> 00:01:24.010 i didn't even know rust existed actually for the most time 00:01:24.010 --> 00:01:28.550 i ignored it thinking it's a garbage collected language but 00:01:28.550 --> 00:01:31.390 then i saw this guy on youtube doing demo scene 00:01:31.390 --> 00:01:34.750 type of things with rust which is this super tiny 00:01:34.750 --> 00:01:37.850 executables that do art and nice visuals 00:01:37.850 --> 00:01:40.650 and music so i was like oh wait a 00:01:40.650 --> 00:01:43.850 second it doesn't have a garbage collector so how 00:01:43.850 --> 00:01:46.670 do they do all of this magic with memory that 00:01:46.670 --> 00:01:52.650 they keep saying so that's how i kind of got into it and use it then for some 00:01:52.650 --> 00:01:59.550 tiny emulators in the beginning and then since then i'm still building a game 00:01:59.550 --> 00:02:04.390 in my spare time with rust i would say that was my big rust project outside 00:02:04.390 --> 00:02:06.170 of work and how i got into it. 00:02:06.170 --> 00:02:12.830 So that's interesting because you come from a c plus plus background and i think 00:02:12.830 --> 00:02:17.790 a lot of listeners also have the same background so what kind of convinced you 00:02:17.790 --> 00:02:21.510 of rust in the end was it just the fact that it didn't have a garbage collector 00:02:21.510 --> 00:02:24.290 or did you also learn to like the language at some point. 00:02:24.290 --> 00:02:32.390 So it was i needed i wasn't opposed to something other than c++ i just hated 00:02:32.390 --> 00:02:38.770 build systems in c++ they only work on windows i could only use Visual Studio. 00:02:39.450 --> 00:02:43.610 With CMake, it was always a pain. So when I saw something with Rusty, 00:02:43.770 --> 00:02:46.470 just cargo new, that was mind-blowing to me. 00:02:46.570 --> 00:02:50.050 I mean, people from JavaScript and these other languages are like, 00:02:50.130 --> 00:02:53.710 yeah, we did that for ages. What are you talking about? But for a C++ guy, 00:02:54.490 --> 00:02:57.150 to do a cargo new or cargo run and things to 00:02:57.150 --> 00:03:00.150 just work that's just mind-blowing at 00:03:00.150 --> 00:03:03.130 the time to me and then i started liking bits of 00:03:03.130 --> 00:03:05.970 the language like enums things like that 00:03:05.970 --> 00:03:09.610 i really really like those but yeah 00:03:09.610 --> 00:03:12.590 the build system was just magical i still 00:03:12.590 --> 00:03:16.270 remember the first time i was watching this guy on youtube ferris 00:03:16.270 --> 00:03:21.890 and he was building some demo scene things and he starts a new he's building 00:03:21.890 --> 00:03:27.010 an emulator for n64 i believe and he starts a new project with cargo new and 00:03:27.010 --> 00:03:32.670 then cargo run i was like what is happening this is just so cool so that's why i got into it. 00:03:32.670 --> 00:03:38.830 Shout out to ferris he's also one of the first streamers i watched when i came to rust and, 00:03:39.370 --> 00:03:42.530 he's just amazing i guess everyone should 00:03:42.530 --> 00:03:46.890 watch watch his stuff on on youtube it's still there highly underrated i would 00:03:46.890 --> 00:03:53.110 even say because he really built an n64 emulator from scratch not sure how far 00:03:53.110 --> 00:04:00.770 he got but the idea itself was so great right and yeah i wish he he would do more rust stuff again. 00:04:00.770 --> 00:04:06.610 Yeah we make a petition to bring him back or at least on the podcast we should 00:04:06.610 --> 00:04:09.710 have him it's this good guy yeah but. 00:04:09.710 --> 00:04:16.770 In c++ you have things like conan or package managers I'm not familiar about 00:04:16.770 --> 00:04:20.350 the state of package management in C++. Can you enlighten us? 00:04:20.910 --> 00:04:26.950 So I never used any of those. I think they weren't that popular with game developers at the time. 00:04:27.090 --> 00:04:35.450 So all I was using is raw big Visual Studio and then create a project and then make all my settings. 00:04:35.970 --> 00:04:40.830 And actually, I made the game with some friends that we ported to Linux and 00:04:40.830 --> 00:04:45.610 none of us, like we only, we use directly make files, not even CMake. 00:04:45.670 --> 00:04:51.810 So just put your files in the make file and then in the right place and then it will work. 00:04:53.470 --> 00:04:56.390 That's i'm not familiar with i never use 00:04:56.390 --> 00:05:02.190 conan people keep bringing it up whenever i mention build systems but i i never 00:05:02.190 --> 00:05:06.750 tried and i don't think they supported at the time windows that well i'm not 00:05:06.750 --> 00:05:14.890 sure what the status was mainly i'm a windows developer i never like only recently i bought a mac i. 00:05:14.890 --> 00:05:20.790 Think there's something to be said about conan not being officially, 00:05:21.710 --> 00:05:29.990 part of the c++ how could i say standard or you know it's it's a separate project 00:05:29.990 --> 00:05:34.910 and i guess there's a couple good things about being separate but in reality, 00:05:35.630 --> 00:05:41.470 for something as central as a package manager in 2025 it's really beneficial 00:05:41.470 --> 00:05:46.270 to have that as part of the normal ecosystem part of the runtime that you install. 00:05:46.270 --> 00:05:49.830 Yeah and it makes for such a nice smoother 00:05:49.830 --> 00:05:53.230 starting experience I would say so it 00:05:53.230 --> 00:05:58.270 is a little bit of a shame but yeah I haven't written C++ since 2018 so I'm 00:05:58.270 --> 00:06:04.690 so happy now because all of my side projects used to be in C++ like all sorts 00:06:04.690 --> 00:06:11.650 of game mini game engines or whatever so now I have used Rust for that and now I use Rust at work. 00:06:12.450 --> 00:06:18.210 As part of your work you also probably review a lot of code how has that review 00:06:18.210 --> 00:06:21.350 process changed ever since you moved from c++ to rust. 00:06:21.350 --> 00:06:25.510 So that actually is really you 00:06:25.510 --> 00:06:28.590 probably know because you've interviewed other people but 00:06:28.590 --> 00:06:31.470 i can tell you i'm so much more relaxed on 00:06:31.470 --> 00:06:34.710 code reviews so for example we do 00:06:34.710 --> 00:06:39.910 still have c++ code in in my team for the thing we deploy on devices we have 00:06:39.910 --> 00:06:48.430 a mini layer that abstracts the system there so that's written in C++ and comes 00:06:48.430 --> 00:06:52.970 with the virtual machines and all that that the UI uses later on. 00:06:53.590 --> 00:06:59.110 So that's in C++. And then in RAS code, it's just like paying attention only 00:06:59.110 --> 00:07:03.810 to you know, code is doing what it's supposed to do, not this. 00:07:03.970 --> 00:07:05.530 What are you doing with this memory? 00:07:05.750 --> 00:07:10.410 Where is it coming from? Where is it going? Even with smart pointers or unique 00:07:10.410 --> 00:07:14.470 pointers, it's so easy to shoot yourself in the foot as a newbie. 00:07:15.170 --> 00:07:19.710 And that it's For reviewing code, it's definitely more relaxing. 00:07:20.110 --> 00:07:23.990 I think it's way easier to write correct code when it. 00:07:25.090 --> 00:07:32.950 Comes to memory access it's just trivial now the only thing i would say which helped rust, 00:07:33.630 --> 00:07:36.830 adoption a lot is that it doesn't require years and 00:07:36.830 --> 00:07:41.530 years of experience so for example if i were to propose c++ for the same thing 00:07:41.530 --> 00:07:46.250 i'm using rust now for performance reasons it would be so hard it's so hard 00:07:46.250 --> 00:07:52.550 to onboard people onto c++ so unis i did c and c++ in unis but today that's 00:07:52.550 --> 00:07:54.210 not true anymore and very few of them, 00:07:54.750 --> 00:07:56.850 the new grads even do C++, 00:07:57.530 --> 00:08:01.330 very few of them so it's really hard to onboard them, it takes so much time 00:08:01.330 --> 00:08:03.810 and with Rust it's so much easier it's just like, 00:08:04.610 --> 00:08:08.810 so basically the team that switched to Rust in Prime Video switched from JavaScript 00:08:08.810 --> 00:08:14.210 to Rust and it's fine I mean it's a new language, it has some onboarding but imagine, 00:08:14.990 --> 00:08:20.030 switching people who only know JavaScript and Java to Rust to C++ it's just 00:08:20.030 --> 00:08:25.450 gonna have It crashes everywhere, basically. 00:08:25.870 --> 00:08:32.730 It's a given. So I would say for adoption from teams that know other languages, 00:08:32.910 --> 00:08:39.270 which isn't C++, it's way easier to onboard new people, even new grads or experts. 00:08:40.480 --> 00:08:45.700 So to summarize, the difference between reviewing C++ code and Rust code is 00:08:45.700 --> 00:08:50.400 that in C++, you need to be really careful with where memory goes. 00:08:50.560 --> 00:08:53.760 Memory management is way more central in the review process, 00:08:53.760 --> 00:08:56.860 whereas in Rust, it's more about the business logic. 00:08:57.180 --> 00:09:04.040 It's more about the data flow, but not on a systems level, more on a business logic level. 00:09:04.540 --> 00:09:08.980 Yes, you're just verifying that this code is achieving what it's supposed to 00:09:08.980 --> 00:09:12.260 achieve, but not at the low level of what are you doing with this memory? 00:09:12.440 --> 00:09:14.860 Is it safe to access it where you're accessing it? 00:09:15.060 --> 00:09:18.720 Are you sharing it weirdly between threads or so on? 00:09:19.180 --> 00:09:25.440 My main problem with learning C++ in the past was that there was so much hidden knowledge. 00:09:25.680 --> 00:09:31.340 There were so many unwritten things that you needed to know about this language. 00:09:31.340 --> 00:09:34.280 It was a big, big language that you needed to learn. 00:09:34.780 --> 00:09:38.080 Whereas in Rust, a lot of things are documented really well. 00:09:38.080 --> 00:09:44.180 And yes there is a learning curve but after this you have pretty good documentation 00:09:44.180 --> 00:09:49.900 you have the compiler helping you along and the language feels smaller still. 00:09:49.900 --> 00:09:53.160 Yeah the thing with c++ is everyone 00:09:53.160 --> 00:09:56.180 picks their own flavor you know and then 00:09:56.180 --> 00:09:59.480 there's all sorts of parts of the language that oh don't 00:09:59.480 --> 00:10:02.580 use that but it's there like as a newbie you 00:10:02.580 --> 00:10:05.500 don't know what you shouldn't touch pointers are still 00:10:05.500 --> 00:10:08.820 there like row pointers they're everywhere they're just and it's 00:10:08.820 --> 00:10:12.260 so easy to just access them to get around 00:10:12.260 --> 00:10:15.260 certain things that maybe they would 00:10:15.260 --> 00:10:19.280 catch so like a unique pointer oh let me just let me just grab the pointer out 00:10:19.280 --> 00:10:23.340 of this unique pointer and do my thing or whatever and it's just maybe share 00:10:23.340 --> 00:10:27.420 it accidentally to some other place and then save it in some other place and 00:10:27.420 --> 00:10:32.940 that's how you get use after free very easy so that's the problem whereas other 00:10:32.940 --> 00:10:34.960 languages which are newer like Rust, 00:10:35.240 --> 00:10:38.440 they don't have this kind of everyone-needs-to-pick-a-subset, 00:10:39.020 --> 00:10:39.580 problem. 00:10:40.680 --> 00:10:43.660 Use the whole language, it's fine. The whole... 00:10:44.270 --> 00:10:49.050 Features are more cohesively implemented so i think that's a big deal. 00:10:49.050 --> 00:10:52.650 Well one could argue that one 00:10:52.650 --> 00:10:56.070 main reason for c++ having all of these caveats 00:10:56.070 --> 00:10:59.150 is that it's language that has historically 00:10:59.150 --> 00:11:02.450 grown over time and also is still compatible 00:11:02.450 --> 00:11:05.670 with c i'm actually not sure if that is still 00:11:05.670 --> 00:11:08.930 true or if that's an urban legend 00:11:08.930 --> 00:11:11.770 by now i would like to hear your opinion on 00:11:11.770 --> 00:11:14.950 this and second i think when you 00:11:14.950 --> 00:11:18.270 build larger systems in c++ you run into 00:11:18.270 --> 00:11:21.850 all of these really gnarly 00:11:21.850 --> 00:11:27.570 data management problems because this will kind of haunt you and bite you later 00:11:27.570 --> 00:11:33.490 it doesn't bite you at the start usually not at compile time so you're kind 00:11:33.490 --> 00:11:38.530 of in a situation where later on you have a larger C++ code base, 00:11:38.710 --> 00:11:41.010 and then suddenly these problems pop up? 00:11:41.970 --> 00:11:47.990 Yeah, I think one, there is definitely the aspect that it grew over time, 00:11:48.170 --> 00:11:50.270 and even multi-threaded programming 00:11:50.270 --> 00:11:54.350 became popular when I was in kind of uni, or just entering uni, 00:11:54.750 --> 00:11:58.910 when like Herb Sutter and those guys were like, there's no free lunch, 00:11:59.030 --> 00:12:00.170 they were writing these articles. 00:12:00.170 --> 00:12:07.830 So it was a language that just couldn't have predicted how the computing evolves 00:12:07.830 --> 00:12:10.130 and towards multi-threaded and all of those things. 00:12:10.370 --> 00:12:15.730 So I think history definitely has a role. It's not like they implemented it 00:12:15.730 --> 00:12:16.870 wrong. They just didn't know. 00:12:17.450 --> 00:12:23.570 So multi-threading wasn't a thing at this scale until way later on, 00:12:23.690 --> 00:12:25.510 the history of the language. 00:12:25.690 --> 00:12:32.190 So there were hard-learned lessons by C++ there, whereas Rust already came with 00:12:32.190 --> 00:12:35.010 this benefit of hindsight and, 00:12:36.010 --> 00:12:43.510 knowledge that was kind of occurred over time so I think. 00:12:45.150 --> 00:12:48.810 Kind of, it's already had way more knowledge. 00:12:48.890 --> 00:12:51.730 So when they designed Rust, I assume they had way more knowledge about, 00:12:51.890 --> 00:12:53.610 okay, what does computing look like today? 00:12:53.810 --> 00:12:59.090 What are the common problems with other things? Let's fix those in a breaking manner. 00:12:59.370 --> 00:13:03.030 Because it's not backwards compatible with C++ in any way, right? 00:13:03.150 --> 00:13:06.090 You only can, they don't have that constraint. 00:13:06.290 --> 00:13:10.950 So I think they did a really good job, the Rust designers and the people who contributed there. 00:13:11.890 --> 00:13:16.950 C++ keeps trying to do all of these things. There's various efforts, 00:13:17.230 --> 00:13:25.610 but honestly, I think they should probably stop adding things to the language. Just, it's fine. 00:13:26.310 --> 00:13:33.970 We keep, you know, you keep adding, they added recently something like option 00:13:33.970 --> 00:13:38.650 or result type of, and it's just so, it looks so ugly. 00:13:38.910 --> 00:13:41.910 Don't understand how people can criticize Rust for looking ugly. 00:13:43.410 --> 00:13:47.530 Like these guys who write c and then you look at c plus plus and c and c plus 00:13:47.530 --> 00:13:52.650 there's just god what's happening it it wasn't that bad when i started honestly 00:13:52.650 --> 00:13:58.410 c plus plus 11 okay that's fine it's just that's a good language just stop adding things to it. 00:13:58.410 --> 00:14:03.210 What do you think about all of these recent security efforts by people like 00:14:03.210 --> 00:14:07.550 bionis duestrup who try to make c plus plus safer is going anywhere? 00:14:08.070 --> 00:14:12.010 I haven't kept up to date with those. I know there's some efforts. 00:14:12.790 --> 00:14:16.570 Herb also, the Microsoft guy, I think Herb Sutter was there. 00:14:16.770 --> 00:14:22.870 He also had some ideas, even with some sort of a smartphone, 00:14:23.030 --> 00:14:25.650 but I'm not up to date with what's happening there. 00:14:26.290 --> 00:14:32.910 And in general, about security in the fields I worked in, the thread models 00:14:32.910 --> 00:14:37.670 were such that you didn't, there were other, performance was definitely on the first, 00:14:38.760 --> 00:14:44.520 order of business rather than security so for video game engines for example 00:14:44.520 --> 00:14:49.460 you're more concerned with that than security especially for a single player 00:14:49.460 --> 00:14:55.260 game where you don't you don't really interact with the outside basically so do. 00:14:55.260 --> 00:15:01.600 You think that performance and security are always at odds or has rust proven that you can have both. 00:15:01.600 --> 00:15:04.380 No you can have both and i think in good 00:15:04.380 --> 00:15:07.380 c++ you can have both as well today from what 00:15:07.380 --> 00:15:10.720 i can see the baseline libraries we 00:15:10.720 --> 00:15:14.380 use are in c the open ssl and those things curl 00:15:14.380 --> 00:15:19.600 everyone uses curl in everywhere basically and security is super important there 00:15:19.600 --> 00:15:25.000 so i think you can get it in both languages but with various degrees of effort 00:15:25.000 --> 00:15:30.400 they're not at odds but they for certain languages they take definitely more 00:15:30.400 --> 00:15:32.320 effort than others now Now. 00:15:32.480 --> 00:15:38.320 I'm not sure if you still write any C++, but if so, or even if you just read C++ code nowadays, 00:15:38.640 --> 00:15:48.360 has your way of dealing with the language changed the writing style or what you consider good C++? 00:15:49.180 --> 00:15:56.540 So I think one thing that helped me when I learned Rust is it kind of formalized 00:15:56.540 --> 00:16:00.880 the way you're supposed to think about memory and memory access patterns. 00:16:01.600 --> 00:16:05.840 So I don't think I was ever reviewing C++ code with the idea of, 00:16:06.140 --> 00:16:09.640 hmm, who has access in a mutable way to this data? 00:16:09.800 --> 00:16:13.920 I just didn't have this concept in my head. I mostly look at stuff like, 00:16:14.060 --> 00:16:16.160 did you free it in the right time, and it's... 00:16:16.950 --> 00:16:22.450 Maybe a vague concept of lifetimes, but not quite as formal as Rust makes it. 00:16:22.630 --> 00:16:27.250 So it definitely gave me at least some vocabulary and more structured way of 00:16:27.250 --> 00:16:30.970 thinking about catching those issues when I review code. 00:16:31.510 --> 00:16:36.730 Because, I mean, in the beginning, I was just using Rust at home for my side 00:16:36.730 --> 00:16:38.290 projects and passion stuff. 00:16:38.490 --> 00:16:42.650 So then when I got to work, I was writing C++ and it made me think a little 00:16:42.650 --> 00:16:45.790 bit more like, oh, what's the order of access here? 00:16:45.790 --> 00:16:49.630 How long does this live and it gives you a bit of vocabulary so even if you 00:16:49.630 --> 00:16:53.650 write c++ i think it makes you a good better c++ programmer yeah. 00:16:53.650 --> 00:16:57.950 And what have you done in rust at prime video. 00:16:57.950 --> 00:17:05.830 So i'm just a guy who rewrites things in rust slowly recently completing almost 00:17:05.830 --> 00:17:12.430 the the whole app rewrite so basically i started working on the so the team 00:17:12.430 --> 00:17:14.450 i work in is the client team, 00:17:15.270 --> 00:17:17.990 which is a little bit special because we work on 00:17:17.990 --> 00:17:21.350 living room devices so basically with the 00:17:21.350 --> 00:17:24.110 way prime video works we have one application that goes on 00:17:24.110 --> 00:17:27.190 all of these devices no matter how performant they 00:17:27.190 --> 00:17:31.610 are so it could be all of the gaming consoles all 00:17:31.610 --> 00:17:35.270 of the like tvs setup boxes streaming sticks 00:17:35.270 --> 00:17:38.110 they have the same application running on them 00:17:38.110 --> 00:17:41.650 which has a tiny c++ layer 00:17:41.650 --> 00:17:44.430 and it used to have a javascript and react layer on top 00:17:44.430 --> 00:17:47.730 and basically my work started 00:17:47.730 --> 00:17:54.070 in that c++ layer and then i moved on and started rewriting parts of the javascript 00:17:54.070 --> 00:18:00.270 bit in rust and web assembly so instead of vending things over the air with 00:18:00.270 --> 00:18:05.490 a javascript bundle now we send a wasm bundle and that people download, 00:18:05.730 --> 00:18:09.370 and we have a WebAssembly virtual machine on the device that runs that code, 00:18:09.710 --> 00:18:11.990 mainly for performance reasons. 00:18:13.770 --> 00:18:20.830 That's kind of a lot of the work I do. So recently, we started with the UI low-level 00:18:20.830 --> 00:18:25.210 bits, like, for example, just the scene management, animations, things like that. 00:18:25.490 --> 00:18:31.690 Those became Rust four years, five years ago. I don't remember. Maybe four. 00:18:31.970 --> 00:18:34.030 My timelines are a bit wonky. 00:18:35.670 --> 00:18:41.830 But we started with a more baseline level. And then today we even wrote our 00:18:41.830 --> 00:18:47.070 own UI SDK in Rust to replace the React part. 00:18:48.070 --> 00:18:54.990 And now our UI client engineers, front-end equivalent, are writing Rust as their 00:18:54.990 --> 00:18:57.510 day job for about a year or so. 00:18:58.270 --> 00:19:04.130 So this application has already been deployed on a lot of devices. 00:19:04.130 --> 00:19:08.830 But you can imagine it's quite quite a few given it runs on all of the gaming 00:19:08.830 --> 00:19:16.470 consoles all of the setup boxes you can think of and tvs samsung lg sony everything 00:19:16.470 --> 00:19:20.110 so yeah and i guess it also runs on. 00:19:20.110 --> 00:19:25.330 Some of the consoles like the playstation 4 and 5 and the xbox i guess. 00:19:25.330 --> 00:19:34.890 Yeah gaming consoles ps4 5 um all of the xboxes xbox one xbox one s there's 00:19:34.890 --> 00:19:38.170 a lot of them these days but yeah so a. 00:19:38.170 --> 00:19:41.130 Few people might even have more than one deployment of. 00:19:41.130 --> 00:19:45.630 Your application somewhere i wouldn't be surprised because at least you have 00:19:45.630 --> 00:19:51.390 a tv which if you bought it since 2017 or so it's probably one of these tvs 00:19:51.390 --> 00:19:55.110 that has apps on them so it will have the prime video app because. 00:19:55.110 --> 00:19:57.430 They they sometimes even come pre-installed right. 00:19:57.430 --> 00:20:00.210 Yes yes so they this was 00:20:00.210 --> 00:20:03.830 one of the problems because as i said like the hardware 00:20:03.830 --> 00:20:06.990 in these devices especially on the cheaper 00:20:06.990 --> 00:20:09.990 tvs or the setup boxes 00:20:09.990 --> 00:20:13.890 isn't super powerful even on the streaming sticks they're powered by usb right 00:20:13.890 --> 00:20:18.830 so you can't have crazy good hardware inside because it's just whatever usb 00:20:18.830 --> 00:20:25.350 connection can feed so basically ideally we would write our code into something 00:20:25.350 --> 00:20:29.150 like C or whatever Rust and, 00:20:30.120 --> 00:20:35.180 From the beginning, let's say Rust wasn't on the table, but we would do everything 00:20:35.180 --> 00:20:40.300 in C++ and have the best performance ever because we already had it for the native layer. 00:20:40.480 --> 00:20:43.000 The problem is on these devices, you don't have app stores. 00:20:43.380 --> 00:20:48.420 So sometimes your app goes with the firmware update. Sometimes it's quite a manual process. 00:20:48.640 --> 00:20:53.840 So that's why we move more towards JavaScript and React so we can keep shipping 00:20:53.840 --> 00:20:56.360 bug fixes and updates to customers. 00:20:56.360 --> 00:21:00.120 As a consequence performance wasn't fantastic 00:21:00.120 --> 00:21:05.600 as you can imagine and we had to put a lot of effort into optimizing that and 00:21:05.600 --> 00:21:13.080 even so it was very difficult so rust and web assembly helped a lot on the performance 00:21:13.080 --> 00:21:17.880 side and it's code that you can still ship and download and has. 00:21:19.200 --> 00:21:22.100 Way better performance than a javascript code would 00:21:22.100 --> 00:21:25.560 have because if you imagine javascript today 00:21:25.560 --> 00:21:28.680 you probably imagine it on a laptop or phone or 00:21:28.680 --> 00:21:34.160 whatever where you have jit enabled but on these devices you maybe don't have 00:21:34.160 --> 00:21:38.940 enough memory to even enable a jit virtual machine like a v8 or whatever so 00:21:38.940 --> 00:21:45.760 non-jitted javascript is just so slow you most people don't know the pain so yeah. 00:21:46.380 --> 00:21:50.080 Yeah that's one thing and the other thing is i'm not even sure about garbage 00:21:50.080 --> 00:21:55.340 collection on these devices how efficiently it works because it might be that 00:21:55.340 --> 00:22:01.260 eventually you run out of memory or it's just very laggy it gets slow when you 00:22:01.260 --> 00:22:04.380 use it for a longer time did you ever encounter that. 00:22:04.380 --> 00:22:14.040 Yes so we we trigger so because garbage collection is such a problem in in in javascript non-jitted, 00:22:15.520 --> 00:22:21.560 we disable garbage collection if we have an animation playing and then we call 00:22:21.560 --> 00:22:27.560 GC on the VM manually whenever we know it's safe to do a little bit of stalls 00:22:27.560 --> 00:22:34.040 like your idle or whatever so that was a huge problem as well for us because. 00:22:34.040 --> 00:22:38.800 These devices run for a very long time and sometimes people have them on for 00:22:38.800 --> 00:22:44.420 hours and maybe even days at times you can't really just you know, 00:22:44.760 --> 00:22:47.600 close it and start it back up again. That's not an option. 00:22:48.280 --> 00:22:49.780 Well, we do that if we... 00:22:52.700 --> 00:22:56.180 When we... This was actually... This was a better experience, 00:22:56.340 --> 00:23:00.680 by the way, than we're going to get there probably, but because you have exceptions, 00:23:00.680 --> 00:23:05.420 if something unexpected happens, you can just catch it and reboot the whole app. 00:23:05.740 --> 00:23:13.040 We lost that with transitioning to Rust, so I kind of miss a mega try-catch kind of, 00:23:14.030 --> 00:23:19.210 situation especially with web assembly where you don't have you don't have stuck 00:23:19.210 --> 00:23:23.090 unwinding and you can't quite do the same things. 00:23:23.090 --> 00:23:26.790 In raw rust you could unwind the stack that's true you could, 00:23:27.450 --> 00:23:33.970 technically panic or throw an error and then unwind on that right or yeah if 00:23:33.970 --> 00:23:37.690 you have a panic you can handle it if you wanted to but if you cross the rust 00:23:37.690 --> 00:23:41.330 to web assembly boundary this is where things get really hard. 00:23:41.810 --> 00:23:46.350 Yes, in WebAssembly, it's not possible today. 00:23:46.870 --> 00:23:49.910 So we lost some features, I would say. 00:23:50.290 --> 00:23:55.050 But at least, it's in general very stable. 00:23:55.590 --> 00:23:59.530 But if you do have a panic, it is a very unpleasant experience. 00:23:59.910 --> 00:24:05.810 And actually, one thing I would say, writing panic-free code in Rust is extremely 00:24:05.810 --> 00:24:13.010 difficult because you can't quite know when a panic would happen. 00:24:13.210 --> 00:24:17.030 It could be in one of your dependencies. It could be anywhere. It's not like in... 00:24:17.610 --> 00:24:20.790 You can't do anything about it. So one of the panics came... 00:24:20.790 --> 00:24:27.490 I remember because we got paged because of this edge case somewhere that had 00:24:27.490 --> 00:24:31.010 to do with the way third-party library worked. 00:24:31.650 --> 00:24:36.110 When the third-party library detected that you shouldn't call certain things in a certain way, 00:24:36.390 --> 00:24:41.570 it would panic it was like this is incorrect but then the customer has no app 00:24:41.570 --> 00:24:48.850 anymore man what did you do so then we had to kind of like fix that and patch 00:24:48.850 --> 00:24:51.250 it and return a result because. 00:24:52.670 --> 00:24:55.910 The degrees of correctness people sometimes 00:24:55.910 --> 00:24:59.130 attach to things i feel they're 00:24:59.130 --> 00:25:04.350 a bit too much at times where we could be a bit more lenient to return a result 00:25:04.350 --> 00:25:11.090 but don't take down the whole thing because in some deep deep library somewhere 00:25:11.090 --> 00:25:16.410 you found some inputs that were a bit dodgy so if. 00:25:16.410 --> 00:25:21.570 I started such a project i'd be concerned that i could never update my deployment 00:25:21.570 --> 00:25:26.450 once it's on this box it will stay there forever and sometimes people don't 00:25:26.450 --> 00:25:34.290 do any updates on their devices so it needs to be sort of perfect or at least very close to robust, 00:25:35.770 --> 00:25:41.510 was that a concern going in and probably you already had the problem before 00:25:41.510 --> 00:25:43.790 rust so it wasn't a new problem. 00:25:43.790 --> 00:25:48.610 Yeah we already had the problem i mean the rust code we download as a web assembly 00:25:48.610 --> 00:25:53.470 bundle so that didn't change this doesn't sit on the device forever this is 00:25:53.470 --> 00:25:54.890 It's just every time you start the app, 00:25:54.990 --> 00:25:59.090 we download a new app if one was released. 00:26:00.380 --> 00:26:08.980 On the C++ side, this is why a lot of low-level code that managed our UI scene, 00:26:09.140 --> 00:26:10.840 like doing the animations, 00:26:11.400 --> 00:26:16.060 creating the scene tree with the nodes, the hierarchy of nodes, 00:26:16.060 --> 00:26:18.140 and all of that was in JavaScript initially. 00:26:19.120 --> 00:26:25.160 Normally, browsers or other UI engines do this in C++ for speed. 00:26:25.340 --> 00:26:29.460 But because I had that fear that, man, whatever I write here, 00:26:29.460 --> 00:26:32.920 we write and stays here mainly forever 00:26:32.920 --> 00:26:35.960 is not quite true because if something happens you 00:26:35.960 --> 00:26:39.320 can ask for an update work with the partners but 00:26:39.320 --> 00:26:42.780 it's a very expensive time consuming process 00:26:42.780 --> 00:26:45.880 every update would 00:26:45.880 --> 00:26:49.500 be different they multiply very fast we 00:26:49.500 --> 00:26:52.360 solved that by minimizing the amount 00:26:52.360 --> 00:26:55.840 of C++ code that you put on device and has to 00:26:55.840 --> 00:27:01.020 be there but then we suffered for performance issues due to the fact that we 00:27:01.020 --> 00:27:05.760 moved it to javascript but in the meantime we got lucky and web assembly got 00:27:05.760 --> 00:27:08.480 invented and we could take that 00:27:08.480 --> 00:27:13.160 technology and deploy it and then invent rust code over the air so now. 00:27:13.160 --> 00:27:16.360 A lot of people have some rust experience maybe 00:27:16.360 --> 00:27:19.720 even some production rust experience but they've never 00:27:19.720 --> 00:27:23.240 really written a platform because to me 00:27:23.240 --> 00:27:26.160 it sounds like what you're building here 00:27:26.160 --> 00:27:31.440 is way more foundational than what people touch on a regular basis do you have 00:27:31.440 --> 00:27:36.840 any tips for people who would start with such a large project or would you even 00:27:36.840 --> 00:27:41.000 disagree and say well it's not really a platform because we already still have 00:27:41.000 --> 00:27:45.500 the old c++ layer below but we just replace javascript with web assembly. 00:27:45.500 --> 00:27:47.420 We well, 00:27:48.780 --> 00:27:54.280 It was easy for us because we had the backup of switching to the JavaScript version, right? 00:27:54.380 --> 00:27:58.180 So when we were doing this project, it always starts as an experimental, 00:27:58.180 --> 00:28:00.240 let's see what could happen. 00:28:01.020 --> 00:28:05.720 It's not exactly like that, but it starts a bit as a research project and you 00:28:05.720 --> 00:28:09.920 have a way to go back to the old app. 00:28:10.140 --> 00:28:14.920 I think one thing I would say about, because we replaced React with our own 00:28:14.920 --> 00:28:18.640 UI SDK eventually, and that is a pretty big project. 00:28:18.780 --> 00:28:20.400 I think you're right. 00:28:20.620 --> 00:28:24.780 It is a huge... It's not foundational because we have only one client, 00:28:25.020 --> 00:28:26.380 one app that uses us, right? 00:28:26.460 --> 00:28:30.620 So it's not like we're React that has to service billions of projects. 00:28:31.260 --> 00:28:36.780 But I don't think you can build these UI SDKs without fully, 00:28:37.220 --> 00:28:42.880 deeply understanding the problems in the things you're wanting to replace, right? 00:28:42.880 --> 00:28:47.280 So this was for us an eight-year journey overall since I've been there and probably 00:28:47.280 --> 00:28:50.240 longer because the app existed from before me. 00:28:50.500 --> 00:28:57.620 And the lessons that you learn are very important in the success of any future replacement. 00:28:57.920 --> 00:29:03.020 So I would be very cautious in allowing, like, for example, if some team comes 00:29:03.020 --> 00:29:06.840 to me and they say they want to rewrite something in Rust, the first thing I 00:29:06.840 --> 00:29:10.720 would test them is like, how deeply do you understand the problems with your current system? 00:29:11.240 --> 00:29:17.300 Or are you just a fan like me? And it's nice. I obviously am happy when these 00:29:17.300 --> 00:29:18.920 things happen because I like the language. 00:29:19.280 --> 00:29:24.420 But I think if you want to be successful here, I would say it's really, 00:29:24.540 --> 00:29:28.500 really important to deeply, deeply understand all the pain points with the previous 00:29:28.500 --> 00:29:30.540 thing and advantages, right? 00:29:30.780 --> 00:29:34.480 Because it's not like we rewrote some JavaScript in Rust and it's better. 00:29:34.660 --> 00:29:41.440 Some things are worse. Like I can't panic anymore, which is really difficult to never panic, okay? 00:29:41.720 --> 00:29:44.860 So access an array at the wrong point, that's it, no more app. 00:29:45.220 --> 00:29:51.000 This didn't happen in JavaScript, you know? So it's a trade-off always, I would say. 00:29:53.160 --> 00:29:59.800 It's so true because you need to understand where you're coming from and you 00:29:59.800 --> 00:30:02.380 need to understand your problem domain. 00:30:02.680 --> 00:30:07.420 If you don't have that knowledge, you're missing out on more than 50% of the 00:30:07.420 --> 00:30:12.620 advantages of porting to any new language, such as Rust. 00:30:13.420 --> 00:30:18.040 And a lot of people, they just think they move to greener pastures, 00:30:18.260 --> 00:30:22.020 whereas in reality, they just switch one problem with another. 00:30:23.680 --> 00:30:28.200 It's really true and like we 00:30:28.200 --> 00:30:31.000 didn't start like this right we started with 00:30:31.000 --> 00:30:33.740 just doing the things we knew rust would do 00:30:33.740 --> 00:30:36.500 a good job at because we already knew we did 00:30:36.500 --> 00:30:39.600 a bad job at in javascript which is managing the tree 00:30:39.600 --> 00:30:42.800 of ui elements so we moved that away from c++ 00:30:42.800 --> 00:30:45.780 into javascript for updatability but we 00:30:45.780 --> 00:30:48.520 knew the trade-off there we built it and we're like 00:30:48.520 --> 00:30:51.420 oh god this is so slow but at least we can update 00:30:51.420 --> 00:30:55.100 it so when we made the decision let's keep 00:30:55.100 --> 00:30:58.260 the react app but put the 00:30:58.260 --> 00:31:01.620 management of ui tree rust and web assembly 00:31:01.620 --> 00:31:05.580 in a different vm and these two things communicate through some 00:31:05.580 --> 00:31:11.380 message bus then it was kind of an easy choice starting from a smaller project 00:31:11.380 --> 00:31:17.240 and it's it has a backup you can always fall back to the thing that works and 00:31:17.240 --> 00:31:22.540 it gives the team reading space to kind of research here and deploy it. 00:31:23.370 --> 00:31:26.630 Without even having a billion teams collaborating. 00:31:27.030 --> 00:31:33.710 So this initial thing, when we moved the UI tree management to Rust, 00:31:33.930 --> 00:31:39.430 I think was done by seven people over a seven, eight month period, 00:31:39.670 --> 00:31:40.670 but they didn't interact. 00:31:40.870 --> 00:31:43.630 The other teams didn't even know we switched the thing underneath them. 00:31:43.910 --> 00:31:46.030 So it was quite nice. 00:31:47.370 --> 00:31:51.610 When we rewrote the UI, everyone knew because there's no other, 00:31:51.610 --> 00:31:55.810 there's no way you impact a lot of people but at least you come from a more experienced, 00:31:57.070 --> 00:32:01.190 you already deployed it in production once you know you have quite a bit of 00:32:01.190 --> 00:32:04.830 experts in the team so it's a bit of an easier sell. 00:32:04.830 --> 00:32:10.770 I guess the transition was also easier because you kept the react part and then 00:32:10.770 --> 00:32:17.170 you had this bridge to web assembly and to rust so you didn't have to replace 00:32:17.170 --> 00:32:19.290 the full thing it wasn't a complete 00:32:19.290 --> 00:32:24.570 rewrite from the get-go it was more or less a gradual adoption yes. 00:32:24.570 --> 00:32:31.950 Yeah we we did one thing at the time like that one screen page at the time basically yeah. 00:32:31.950 --> 00:32:39.050 Interesting that you went page by page sort of because the way i thought about 00:32:39.050 --> 00:32:46.670 it was you kind of started with animations or was that not the entry point? 00:32:46.950 --> 00:32:51.810 Yeah, we started with the animations and those things, but the pages were still 00:32:51.810 --> 00:32:54.230 in React. But by the time we started, 00:32:54.910 --> 00:33:00.270 moved pages to Rust. It was one page at a time because we didn't want to have 00:33:00.270 --> 00:33:08.630 on the same screen both JavaScript code changing the screen and Rust code changing the screen. 00:33:08.830 --> 00:33:15.910 So the way they're separated at the lower level in the engine, 00:33:16.310 --> 00:33:17.990 it's two different worlds. 00:33:18.550 --> 00:33:21.630 Like in the ECS world, they're completely 00:33:21.630 --> 00:33:24.430 separate entity component systems for those 00:33:24.430 --> 00:33:27.370 worlds we use entity component systems there 00:33:27.370 --> 00:33:30.450 which is what bevy and those engines use 00:33:30.450 --> 00:33:34.430 in rust so it's a bit like a 2d engine but like two different separate worlds 00:33:34.430 --> 00:33:40.010 for us so we didn't want to deal with one thing if you do that which is very 00:33:40.010 --> 00:33:45.490 problematic is focus management so focusing a thing that may have been a react 00:33:45.490 --> 00:33:49.070 ui element from a rust ui element it's just, 00:33:49.650 --> 00:33:55.230 very brittle to keep track of all of those things so it was simpler for us to 00:33:55.230 --> 00:34:02.350 move a whole page at a time it simplifies things which helps delivering it faster then that. 00:34:02.350 --> 00:34:06.630 Means that if you use the app you might have switched between javascript and 00:34:06.630 --> 00:34:09.850 rust multiple times throughout the interaction. 00:34:10.390 --> 00:34:18.850 Yeah, yeah. So we started with three pages, and then we kind of ordered them 00:34:18.850 --> 00:34:21.010 in order of use and complexity. 00:34:21.750 --> 00:34:25.490 This was one of the things that was called out by people. 00:34:25.750 --> 00:34:31.170 In Amazon, we review a lot of docs with directors, VPs, and so on when we do these big changes. 00:34:31.510 --> 00:34:35.490 And a lot of people are a little bit concerned that it will look jarring to 00:34:35.490 --> 00:34:38.990 go from the support smooth Rust pages to the old JavaScript pages. 00:34:38.990 --> 00:34:43.030 And how do we make sure it's not like shocking. 00:34:43.630 --> 00:34:50.110 But it was fine in the end. I mean, it was unavoidable. 00:34:51.370 --> 00:34:57.830 Did you start with the slowest pages or with the least used pages or what was the strategy? 00:34:58.130 --> 00:35:03.730 Well, we started from the, when you start the app, you see the user profiles. 00:35:04.010 --> 00:35:08.510 So first we replaced that profiles page because of a few reasons. 00:35:08.510 --> 00:35:10.470 It allowed us to put SoundCloud in production. 00:35:10.770 --> 00:35:15.730 It wasn't a complicated page because it's like three buttons with a bit of animations. 00:35:16.790 --> 00:35:21.690 So we didn't require that much UI engine bits to be built. 00:35:22.030 --> 00:35:25.650 One thing to remember is this UI engine didn't exist, right? 00:35:25.870 --> 00:35:31.310 So it was developed at the same time with the app pages. So we picked something 00:35:31.310 --> 00:35:37.610 that didn't have too many widgets because the UI engine team that made this 00:35:37.610 --> 00:35:40.070 kind of React in Rust equivalent, 00:35:42.090 --> 00:35:44.970 built like the concept of a button of focus 00:35:44.970 --> 00:35:47.750 management and that's it and then 00:35:47.750 --> 00:35:50.670 we started with that profiles page and once you selected it 00:35:50.670 --> 00:35:53.470 you went you go to the main app page where 00:35:53.470 --> 00:35:56.670 you have all your movies and we call it the collections page 00:35:56.670 --> 00:36:03.090 so the main app page was the second one that we replaced and then from there 00:36:03.090 --> 00:36:08.090 that was the most complex one and then from there we just replace things on 00:36:08.090 --> 00:36:15.350 the journey to playback basically it was more in order of complexity plus use i would say do. 00:36:15.350 --> 00:36:21.070 You have any metrics um in terms of usage of the app did you see the number 00:36:21.070 --> 00:36:24.430 of panics spike or the latency go down. 00:36:24.430 --> 00:36:30.830 We saw latency go significantly down especially input latency on the worst devices 00:36:30.830 --> 00:36:41.030 went from stuff like 400 something milliseconds on the worst device to 30 milliseconds. 00:36:41.510 --> 00:36:52.450 So it was a huge drop in input latency, mainly because the way React works is 00:36:52.450 --> 00:36:54.630 It's a very complicated way. 00:36:55.110 --> 00:37:00.990 And it does a lot of work to figure things out and to figure out like divs in 00:37:00.990 --> 00:37:04.050 the tree and layout is also very slow. 00:37:04.370 --> 00:37:11.510 So we simplified a lot on that critical path while allowing for certain cute 00:37:11.510 --> 00:37:15.790 features like every property could be animated and things like that. 00:37:16.230 --> 00:37:20.490 So simplifying those allowed us to have quite a huge gain in performance. 00:37:20.490 --> 00:37:26.790 That plus Rust being just a, Rust and WebAssembly just being a faster option 00:37:26.790 --> 00:37:29.830 than equivalent JavaScript code. 00:37:29.950 --> 00:37:33.030 So it was, I would say, two optimizations. 00:37:33.550 --> 00:37:39.110 One in the logic that we simplified, and then you get the base layer performance boost from Rust. 00:37:40.580 --> 00:37:46.420 And I can certainly relate to that because at least to me, when I use any of 00:37:46.420 --> 00:37:53.400 these applications, the input feels like some of the slowest parts of the entire application. 00:37:54.280 --> 00:38:00.220 It's like annoying if you type a letter, for example, and it takes ages for 00:38:00.220 --> 00:38:03.800 it to appear and annoys me to no end. 00:38:03.920 --> 00:38:07.800 So it must have felt like a free upgrade of my hardware somehow. 00:38:07.800 --> 00:38:12.040 No, people did message me that they knew I worked. So it's like, 00:38:12.120 --> 00:38:14.860 oh, you finally updated it on my TV. It's like, yeah. 00:38:15.800 --> 00:38:20.540 So you could definitely tell, especially since this rollout is usually gradual 00:38:20.540 --> 00:38:24.660 because we wanted to also monitor if they impact like business metrics, 00:38:24.660 --> 00:38:28.420 if people use the app more, watch more video. 00:38:28.580 --> 00:38:31.300 And those experiments are a bit more long running. 00:38:32.240 --> 00:38:35.960 So not everyone got it at the same time. did. 00:38:35.960 --> 00:38:42.340 People only notice it based on speed or were there also any ui differences so 00:38:42.340 --> 00:38:47.800 did you tackle updating the ui at the same time or did you try to be one to 00:38:47.800 --> 00:38:49.600 one compatible with the old version. 00:38:49.600 --> 00:38:58.180 We were well neither so both the old version so both javascript and rust so 00:38:58.180 --> 00:39:02.940 the prime video app went through a redesign and the redesign happened on both 00:39:02.940 --> 00:39:07.320 JavaScript and then the Rust bit implemented the new look only. 00:39:08.540 --> 00:39:13.320 So they were both, both of the UIs were getting kind of a redesign and then 00:39:13.320 --> 00:39:16.300 Rust came and just did the new thing. 00:39:16.620 --> 00:39:23.200 We didn't try to do much more than add a few animations I think here, but very few. 00:39:23.480 --> 00:39:28.620 So you have a bit of a, because we wanted to showcase layout animations, 00:39:28.940 --> 00:39:30.140 those are animations that, 00:39:30.920 --> 00:39:33.860 move an element in such a way that it requires you 00:39:33.860 --> 00:39:37.360 to recompute the layout of the other elements in 00:39:37.360 --> 00:39:40.180 the scene just to showcase that there it's really 00:39:40.180 --> 00:39:43.000 fast on the new ui engine in 00:39:43.000 --> 00:39:46.040 rust but that's just i think in one or 00:39:46.040 --> 00:39:48.980 two places used or it was we try 00:39:48.980 --> 00:39:53.760 to keep them as close as possible because then it kind of invalidates all the 00:39:53.760 --> 00:39:58.800 you can't quite attribute oh why did people use the up more on rust maybe it 00:39:58.800 --> 00:40:02.900 was because of the animation or was it because of the it's hard to find the 00:40:02.900 --> 00:40:08.840 cause so we try to be as one-to-one with the javascript version as possible right. 00:40:09.440 --> 00:40:14.840 The new version had better performance and i'm assuming it also used less memory 00:40:14.840 --> 00:40:21.080 did that mean it allowed you to do more with the devices eventually because 00:40:21.080 --> 00:40:25.000 now you have more headspace Now you could experiment more. 00:40:25.420 --> 00:40:34.060 On the memory side, we use more, actually, because we have still the JavaScript version. 00:40:34.690 --> 00:40:37.750 In the same like we didn't 00:40:37.750 --> 00:40:40.590 delete javascript code if we have it in Rust because 00:40:40.590 --> 00:40:43.770 we kept it there if we needed to switch to a 00:40:43.770 --> 00:40:47.050 fallback javascript full javascript 00:40:47.050 --> 00:40:50.230 experience so overall with memory 00:40:50.230 --> 00:40:53.950 that became quite a bit of um quite a 00:40:53.950 --> 00:40:56.710 bit higher i'd say because there is now it's like two 00:40:56.710 --> 00:41:00.050 apps there obviously as we port things 00:41:00.050 --> 00:41:03.130 over the js part will be deleted 00:41:03.130 --> 00:41:06.330 or minimized for things like 00:41:06.330 --> 00:41:10.250 just having logs there or whatever it's another 00:41:10.250 --> 00:41:13.950 kind of free thread that you can shove things 00:41:13.950 --> 00:41:19.770 to with a message bus and then they can do some work for example right now we 00:41:19.770 --> 00:41:24.630 send metrics and logs from javascript to our backends so from rust app we collect 00:41:24.630 --> 00:41:28.710 them we send them to javascript but JavaScript massages them in the right JSON 00:41:28.710 --> 00:41:32.070 format and then reports things from time to time. 00:41:32.650 --> 00:41:37.290 So that bit probably is not as critical to delete, but yeah. 00:41:37.990 --> 00:41:43.230 So running JavaScript and WebAssembly VMs in separate threads is interesting. 00:41:43.510 --> 00:41:48.330 And how do you handle synchronization and stake consistency between those two worlds? 00:41:48.490 --> 00:41:53.410 You mentioned messagebusses maybe one or multiple, I don't know. 00:41:53.410 --> 00:41:57.930 Is that the way you generally handle memory synchronization, 00:41:58.150 --> 00:41:59.430 or are there some other patterns? 00:41:59.910 --> 00:42:02.930 We don't quite fully share memory. 00:42:03.050 --> 00:42:09.550 Well, you can't fully share memory because the way of WebAssembly memory model is. 00:42:10.010 --> 00:42:17.170 So we just send messages between the two virtual machines, and through messages, 00:42:17.170 --> 00:42:18.850 they synchronize their state. 00:42:19.310 --> 00:42:23.290 So, for example, let's say you want to synchronize the history. 00:42:24.310 --> 00:42:27.290 Stack like if you press back where 00:42:27.290 --> 00:42:30.570 where would that go you just send a message that hey 00:42:30.570 --> 00:42:33.910 javascript updated and so on and then it's just 00:42:33.910 --> 00:42:36.730 message passing that's that's all it is 00:42:36.730 --> 00:42:42.910 and similarly javascript can tell the Rust vm it's usually one of them is the 00:42:42.910 --> 00:42:47.290 source and the other one is just synchronizing with the source so there's only 00:42:47.290 --> 00:42:54.270 one source of truth that propagates its state to the virtual machine that just follows i. 00:42:54.270 --> 00:42:58.610 Guess one of the big advantages of using a message bus here and please correct 00:42:58.610 --> 00:43:04.610 me if i'm wrong is that you avoid interrupting either world so for example if 00:43:04.610 --> 00:43:11.210 the wasm thread does any work it it does not need to react immediately to a message coming in right. 00:43:11.210 --> 00:43:17.210 No yeah and it's also way easier to implement than anything else basically so 00:43:17.210 --> 00:43:22.570 you can always poll a message whenever you want, whenever you have time to deal with them. 00:43:22.970 --> 00:43:28.490 Exactly. So you don't, they're not, obviously if it takes a long time to process 00:43:28.490 --> 00:43:30.290 a message, you've blocked the UI there. 00:43:30.490 --> 00:43:35.910 So it still has those problems, but it gives you a bit of wiggle room on how you, yeah. 00:43:36.130 --> 00:43:41.170 And while you did that transition, did you fix any bugs? 00:43:41.330 --> 00:43:44.410 Did you find any logic bugs in JavaScript? 00:43:45.010 --> 00:43:49.530 Any incorrect behavior that you were able to clean up then? Or even just technical 00:43:49.530 --> 00:43:52.050 depth as well to reduce the maintenance burden? 00:43:52.990 --> 00:43:58.030 No, but I can tell you one thing I did. When I was porting things from JavaScript 00:43:58.030 --> 00:44:01.670 to Rust, I found a place in the UI that was a bit dodgy. 00:44:01.870 --> 00:44:07.170 And I did it the right way. I think it was to how some transparency was being 00:44:07.170 --> 00:44:08.590 calculated, or I don't remember. 00:44:09.410 --> 00:44:15.290 And I broke the app. So basically, then I became very careful into telling people, 00:44:15.510 --> 00:44:19.790 please maintain whatever JavaScript was doing. That's the ground truth. 00:44:20.030 --> 00:44:22.910 We're not thinking here, we're porting it over. 00:44:23.310 --> 00:44:28.350 Let's not try to... It's really easy to fall into the trap of, 00:44:28.510 --> 00:44:31.710 oh, let me do it the right way, but then you don't realize that other things 00:44:31.710 --> 00:44:34.330 on top of you depend on it being wrong. 00:44:34.650 --> 00:44:40.830 It's that whole classic meme of whatever the API is doing is what the API should do. 00:44:40.830 --> 00:44:43.690 And yeah we got bit by 00:44:43.690 --> 00:44:46.510 that a couple of times me specifically because 00:44:46.510 --> 00:44:49.230 i was feeling so smug about it's like ah look at this 00:44:49.230 --> 00:44:52.770 silly thing i fixed it and it's like hey Alex 00:44:52.770 --> 00:45:00.210 did you notice this thing doesn't work at all no no so yeah it was with i would 00:45:00.210 --> 00:45:06.610 say because i rewrote a couple of these systems one of my lessons learned is 00:45:06.610 --> 00:45:10.590 to just move it with whatever behavior it has. 00:45:11.200 --> 00:45:14.260 Try not to be smart about that and then 00:45:14.260 --> 00:45:17.080 you can you can iterate on things 00:45:17.080 --> 00:45:20.240 later but it already these projects 00:45:20.240 --> 00:45:23.200 of rewriting things i feel they take longer 00:45:23.200 --> 00:45:28.300 than people expect especially since they're usually quite optimistic like oh 00:45:28.300 --> 00:45:33.480 what's what's so complicated about a ui tree you know it's just a tree has some 00:45:33.480 --> 00:45:37.940 properties you update it you traverse the tree you update the children and can 00:45:37.940 --> 00:45:41.520 do it in a week if you're very naive about it. 00:45:41.640 --> 00:45:46.680 But then it turns out it takes a year and then every little bug in the thing 00:45:46.680 --> 00:45:52.300 needs to be translated because there's stuff that depends on those bugs that 00:45:52.300 --> 00:45:53.800 people put workarounds for. 00:45:54.020 --> 00:45:57.900 And then they work differently if you change your code. 00:45:58.060 --> 00:46:04.640 So I'll just keep things as they are when I translate them to a new language 00:46:04.640 --> 00:46:06.800 and then iterate on them, I would say. 00:46:07.080 --> 00:46:11.120 I would advise against trying to fix anything. 00:46:12.240 --> 00:46:17.420 Yeah, I fully agree with that. Because otherwise you would tackle two problems at once. 00:46:18.600 --> 00:46:22.220 So I know a lot of companies who did the same. 00:46:22.440 --> 00:46:27.660 They started to put over the old logic, including the bugs and everything, 00:46:27.880 --> 00:46:29.220 and then started to clean up. 00:46:30.260 --> 00:46:33.940 That's a way more productive way to work in that area. 00:46:34.820 --> 00:46:39.300 Especially for people that are not know like especially for people that are 00:46:39.300 --> 00:46:45.380 not comfortable or not experienced with porting code it's a skill that you need 00:46:45.380 --> 00:46:49.460 to learn it's it's a different skill than writing code from scratch because 00:46:49.460 --> 00:46:54.680 you're taking code that already existed and is running in production yeah. 00:46:54.680 --> 00:47:00.340 And what's correct in this scenario is the behavior that is already there They're 00:47:00.340 --> 00:47:02.340 not what you think should happen. 00:47:02.700 --> 00:47:08.640 So what correct means changes if you're replacing a system, I'd say. 00:47:09.700 --> 00:47:14.920 And speaking of old behavior, you must still be able to handle old devices as 00:47:14.920 --> 00:47:17.860 well that don't support WebAssembly. 00:47:18.020 --> 00:47:23.340 Do you have two code bases or two separate code paths or some sort of fallback? 00:47:23.340 --> 00:47:27.280 We still have the javascript pages 00:47:27.280 --> 00:47:31.120 that are maintained in parallel i 00:47:31.120 --> 00:47:38.260 would say they're not getting all of the new features like all of the new animations 00:47:38.260 --> 00:47:43.960 and whatnot but because there's significant even at low percentages worldwide 00:47:43.960 --> 00:47:48.060 those are a lot of customers that are still, 00:47:48.800 --> 00:47:52.940 due to, well, historical reasons, they're getting only the JavaScript pages 00:47:52.940 --> 00:47:55.420 because we didn't put Rust on the C++ layer yet. 00:47:56.120 --> 00:48:00.000 So we still have to maintain those. And we're going through efforts of, 00:48:00.970 --> 00:48:07.550 kind of backporting this native WebAssembly VM so we can allow them to run WebAssembly. 00:48:08.090 --> 00:48:11.370 But for now, they're both maintained. 00:48:12.850 --> 00:48:17.270 Okay, so that means we are in 2025 now. 00:48:17.410 --> 00:48:21.790 How big is the code base nowadays and what's the team size? 00:48:22.150 --> 00:48:29.170 So team size is around 100 or more contributors to the whole project. 00:48:29.170 --> 00:48:37.270 We have a lot of people from backend teams that just want to surface some messaging 00:48:37.270 --> 00:48:41.230 or something like that, and they have to write a bit of UI code. 00:48:41.510 --> 00:48:46.990 So that's why the number is so high. And maybe on the code base size, 00:48:47.350 --> 00:48:53.310 below 100,000 lines of code and more than 50, I would say. 00:48:54.310 --> 00:49:00.450 I didn't check it recently. Also, it's a little bit hard to check because we 00:49:00.450 --> 00:49:06.510 rely heavily on macros because we replaced React and we wanted to give people 00:49:06.510 --> 00:49:11.710 this kind of JSX-like experience of compositing UI. 00:49:12.130 --> 00:49:15.170 So that expands quite a lot. 00:49:15.530 --> 00:49:22.010 So maybe you write five lines of a macro where you compose some UI elements, 00:49:22.030 --> 00:49:27.150 but then the code generated by that is quite significant. And so it's hard to tell, yeah. 00:49:28.190 --> 00:49:32.990 How would that feel like if I were to write a component in that system? 00:49:33.510 --> 00:49:38.890 Would it be easy for me even if I wasn't a Rust developer? Or would it be very specific? 00:49:39.170 --> 00:49:43.370 You mentioned the macro and it being a bit JSX-like. 00:49:43.850 --> 00:49:47.750 Would I be able to understand as a JavaScript developer? Or would I need to 00:49:47.750 --> 00:49:49.530 learn Rust for real first? 00:49:50.030 --> 00:49:58.590 So I think on the UI programming side, So if you're a front-end type of engineer, 00:49:59.090 --> 00:50:05.810 you don't need to learn that much Rust to be able to write code. 00:50:06.050 --> 00:50:11.370 The reason for this is that we are very relaxed with copying data in the UI 00:50:11.370 --> 00:50:15.130 side, and we wrap stuff in RC pointers and things like that. 00:50:15.170 --> 00:50:21.710 So they don't quite hit ownership issues that you are usually having to deal 00:50:21.710 --> 00:50:26.910 with when you write new Rust code. It's not super optimal, but it doesn't matter 00:50:26.910 --> 00:50:29.270 compared to JS is way faster anyway. 00:50:29.730 --> 00:50:37.950 So we used to use Leptos for this framework for effects and signals. 00:50:37.950 --> 00:50:45.750 And this is very familiar to people who know React or SolidJS and those type of things. 00:50:46.070 --> 00:50:48.930 So in that sense, it's a bit easier. 00:50:49.950 --> 00:50:56.690 I think in general, when you go to Rust from a higher level language, it's easier because, 00:50:57.490 --> 00:51:01.890 I don't know what's happening in that browser underneath me when I write React code. 00:51:02.050 --> 00:51:06.230 This code sometimes is being called once every time I click a button. 00:51:06.390 --> 00:51:10.590 This other code is being... Because it's not like you read the code and that's how it's executed. 00:51:10.950 --> 00:51:16.010 It's always they go into different places. So I think in our framework, 00:51:16.010 --> 00:51:18.470 if you read the code, this is how things are happening. 00:51:18.650 --> 00:51:22.250 We just expand it so you don't have to write a lot of things by hand. 00:51:22.250 --> 00:51:27.810 But there's no magic, except maybe in the effects and signals, 00:51:27.810 --> 00:51:33.930 which are still a little bit like magic because they're functions that are called 00:51:33.930 --> 00:51:36.030 when you set a value on another thing. 00:51:36.310 --> 00:51:39.370 How does it know to call the function? It's just a little bit, 00:51:39.490 --> 00:51:44.470 it's not quite intuitive, but it is intuitive to people who know React because 00:51:44.470 --> 00:51:48.970 they got used to like effects and this type of behavior. 00:51:48.970 --> 00:51:53.070 For me, it was just so weird to have this kind of, oh, whatever, 00:51:53.370 --> 00:51:56.970 what's happening here? And it turns out what's happening is very ugly. 00:51:58.410 --> 00:52:04.890 It's horribly ugly, but, you know, it's the way reactive UI works, right? 00:52:05.130 --> 00:52:11.870 Did you have to port that stuff over to Rust or did you find a different way to solve that? 00:52:12.010 --> 00:52:16.770 So that is already done by Leptos and there's a few reactive frameworks that 00:52:16.770 --> 00:52:24.950 we relied on. we eventually moved a bit away from it because we wanted to optimize certain things. 00:52:25.170 --> 00:52:29.770 And I think the open source projects is going a bit in a different direction. 00:52:30.530 --> 00:52:34.910 So for simplicity, we had to change some stuff there. 00:52:35.030 --> 00:52:37.950 But basically, there are libraries who did this. 00:52:38.030 --> 00:52:45.230 And I think those type of libraries pushed what's possible with Rust UI in general 00:52:45.230 --> 00:52:50.970 and making it more pleasant to use for people who are front-end engineers. 00:52:51.670 --> 00:52:56.990 Do you use a fork of Laptos now that you maintain yourself? And which libraries did you mean? 00:52:57.830 --> 00:53:01.110 So there's a few reactive ones. I don't remember the name. 00:53:01.290 --> 00:53:09.250 We used to use Leptos and then forked it and then wrote our own thing on top instead. 00:53:10.590 --> 00:53:15.730 And the libraries that you mentioned, the ones that allow you to do the signal 00:53:15.730 --> 00:53:17.090 or event handling in Rust? 00:53:17.310 --> 00:53:22.510 No, this is the signal. So Leptos is basically signals, effects, 00:53:22.970 --> 00:53:29.630 things like that. they do provide also a way to declaratively do your ui but 00:53:29.630 --> 00:53:31.890 we didn't use that we only use the signals bit. 00:53:31.890 --> 00:53:35.150 Very early on in our conversation we 00:53:35.150 --> 00:53:38.510 talked about game development and i 00:53:38.510 --> 00:53:41.610 wonder how much of that design was inspired 00:53:41.610 --> 00:53:44.710 by game development or how can 00:53:44.710 --> 00:53:47.870 you compare those two worlds i understand that 00:53:47.870 --> 00:53:53.110 this is not a game but it feels like you have things that are similar to games 00:53:53.110 --> 00:54:01.170 like reacting to events some sort of ui some sort of event loop and maybe can 00:54:01.170 --> 00:54:06.110 you compare those two worlds and how did you model that i'd. 00:54:06.110 --> 00:54:14.830 Say basically it's a lot like a game engine under the hood so right now bevy 00:54:14.830 --> 00:54:22.850 popularized this but we did use we also use entity component systems for all our low-level bits. 00:54:23.090 --> 00:54:30.570 So while we kept front-end engineers happy with leptos-like things with effects 00:54:30.570 --> 00:54:34.290 and signals and this reactive way of writing UI, 00:54:34.570 --> 00:54:38.110 under the hood, it's basically a game engine, more or less. 00:54:38.310 --> 00:54:42.090 So it has entity component systems, it has systems. 00:54:42.390 --> 00:54:48.270 Every UI element becomes an entity, and we attach components to that entity 00:54:48.270 --> 00:54:50.570 based on what behavior it needs to have. 00:54:50.710 --> 00:54:56.150 And then we have systems like the focus system that looks at every focusable 00:54:56.150 --> 00:55:01.350 entity and takes the focusable component and does stuff based on your input. 00:55:01.590 --> 00:55:04.070 So it's very, very close to game engine. 00:55:05.090 --> 00:55:13.030 One thing we maybe didn't add and are added later, which I think makes it a 00:55:13.030 --> 00:55:17.810 bit different than a game engine, is async stuff. 00:55:18.230 --> 00:55:25.730 We didn't allow async at all, basically, because you had no runtime. 00:55:26.370 --> 00:55:30.010 So you can't do anything with a future, right? 00:55:30.290 --> 00:55:35.730 So I wish we had added stuff earlier. and that's not a concept you usually find 00:55:35.730 --> 00:55:42.390 in video games, at least not the way you write it in Rust today with async whatever 00:55:42.390 --> 00:55:44.330 and tokio and all of these things. 00:55:44.510 --> 00:55:49.890 It's more like something you find on the back end because async programing is 00:55:49.890 --> 00:55:55.370 like waiting on multiple things, whereas what a game does is processing multiple 00:55:55.370 --> 00:55:56.690 things. So it's more like thread. 00:55:57.890 --> 00:56:02.030 You create some threads and you do work on those threads instead of waiting. 00:56:02.030 --> 00:56:03.310 So that. 00:56:03.310 --> 00:56:05.910 Was a concept that was added a bit later. 00:56:05.910 --> 00:56:09.190 So my understanding is that with 00:56:09.190 --> 00:56:12.010 entity component systems what you want to do is 00:56:12.010 --> 00:56:15.410 to work in parallel on the same entity just looking 00:56:15.410 --> 00:56:18.250 at it from different angles a certain 00:56:18.250 --> 00:56:22.050 facet of say elements and 00:56:22.050 --> 00:56:25.270 you only deal with these facets but you do deal 00:56:25.270 --> 00:56:28.410 with them in parallel whereas in a 00:56:28.410 --> 00:56:31.510 classical let's say io bound 00:56:31.510 --> 00:56:34.950 system you deal with things concurrently 00:56:34.950 --> 00:56:40.230 as they come in and so is that where those two worlds collide where you have 00:56:40.230 --> 00:56:46.150 events that are handled asynchronously or concurrently whereas you kind of want 00:56:46.150 --> 00:56:50.650 to update the ui in parallel or you want to update all of these properties in 00:56:50.650 --> 00:56:52.270 parallel with your entities? 00:56:52.990 --> 00:57:01.050 So the place where they kind of interact is when you have UI elements that are 00:57:01.050 --> 00:57:02.450 waiting on the network call. 00:57:03.230 --> 00:57:10.810 This is not very common in video games, but it's very common in UI applications 00:57:10.810 --> 00:57:14.470 that most of their data comes from some backend. 00:57:15.550 --> 00:57:20.310 It is a little bit on games because you get data from disk that you process 00:57:20.310 --> 00:57:23.950 and then it gets to show up on your screen. 00:57:24.890 --> 00:57:32.090 But async programming is a bit foreign to game developers as understood by this 00:57:32.090 --> 00:57:37.510 kind of idea of a promise or a future that you pull on and it's not a thing 00:57:37.510 --> 00:57:40.770 that happens often in games, I think. 00:57:41.390 --> 00:57:46.990 So because our UI engine is very driven with responses from the backend, 00:57:46.990 --> 00:57:51.170 And I think we could have added async stuff a bit earlier. 00:57:52.370 --> 00:57:57.070 I personally, so I was the tech lead on this project. And I think it was a bit 00:57:57.070 --> 00:58:01.670 my fault because I didn't quite fully understand async because it's just, 00:58:02.720 --> 00:58:09.020 I think async code is very useful, but it's also very hard to read code and 00:58:09.020 --> 00:58:12.060 have a mental model of how it gets executed. 00:58:12.400 --> 00:58:17.340 And for this reason, I decided to, okay, let's keep it a bit simple, 00:58:17.480 --> 00:58:21.060 not have any async, and then give you some callbacks. 00:58:21.420 --> 00:58:27.100 But then it turns out actually some things would be simpler with async if you don't overdo it. 00:58:27.280 --> 00:58:33.600 So we added it eventually recently. So now you can declare a UI component that 00:58:33.600 --> 00:58:35.360 shows a spinner or something, 00:58:35.440 --> 00:58:41.540 and it will automatically be replaced by something else when we download some 00:58:41.540 --> 00:58:44.000 data. And that's done with async now. 00:58:44.440 --> 00:58:48.360 We call it deferred components, I think they're called in the wild. 00:58:48.620 --> 00:58:54.020 But for those scenarios, I think that's where it differs from a game engine. 00:58:54.020 --> 00:58:58.940 But other than that all of the graphics all of the traversal of things all of 00:58:58.940 --> 00:59:05.260 the entity component systems that we have are very similar to a game engine yeah. 00:59:05.260 --> 00:59:08.300 I could also see two approaches 00:59:08.300 --> 00:59:11.200 to that problem one would be the simple way 00:59:11.200 --> 00:59:14.340 to say you have a proxy around your component 00:59:14.340 --> 00:59:17.680 and the component itself is completely sync whereas 00:59:17.680 --> 00:59:22.860 the data proxy that is around it kind of is async and then passes in the data 00:59:22.860 --> 00:59:27.420 that it receives right that is one way and the other way would be to say let's 00:59:27.420 --> 00:59:33.140 completely embrace async and do everything async even in the components and 00:59:33.140 --> 00:59:35.960 it feels like you went from one to the other at some point or. 00:59:35.960 --> 00:59:38.940 You changed your mind on this we didn't so 00:59:38.940 --> 00:59:42.040 we went with the first one that you described so basically 00:59:42.040 --> 00:59:45.060 you kind of wrap it into something and then 00:59:45.060 --> 00:59:48.680 because it's a async block 00:59:48.680 --> 00:59:51.540 you can write your code and await on 00:59:51.540 --> 00:59:54.520 network calls a bit nicer create the 00:59:54.520 --> 00:59:58.000 ui component but then the ui itself will be just 00:59:58.000 --> 01:00:03.800 it replaces something in the tree and that's just normal code once it ends executing 01:00:03.800 --> 01:00:09.880 that future this is the difficulty with async code is very kind of viral it 01:00:09.880 --> 01:00:17.300 just spreads everywhere and then you need to have this sync points with non-async code or, 01:00:19.530 --> 01:00:23.170 You either go all in and it's async everywhere, but we didn't do that. 01:00:23.310 --> 01:00:27.250 We just allow a sort of a wrapper. It goes into a task that executes it. 01:00:27.590 --> 01:00:31.070 And at the end of the day, the UI code you're going to get is going to be normal 01:00:31.070 --> 01:00:35.230 code. It goes out of that future at some point. 01:00:36.510 --> 01:00:42.070 Usually once we're done with the processing of a network call or some message 01:00:42.070 --> 01:00:45.670 pass that waits on JavaScript or something. yeah. 01:00:45.670 --> 01:00:51.090 In this case it sounds like it was easier to keep the lower level components 01:00:51.090 --> 01:00:56.170 the leaf nodes sync and have an async runtime around it. 01:00:56.170 --> 01:01:04.970 Yeah yeah because ultimately in ui a lot of patterns are like set timeout and 01:01:04.970 --> 01:01:12.130 those type of tasks can easily be async tasks as well that you then run from time to time. 01:01:12.350 --> 01:01:17.410 And that's another place where we... You just, as a UI engineer, 01:01:17.410 --> 01:01:22.150 you have this way of creating a function and shoving it in our executor. 01:01:22.310 --> 01:01:26.690 And then whatever comes out of it might be attached to the UI tree later. 01:01:26.690 --> 01:01:33.450 But it's kind of like an extra feature that you get rather than all the UI code is async everywhere. 01:01:34.410 --> 01:01:41.630 Do you port a lot of promises from JavaScript to Rust or was it not the case? 01:01:42.830 --> 01:01:44.630 There were some around, 01:01:47.010 --> 01:01:54.570 things that deal with network usually are like that with fetch and then do something 01:01:54.570 --> 01:01:56.050 and then you wait on those. 01:01:56.550 --> 01:02:00.630 We have just a way right 01:02:00.630 --> 01:02:05.810 now that you can say you write an async function you do your way however you 01:02:05.810 --> 01:02:11.710 want and then you take that future and send it to us with our api which is a 01:02:11.710 --> 01:02:20.390 task and this is the task and then we pull it whenever we have time usually per frame but yeah yeah. 01:02:20.390 --> 01:02:26.170 That means you don't usually use the vanilla futures you have async functions 01:02:26.170 --> 01:02:29.390 for the most part and then you, 01:02:30.050 --> 01:02:34.510 pull them from the outside but you don't build up your own future as you would 01:02:34.510 --> 01:02:37.470 with let's say a vanilla JavaScript promise. 01:02:37.730 --> 01:02:41.690 No, no, no. Yeah, it's an async function that returns the, yeah. 01:02:42.550 --> 01:02:50.430 And will you be able to use some of the newer features of CPUs nowadays with SIMD, for example? 01:02:51.350 --> 01:02:58.330 So we do work a lot with the WebAssembly. So we contribute to the WebAssembly VM. This is on GitHub. 01:02:58.770 --> 01:03:00.590 It's a WebAssembly micro runtime. 01:03:01.210 --> 01:03:07.430 So we were quite involved there with the community and also in the WebAssembly 01:03:07.430 --> 01:03:09.770 community working group. 01:03:09.770 --> 01:03:12.790 So we worked there on 01:03:12.790 --> 01:03:16.890 adding threads to web assembly simd is 01:03:16.890 --> 01:03:19.890 another thing that we are quite keen on that was 01:03:19.890 --> 01:03:23.710 already done so with these 01:03:23.710 --> 01:03:26.750 things it's a little we're suffering from our 01:03:26.750 --> 01:03:31.330 own eagerness of using things because you want 01:03:31.330 --> 01:03:35.350 to use the latest things and you're contributing to them but then it kind of 01:03:35.350 --> 01:03:42.090 creates a build time decision on our side because the WebAssembly bytecode is 01:03:42.090 --> 01:03:46.970 not backwards compatible or guaranteed to be backwards compatible, let's say. 01:03:47.210 --> 01:03:52.250 Well, maybe with... No, with SIMD it's not, so you need the feature enabled. 01:03:52.570 --> 01:03:57.670 With threads, you're missing some functions, so you could make it work, but it's not great. 01:03:58.390 --> 01:04:03.530 So we're a little bit careful of how many of those builds we issue from our 01:04:03.530 --> 01:04:06.590 pipeline because we don't want to overly complicate our pipeline. 01:04:07.190 --> 01:04:11.830 So it's a bit, it's a tricky thing. 01:04:12.310 --> 01:04:19.170 We are, it didn't help that much because we're not doing a lot of things that 01:04:19.170 --> 01:04:22.370 were CMD, for example, could help us with. 01:04:23.480 --> 01:04:26.960 Maybe if we re-architect some of our lower-level systems. 01:04:27.560 --> 01:04:33.720 But we are definitely looking into it and contributing. So yeah, 01:04:33.980 --> 01:04:35.420 we're interested in those. 01:04:36.380 --> 01:04:39.540 It's definitely a hot new space. 01:04:40.140 --> 01:04:46.640 And there's a lot of things going on. I also know that there is a concept around 01:04:46.640 --> 01:04:48.520 garbage collection in WebAssembly now. 01:04:48.660 --> 01:04:53.340 Or at least there's a work-in-progress sort of draft for this. 01:04:53.480 --> 01:04:58.840 That's kind of crazy as well and i'm not sure if there's any work being done 01:04:58.840 --> 01:05:03.660 around panic handling in web assembly which would also be kind of interesting right. 01:05:03.660 --> 01:05:09.140 As far as i know there was some work done on stack unwinding which is the missing 01:05:09.140 --> 01:05:11.140 bit that we have on around panics, 01:05:11.820 --> 01:05:16.760 there's kind of these two web assembly is funny because there's these two or 01:05:16.760 --> 01:05:20.920 more factions that work on it and they have completely different use cases. 01:05:21.180 --> 01:05:25.640 So the browser vendors, they care about completely different things than people 01:05:25.640 --> 01:05:27.580 in the IoT space care about. 01:05:27.900 --> 01:05:35.500 So no one who works on IoT-like devices that I know is excited about garbage 01:05:35.500 --> 01:05:40.020 collection because it's just extra stuff, right? 01:05:40.200 --> 01:05:47.280 It's useful as a concept and it's useful when you talk about maybe communicating with But it's just, 01:05:47.440 --> 01:05:54.620 it is a project that I think was useful for so many use cases that it now suffers 01:05:54.620 --> 01:05:58.120 a little bit because of it, because it's pulled into all of these different directions. 01:05:58.120 --> 01:06:03.360 So there's also compute-at-edge type of things that use WebAssembly, 01:06:03.400 --> 01:06:08.540 and those situations have completely different hardware and feature requirements 01:06:08.540 --> 01:06:15.960 than someone using it on an IoT device, for example, or even us on TVs. 01:06:15.960 --> 01:06:20.440 We're kind of in the middle because they're not IoT devices, but more powerful. 01:06:21.140 --> 01:06:27.580 So simpler is, in my view, better. And then there's the different use with the web browser. 01:06:27.780 --> 01:06:34.400 So right now WebAssembly is this hot kind of hotbed for solutions in all of 01:06:34.400 --> 01:06:37.980 these different fields. I'm quite curious how it will evolve. 01:06:38.780 --> 01:06:44.120 But yeah, it is definitely evolving quite fast, I would say, 01:06:44.480 --> 01:06:47.160 compared to other standards or things. 01:06:48.080 --> 01:06:53.300 All right. Finally, you're a principal engineer. And you have a lot of experience 01:06:53.300 --> 01:07:00.500 now with working with these worlds, JavaScript, Rust, C++, WebAssembly. 01:07:01.440 --> 01:07:09.480 My question is, how do you train a team to be productive in Rust without losing any momentum? 01:07:09.960 --> 01:07:15.180 What are your learnings? What would you do differently now if you started over? 01:07:15.660 --> 01:07:18.980 How do you get a team from zero to production with Rust? 01:07:18.980 --> 01:07:24.140 This is really difficult. One thing I would say, and I found it successful, 01:07:24.540 --> 01:07:32.220 is starting with a small project that's quite self-contained was critical for 01:07:32.220 --> 01:07:33.740 us. And that's for two things. 01:07:34.360 --> 01:07:41.000 First thing, you get used to the language. And you get used to even things that 01:07:41.000 --> 01:07:44.420 people don't quite think about when it comes to rewriting things in Rust. 01:07:44.680 --> 01:07:49.420 At Amazon, we have fairly good, let's say, ecosystem inside of the company. 01:07:49.760 --> 01:07:53.920 But if you're at a company that doesn't have that ecosystem to rely on, 01:07:54.060 --> 01:08:00.260 to deploy some RAS code, even deploying something small and keeping itself contained 01:08:00.260 --> 01:08:04.560 is going to pave the way for other people to be more productive in the future. 01:08:04.820 --> 01:08:11.440 And that's why I think starting for a small project is what I would definitely advocate. 01:08:11.960 --> 01:08:18.560 Another thing that I found successful is finding a place where latency or speed 01:08:18.560 --> 01:08:24.100 matters to whoever your customer is in there. 01:08:24.720 --> 01:08:28.500 So for us, for example, having smooth animations mattered. 01:08:28.760 --> 01:08:32.780 And that's how the first project started for us. Just do the low-level tree 01:08:32.780 --> 01:08:38.260 management, move these UI elements fast, and then people were on board with 01:08:38.260 --> 01:08:42.980 it because we thought, yes, of course, smooth animations, What do we have to lose? 01:08:43.400 --> 01:08:47.020 And that some companies maybe have different maths. 01:08:47.400 --> 01:08:49.120 They do. But for example, if 01:08:49.120 --> 01:08:53.860 you think about at our scale, a team of seven engineers for a year is not. 01:08:54.600 --> 01:08:58.540 Crazy investment maybe it's 01:08:58.540 --> 01:09:01.620 going to be one or two people for other companies for six 01:09:01.620 --> 01:09:04.480 months but it will pay off right if you 01:09:04.480 --> 01:09:08.020 give people time and if you have a problem that could 01:09:08.020 --> 01:09:14.620 be solved by rust i would say usually around stuff like processing lots of data 01:09:14.620 --> 01:09:20.640 or doing something around that and that's kind of an easy way to get your foot 01:09:20.640 --> 01:09:26.120 through the door how do you use it at scale is then a little bit more tricky. 01:09:26.300 --> 01:09:30.260 And I think here, for example, we could have done better. 01:09:30.460 --> 01:09:37.240 So once we did the port of the UI engine, even then, the team didn't start with 100 people. 01:09:37.420 --> 01:09:41.280 It started with maybe 15, and we grew and grew and grew. 01:09:41.560 --> 01:09:47.520 The problem is, after the first two pages, we were so successful that we reminded it, okay, now, 01:09:47.980 --> 01:09:51.920 everyone, you're using Rust because look at this, it's cooler, 01:09:52.100 --> 01:09:57.880 it's faster, it's smoother, is people use it more at home, use the app more, 01:09:58.060 --> 01:10:00.080 they watch more stuff, they don't abandon sessions. 01:10:00.660 --> 01:10:03.460 You know, as we saw boosts everywhere. 01:10:03.980 --> 01:10:09.940 But the problem is then you have so much more new people than people who can 01:10:09.940 --> 01:10:14.280 help them in cold reviews and then it becomes a little bit more difficult. 01:10:14.280 --> 01:10:18.860 So I think they're having a more gradual adoption curve would have been nicer. 01:10:19.120 --> 01:10:23.260 And that's the thing you asked me what I would do better. I would advocate for 01:10:23.260 --> 01:10:28.100 us to put the brakes a little bit on moving absolutely everything all at once. 01:10:29.680 --> 01:10:33.760 Just iteratively delivering, because it's a little bit like throughout... 01:10:34.280 --> 01:10:38.480 What I would say is this kind of things where you propose something, 01:10:38.640 --> 01:10:42.500 like let's move this thing to Rust or let's build this new thing in Rust and 01:10:42.500 --> 01:10:46.380 so on, or in any other language. It's a little bit like... 01:10:47.580 --> 01:10:50.560 Betting a little bit of your the trust that 01:10:50.560 --> 01:10:54.260 you have in your group or your peers so 01:10:54.260 --> 01:10:57.400 you gain a bit of trust you ship something you gain 01:10:57.400 --> 01:11:02.560 more trust you you bet your trust again that you have accumulated on a new project 01:11:02.560 --> 01:11:07.600 and this is how you can keep the momentum growing as a technologist in my view 01:11:07.600 --> 01:11:13.680 and yeah i mean there's also like maybe i'm survivorship biased here but if 01:11:13.680 --> 01:11:16.980 you make the right bets, it's fairly, it's usually fine. 01:11:17.960 --> 01:11:22.100 If you bet all of your trust on some mega project that then never ships, 01:11:22.340 --> 01:11:24.020 then it's going to be really difficult. 01:11:24.300 --> 01:11:29.660 So I just take more careful bets where I know I have a fairly good chance of delivering. 01:11:30.280 --> 01:11:35.060 On the training side, actually, this is a thing we're still figuring it out 01:11:35.060 --> 01:11:37.760 because people learn so differently. 01:11:38.100 --> 01:11:41.700 And I'm kind of oblivious to this because i'm the 01:11:41.700 --> 01:11:44.560 guy i don't know read the book guys what's the problem just 01:11:44.560 --> 01:11:47.980 it's a book we have a time to read it but not 01:11:47.980 --> 01:11:50.720 everyone learns the same way so i think having a sort 01:11:50.720 --> 01:11:56.480 of more a better way would be to we're trying to do now some courses and things 01:11:56.480 --> 01:12:00.700 like that a bit more applied to our ui and we have some documentation but it's 01:12:00.700 --> 01:12:06.140 more like it's it's knowledge that you get from your peers today it's not like 01:12:06.140 --> 01:12:08.880 oh i'm consulting the documentation and i'm learning. 01:12:09.080 --> 01:12:13.160 It's like some experienced guy told me in the code review this is better, 01:12:13.180 --> 01:12:15.360 so I'm doing it now and then I'll tell someone else. 01:12:15.800 --> 01:12:22.220 So this is kind of like, I don't know, knowledge that's trying to make a bit more, 01:12:23.400 --> 01:12:28.860 to spread it in a bit more organized way through some courses and things like that. 01:12:29.060 --> 01:12:34.500 Another thing I would say in Rust-specific case that really helped us, 01:12:35.760 --> 01:12:38.760 especially everyone's moving fast. We want to shift fast. 01:12:38.940 --> 01:12:41.420 Everyone, who has time to write documentation, right? 01:12:43.500 --> 01:12:51.420 One thing we did well was we heavily, heavily relied on the ability to write examples in Rust. 01:12:52.340 --> 01:12:55.060 So example.rs, right? 01:12:55.320 --> 01:13:01.440 So everything has an example. All the UI elements, we have hundreds of examples in our... 01:13:02.200 --> 01:13:07.400 People usually use VS Code or Z, I ported our launch JSON to Z. 01:13:07.680 --> 01:13:11.420 But basically, we have hundreds of examples you can pick from, 01:13:11.640 --> 01:13:13.040 run, and see how they behave. 01:13:13.180 --> 01:13:17.720 Put some breakpoints in a very self-contained place. So I think that helps for 01:13:17.720 --> 01:13:22.360 onboarding new people, having like a big amount of examples, 01:13:22.560 --> 01:13:23.960 especially with bigger projects. 01:13:24.200 --> 01:13:28.320 So you can kind of get an idea of what's happening easier. 01:13:29.020 --> 01:13:33.940 So that's what I would say. But yeah, starting small and then gaining a bit 01:13:33.940 --> 01:13:37.520 of momentum, doing a bit more and a bit more is definitely the way. 01:13:37.980 --> 01:13:41.780 Definitely structured training is really important. I guess that's also why 01:13:41.780 --> 01:13:43.960 we are here. So that's kind of nice. 01:13:44.820 --> 01:13:49.680 What do you look for in Rust engineers? You must have interviewed quite a lot. 01:13:50.100 --> 01:13:54.880 How do you know they can quickly pick up the language? 01:13:54.880 --> 01:14:01.080 What are some traits that you see in the real world where you can judge how 01:14:01.080 --> 01:14:05.520 people will, if people will be successful with that language or not? 01:14:05.920 --> 01:14:12.880 Oh, this is actually quite interesting because we don't test people for language skills on hiring. 01:14:13.600 --> 01:14:20.340 So my opinion as long as you don't some people just don't like the language 01:14:20.340 --> 01:14:26.140 and then you move teams or whatever but basically I don't think it's that special I think the whole, 01:14:27.380 --> 01:14:32.540 Rust is hard to learn thing is just something that is a meme about this language. 01:14:32.760 --> 01:14:39.200 I never quite got it because I kept trying to learn C++ for 10 years, man. 01:14:40.120 --> 01:14:46.420 You have no idea what hard is. If you're telling me Rust is hard, try C++. 01:14:48.140 --> 01:14:51.220 So in my view, I think that's a non-issue. 01:14:51.220 --> 01:14:56.260 And in general, so at Amazon, and actually all of, I think all of the big tech 01:14:56.260 --> 01:15:02.720 companies don't quite test on specific language skills, because I think it's 01:15:02.720 --> 01:15:05.440 assumed that you'll figure it out and learn and so on. 01:15:05.620 --> 01:15:09.760 And also, it's also assumed that you'll probably work with more than one language 01:15:09.760 --> 01:15:11.340 throughout your stay there. 01:15:11.640 --> 01:15:17.000 So it's not necessarily something that, but obviously, if you're willing to 01:15:17.000 --> 01:15:23.980 learn, that's quite important. because if you're resisting this type of change, you don't learn. 01:15:25.280 --> 01:15:28.020 It's going to be harder for you to adapt, right? 01:15:28.680 --> 01:15:32.280 Maybe some people just want to read JavaScript, right? So then in that case, 01:15:32.520 --> 01:15:34.280 they can move teams or whatever. 01:15:34.840 --> 01:15:39.100 It's a big company. But I would say overall, we don't do anything special for 01:15:39.100 --> 01:15:43.160 Rust training and for testing Rust skills. 01:15:44.700 --> 01:15:49.480 That's clever. Yeah, I fully understand. And traditionally, the final question 01:15:49.480 --> 01:15:52.320 is, what is your statement to the Rust community? 01:15:52.880 --> 01:15:57.340 Well, first of all, I'm really, really happy with everything. 01:15:57.340 --> 01:16:01.780 We have such a great array of crates out there to learn from. 01:16:02.500 --> 01:16:05.760 It's one of the communities, by the way, I really want to say this, 01:16:05.860 --> 01:16:08.360 that's been friendly as it grew. 01:16:08.720 --> 01:16:14.540 And I've never quite noticed this in another niche community that becomes more popular. 01:16:14.540 --> 01:16:17.380 It usually kind of degrades but 01:16:17.380 --> 01:16:20.200 the quality in interaction has been 01:16:20.200 --> 01:16:23.320 very consistent with the Rust community so I really 01:16:23.320 --> 01:16:26.440 like that one thing I wish we were focusing on 01:16:26.440 --> 01:16:30.620 more in the ecosystem is build speeds 01:16:30.620 --> 01:16:37.560 like iteration times just getting those down I think right now we more or less 01:16:37.560 --> 01:16:43.200 accepted that Rust will be slower to compile or whatever but I think there's 01:16:43.200 --> 01:16:48.140 examples out there in other languages where people have gotten great results, 01:16:48.500 --> 01:16:53.140 like Zig, for example, is pushing a lot on the build speed and the iteration time. 01:16:53.360 --> 01:16:59.060 And I think we all win if we can iterate in our codes super fast, 01:16:59.200 --> 01:17:02.200 like press whatever button you're pressing to run your code. 01:17:02.460 --> 01:17:07.340 And if that's like two seconds, it's just magical. Once it gets past a certain 01:17:07.340 --> 01:17:10.280 threshold, my productivity just, I get distracted. 01:17:10.660 --> 01:17:15.520 So I would just work a lot and investing on if I were like. 01:17:16.760 --> 01:17:22.000 If I had the power to invest in anything in the Rust community I would say this 01:17:22.000 --> 01:17:26.760 would be a really good place to invest in iteration times and build speed to decrease them. 01:17:27.810 --> 01:17:32.350 Who knows maybe at some point we will use cargo sick build for building our 01:17:32.350 --> 01:17:37.530 Rust projects because there is a sick compiler back end for Rust nowadays yeah. 01:17:37.530 --> 01:17:43.270 Especially for debug code where you don't need the whole optimizations from llvm and so on yeah. 01:17:43.270 --> 01:17:52.050 Yes i would like that Alex thanks a lot for taking the time i really enjoyed 01:17:52.050 --> 01:17:55.950 the calm and level-headed conversation and thanks for the interview. 01:17:56.450 --> 01:17:59.710 Thank you. This was lovely. Thanks so much for having me. I loved it. 01:18:01.230 --> 01:18:05.190 Rust in Production is a podcast by corrode. It is hosted by me, 01:18:05.450 --> 01:18:08.010 Matthias Endler, and produced by Simon Brüggen. 01:18:08.550 --> 01:18:13.250 For show notes, transcripts, and to learn more about how we can help your company 01:18:13.250 --> 01:18:15.830 make the most of Rust, visit corrode.dev. 01:18:16.450 --> 01:18:19.210 Thanks for listening to Rust in Production.