WEBVTT 00:00:01.550 --> 00:00:05.590 Welcome to another episode of Rust in Production, a podcast about companies 00:00:05.590 --> 00:00:07.990 who use Rust to shape the future of infrastructure. 00:00:08.330 --> 00:00:12.670 My name is Matthias Endler from corrode, and today I'm joined by Danilo Krummrich 00:00:12.670 --> 00:00:17.010 from the Linux Kernel Project to talk about bringing Rust into the kernel. 00:00:19.310 --> 00:00:23.390 Danilo, thanks so much for taking the time for the interview today. 00:00:23.650 --> 00:00:27.710 Can you quickly say a few words about yourself and about Red Hat, 00:00:27.890 --> 00:00:28.730 the company you work for? 00:00:28.730 --> 00:00:34.490 Of course, thanks for the invitation. Yeah, so my name is Danilo and I'm a Linux 00:00:34.490 --> 00:00:39.170 kernel engineer working at Red Hat in the accelerators and GPU group. 00:00:39.610 --> 00:00:44.270 And I'm also a maintainer in the Linux kernel of various subsystems, 00:00:44.330 --> 00:00:48.110 components and drivers and member of the Rust4Linux team. 00:00:48.730 --> 00:00:54.710 It's almost intimidating to think that you work on such an important project 00:00:54.710 --> 00:00:57.290 and also on such a low level. 00:00:57.470 --> 00:01:03.510 I guess a lot of people that are listening in either use Linux or get in contact 00:01:03.510 --> 00:01:07.950 with Linux on a daily basis because, you know, we use a ton of servers nowadays 00:01:07.950 --> 00:01:09.550 and a lot of these run on Linux. 00:01:09.730 --> 00:01:14.170 What does it feel like to work on that level and how did you even get started 00:01:14.170 --> 00:01:15.710 with kernel development? 00:01:15.710 --> 00:01:20.310 I don't think it necessarily feels different than working on any other software 00:01:20.310 --> 00:01:22.410 project, I would guess. I cannot really tell. 00:01:22.630 --> 00:01:26.930 I worked on a few other projects in the past 10 years ago. I worked a little 00:01:26.930 --> 00:01:29.610 bit on the AOSP, so the Android open source project. 00:01:30.050 --> 00:01:35.170 I worked there at kernel level as well, but also some user space portions. 00:01:36.270 --> 00:01:41.770 So i was always interested in the kernel since i finished my exams at university 00:01:41.770 --> 00:01:47.610 and when i joined the first company that i also worked at when i was a student 00:01:47.610 --> 00:01:52.050 i i just told people hey i'm interested in the kernel i want to work at that 00:01:52.050 --> 00:01:54.730 just get me to the just get me to the base layer team, 00:01:55.770 --> 00:02:01.170 and then i just started working there and and especially i started learning 00:02:01.170 --> 00:02:05.330 a lot the learning curve was very, very high. 00:02:05.610 --> 00:02:10.270 When was the first time you personally heard of the Rust4Linux project? 00:02:10.830 --> 00:02:14.630 When was the first time you heard about, I mean, discussions, 00:02:14.630 --> 00:02:16.770 adding Rust to the Linux kernel? 00:02:17.010 --> 00:02:22.230 I would say almost from the beginning, because I'm following what's going on 00:02:22.230 --> 00:02:24.890 in the project, what's discussed in mailing lists. 00:02:25.330 --> 00:02:30.110 But this was more reading along. It was not that I I was immediately, 00:02:30.530 --> 00:02:36.670 okay, this is definitely something we need to, I need to get involved in. But I was aware. 00:02:37.150 --> 00:02:38.570 Did you know Rust back then? 00:02:39.730 --> 00:02:42.910 So this was all new for you, but how did it feel like? 00:02:43.210 --> 00:02:48.890 Can you remember the time, the feeling that maybe thinking about the adoption 00:02:48.890 --> 00:02:53.630 of Rust and was that the time when you also looked at the language for the first time? 00:02:54.390 --> 00:02:59.650 Can you talk about your first impressions of the language coming from a mostly 00:02:59.650 --> 00:03:00.790 C background, I'm guessing? 00:03:01.570 --> 00:03:05.730 It was a bit intimidating, I would say. The C language is very, very simple. 00:03:05.810 --> 00:03:09.490 And in the kernel, we are doing very, very complicated things. 00:03:09.490 --> 00:03:11.070 With a very, very simple language. 00:03:12.070 --> 00:03:17.730 With Rust, it's different. Rust is a very difficult language, I would say. 00:03:19.050 --> 00:03:25.130 But at the same time, it makes some of the problems we have to solve much easier to solve. 00:03:25.330 --> 00:03:27.550 So I guess that's a good trade-off. 00:03:27.910 --> 00:03:32.430 Did you ever second-guess the adoption of Rust in a Linux kernel, 00:03:32.470 --> 00:03:36.430 or was it always clear to you that there were very obvious benefits? 00:03:37.670 --> 00:03:44.150 So after I got used to the concepts of Rust and understood what they are about 00:03:44.150 --> 00:03:48.950 and understood how I can utilize them, I don't have any doubt. 00:03:49.680 --> 00:03:53.620 This is really i'm very or let it say the other way around i'm very very convinced 00:03:53.620 --> 00:03:59.220 that rust definitely has a big future in the kernel how. 00:03:59.220 --> 00:04:02.520 Long did it take to build out the entire infrastructure infrastructure that 00:04:02.520 --> 00:04:09.340 is needed for such a project inside the kernel and outside the kernel from the rust side i mean. 00:04:09.340 --> 00:04:12.240 So i think a lot 00:04:12.240 --> 00:04:15.280 of initial so a lot of the initial work was 00:04:15.280 --> 00:04:19.640 and i cannot say a lot about that because that was not the time where i joined 00:04:19.640 --> 00:04:24.720 the project this was before i joined the project but i think and if i'm forgetting 00:04:24.720 --> 00:04:28.840 someone i'm very sorry about it but i i think it was mainly Miguel and and Wedson 00:04:28.840 --> 00:04:32.780 so Miguel Ojeda and Wedson Almeida who, 00:04:34.240 --> 00:04:40.520 started the Rust4Linux project and in a downstream branch worked out a lot 00:04:40.520 --> 00:04:41.460 of infrastructure, 00:04:42.460 --> 00:04:47.700 And the initial support in the Linux kernel, build system integration, and so on. 00:04:48.280 --> 00:04:51.320 And I think the initial lift there 00:04:51.320 --> 00:04:55.780 was to convince people that it's worth adding it to the Linux kernel. 00:04:55.940 --> 00:04:59.860 There were approaches adding a second language into the kernel in the past. 00:05:00.060 --> 00:05:03.340 I think C++ was tried a couple of times, and it failed badly. 00:05:03.920 --> 00:05:09.540 So adding a new language to the Linux kernel is definitely not an easy thing to attempt. 00:05:09.960 --> 00:05:16.220 And they were successful. So I think this milestone was really one of the big ones. 00:05:16.760 --> 00:05:22.460 Why has Rust succeeded, quote unquote, to be adopted in the Linux kernel and C++ not? 00:05:23.120 --> 00:05:29.380 So I think that a subset of C++ would have been a great addition to the kernel. 00:05:29.680 --> 00:05:33.220 Just to name two examples, the first one being inheritance. 00:05:33.800 --> 00:05:40.960 The C kernel code relies a lot on the inheritance pattern. So getting some language 00:05:40.960 --> 00:05:44.500 support for that would definitely been a great improvement. 00:05:44.880 --> 00:05:54.140 And the second one that I think is very, very useful is as simple as field visibilities of structures. 00:05:54.460 --> 00:06:01.780 So we have the case in the kernel that generic components obviously have fields that are, 00:06:02.550 --> 00:06:05.830 or should be accessed by design by 00:06:05.830 --> 00:06:11.190 the users but there is also private fields and what we often end up is that 00:06:11.190 --> 00:06:17.110 drivers instead of representing their needs by contributions to the common component 00:06:17.110 --> 00:06:25.710 in the kernel instead abuse private fields of the structure and basically peek into component. 00:06:27.470 --> 00:06:32.590 Internals, which obviously is not considered by the component maintainers in 00:06:32.590 --> 00:06:39.130 the end and hence causes problems in maintainability and can actually lead to bugs in the kernel. 00:06:39.710 --> 00:06:43.450 So those are problems that are solved by Rust as well. 00:06:43.850 --> 00:06:49.010 And I think overall, Rust is just the much better fit for the kernel because 00:06:49.010 --> 00:06:53.550 we have much less things that are not a good fit for the kernel. 00:06:53.810 --> 00:07:01.490 Plus, it has all the additional features that help with memory safety that C++ does not have. 00:07:01.810 --> 00:07:07.590 However, it also brings some disadvantages with it, which is the common practice 00:07:07.590 --> 00:07:16.330 in Rust to just unwrap results or unwrap options. So basically operations that panic the program. 00:07:16.490 --> 00:07:21.050 And in user space, that's the same option in a lot of cases. 00:07:21.050 --> 00:07:25.570 But in the kernel, it almost never is a sane option because a panic in the kernel 00:07:25.570 --> 00:07:30.390 is really just the last resort when the system is in a state where it's basically 00:07:30.390 --> 00:07:36.870 non-recoverable and otherwise you would corrupt memory or in the worst case 00:07:36.870 --> 00:07:38.230 even corrupt file systems. 00:07:38.610 --> 00:07:45.270 So this is something that we have to take care of when using Rust in the kernel 00:07:45.270 --> 00:07:51.450 that we're making it hard for people to accidentally trigger panics in the kernel. 00:07:52.520 --> 00:07:59.220 So what was your first task when starting with Rust4Linux? 00:07:59.460 --> 00:08:04.520 Was it a thing that you voluntarily picked up or was there a need at work where 00:08:04.520 --> 00:08:07.680 you needed to start the Rust, where you needed to touch the Rust part? 00:08:09.000 --> 00:08:16.620 I think it really started with me taking maintainership of the Nouveau driver. 00:08:16.620 --> 00:08:26.480 And it's also a result out of the way forward with Nouveau and the introduction of Nova. 00:08:27.260 --> 00:08:33.760 So with the Nouveau driver, we have a lot of issues which are... 00:08:33.760 --> 00:08:40.720 So back then when I started at Red Hat, I started working on a component that's 00:08:40.720 --> 00:08:44.780 named GPUVM for the DRM subsystem. 00:08:44.780 --> 00:08:50.180 I mean, it's basically managing the GPU's virtual address space and doing some 00:08:50.180 --> 00:08:53.620 more things in the context of providing 00:08:53.620 --> 00:08:58.280 helpers for drivers to implement Vulkan-compatible user space APIs. 00:08:58.680 --> 00:09:05.080 So APIs that are given out to user space drivers, for instance, Mesa. 00:09:07.280 --> 00:09:14.400 So the work of GPU VM I did in the context of adding a new UAPI to Nouveau, 00:09:14.580 --> 00:09:16.260 which is Vulkan-compatible. 00:09:16.860 --> 00:09:23.540 And in order to get that going in Nouveau, we needed to do quite some changes 00:09:23.540 --> 00:09:27.060 on how the page table management of the GPU itself works. 00:09:27.580 --> 00:09:32.640 But it was never possible to do it to the full and correct extent. 00:09:32.640 --> 00:09:34.460 We still have problems there. 00:09:34.600 --> 00:09:41.360 And that's because the design of how Nouveau is written just doesn't really 00:09:41.360 --> 00:09:48.240 work out well for this kind of requirements that we get from a Vulkan-compatible UAPI. 00:09:48.240 --> 00:09:53.500 So those design reworks would be a lot of effort, 00:09:53.740 --> 00:09:57.460 especially considering that Nouveau 00:09:57.460 --> 00:10:03.720 supports a huge range of GPUs and GPU architectures and generations. 00:10:04.420 --> 00:10:11.740 So you can basically split them up into the generations before NVIDIA introduced 00:10:11.740 --> 00:10:14.520 GSP and after NVIDIA introduced GSP. 00:10:14.520 --> 00:10:21.260 So GSP means GPU system processor, and it's a firmware processor that lives within the GPU. 00:10:21.760 --> 00:10:26.800 And instead of programming the hardware directly, if you have a GSP GPU, 00:10:27.060 --> 00:10:34.340 you can just talk to the firmware through a ring buffer and a firmware-specific 00:10:34.340 --> 00:10:36.700 protocol and instruct the firmware 00:10:36.700 --> 00:10:40.600 to do the things that you would usually program through registers. 00:10:42.620 --> 00:10:50.680 And by reworking all those layers to get a Vulkan compatible UAPI we would need 00:10:50.680 --> 00:10:57.220 to consider the whole range of GPUs, the ones pre-GSP and the ones since GSP introduction, 00:10:58.320 --> 00:11:03.900 so that would be a lot of effort just to rework something we also have other 00:11:03.900 --> 00:11:08.440 problems with and the other problems we have in Nouveau are. 00:11:09.920 --> 00:11:12.640 How well it's documented and how 00:11:12.640 --> 00:11:15.960 it's designed so Nouveau consists out of lots 00:11:15.960 --> 00:11:19.600 and lots of v tables with callbacks that 00:11:19.600 --> 00:11:22.740 are not really documented in terms of ownership 00:11:22.740 --> 00:11:26.160 and lifetimes of the objects that are allocated returned and 00:11:26.160 --> 00:11:29.220 and handled which is which 00:11:29.220 --> 00:11:32.380 is one of the huge problems and one problem 00:11:32.380 --> 00:11:35.600 that is really important to me personally to 00:11:35.600 --> 00:11:38.620 address is that Nouveau 00:11:38.620 --> 00:11:42.880 is just not really accessible to users we don't 00:11:42.880 --> 00:11:46.120 have a lot of contributors and i 00:11:46.120 --> 00:11:51.120 think the reason for that is because just no one really understands how it works 00:11:51.120 --> 00:11:57.500 and doesn't get comfortable working with that driver and for an open source 00:11:57.500 --> 00:12:03.720 project that's not really what you want so what we, 00:12:04.420 --> 00:12:09.500 there ended up with was to decide at a certain point we should do a new driver 00:12:09.500 --> 00:12:16.000 we should do a driver that is gsp only and if we do a new driver now we saw 00:12:16.000 --> 00:12:17.980 that Rust4Linux is progressing. 00:12:19.180 --> 00:12:24.860 We thought okay if we do a new driver now we have the choice do we do another c driver, 00:12:25.520 --> 00:12:37.140 or do we do a Rust driver where the language solves a lot of problems for us that DRM drivers have. 00:12:37.300 --> 00:12:40.480 So DRM drivers, GPU drivers are usually very, very complicated, 00:12:41.360 --> 00:12:47.600 suffer from race conditions, suffer from memory issues and just because they're 00:12:47.600 --> 00:12:49.760 complicated and it's hard to get it right. 00:12:50.060 --> 00:12:56.420 And Rust helps a lot with that. So we decided to go for a Rust driver and or 00:12:56.420 --> 00:13:00.420 actually first step evaluate if we want to do a rust drive and this is where 00:13:00.420 --> 00:13:06.120 i got in touch with rust in the kernel the first time because i started doing this 00:13:06.120 --> 00:13:08.920 evaluation of is a rust driver, 00:13:10.040 --> 00:13:17.780 for the successor of Nouveau doable is it is it reasonable and what are the 00:13:17.780 --> 00:13:21.280 things that we need to consider what are the things that we need to do to get it going. 00:13:21.280 --> 00:13:24.440 And so at this point in time you're 00:13:24.440 --> 00:13:30.100 thinking is rust a better choice for writing a completely new driver for nvidia 00:13:30.100 --> 00:13:36.260 graphics cards for the linux kernel and moving away from the legacy codebase which 00:13:36.260 --> 00:13:40.960 was Nouveau back in the day i think Nouveau not even sure when it started but 00:13:40.960 --> 00:13:42.380 it's been around for a while right. 00:13:43.060 --> 00:13:47.880 Yeah i think it's it's around for 15 20 years. 00:13:47.880 --> 00:13:51.220 Yeah okay so that 00:13:51.220 --> 00:13:56.880 must mean the code base is relatively evolved it has evolved and it's probably 00:13:56.880 --> 00:14:02.300 complicated to maintain i'm not sure about the status but i'm just guessing 00:14:02.300 --> 00:14:07.860 that with 15 years or 20 years of development time it might not be the easiest 00:14:07.860 --> 00:14:10.940 codebase to work with especially for beginners yeah. 00:14:10.940 --> 00:14:13.660 It absolutely grew over a long period of 00:14:13.660 --> 00:14:19.760 time and yeah so the lack of documentation and the fact that Nouveau was maintained 00:14:19.760 --> 00:14:26.780 for a long time by a single person and all the knowledge about how things are 00:14:26.780 --> 00:14:31.500 meant to work within the driver only really exists in the head of this person 00:14:31.500 --> 00:14:32.940 doesn't really help either. 00:14:33.920 --> 00:14:37.560 Okay, but when you describe it like that, it sounds really daunting to start 00:14:37.560 --> 00:14:42.320 a new driver because you kind of need to get that business logic, 00:14:42.520 --> 00:14:45.260 port it over first to a new language, 00:14:45.560 --> 00:14:51.520 second from an older code base, which maybe you are not 100% familiar with. 00:14:51.660 --> 00:14:53.920 I'm not sure. Maybe you were. Yeah. 00:14:54.870 --> 00:15:00.890 There might be a lot of internals or assumptions or implications in the code, 00:15:00.890 --> 00:15:03.750 which will make that port harder as well. 00:15:04.070 --> 00:15:08.650 And on top of it, you also have to understand the structure of the Linux kernel 00:15:08.650 --> 00:15:13.530 itself and how subsystems and drivers interact and so on. 00:15:13.610 --> 00:15:16.490 So it sounds like a very challenging task. 00:15:16.690 --> 00:15:21.930 Oh yeah, it is challenging, no question. But it's not like we're potting over the logic. 00:15:21.930 --> 00:15:26.610 The good thing is that we're starting from GSP only, 00:15:27.070 --> 00:15:35.290 which means that not only, but to a large extent, we really have to understand 00:15:35.290 --> 00:15:37.130 how the firmware interface works. 00:15:37.370 --> 00:15:43.030 And this is absolutely doable and doesn't really require to understand every 00:15:43.030 --> 00:15:45.910 single bit of what the Nouveau code does. 00:15:46.110 --> 00:15:51.670 And so you can start to port over that part alone in isolation. 00:15:51.930 --> 00:15:57.770 Yeah, so what we went for is we basically split the Nova driver into two separate drivers. 00:15:58.270 --> 00:16:02.570 One of them basically provides, which we call NovaCore, 00:16:02.970 --> 00:16:09.250 provides the hardware and firmware abstraction layer, which is needed not only 00:16:09.250 --> 00:16:15.930 by Nova DRM, which is the second Nova driver, basically, and implements all the DRM parts. 00:16:15.930 --> 00:16:22.170 So where DRM is direct rendering manager, it's the subsystem in the kernel that 00:16:22.170 --> 00:16:24.410 handles GPUs and accelerators. 00:16:25.770 --> 00:16:31.010 So this is then the second driver and it sits on top of NovaCore and it's, 00:16:32.060 --> 00:16:36.140 utilizes the hardware through this abstraction layer this 00:16:36.140 --> 00:16:39.720 is necessary for two reasons the first is that 00:16:39.720 --> 00:16:43.340 there is also the nvidia vgpu 00:16:43.340 --> 00:16:47.220 driver which is currently making its way upstream which 00:16:47.220 --> 00:16:50.140 is a virtualization layer or actually it's 00:16:50.140 --> 00:16:53.780 only really the manager of the virtualization layer because 00:16:53.780 --> 00:16:57.700 it manages the pci virtual 00:16:57.700 --> 00:17:01.220 functions so the gpu has has one 00:17:01.220 --> 00:17:04.260 physical function that is exposed through 00:17:04.260 --> 00:17:07.960 to the system to the host kernel which is 00:17:07.960 --> 00:17:11.220 where a normal driver would sit on top of but additionally 00:17:11.220 --> 00:17:14.240 there are protocols in the pci layer that allow 00:17:14.240 --> 00:17:17.280 the gpu to expose also 00:17:17.280 --> 00:17:20.220 virtual functions which looks like as if 00:17:20.220 --> 00:17:23.180 they were additional gpus but actually they 00:17:23.180 --> 00:17:28.020 are virtualized within the graphics card and vgpu 00:17:28.020 --> 00:17:34.640 manages those through nova core by utilizing nova core to expose those virtual 00:17:34.640 --> 00:17:39.360 functions to virtual machines and then virtual machines can run nova core again 00:17:39.360 --> 00:17:44.680 and then Nova DRM on top to actually expose the full graphics stack. 00:17:45.800 --> 00:17:51.580 So that's one reason why we have this driver split, because we have basically 00:17:51.580 --> 00:17:55.380 multiple clients for the firmware and hardware abstraction layer that lives within NovaCore. 00:17:56.340 --> 00:18:02.660 The second reason is the firmware API that is exposed by the GPU, 00:18:03.660 --> 00:18:09.620 is not stable, or at least we cannot rely on this API to be stable. 00:18:10.240 --> 00:18:13.860 And that's one of the things where Rust helps a lot as well, 00:18:14.040 --> 00:18:20.480 because abstracting the firmware API is much easier with the powerful Rust type 00:18:20.480 --> 00:18:25.380 system and things like PROC macros compared to C where we, 00:18:25.640 --> 00:18:31.220 in the end, would need to build endless VTables to differentiate between the 00:18:31.220 --> 00:18:35.820 different firmware versions And then you can build more VTables to gather the 00:18:35.820 --> 00:18:39.100 common code that's maybe similar between certain versions, 00:18:39.100 --> 00:18:44.520 which becomes a huge mess and is one of the reasons why Nouveau is also not sustainable. 00:18:46.600 --> 00:18:51.500 And in Rust, that would be a proc macro, which could be expanded, 00:18:51.860 --> 00:18:55.420 which might end up being the same Vtable, but you don't have to write it yourself. 00:18:55.820 --> 00:18:57.000 Yes. Yes. 00:18:57.900 --> 00:19:01.000 Now, a bit of a silly question, but bear with me for a second. 00:19:02.000 --> 00:19:06.780 How do you even start with developing such a driver in the kernel? 00:19:06.920 --> 00:19:08.960 You can't just possibly say cargo new. 00:19:10.140 --> 00:19:12.560 How do you initiate a new driver? 00:19:12.560 --> 00:19:18.540 From the technical side of things, it's very easy because the major work of 00:19:18.540 --> 00:19:23.260 integrating the Rust compiler into the Linux kernel has already been done before 00:19:23.260 --> 00:19:24.780 I joined the Rust4Linux project. 00:19:25.100 --> 00:19:28.900 And this is also what the Rust tree in the kernel is actually about. 00:19:29.060 --> 00:19:31.500 It's about compiler and build system support. 00:19:31.700 --> 00:19:35.340 It carries a lot of other things and carries a lot of other base infrastructure 00:19:35.340 --> 00:19:36.780 that we have in the kernel as well. 00:19:37.080 --> 00:19:40.660 But its main focus is compiler and build system. 00:19:41.400 --> 00:19:44.820 So you basically create a 00:19:44.820 --> 00:19:47.620 new driver the exact same way as you would create 00:19:47.620 --> 00:19:50.500 a new driver in c in the kernel which 00:19:50.500 --> 00:19:53.860 is you create a kconfig file create a 00:19:53.860 --> 00:19:56.600 new config option for the kernel to include this new 00:19:56.600 --> 00:19:59.580 component which is i don't know 10 00:19:59.580 --> 00:20:02.480 15 20 lines of of code most of 00:20:02.480 --> 00:20:05.340 that being just module description and then 00:20:05.340 --> 00:20:10.300 you have a make file where you add the the 00:20:10.300 --> 00:20:17.780 core rs file which then pulls in all other modules to the global variable that 00:20:17.780 --> 00:20:23.720 grabs all the all the source files for the cabled build system and that's it 00:20:23.720 --> 00:20:29.920 and then it builds so this part is is actually pretty easy We're not using Cargo. 00:20:30.040 --> 00:20:32.980 So we're integrating the Rust compiler into kbuild. 00:20:33.970 --> 00:20:36.610 Okay, but then you have a driver which does essentially nothing. 00:20:36.610 --> 00:20:40.330 You still have to communicate with the outside world. How does that look like? 00:20:40.690 --> 00:20:43.650 Exactly. So then you basically have a driver stub. 00:20:43.930 --> 00:20:47.450 Actually, you don't even have a driver stub. The only thing you really have 00:20:47.450 --> 00:20:52.690 is a so-called kernel module, which has an entry point and an exit point, 00:20:52.870 --> 00:20:55.950 which is the init function and the exit function. 00:20:56.150 --> 00:21:01.170 That's it. In the Rust abstraction, it's basically an init function and the 00:21:01.170 --> 00:21:04.270 exit function is basically just drop off the module structure. 00:21:05.450 --> 00:21:09.150 But that's all you got. Now you need some driver infrastructure. 00:21:09.150 --> 00:21:15.850 And that's where the first challenge was for the Nova project because a lot 00:21:15.850 --> 00:21:19.130 of work has been done before I joined the Rust4Linux project. 00:21:19.130 --> 00:21:25.250 Like the initial integration of the Rust compiler into the build system, 00:21:25.590 --> 00:21:30.610 module support, lots of generic infrastructure that you need. 00:21:30.830 --> 00:21:38.770 So abstractions for handling C reference counted structures on the Rust side, for instance. 00:21:39.190 --> 00:21:44.310 Some locking stuff was there. So really just the absolute fundamental basics. 00:21:44.830 --> 00:21:49.330 But there was no driver infrastructure. So you weren't able to write drivers. 00:21:49.610 --> 00:21:55.230 So the first thing that we actually needed to come up with was core driver infrastructure, 00:21:55.230 --> 00:22:04.210 which is the representation of a device, representation of a driver, 00:22:04.850 --> 00:22:10.570 representation of a bus, and all the glue code to connect things together. 00:22:11.560 --> 00:22:14.420 But i think before i talk more about that we would 00:22:14.420 --> 00:22:17.480 need to go one step back and 00:22:17.480 --> 00:22:20.620 talk about what we call abstractions in 00:22:20.620 --> 00:22:24.460 Rust4Linux right yeah so the 00:22:24.460 --> 00:22:27.820 approach we we take and what rust focuses on 00:22:27.820 --> 00:22:31.200 in the kernel is really drivers this is 00:22:31.200 --> 00:22:34.280 where we can get most of the value of the language in 00:22:34.280 --> 00:22:37.180 the short term i think also because 00:22:37.180 --> 00:22:40.460 of the architecture of Linux. Linux is a monolithic kernel 00:22:40.460 --> 00:22:43.920 and therefore all drivers also 00:22:43.920 --> 00:22:47.200 run in kernel space and if a driver messes things 00:22:47.200 --> 00:22:50.160 up it breaks the whole machine right 00:22:50.160 --> 00:22:59.940 so if we are able to provide support for writing drivers in rust a huge part 00:22:59.940 --> 00:23:09.880 for safety and and reliability of the kernel is is is achieved also if we consider that. 00:23:10.760 --> 00:23:18.660 Subsystem code, so really kernel core code, is reviewed and tested much more 00:23:18.660 --> 00:23:20.620 thoroughly than driver code is. 00:23:21.080 --> 00:23:26.900 So the most value in the shortest time really get from drivers. 00:23:27.700 --> 00:23:33.440 And the mechanism that people have worked out to interface with C infrastructure 00:23:33.440 --> 00:23:38.620 is to write so-called abstractions. So we're not directly calling C functions 00:23:38.620 --> 00:23:41.100 from Rust because that would be fundamentally unsafe. 00:23:42.220 --> 00:23:45.760 Because then we would need to deal with raw pointers all the time. 00:23:45.900 --> 00:23:48.960 And then we wouldn't really get a lot of the benefits from Rust. 00:23:49.140 --> 00:23:54.120 So what we're doing instead is we're building abstractions, which means that 00:23:54.120 --> 00:23:59.900 for a C structure and its corresponding functions to Rust. 00:24:02.260 --> 00:24:06.580 Achieve the functionality that's intended by the structure. 00:24:06.580 --> 00:24:13.660 We write a Rust structure that in some way embeds the C structure or embeds 00:24:13.660 --> 00:24:19.640 a pointer to the C structure. It really depends on the actual case. 00:24:20.100 --> 00:24:28.980 And then we write some Rust code around that API. So you can only use that API in a way that is safe. 00:24:29.880 --> 00:24:35.980 So, for instance, if you obtain a device, a representation of a device in the 00:24:35.980 --> 00:24:41.600 system, you would not get a pointer, but rather a reference to the device. 00:24:42.040 --> 00:24:46.200 And instead of now dealing with only the reference of the device, 00:24:46.380 --> 00:24:50.740 you also have certain device states. So, in Rust, that's just type states. 00:24:51.120 --> 00:24:54.360 And in the kernel, a device can be in multiple states, 00:24:54.380 --> 00:24:58.300 and we represent that by type states, and then only allow the functions that 00:24:58.300 --> 00:25:04.300 are applicable for a certain state if the user can or if the user calls it on 00:25:04.300 --> 00:25:09.440 the abstraction type with the corresponding type state. 00:25:11.260 --> 00:25:19.860 Okay, that sounds super cool because type state is a very nice way to model state machines, 00:25:19.980 --> 00:25:25.540 obviously, but I don't know if you had an equivalent on the C side before or 00:25:25.540 --> 00:25:32.220 if that was a new integration or if that was a new abstraction that you added in that process. 00:25:33.430 --> 00:25:39.610 So on the C side, if we stay at the device example, that's not existent on the C side. 00:25:39.710 --> 00:25:42.390 On the C side, it's really just a raw struct device pointer, 00:25:42.790 --> 00:25:50.670 and it's on the user to use it in the correct way. 00:25:51.150 --> 00:25:55.650 Okay. Well, I made it sound like it was an obviously good idea, 00:25:55.690 --> 00:26:00.970 but maybe we can also take a step back here and maybe describe what is the benefit 00:26:00.970 --> 00:26:05.510 of the type state pattern, and maybe even what is it in the first place. 00:26:06.270 --> 00:26:15.490 Yeah. So let me explain real quick how device, driver, and bus fits into the bigger picture. 00:26:15.770 --> 00:26:23.410 So if you want to write a driver in the kernel, you have to implement the driver 00:26:23.410 --> 00:26:26.850 structure of the particular subsystem. Let's say PCI. 00:26:27.170 --> 00:26:33.570 So you implement the PCI structure, which requires you to implement a couple of callbacks. 00:26:33.770 --> 00:26:36.290 Lots of them are optional. Some of them are mandatory. 00:26:37.050 --> 00:26:40.810 The two most interesting callbacks are probe and remove. 00:26:41.530 --> 00:26:47.070 The probe callback of the driver is called once the bus, in this case the PCI 00:26:47.070 --> 00:26:51.150 bus, detects a device in the system that matches the driver. 00:26:51.350 --> 00:26:56.190 So the bus basically detects there is a new device and then it checks what is 00:26:56.190 --> 00:27:00.570 the vendor ID of the PCI device, What is the device ID of the PCI device? 00:27:00.850 --> 00:27:03.670 It checks a couple more things, of course. This is a bit oversimplified. 00:27:04.390 --> 00:27:11.030 But then it looks for a driver in the system that is registered that matches those attributes. 00:27:11.310 --> 00:27:14.790 And if it finds a driver in the system that matches the attributes, 00:27:15.010 --> 00:27:22.770 it calls the probe function that the driver registered and passes in the PCI 00:27:22.770 --> 00:27:25.850 device structure in the probe function. 00:27:25.850 --> 00:27:29.890 And then the driver can start operating the device. 00:27:30.170 --> 00:27:36.410 It can obtain device resources such as I.O. memory or interrupt handlers and 00:27:36.410 --> 00:27:37.810 just do its business logic. 00:27:38.860 --> 00:27:44.060 It gets a mutable pointer to that device, or what would be the input? 00:27:44.260 --> 00:27:52.100 So in C, it's really just a pointer. If we're in Rust, it's a reference to a 00:27:52.100 --> 00:27:56.540 PCI device structure with a certain type state. 00:27:56.900 --> 00:28:01.940 So for probe, so actually for all the callbacks that come from a bus, so like probe, remove. 00:28:02.140 --> 00:28:05.640 So remove is obviously when the driver is unbound from the device, 00:28:05.740 --> 00:28:08.100 when the device is unbound from the driver this way around. 00:28:08.860 --> 00:28:14.420 Um, but in the bus callbacks, it gets the type state core. 00:28:14.600 --> 00:28:20.140 So then the device has the core type state, and then you have access to certain 00:28:20.140 --> 00:28:24.480 functions of the PCI device that you would otherwise not have. 00:28:24.780 --> 00:28:31.340 And that's important because in the bus callbacks, a global bus lock is taken, 00:28:31.360 --> 00:28:37.540 and that allows you to modify certain fields within the device structure that 00:28:37.540 --> 00:28:39.560 are needed for instance to, 00:28:40.420 --> 00:28:45.520 enable i o memory in the first place or bus mastering and, 00:28:46.950 --> 00:28:53.790 So by having this type state, we can ensure that those functions that are only 00:28:53.790 --> 00:28:58.470 allowed to be called from bus callbacks are not called from anywhere else. 00:28:58.610 --> 00:29:02.510 And the way they could be called from anywhere else is because device structures 00:29:02.510 --> 00:29:04.670 are fundamentally reference counted in the kernel. 00:29:04.850 --> 00:29:08.070 So everyone can hold on to a reference of that thing. 00:29:08.290 --> 00:29:11.830 So it is theoretically possible, but in Rust, it's not. 00:29:13.630 --> 00:29:17.130 Yeah so a simplified version of that would be you 00:29:17.130 --> 00:29:20.170 get a device you are allowed to call a 00:29:20.170 --> 00:29:22.930 couple of methods on it which give you a 00:29:22.930 --> 00:29:26.350 new state of that device a thing that you 00:29:26.350 --> 00:29:29.190 can hold on to but then crucially you cannot call 00:29:29.190 --> 00:29:32.130 methods that would be invalid or you try 00:29:32.130 --> 00:29:34.950 to make invalid state irrepresentable this way 00:29:34.950 --> 00:29:38.370 yes that's pretty cool it feels 00:29:38.370 --> 00:29:42.870 like the type state pattern is a very core part of the abstractions that you 00:29:42.870 --> 00:29:48.390 needed for kernel development from rust are there any other components that 00:29:48.390 --> 00:29:50.570 you needed to build out which come 00:29:50.570 --> 00:29:55.470 to mind anything that other systems programmers could also learn from so. 00:29:55.470 --> 00:30:02.230 I think one example may be how we deal with reference counted structures on the C side. 00:30:02.850 --> 00:30:10.330 So in C, reference counting works that structures embed a struct kref, 00:30:10.610 --> 00:30:17.190 which is in the end an atomic counter that gives you a release callback once 00:30:17.190 --> 00:30:21.010 it drops to zero, and then you have to implement a release callback in a certain 00:30:21.010 --> 00:30:22.410 way to clean up that structure. 00:30:23.710 --> 00:30:28.170 And this is a pattern that we have very often. Things in the kernel are very 00:30:28.170 --> 00:30:29.190 often reference counted. 00:30:30.240 --> 00:30:33.700 And in C abstractions, we had to deal with that. 00:30:33.920 --> 00:30:38.300 So what people did were writing a trait that is called always ref counted, 00:30:38.600 --> 00:30:46.500 which implemented by the corresponding Rust structure representation of the 00:30:46.500 --> 00:30:47.820 reference counted C struct, 00:30:48.140 --> 00:30:53.880 just provides common helpers for reference counting and a common interface for reference counting, 00:30:53.880 --> 00:30:57.760 which is the a ref type with 00:30:57.760 --> 00:31:00.700 with a generic so you basically 00:31:00.700 --> 00:31:04.440 have your rust abstraction structure let's say device and since 00:31:04.440 --> 00:31:09.260 the device is fundamentally reference counted you then end up with an a ref 00:31:09.260 --> 00:31:16.860 device and then this a ref knows that there's always reference counted implemented 00:31:16.860 --> 00:31:22.020 for the device because it's a trade bound and then we can we can make those parts common. 00:31:23.180 --> 00:31:26.360 Well typically in a pure rust 00:31:26.360 --> 00:31:30.960 world you would probably lean into ownership a little more like a lot of things 00:31:30.960 --> 00:31:36.780 just move between different components in rust but i'm assuming that that will 00:31:36.780 --> 00:31:41.680 be very hard to do in an inux kernel because the rest of the code expects references 00:31:41.680 --> 00:31:46.180 and passes references around or ref counted structures to be more precise. 00:31:46.180 --> 00:31:51.220 We try to avoid reference counting if it's not absolutely necessary. 00:31:51.740 --> 00:31:56.140 But if the C structure already does it, there's usually a very good reason for it. 00:31:56.780 --> 00:32:01.460 Um, so this is something we didn't just have to adapt to. 00:32:02.240 --> 00:32:06.220 I may have one other good example that is worth mentioning in this context, 00:32:06.220 --> 00:32:13.560 which is you mentioned in Rust, you usually use move semantics and try to not 00:32:13.560 --> 00:32:17.040 reference count things or put things on the heap, um, for that. 00:32:17.540 --> 00:32:23.720 And this is another implication of writing abstractions to existing C code, 00:32:23.920 --> 00:32:31.860