It's Rust in Production, a podcast about companies who use Rust to shape the future of infrastructure. My name is Matthias Endler from corrode, and today we talk to Sebastian Scholz from Gama Space about controlling solar sails and satellites with Rust. Sebastian, thanks very much for taking the time for the interview today. Can you say a few words about yourself and about Gama? Yes, thank you very much for having me, Matthias. I'm very excited to be here. My name is Sebastian Scholz, and I'm an electronics and embedded system engineer at Gama Space. We are based in Paris, and I've been with Gama Space since January 2023. Our mission, our Gama Space's mission, is to develop and provide a propulsion technology called solar sails in space. And to do that, we are using rust in various places. And that's what I'm here to talk about. Now, not everyone might be familiar what a solar sail is. Can you describe the principle behind it? Yes. So the concept is actually rather simple. You know, when you have a sail on a boat, you have the wind blowing into it. In space, you don't have that, but you have the sun. And the photons of the sun provide some amount of pressure on a sail. So if you have reflective space, the reflections of the photon give you an impulse and so provide momentum to your sail. So basically, it's a way of moving in space, but without fuel and just relying on the sun. Wow. Basically, free propulsion. Basically, free propulsion. That's exactly it. The main advantage of it, of combustion propulsion, is that you don't need to bring your fuel with you. So, as you might know, it's still kind of hard to get mass into space. So, by saving all of this potential fuel mass, it's going to be easier and cheaper to go into space. And the trade-off is speed, I'm assuming, right? Yes, of course, there are some downsides compared to a normal combustion engine. The impulse, the specific impulse that you get is rather low. So that means the acceleration is extremely low, actually, in the area of a few newtons versus kilonewtons that you get from normal propulsion. So that means you have to plan your trajectories accordingly. But, I mean, the benefit is that this propulsion is infinite compared to the fuel, because at some point fuel runs out. But the sun, it's going to take a very long time for it to run out. Unless you're too far away from the sun. Yeah, but even if you're too far away from the sun, you still get some amount of propulsion. I mean, it's not like you're on an empty tank. You still have some amount of movability even if the sun is very low. But if you're on the outside of the solar system, you're going to have to move a bit slower. Yes, that's absolutely true. It's not a physics podcast, but at the same time, I'm always curious about space. It's a thing that's fascinated me since I was a child. And a lot of people, they requested an episode with you and with the project that you do, which we're here to talk about, because you also happen to use Rust. Yes, quite extensively, as a matter of fact. So for the solar sails, we have launched in January 2023. So right around the time when I joined the company, we've launched a satellite, a demonstration mission for this technology, the solar sails. And the onboard data handling system, our part of the satellite, is more or less completely written in rust. So we have a few sealers underneath the hood, but the main code and all of the supporting test equipment was written using Rust. Now, isn't it extremely risky to use Rust for such a mission? You don't really have that many tries anyway. It's not like redeploying a backend or so. Yes, there's one of the great dangers if you develop something for space. You get one shot with your satellites and if you mess it up it's not like you could just go up and try to reboot it you know you you really have only one shot but the mindset that we have at Gama is the mindset of the new space companies so we try to move fast and fail fail fast so we try a bunch of new things and if we do fail it's unfortunate of course but it's not a big deal That's kind of how we do learn. And so what we wanted to do with this mission, it's a demonstration mission. So we wanted to demonstrate our technology, but we also wanted to try out new things. And so using Rust as the main driver for our onboard data handling system was one of these decisions that we wanted to try out to see if it's going to be usable for the future as well. And before we talk about the onboard data handling system, what do you define as new space and what are maybe other companies in that realm? Yeah, so the main one that is probably known to everyone is SpaceX in that regard. And you can definitely see in the way they work that they like to fail fast as well. I mean, just look at the Starship launches. The development of this kind of rocket is extremely difficult, but they're doing it very fast and they're failing very fast as well. I mean, many of these launches end with spectacular explosions. And so the mentality is, yeah, OK, we're going to have to spend a lot of resources, but we're going to learn a lot of it as well. You're doing the failure so yeah that's that's just one way of moving forward versus in the past the kind of old space which is the big entities like nasa they want to do it once and want to do it correctly and i mean this mindset is also very important if you're doing like deep space missions as well where it takes a really long time to get to places for example i mean as well if you want to go to Mars. You kind of get one shot and if you don't do it correctly, you're going to have to wait a couple 10 or more years to get back. Okay, it's a test mission and yet it is foundational software that you're writing because you might as well just use some of the more established or some of the more common languages like C, C++ or maybe even ADA for the test mission for Gama Alpha but you chose rust why. So this mainly boils down to our software engineer at the time so this was before i started working at the company i i didn't mention but the Gama Space itself is only about five years old now so it was founded in the 2020s and we well We kind of move relatively fast and the team only grows relatively so. So right now we are, I think, 21 people in the company. And at that point, when we launched Alpha, it was more like 12. And most of our team is actually in mechanics because as you can imagine, I mean, the folding and creation of the sale is its own topic. It's one of the main things that we do. And so we only had one team electronics and embedded systems engineer the one that was responsible for the creation of of the on-board data handling system at the time which is a colleague of mine called Chris and his background was mainly in c plus plus and so he decided but he he knew that c plus plus was not well suited for well i mean it was difficult to use to get a reliable piece of software for the space environments. And so when he looked into options and other kind of languages to use, he decided that Rust would be a perfect fit for our case. He had a little bit of prior experiences in it as well. And so that's why we didn't choose Ada, for example. And how did he evaluate Rust? I know that you were not there, but maybe you talked to them about it. Why did they choose it? Yes. So basically, he did a trade-off of Rust versus other languages. And I mean, it came out clearly on top. The memory safety, just eradicating a whole bunch of errors that you can have with C and C++ is really critical for space applications. It's these kind of avoidable errors that can be caught before you deploy your code, that can just be denied by the compiler. And in general, the help that the compiler gives you compared to other languages in writing safe and good code, this I think was the main driver. And, of course, the ability to still go into the embedded systems level, still be real-time and be reliable. How can I imagine day-to-day at Gama? Do you use any special software that us normal, let's say, earthly Rust developers don't use? Or would you say, no, it's mostly the same components, it's mostly the same code structure, it's the same CI/CD platform like any other project? Yeah, actually, it's very, very similar. So in a bunch of companies, you would find setups with GitLab, for example, and so do we have it. We use a lot of crates that are just out there for many people to use, like ZeroCopy, for example, and other things, probe-rs for our debugging environment. It's a little bit more specific because we work with hardware. So in many software companies, you have your code, you have unit tests, you have integration tests that you run on your CI, and that's that. We have to interface with actual real hardware. And so maybe one of the key differences is that we have a flat-sat for the Gama Alpha product, so for theGama Alpha satellite. A flat-sat, I should say, is basically the electronics of the satellite unwrapped, laid out on a flat on a table, hence the name flat set. And it's a one-to-one, pretty much a one-to-one copy of the actual hardware of the satellite and we integrated it in our CI so that our tests, we have mainly integration tests and they run on the actual hardware and then we have probes attached to look at the actual output of the system and make sure that everything is all right. It's kind of cool that you use probe-rs especially because, All of the embedded, or I call it embedded, but like all of the low-level projects that I know of use it to some extent. Was it a natural choice for you? Did you evaluate anything else? Is there even anything else in that space? Or is probe-rs the de facto standard now? Yeah, it took us a while to get all of these systems up and running. And in the process of doing so, we did evaluate multiple things. But there was not a lot of choice for this sort of thing. We could have also just used regular C debuggers, as in just start a different process with the debugger and then interface with that. That was one of the other options that we looked at. I mean, that would basically have been the normal C approach, you know, like just use a GDB, for example, as a debugger. But the benefit of probe-rs was that, of course, it was written in Rust, and it allowed for a lot more fine-grained control of what we wanted to do with our integration tests. For example, we have a system to compile, test Rust binaries, and then upload them via probe-rs onto the actual hardware, and then look at the output and look for certain signals to make sure that the test that this binary represents passes. So that was very easy to do in Rust and using probe-rs. And how does that system look like? How does it work? What's the end-to-end development flow? So, So I'm guessing you develop code in your IDE. I'm not sure which IDE you use. Yeah, so we use VS Code. And so if we want to develop a new feature or work on some code, the typical workflow would be, so you write your code, you push it onto a new branch, and as soon as you push, our CI starts, and it reserves a slot on the flat side. The interesting thing is that we also use the flat side for manual debugging. So we have kind of a little bit of a system where you can reserve a slot on the flat set. During that time, the CI can't run and has to wait. But if the CI is running, you have to wait for a bit until it's finished. And so we kind of have a little server, which unsurprisingly is also written in Rust. Yes, that kind of manages the access to the flat set. And you wrote that yourself? Yeah, we wrote it ourselves. I think, oh, it's been a while, but I think it uses Hyper as a web server. I think it's called Hyper. Yeah. And basically, it manages access to it. At that time, so it has a web interface, and at that time, we wanted to kind of get fancy, and I have a little bit of a background in Flutter development as well. So the web interface is based on Flutter, but the actual logic and the serving of the files is in Rust. Oh, wow. that's a really cool project in and of itself because because i really wondered how would you how would you test your hardware components you kind of want to abstract that away you don't want to unplug and plug it back in again you kind of want to have a system for that and i'm assuming that there's not much on the market especially for flat sets like this it's probably a, Still an embedded setup, just like any other embedded setups. But the moment you go into the details here, you will find that you probably need a custom solution, right? Exactly. So this is one of the unfortunate truth. If you do develop your own system, there's no one solution fits all kind of a thing. So we did try to find generic solutions for embedded testing. And I don't know how it looks right now, but a couple of years ago, two and a half years ago, So there wasn't really a lot that helped you write actual tests for the embedded target and run them. So yes, this solution of the integration tests and the test suite itself is also a custom solution that we developed in Rust as well. So the whole integration test suite is also written in Rust and developed in-house. But the integration tests are still in a tests folder, just like you would expect from any normal Rust project, right? Yes, pretty much. so basically it's a binary that has all of the tests in it and then we just upload this binary to the hardware and then let it run and we look at the output via communication protocol I think for that we use just UART very common low-level communication protocol. Now earlier you mentioned correctness in a different context but I think it fits in here as well because couldn't you test a lot of the logic with unit tests? Or is it so that you kind of want to always test the inputs and outputs as well because it's kind of a monolithic service? Yeah, we have some unit tests and for smaller functionalities we do use those to test. The theme that we have in our code is we have one, big project but it's it's built up in smaller crates so the big kind of onboard data handling system software uses a lot of smaller crates that we also develop and the smaller crates they have unit tests for their functionality but for the big onboard data handling system it's harder to test these things because they're very much hardware specific so for example if i want to test that our persistence works so we are able to store data in some sort of flash memory i can't just write a unit test for it because i need the actual hardware i sure i could try to emulate it somehow, but that's a bunch of extra work and i'm not guaranteed that it is going to actually work on the hardware itself so in an embedded environment you you kind of can't, avoid doing integration tests because you need to test on the hardware so you're just going to test the whole system as a whole yeah. Well when you say that you store files on flash in a normal scenario it wouldn't be a problem if the flash storage fails you can't just you know fix it because it's maybe right next to you but it's harder to do in space do you have any failover for that sort of hardware failure? And if so, how do you test that? Yeah. So in fact, all of our flash memory, first of all, it's a very specific chip usually that has some built-in error detection and correction. So on a hardware level, we are protected against it. But on top of that, for specific parts of code that is stored on this flash, because that's the important thing. Code needs to be stored somewhere. And so if that code gets corrupted, you're going to be in big trouble. So one of the steps that we do in our bootloaders is that we check this code for errors. For example, for bit flips. In space, you have radiation. And so just random bits in your data and your code in that instance can just randomly fit, which can mess up the entire code flow. So we have steps. We have code written to verify that the stored code that we have in our flash memory is correct and that can potentially even recover from errors. So there's something called a hemming code, which means you basically reserve some extra space for redundant bits and you can use those bits later to detect and to recover from flipped bits. And this kind of code we have integration tests that uses prop.is to manually write into the flash into memory even and to flip some bits, And then we'll let it boot, see that it handles these kind of corruptions that we introduce correctly and fixes itself, yes. And that's also a custom solution? Yes. I mean, this is one of the tests that we run during our integration tests. So, yes, the integration test suite has a component that allows it to write into this flash memory. And that is developed by us, and that's a custom solution. Indeed let's talk a little bit about your background where you came from what your other strong languages were and how you got onboarded on the project and your first experiences with the code base a lot of people might be curious about you know exactly this process of working for quote-unquote a space company so. During my studies when i joined I joined a lot of projects that do things in space. So for example, one of the cool things that we did back then was something called Bexus and Rexus, which is a European project for students that allow you to do some sort of project, some sort of scientific mission on a small suborbital rocket or weather balloon that they launched from Sweden. And so that's kind of where I got exposed to my first real mission Bayes mission experience using those projects and we used a bunch of different languages though mainly C and Python and I must admit before I joined Gama and in my university days I have never heard of or used Rust so I'm kind of a late joiner in that regard but after finishing my studies I joined Gama then in January 2023 and that's The point when I started to learn about Rust and when I kind of fell in love with it because of the security guarantees that it gives you. You mentioned earlier that all of the stuff that we have talked before must have been in place before. And that's actually not the case. This is one of the reasons I got onboarded because at the time it was literally just Chris responsible for the software. And so we really needed more support in the electronics and in the software department, embedded software department. And so that's where I came in. Although I couldn't directly start working on the onboard data handling system. Well, I did some small contributions here and there. But when I started, I got assigned to a kind of a side project. So you have to imagine the satellite has just launched a few days ago. I come into the company, I get used to everyone, everything, meet everyone, and then, ah, okay, we have kind of a validation of our satellite. We want to do a zero-g flight. So that is a flight where the plane. Does a parable so it goes up and then it flies in a way so that you you cancel out gravity so you for a few seconds like 30 seconds you are weightless and we wanted to use that and develop a system to have a small scale solar sail deployed and to see how it how it behaves in COG because we they had simulations at the time but there was no practical experience and so we wanted to we wanted to see if our simulations actually match what we see in real life and so i got assigned to build this specific project and the code for it and of course because we were using rust this was also done in rust so they it's kind of it was kind of a relative simple thing we just had three motors and we were controlling a a disk inside of a box and on the disk so we were able to spin the disk and then release something that released our small-scale sail. And the main part of this was just to do this control, kind of account for vibrations of the plane because those kind of disturb the data and then record the data, of course. There was a lot of data coming in because we wanted to record as much as possible from this experience. We had cameras that captured the entire thing, but we also had various sensors, accelerators, and stuff like that. So that was kind of my first... My my foot into into the company and that's kind of also still what we do today if we if somebody else joins the company they're gonna have to get used to the way we work around here they're gonna have to get used to rust potentially and so they they usually get a small project on the site where they can they can work on it's still important to us of course and then they they kind of join the main force, like that and we have a bunch of back and forth where we because you're learning the way of how to write code not only for Rust but also for space there's a bunch of things that we, how to's that we have in the company that we've gathered over the years on how to write Rust safely as possible because unfortunately, it's still possible to screw up in Rust. Is it? Yes. So, yeah, one of the main things in the beginning, I thought, ah, it's so cool. Like, I can't get wrong with this. If my code compiles, it's perfect. And it's true in most of the cases, but some logic bugs can still sneak in. Mostly if you don't write your code in a way that prevents them because that's one of the cool things about Rust and what we use extensively here as well, if you write your code, Using the strong type guarantees, you can make it so that any invariants that need to hold true, so any preconditions that your code has and that need to be true for it to work correctly, you can express that in the type system. Maybe not all of them, but most of them. And so, for example, we use that extensively for configuration. We have things, we have code that can only run once the hardware is initialized correctly. And so we pass around serial-sized tokens, types that can only get created in one place in the system, and that's after initialization of that specific hardware. And so we use, for example, this sort of a token in other places to prove, yes, this code will only ever run after initializing that specific hardware. Couldn't you also use a type state pattern for that? Well if you do that type state pattern you kind of, You kind of make code dependent. You encapsulate the code. Not encapsulate. You make it dependent on each other a lot more, I feel like. So instead of writing a huge state machine, basically, like that, you can write your code more freely, just at any given structure that you want, just passing around these types of keys. Yeah, I never heard about that types with keys pattern, the token pattern that you described. Is that a thing that you invented, or does it come from any other project? Well, it's definitely used in the embedded world a lot. Maybe not in the same kind of way, but we've got the idea from crates like Embassy and actually various hardware abstraction layer crates that if you, in the beginning, they give you one struct that contains all of the peripherals but it's just zero-sized types for these peripherals and the idea behind that is that you can then when you want to use one of these peripherals you have to consume that type. And that prevents you from using the same peripheral twice, for example pins if you think about a microcontroller it has pins that you can toggle on or off and if you decided at one point yes okay i'm gonna use this pin to i don't know to talk to a motor for example and then later you think okay right i have a i have another use for a pin maybe i don't know i want to toggle an led or something let me see ah maybe pin one is still available i'm gonna try to use it ah no i've already used this before and rust tells you this because you moved the type you moved the variable into the function that used it and so you can't use it later and so that's that's kind of the the same idea. You have a token for each peripheral, for each pin that you can only use once. And you This is one of the things I really love about Rust. It's the ability to detect these kind of configuration errors at compile time. I mean, the compiler literally prevents you from writing codes that is wrongly configured that would be valid code just from a processor perspective. I mean, it's okay if you use the same pin for two different things. You can write to the registers and it's not going to care. But the end result is that the pin is not going to do what you want because you're using it for two things at the same time. And by expressing this notion of only one part of your program can use this specific thing via a token, the compiler can ensure that this, in fact, is the case. Now I understand part about loosely coupled components as well because I'm not sure, but I guess Embassy used the type state pattern and a lot of generics for the pins before. I'm not sure if they dialed it down a bit or dialed it back because, sometimes want to repurpose pins and if you set up your system once it's not as flexible as if you were to be able to you know change that dynamically sort of and at the same time you just use the normal ownership rules in rust to enforce the same behavior so you can still encode it in the type system without really having a lot of overhead and a lot of complexity on the on the type system level. Exactly yes we as i said we use this in a bunch of places also for example for wrapping c libraries it's kind of an interesting thing we during our Gama Alpha project we used a communication protocol called CSP and unfortunately at the time there was no pure rust implementation of it so we had no choice but to wrap well either implement the whole thing ourselves but that would have taken quite a bit of time which we didn't have or to wrap the the c library and the c library has a bunch of invariants when you use it you have one of them is you have to call a function called csp_init before you do anything else and so this kind of a thing once again we we have a series struct in rust called CSP it has an init function and it gives you is the only place where you can get an instance of this CSP struct and then the CSP struct allows you to call other functions on itself but you need a self so you need an instance of the struct and so that proves that you have initialized the CSP the underlying c library. Because that's sort of the only place where you can get that token. From there's only one place to get this token and in that place we initialized the library just before. So it's kind of a way of you have one place in your code where you know about these invariants and you only need to take, to express these invariants at that place so if i for example if i have function of this library and i know it needs to be initialized before all i need to do is i need to take a reference to to this token and that guarantees me yes the library has to be initialized and so as a user of this library which is also us but sometimes we can forget maybe you forget that you have to initialize the library before or you're just you know you're moving one line of code a few lines up by accident before the initialization because you think, ah, this might be, more efficient that way or something like that. You forget about these environments much too easy. But by encoding them into the type system and by encoding them into the function signatures, we can still enforce it and we can let the compiler help us in not forgetting about it. That's pretty impressive. At the risk of going a bit meta here, how do you come up with these patterns? Do you find them online in certain other crates? Or do you think really hard about those problems and then start to encode those invariants? It's not really discoverable. Yeah. So I think the way we discover them is by failing, by writing code that is invalid and where we then introduce certain bugs. And then later discovering, ah, yeah, okay, we need to prevent us from ever writing that code again. And so... For example, with the case of the library, it was literally the case that we called the function before we initialized it. And so that's how we got the idea of, okay, we need to prevent that. How do we prevent that? Let's use this token system. Invest. Do you write down those correctness guidelines somewhere? Well, we do have a bunch of rules like that, but the point is you only have to write them in code. Right so the once once you corrected that mistake once you made sure via the type system that it's impossible to use a function in an incorrect state you don't have to you don't have to write anything anymore because the compiler will enforce it. Well i guess you review a lot of rust code nowadays a lot of rust code that should or cannot fail, how do you review rust code for correctness what do you look out for what are some common patterns that you found useful here. Yeah for reviewing i like to to think about all possible states that a certain rust code can be in and all possible inputs it's kind of like like fuzzing you know where you but but manually fuzzing we we like to split up our our code and our metric as into small chunks and so it's relatively easy usually for us to to think about all the possible inputs and what can go wrong and then i like to make sure that we use the type system to its full extent and to encode any preconditions as i said into into the types. It's kind of like if you have a precondition for using a value that can't be zero, in Rust, you have the non-zero type. So Rust itself uses a lot of these types for preconditions as well. Same with the C string, a string that has to be zero, null byte terminated. So Rust itself gives us a lot of these ideas as well. And so when I examine code, I'm just trying to think of all of these different invariants that need to hold true for a specific code. I'm trying to find them out and make sure that they're expressed as much as possible. Are you a friend of debug assertions? Yes. Well, I mean, we use them extensively as well, but only in places where we can't guarantee something just with the type system. I mean, there's always certain things that you can't guarantee at compile time, especially if you're interacting with hardware, right? Some of these things you just can't do with just the type system. For example? Some of the runtime state that can't be known at compile time, we, for example, have our own heap. And so the idea of a heap is that you have a bunch of memory available and you at runtime allocate things out of that heap as needed. And you can never kind of really be sure how much you allocate just from looking at your code because it depends on timing. It depends on when certain pieces of code need certain amount of memories. And so for those kind of, So if we request a piece of memory, but we don't have any more, for example, like this, there has to be still debug assertions. And in fact, Rust itself brings a lot of debug assertions as well. Every time you access an array, for example, or a slice, you have debug assertions that make sure that you don't go over any of the limits in memory. And that's one of the strong systems of Rust as well. That's the memory safety all of these checks at runtime as well of course you can disable them but this is for us it's a tradeoff of size versus security and for the Gama Alpha project at least we were able to leave debug assertions on. Because that's another part of the space industry. Usually you work with hardware that is not like your modern laptops and computers, where you have basically unlimited memory and space. No, our programs need to fit in tiny flash memories. And so removing debug assertions can be one of the ways where you can have more actual code, but with a smaller size. And by tiny, you mean how large? Think like a couple hundred kilobytes oh wow yeah it kind of depends i mean for for the alpha satellite we had i think so the processor itself had an integrated flash memory of 256 kilobytes and then we had an external chip with i think an additional one megabyte of flash if i'm not mistaken so not a lot of space and some of the some of the space you need for for data as well because yeah in you talked about this in a previous episode you talked with i think ksat a provider of ground stations so you might be familiar already but the satellite doesn't have a direct communication to the ground all of the time we also need to store data on the satellite itself before we can pass it down to the ground stations. Okay. It turns out there's not a lot of space in space. You work on tiny hardware. And shout out to Vegard as well. It was an amazing episode too. Now, Rust has fallible allocations for vectors nowadays. But I'm guessing that you even want to avoid allocations in the first place, especially the dynamic allocations. And Oxide, for example, has a scheduler called Hubris and Steve, please correct me if I'm wrong here but Steve Klapnik, that is. I think what they did was they used static allocation, static memory and every payload that they run on that scheduler has a fixed size known at compile time which allows you to have predictable allocations. Do you use that pattern as well? if you can? So having a fixed struct and knowing that only this struct will need to be allocated, quote unquote, on the stack? Yes, as much as possible. So whenever we can use it, we try to avoid the allocations and just use stack variables. Or we have our own stack allocated statically and use elements out of that. But sometimes it's just not possible to know at compile time, what kind of, how much memory you need to use. For example, think about the telecommand stack. So for the Alpha Satellite, we were able to send telecommands to it. And on the satellite itself, you don't know how many telecommands will come in at a given time and how long it takes for each telecommand to be worked on. And so we had a cube. Of telecommands and this queue of course has an upper limit but the the memory out of it was kind of heap allocated you know because it was it was it had an upper limit but it was taken out of a big slab but in in all other cases where we can we really try to avoid using the heap i mean that's the no std environment for you you don't get heap for free you need to provide your own heap if you need it and so you're very very specific about the places that you that you need a heap and you try to avoid it in all other cases one thing that can help with that a lot is a crate called zero copy for example in our telecommand and telemetry system we use it very extensively so the the way it works is you you define your structs the data that you want to send, want to be able to send, and we use zero copy to convert them or let them be converted safely into bytes and from bytes without the use of an extra buffer, without the use of an extra allocation. Just the same amount of memory. It's basically a safer transmute. And so, because we need to send the data as bytes, obviously, so we need to do this conversion to and from bytes and zero copy is a great help for that sort of work if. I understand correctly that means you don't initialize the struct you have a view into this byte block in this block of bytes and zero copy kind of makes that safe for you. Yes in in a way so there's two ways of using it first of all the easy way is you already have a struct and you want to convert it into bytes and send those bytes over the network. ZeroCopy helps you with that. But then the other way around... You receive a bunch of bytes, and then you want to pass these bytes as a struct. And ZeroCopy makes this passing, this conversion from just a view of the bytes into a reference to the actual struct possible and save. By making sure that there's no invariants that can't be uphold. Or in newer versions, at the time they didn't have that, but in newer versions you have a try-variant of all of their functions, which allows to test for certain invariants. For example, if you want to receive a Boolean, a Boolean can either be 1 or 0, but a byte can be 0 to 255, right? So you have a bunch more values that normally shouldn't be allowed, and these need to be verified. And that's also one of the critical things. In C, I saw a bunch of code where you just convert from your bytes into integers and then you use those integers somewhere else and so it's very easy to mess up. With just not validating your inputs enough. And especially if you accept input over satellite connection, there's a bunch of bits that can flip. There's a bunch of stuff that can just go wrong. And so you really need to be sure to pass it correctly. And once again, we can use Rust's type system by making the constructors to all of these payload structs private and letting it just take bytes and doing all of this validation in this new function, in the constructor, and returning an option of ourself or a result of ourself. And so if any of the preconditions, they're just local, they're right next to the struct, so it's easy to verify that all of the preconditions that we need when converting from a byte string are checked for. And then afterwards, you're sure if you have this struct, its data is validated. Well, that's really nice. It's a nice example of making illegal state impossible to represent and parsing instead of validating. That's kind of cool. You said that zero copy is a bit like a safe transmute. Would that mean you can ditch transmute altogether and always use zero copy? Or are there any cases where zero copy just doesn't provide a safe interface? Actually, we have no instances of transmute. And it's on the list of things that we really try to avoid. And so far, we have not come across a use case that is not covered by a zero copy. So, of course, sometimes we have to convert between different types in Rust, and we use as for that. But those are safer conversions than just using transmute. And the fact that the type system guarantees no just random conversion between different types is really helpful to reason about these invariants in your code. So, for example, just think about in C, I can create any struct at any point in any time. In Rust, you need unsafe to do it. And it's a big kind of flag. If you see unsafe in the code, it helps you during the review because when you use unsafe, you always need to prove, at least with a comment, that certain invariants are holding up. And that makes it very easy to prove that your code can be safe, even if you're using unsafe. So unfortunately, we still have C libraries underneath, which are all unsafe. I mean, it's C code, so there's no help from the Rust compiler there. And so interfacing with those does require unsafe boundaries, unsafe blocks. But when we write our wrapper functions for these functions in the C libraries, we make sure that all of the unsafe free conditions that need to be met for this library to be called are met. And one of the examples is, as I said, the CSP library, where we make sure that it's initialized before we use any of the functions. Okay, you try to minimize unsafe code. Do you also try to minimize unwraps? Are there any better patterns that you use? Yeah. So unwraps, direct unwraps are also somewhat forbidden in our code. The best that we can do are expects, where you provide a message and a reasoning of why this certain thing can never fail. And there's still a few pain points in Rust with this, where you do need to unwrap certain things. For example, think about creating a non-zero value. You can do this unwrapping or this expecting in const code, which is very nice. So you can make it a compile time error. Yet still, it's unfortunate that certain things can't be expressed as you want them and need to be unwrapped. But we are constantly fighting that. For example, in cases where we do need a buffer for certain things and we know beforehand that we need a certain size, you can, Add certain trade bounds using the unstable const generic features that prevent you from needing to use Unwrap and then just giving you the actual value that you want without using a result or an option. And that's only possible because you let the compiler validate these preconditions at compile time. In my mind, you learned Rust in hardcore mode. it's in the space industry as a startup straight out of university with a python background and then also needing to ship features really early on you know working there what would you recommend others to do on how to learn rust effectively learning quickly learn it well how how would flatten the learning curve? Flattening the learning curve, that's a big kind of issue in Rust, I would say, compared to Python, for example. I think just trying things out and learning by example is the easiest way to do. Unfortunately, I don't have a fix, like a solution for everyone. But the thing is, struggling a bit with the compiler is a bit frustrating in the beginning, but it's a very valuable learning experience. So if you start out using Rust and you're having trouble with lifetimes, with borrowing, whatever, it is the compiler trying to help you. And once you realize this, once you understand that the structures and the ways you did it before are a little bit flawed in regard, once you understand this, it does help you. Unfortunately, it is a hard pill to swallow sometimes, but it's very useful. I, myself, after learning Rust this way, have come back to various Python projects and found out that code that I thought was perfectly fine, that I wrote many years ago, actually had subtle problems that could occur under some circumstances. Not all of the time, but sometimes. And so these, for example, if you have multiple threads and you access certain variables for multiple threads in Python, there's very little help for you to avoid... Accessing a variable at the same time as in fact there's none you need to do that yourself, and in rust you can let the compiler help you with that and initially of course it's it's harder because you need to prove that you're doing you're doing these things in a safe way but it does help you to avoid certain logic bugs that would be easily to would be easy to make otherwise. So see the compiler as an ally. Exactly Try to take a step back and take your ego out look at what the compiler is trying to tell you and try to improve. Actually that's one of the very valuable lessons Rust is amazing and one of the main things is the compiler helping you so much and once you understand that it's not you versus the compiler but it's you and the compiler versus bugs, then you have a much easier time moving forward, even if it takes a bit of time to get there, admittedly. 100% sure that there are people out there who would like to send in a job application right away or would like to work in that space. What are some next projects for Gama? Things that the team will be working on? Maybe also on the Rust side, what are the next challenges? What's coming up? Yeah, so if people are interested in joining Gama, we do have some job applications usually on our website, gamaspace.com so that's where people can find us. So right now the Gama Alpha satellite actually has been, the work on it has been finished for more than a year and we are moving forward with a different kind of sail, a so-called drag sail which is a is a way of de-orbiting a satellite so bringing a satellite into lower atmosphere where it can burn up so that it doesn't take up any space anymore in space because even though space is rust the space around earth is not and yes so there's there's a need for cleaning that up. Does that mean Rust does not have a garbage collector but you use it as a garbage In a way i guess although i must say so for this product that we are developing there and actually that has already been sold in q1 this year we decided that we needed a lot and like a real lot of reliability. So this is one of the main drivers of why you need to have this kind of a system. Your satellite is five years or 10 years in space. There's a lot of radiation, there's a lot of events and it can fail at any moment. And so our component needs to be as reliable as possible. And for that, we decided to have actually no code whatsoever on this product itself. But that doesn't mean that Rust isn't involved at all. In fact, all of our test equipment, we have a dedicated little microcontroller and setup for testing our equipment, is still 100% written in Rust. Or as much as possible. But the actual drag cell that we're developing right now is purely based in electronics. The version that we're doing right now in the future, so the version we're doing right now is for small satellites, but in the future, the next big project will likely be a larger version of it using a lot more complicated hardware, which will require once again software. And for that, we're going to be able to reuse most of the codes that we've written for the Alpha satellites, so all of the software stack, the different bootloaders that we have, the onboard data handling system, most of it can be rewritten, and so it can be repurposed, basically. The hardware is going to change a bit, but the. Because we split up everything into multiple crates, we have a crate specifically for drivers of the hardware. So that is the thing that mostly is going to change. But other logic on top of that can stay the same. In fact, that's one of the reasons why we split up the organization of our code into so many crates. We have, I think, 59 crates in total in our internal crate registry. Not all of them are directly related to the onboard data handling system. Some of them are also for the test equipment on ground. But splitting up the codes into well-defined little bits allows us to reuse most of them in space. It was kind of the idea from the very beginning. Traditionally, the final question is, what's your message to the Rust community? Well i think that rust itself is an amazing tool and the rust community is doing an amazing job in in providing it in in caring for it and also providing no std crates that's it's very important for for our purpose and i think for for those of you in the rust community that that provide these crates, let's try to work on letting the compiler help us as much as possible. So leaning into constgenerics, leaning into the ability of having these marker types, because anything that we can handle at compile time is something that doesn't fail at runtime. And failing at runtime in a mission-critical environment like space is never a good idea. So So if we can, let's try to work towards letting the compiler help us as much as possible. Sebastian, thanks a bunch for taking the time. Thank you very much. And with that, we're closing out Season 5 of Rust in Production. Thanks to all our amazing guests and thanks to you for listening. A special shout-out to Simon Brüggen, who's been editing the show since Episode 1. We'll be back soon with more production stories. In the meantime, if you're thinking about migrating to Rust, we'd love to help. Head over to corrode.dev to learn more. And hey if you enjoyed the show the best way to support us is to leave a review on your favorite podcast platform or share an episode with a colleague or friend it really makes a difference have a nice break we'll see you in season six and thanks for listening to Rust in Production.