COMPOSITES LOUNGE - Das Online Experten Netzwerk

Ilkay Özkisaoglu
Since 04/2021 149 Episoden

#138 Examining with Dr. Phil Gralla on how "Hardware in the Loop" significantly increases data analysis

29.08.2024 40 min Staffel 5 Episode 95

Zusammenfassung & Show Notes

Welcome to our two part episode where Composites Lounge invited Dr Dr. Phil Gralla of sensXPERT - Optimizing Plastics Manufacturing to explain how "Hardware in the Loop" (HIL) increases data analysis.

Ilkay Özkisaoglu found this a fascinating way to utilize data and if you are not familiar with HIL and how your manufacturing of plastics and composites can benefit from it, listen to this episode.

sensXPERT opened its doors end of April 2024 and it took a while to compile this live demo, because we wanted to create the best possible experience, know-how transfer and you to understand the practice of AI, more precisely ML applied to inmold plastics and composites components.

There are also some developments that are contactless, since requirements in trenchless technologies assume a test from a certain distance after installation.

A little more background on HIL to warm you up on this particular episodes, because these are truly technical:

What is hardware-in-the-loop (HIL)?

According to MathWorks.com "HIL (Hardware-in-the-loop) simulation is a technique for validating your control algorithm to be run on a specific target controller by creating a real-time virtual environment that represents the physical system to be controlled. HIL allows you to test the behavior of your control algorithms without physical prototypes.

How does HIL simulation work?

You create and simulate a real-time virtual implementation of physical components—such as a production plant with sensors and actuators—on a target computer.
You run the control algorithm on an embedded controller and run the model of the plant or environment in real time on a target computer connected to the controller. The embedded controller interacts with the plant model simulation through multiple I/O channels.
You refine software representations of your components and gradually replace parts of the system environment with the actual hardware components.

With this approach, HIL simulation can avoid expensive iterations in hardware manufacturing.

Where are HIL simulations used?

HIL simulations are particularly useful when testing your control algorithm on the real physical system would be expensive or dangerous. HIL simulations are commonly used in automotive, aerospace and defense, industrial automation and engineering to test embedded designs.

Examples of commonly used HIL simulations include:

Aerospace and defense: flight simulators and flight dynamics control where it would be too complex to test the control algorithm on the actual aircraft

Automotive: vehicle dynamics and control where it would not make sense to test functionality in the early stages on the road

Industrial automation: plant control testing when stopping production or assembly lines to test control algorithms would mean high resource costs and business losses" (Mathworks.com online accessed 23 Aug 2024, Link in the comments)

Transkript

Good morning. I'm here to have fun with sensXPERT here in Schwanthalerstraße in Munich. As you know, guys, I've been with Composites Lounge, #Composites360onTour my vlogging style. I've been at Hannover Messe this week. I've been at TechTextil. Then I had birthday. I'm 53 years old and today on the Saturday I have an expert here and I would like to interview him on what's going on with testing materials, particularly in plastics and composites. So and if you are watching now from the United States, my expert here is Doctor Phil. But it's not Doctor Phil, you know, from TV, who was the psychologist. This is Dr Phil Gralla and with Dr Phil Gralla we will dive now deep into analyzing systems and inspection systems. I'm here in the laboratory. We have not a polished environment here. This is a laboratory. And in the laboratory, as you know, you have a lot of devices, a lot of cables, a lot of electronics here. You have shelves where parts are into it. And I love it, because this is engineering, and engineering is the heart of our material science. So let's deep dive with Dr Phil Gralla. And now we are turning back to our inspection of materials here. And I'm really keen to see what the result is. -So the first one is our dielectric sensors because we are measuring in process and this one is an example. Well, actually it is not just an example. This is a real sensor you can see here. This is the sensor we are using. And for the lower frequencies, especially used for thermosets, this one will be integrated into the mold. So it needs contact to the material. That's a requirement we have. And for injection molding we would place this one directly in the mold as part of the mold. You can put it in later after you already finished the mold, but it needs to be physically altered. To demonstrate how it works and what we can see. We actually go to this one. This is a laboratory press. So this is not a real mold. It's so small. But we use this one for testing and analyzing. And we have. As we see before, we have our plate. We have the sensor. Now we have this ring we put on top of it. Need to close it a little. And into this kind of mold because it is, we can just put our material. So I will have some epoxy and mix it, put it inside. And then we do our isolation. We close. And then we start our program. It will go down, heat a little and apply some pressure. And we can see what actually happens with the material. And this is a very interesting part, because if you look we have also a DSC, a typical laboratory equipment for testing material. And this is kind of a golden standard. A lot of people use it for analyzing material beforehand. The problem is you always need to take out part of your product, of your final composite, grind it very small, and then you can analyze inside. So you cannot do it in process only after or before. And here we can actually see in the process. We have two tools. One is for controlling the press. So this is the first one I will actually start where we have here always our press. Not only do it right now. Later we will start it with a heat and go down. And since we switched to our new sensor, we actually don't see anything here. There's nothing connected. This is only for controlling and for checking what happens, we use this tool. It's a developing tool that we have for our newest kind of sensor. So this is something where I'm quite involved in gathering the data, displaying the data to make sense of what is coming out. And so this is a little tool we wrote in STL, pure STL in C++. We didn't use anything that was already done. And the reason is that we get a lot of data, especially when starting. So we have a analyzer board which can already decimate the data, which is good because otherwise we have to stream gigabyte per cycle. And when we have this one most of our graphics tools were not able to keep up. So we had to make some small adjustments. That's why we have this tool to actually be fast enough, gathering the data and displaying it with some ring buffers and mathematical tricks to make it fast enough to process all the data coming in and analyzing it. We have Fourier transformations actually on FPGAs directly to get rid of the noise to extract the most important data. And all this one is then being transferred in CAN interface. Why CAN? Well, our devices are also made to go for sewer rehabilitation. And in sewer rehabilitation, we have cable lengths of around 300m and temperature around 120 to 140. So we cannot work with a regular network cable. We can also not use any like Wi-Fi controlled elements. That's why we actually use this CAN technology, which for some people that are into cars might know. That's also how your components in the car communicate with each other. All right. So I mixed some epoxy here. No, actually resin. And we can fill something in. Okay. So we fill some of our material inside here. And we want to see the curing. So what is actually happening whiile it cures. So we'll close this one for a moment and we can focus on this machine. Then I will close this one. You can stay focused here. I will start the press. Okay, the room has to heat up a little so it's not inside. And now we can start. We already started our press. It's heating up and pressing the material down to see actually what happens with the material. Right now, we don't see anything. And that would be quite normal in your process as well. So what we can measure is of course some temperature if you have temperature sensors, the machine and the pressure itself. But this does not tell us directly what happens with the material. We can use the sensors that I showed you at the beginning and to see the viscosity of the material, actually the ion viscosity. You have a dielectric field applied with a frequency, and we check how our material, the ions, ionized and how they align. Like how do they respond to the frequency. And this tells us how the material is curing and the data is gathered live. So you actually have the possibility. The process also is 15 minutes. All right. So I showed an example about our press here - the laboratory press. So this is a very small device if compared to injection molding. And for the next step we actually want to have a look at the machines that we are using. our IPCs. So the idea from sensXPERT was and still is to bring technology and analysis from the lab to our real production and use it in production. So we can not just use the press that you have seen, but instead we have to integrate our technology into already existing processes. And to do that, we have our sensors. But the second part that is important is our edge device or we call it the edge device. The edge device consists of two parts. One is actually an analyzer. So it takes the analog signals and the signals from the different sensors to transform them to something we can use. And the second part is an IPC. An IPC is a computer basically without any screen which does all the processing. Now you might wonder if you say, we want to do something in process and we can show what happens. How does that work if there is no screen? And the trick is you can access this IPC on the local network. It has two network connections, one for local, one for online, if you want to upload data to the cloud, which I do recommend because then we can actually work with historical data. You can access this and then basically like a web page open it. And there you have all the information live. We will show that later so that you have an idea of what's happening. And for everyone who can actually see it, the lab here is small like it's for 4 or 5 people. It's one room, so there is no mold machine next to me. But what we have instead is a mini HIL and a NI-HIL. What are these? Well, HIL stands for a hardware in the loop. And this hardware in the loop we use to simulate the real machine, but by actually sending signals. That's why it's called hardware in the loop. So this one sends us trigger signals, sends us analog signals that simulate being a real machine. So our IPC and our analyzer has no idea that he is somewhere else. And this one is done for testing. So we can in here test different scenarios. We can also check if there is something happening at the customer that we replicate. What is it doing? And for some of you who might have been at a fair and seen measurements and have wondered that where this one is coming from, if it's live measurement, it's probably coming from one of our HILs. This little blue box here is our so-called IPC, which actually is two parts. On one side is an analyzer. On the other side we have our IPC and they come as a box combined, makes it easier for install. And our sensors are being connected to this channel one and two. So you can have two sensors to the same IPC. Either if you would like to have two measurements at the same part, or you might have other purposes, or you can also only use one of them. Then you see here I talked about it about before, two LANs. Why two? One for local access, if you do not want to have it in your regular network, and the other one we use to have connection to our cloud. We do use the cloud to upload data, to retrain our models, to monitor models, and also to give the option for every customer to get an overview about different installations or one installation over the time. How is it performing? How is it performing in comparison? You can also detect if things are out of order. So that is the purpose of cloud in comparison to what is happening on this IPC. And for our IPC that is already all. It's the box. You connect it and to see what happens inside, you will need to take a different computer. Maybe you have one in your network or a laptop, and then you can work on this one to see what happens. What I will do to show you what actually happens inside this box during a measurement, we have connected this one to one of our HILs. So it is simulating a production by sending signals or the box doesn't know, it's not a real production. For it, it is in a production and gets an analog signals. So it is really a hardware-made simulation. And it is gathering this information. And we can see live what happens. What can you see and what can you use. I have a question to you. The box here. There's a lot of technology inside it. But who would be responsible for this box at your customer. Is that the IT department? Is that a production engineer? Describe me: Who at your customer will be operating or setting this up and operating this. We have to distinguish between installation and operation. Prior to the installation, we have a couple of requirements that must be met. One is how is this one connected to a network and if it has internet access. Why is the internet access important? Well, first of all, only this way we can upload data reliable to the cloud. But second is, if there is something that you would like us to check, we can use internet access to directly access the box by ourself and to help you operate this one and to answer questions. Now during the production your IT should not be involved anymore. So everything is set up. We have a team from our application & services helping with the installation coming to make sure everything is set up correctly. When they have done their work and gave you an introduction, the person at the machine floor themself can operate this machine and the reason is that you do not have to do much. So there's only very few buttons. They're mostly online. Well, it's not really online looks online because it's in your web browser, but you are connected directly to this computer and you can check live your measurements, the relevant data that has been collected. You can see the status of your sensor, also of the IPC itself, and you will get predictions. And that is actually an interesting topic. Predictions. Now it's the mathematician in me coming out. Just gathering data is not enough. So we have to make sense of this data. And a lot of times just knowing what is happening inside your process helps you to define your process, or maybe to make sure that you are confident the parts of your producing are correct. They do not have any issues. That is good, but the other part is you can use it to detect something is out of order. Something is already not happening correctly to ideally have time to react to it. So you might see we need more time or more energy to have some more heat. And the second part is if you produce parts, usually you like to bake it a little bit longer just to make sure. And how would it be if you now - actually, we already succeeded our goal of a certain degree of cure or a certain TG and we can stop, we can open our mold now and already continue. So this is something where we say it's a cycle time reduction. So you can use this technology to reduce the cycle and open the mold when your part is already ready. That sounds nice. How long time do we have to react? We can send a signal trigger ourselves when it is reached and you are done. Some processes are not alone. You do something secondary next steps. So the earlier you know that you open the mold in maybe a few minutes, depending on the process, or even if you have a long process in an hour, you have time to set up everything else. This is where prediction is important. So we use machine learning models to predict the degree of cure what is currently the cure, but also to tell you how your degree of cure will look in five minutes and ten minutes and 40 minutes. Depending on your process, we make predictions into the future where you can see how your process is behaving and when it will be ready, when your part is done or as done as you defined, and be ready for the next step. And this is the beauty of this predictions, where now you have the possibility to really not just open something, but actually react in time and also be prepared what is happening at which time points. I'm here to have some deep dive into deep learning, decision trees and algorithms. A world that I'm not necessarily very iterate on, not skillful on. And with Dr Phil, I had very inspiring talks the past days about mathematics, algorithms, deep learning, deep machine learning. And Phil, I would like to ask you to explain our community before we dive into your laboratory equipment here. Because, community, you have to know we have set up a few things and the machines are working right now. Crystallization. Hardening of the composites takes place within 15 minutes. And we thought we use the time to talk about algorithms and machine learning. Now, Phil, one of the questions is I'm reading everywhere now machine learning, deep learning, everything is machine learning, deep learning, neural networks. Tell us and give us some some insights from an expert perspective: Is everything that we are presented with deep learning really deep learning? Or is it that people maybe do some simple decision trees and just label it AI? Yes, that's a very good topic. So, most likely not everything that you hear deep learning is actually deep learning. And we have a saying here in the company from us, we say everything is statistics. And that's also the most important part about this one. If we look into how do machines make decisions. And that's at the core, what we're looking at is statistics. At the at the end, it is statistics. And we have different way to do it. We have very classical things like decision trees where information is taken and based on this information you either go A or B and then you keep going. You have this one decision okay. What is the next data? You do again. And this is already some kind of AI but it is not deep learned. You might make simple steps deep learning if you want. The other part is support vector machines here for over 20 years being used a lot still and they're not as fancy as deep learning, but they have the advantages for not using as much data. And that's also a point where, don't get me wrong, deep learning is very exciting, especially for a mathematician. If I look at ChatGPT, if I look at image generation, it is cool. I really like it, but it might not be what we need, especially not if we are working with data that's way different. If we look at our cycle data, if we look at material data, you cannot go online on GitHub and just download a sample of a million annotated data and start training. You most likely have to do it yourself. And then if I go to a customer, be it like aviation and tell you you have to produce 100 wings for a first start, and I need half a million to actually train a model, I'm pretty sure I'm not going to sell you this algorithm anymore. So there is something where it is a lot of times a good decision to actually take a step back and look at the whole toolset that we have. And there we go back to regression models. We have auto regression models with random things. That's something we are using, especially if we have little data for analyzing and making predictions in the future. We have the typical decision trees. We have ensembles where we use a lot of different machine learning algorithms that are all good in their own way, and together they form a good decision. You can compare that if you are getting sick and you not only going to one doctor, but you have a general doctor and he says you go better to heart doctor. And then you go to the heart doctor and he checks and he's like, yes, but we need someone to do the surgery. So you have to go the next step to a surgery doctor. And these techniques that we can use in the same way, where we have a more general model that points us into the direction which one to take next. None of them is perfect, but the combination together makes it. And that's something where we a lot of times look at it and also why we make our own models, because this is the approach we take. We look into the material, into the information we have, and then choose models that actually are appropriate for our application and also feed the requirements. And that is something always keep in mind we do not need a model that can tell us and translate languages, because that's not what we're doing. We do not need a model that can make images. We are not doing images. We need a model that understands composites, thermoplastics and thermosets. And it can only do this one. I'm happy it does not have to talk with me. It only has to fulfill a job. And that's something to always keep in mind. It's very nice to have a hype and it makes my work much easier talking to customers if they come to me requesting models. But don't be surprised if it is not deep learning but some other technique instead. -Dr Phil, now, the composites industry and the plastics industry are very, very conservative industries. With conservative, I mean they have capital expenditures, CapEx, investments in plants and operations and machines. It could be - I'm just just throwing this out out of my own observations- it could be that engineers that are operating these machines, setting up these machines, they are looking for a fast output, higher output, that they are a bit scared, challenged with all these new forms of... I will call it support. Yeah, because it will not take over the job then and make the job better for the human being. It will support the operations. What would be your one tip for people that are scared about applying AI, deep learning or even simple decision trees in their manufacturing? Okay, so one thing is, like you already pointed out, it is a tool to help you, and you probably have used tools that are much closer to machine learning than you expected beforehand. Computers do it all the time, making decisions even though they don't use deep learning. And if you have something like a PID controller inside that is already mathematics, making decisions for you to have this control of your machine. And we are comfortable with this, and I think that is something that to keep in mind, your machine is already doing this and we just apply it now on more steps. And the first thing you should have is get rid of this idea that machine learning/AI has to be like a human. It doesn't have to be. And if you get rid of this idea that we are trying to replicate a human, but instead take it as another tool, another step, like a PID controller. I think it makes it much easier to feel comfortable with what is happening, and also understand why decisions are not made the same way as I as a human would make it. But it is helping you to control a very difficult process. -That's a really a great conclusion. It's the mindset of the engineers has probably to adapt and understand we are not replacing any workforce, but we are supporting a more efficient production. And this is all what sensXPERT is about process optimization. -And it helps you understand the process. Because even if you have 20 sensors and you see exactly what's happening, it's overwhelming. It's something that I noticed before when people were asking, why do I take so long with analyzing some data and make some nice plots? And I said, because I have to extract data, if you look at what we are collecting, we have 100 graphs, 100 plots for each cycle. No human can understand that easily anymore. We have to make it relevant, to make it understandable. And this is already a tool which we use without realizing how much mathematics, how much of the machine learning is already behind it, to just make sense of all the information that is being gathered. -So, Dr Phil, I have one final question on this small excursion, which I thank you a lot for highlighting this. The last question I'm having for this part is, I'm assuming, just assuming, seeing, walking around the manufacturing processes of our customers and inspection. You know, they invite me to look at the production and I have a lot of insights and I'm seeing everywhere machine control rooms. I guess there's a lot of data today already available. Now, if you think that all this data is available and nothing happens with it, tell us what is the chance or the opportunity that is missed by not utilizing this data? That's a very good question. And I do have an example about this one, which is actually not related to sensXPERT, but work I'm doing. And when I was a student, we had a company we were working with and they were asking for: We need a new robot, which takes very heavy parts from one machine to the next. And they had all these information already when machines are done, when they're ready, and no one actually had a clear look at it. So we started looking into it, and the first conclusion was like, you do not need the second robot if you wanted one, but actually a lot of your machines are stopping - not doing something. So by just more efficiently sorting where the robot is going and where he's putting stuff, we already had a 40% increase of production. And then you could talk about let's add some robots. But that's something where we realize that the efficiency sometimes it's already there, but it just gets lost. No one has a whole picture about the whole process. Also, here at sensXPERT, we look at only the curing of the material, but there is more steps involved coming afterwards. So if you want to have a whole process optimization, this is only one step. And understanding how machines feed into each other, understanding early enough when something is happening, like maybe a part that you have is not perfectly excellent, there's some mis functioning. Maybe it breaks early, maybe it didn't bind correctly or it didn't get hard enough. All of these things can have many causes, and some of the stuff you might already monitor without realizing it. So it can help to make use of all these collected data by applying proper tools. -Wonderful, a nice conclusion. Thank you so much for listening to our short episode. And now we are turning back to our inspection of materials here, and I'm really keen to see what the result is here. You actually have a look at my own working laptop, but what we did is: I connected via remote onto the network where our blue box is right now. So this is the box I showed before. We have our IPC, the blue box, and it's connected to one of our HILs. And here we can now see what happens while we have our measurements by the box is running what you can actually see. And this application that is running in your web browser when you connect to the IPC and we always see our signals. So what is coming. So now it just switched to a new cycle. And it will gather something. We see temperatures. We can also see the ion viscosity here called a master curve. You have to know when we do the measurements. It's not just one frequency we're measuring. We measure a lot of frequencies and we only display the one relevant. Interesting here we can now see our temperature how that is starting in the process which material you have it will change. The sensor for the temperature is actually inside our own sensor as well. So this is additional information that we capture. The ion viscosity. And what will happen is when this one keeps running we take the data the live data and also make a prediction. At some point, you will see that here will be a prediction telling you how it will look like into the future. And we have information about the current measurements, what is being measured on which channel. You can select different measurement methods if you want to select which frequencies you have, what you're looking for. We have our critical points. So this is something if you do not want to have the whole timeline, it's certain points in the timeline that are of value. That's called the critical points. The minimum. You have maximums. You have the change in the in your derivative. All this information. And then also here is of course this prediction where we can see when something ends. This is how you should look like if everything is set up. But you have the options to log in and actually work on all the information. I can do this one here. It's already saved. So you can make changes. You can see about your hardware. We have overviews which hardware is being used. You can make setups. You can do calibrations. We have for the measurements that we can see what is available and which signals do we want to catch. We can also set up for our measurement methods. All these kinds of stuff can be done. Why would you do this one? Well, you might have more than one process. So if you use this one to manufacture different parts, then you can do use different methods. If you have different materials, then it makes sense to adjust the frequencies we're looking at. If you have some are more for lower frequencies, other ones go a little bit better with higher frequencies. That's also where we can help which frequencies make sense, which one you should measure. It depends on the material, on the cycle time. And that is two main reasons why you would change the frequencies. For this part, you might notice that I do not start any measurements. So here we can already see a change to our laboratory equipment - that this does not need to be done manually but instead we actually get signals from the machine where we get information about opening, closing, start of a cycle. So this one will react accordingly and by itself. Then start measurements, save measurements, upload the measurements when a cycle ends. There is the option to also trigger an end of a measurement. We call this one an open mold command, because for most application it means stop and open the mold and this one is implemented so you can even send a signal if you define beforehand what criteria you need to match to actually send this one. I would like to show one more thing, and I think we have to switch to the other side. All right. So this is it from my side. We showed some of the machine. We had to look at our IPC. You could see some of the real data being captured. And I have one more thing, because when you were carefully listening, you might have noticed I was talking about frequency, frequency bands and also being in contact. Most of these things apply for thermosets. That is where we are coming from. That is the main focus we have. Our cycle times are long enough that we can go with this low frequency. And the question is what if you work with thermoplastics. Two things: So one part is we are too slow, right? And we might not see enough. But the good part is and that's what I would like to show, is we do have our standard sensor that I showed already. And this one is actually modified. And this is modified in a way that we use high frequency or to be more precise, radio frequency. So we can go into the gigahertz frequency spectrum, which allows us to work with processes that are much faster. That is especially important for working with the thermoplastics. The other part is what it allows us. And that's the second thing I would show, is what if you have a process that actually cannot be measured in contact, maybe you have extrusion, maybe you have some kind of rehabilitation of sewage systems. There you cannot get in contact. And this is where we are working with a new sensor we are developing. It's also based on this radio frequency. And the neat part about this one is we do not need to be in contact anymore. We can have air in between so we can see it looks a little bit different. Right? You still have the circular, but it's two parts and this one can measure with a distance. Air can be in between. And we can now start measuring different kind of processes which do not allow to be in contact for our sensor with the material in the production. This one looks very big. There's different versions of it under development. We do have first customers using it already, and I'm very excited that this one is progressing the way it does and hopefully soon we are a lot more out of our thermostets, but also in the thermoplastics. So while inspecting the lab and showing Ilkay what we have here, he came across this little device and was wondering what it is for. He guessed correctly that it is sucking air and does something with it. We call it our elephant, and when here in the lab we work with different materials to make sure that we have the safety, that no one gets poisoned. We use this elephant that can move around. You can adjust the size, go up, down and work with hazardous material without getting in harm. Don't be surprised. I will not do this one. I do some of the machines, but most times I hide in my own room on the other side and work on the computer. So when walking through the lab, we have seen on this wall some nice pattern and we were just discussing what this one is. This one actually is a real measurement from an epoxy. And here you can see in red the temperature and in the other colors we can see the frequencies and ion viscosity. And we are very proud of this one as it is looking nice and gives us a feeling every morning to show what we are working with. On the other side, right at the beginning, there is even a y-axis, which gives you some indication that this is a real measurement. And indeed it is, so we are very happy to have it here. Okay, so now measurements. Now the time is over. We also see in the measurement that nothing is curing anymore. So we can open up our mold. Let's go until it's completely open. So when it is open not moving anymore, then we can actually remove our safety glass. Okay. It should unlock. And there we are. Good. Now let's put it all apart and we can already, well I can see it already that it is cured. And when I find the tool. It must be somewhere. There it is. Now we can unscrew it so that we can also see that it actually did cure. And is it hot? Yeah, that does feel like it.