Impact of Digital Transformation and AI in Space Systems Engineering

Today we'll take a look at how digital engineering including careful and intelligent use of AI can help provide an answer to this problem. This presentation explores the confluence or the merging of systems engineering and digital engineering in the space industry. First I'll do my best to provide a working definition of what systems engineering is and what good systems engineering looks like so that we can provide a context for how digital engineering can help move systems engineering towards good and keep it there.

Next we'll take a dive into model-based systems engineering which has exploded in popularity over the last 10 years or so in the space industry. And while model-based systems engineering or MBSE, has definitely benefited many companies in the industry, there is an emerging push towards an even more complete realization of digital engineering. Sometimes this is synonymous with what companies are calling digital transformation. So we'll talk about how this fully realized digital engineering vision is different from model-based systems engineering alone and what that vision can help accomplish. And finally, no digital engineering talk today is complete without tackling and mentioning the role of artificial intelligence. AI is slowly but surely creeping into the space industry and we'll take a look at how and where it's being used, as well as some of the directions of future research around the world.

So let's start by asking what makes systems engineering, systems engineering? Throughout my career in industry I did many job roles from test and integration, to mission operations, to algorithm design, to being an autonomy architect at one point, and I even had one job where my job title was actually systems engineer but I described myself as a systems engineer for all of these roles. Now why is that? Well simple, all of those jobs have some aspect of needing to consider the interaction of multiple subsystems.

Let me give you a simple example of this type of interaction that helps describe the fundamental essence of systems engineering. So on the screen you see a satellite and on this satellite there is a valve. Now the valve of course, naturally is a part of the mechanical subsystem. However this particular valve actuates a thruster which also makes it a very, very key part of the propulsion subsystem. The means of actuation is through electrical power, which also makes it a part of the electrical power subsystem. In order to actuate the valve and read its position back, you need commands and data which makes it part of the command and data handling subsystem. Because thrusters are a critical function onboard the spacecraft, this also means the valve is a part of the safety domain and of course for it all to work properly in orbit, it needs to be part of operations as well.

Now it's the job of systems engineering to keep all of these stakeholders aligned in the same way and make sure that when that valve gets to orbit it actually performs its intended function. Now while systems engineering operates across many subsystems it also has a huge impact across time, across the entire program lifecycle. Systems engineers of course, provide technical leadership on a program. I would argue that just about any chief engineer regardless of their domain expertise is at heart a systems engineer.

Systems engineers define and refine the operational concept, what the mission will actually do, they then translate that into requirements that can actually be formulated into a system. They allocate those requirements to components by defining architectures and interfaces between the components. They lead trade studies to optimize the design and make decisions about which hardware and software is selected. They're responsible for performing integration and test, which is a huge part of the development lifecycle. And finally, they operate the vehicle. Now they're also responsible for managing risk across the program, which means they're often blamed when things don't go right and of course also for making sure that all the documentation is done properly to meet whatever qualification or certification requirements the program might have.

Now while systems engineering oversight is broad, unfortunately the current state of engineering in the space systems is that much of the subsystems engineering is highly siloed. And we recognize this, earlier in my career, I had the privilege to work on a small research and development program. Where we were working to develop a new satellite and the management of that program recognized that it was a real issue traditionally to have these different subsystems engineers talk to each other. So what they did is they put the whole team, all the different subsystems in the same very large room hoping that this would spur discussion and working together.

I was working as an autonomy person, the person next to me was drawing CAD drawings, he was a mechanical designer. But even in this very close, physically close environment, I found that people still kind of put their heads down and did their own work. Some of these things, these traditional ways of doing things are very difficult to break. And of course this also impacts how effectively we can do systems engineering because good systems engineering requires a concept called systems thinking. And systems thinking requires breaking down the silos.

Systems thinking considers the interconnectedness of the components in the system. And a fundamental concept of systems thinking is that a component can behave differently as part of a system than it does on its own. On its own the valve that I mentioned earlier opens and closes. On the spacecraft, it moves the vehicle. Now that's a fairly obvious example.

Imagine an antenna like this one, you may think that this is purely a communications device but I will argue if you deploy this on a spacecraft it will also move the vehicle. If you're in low Earth orbit, this antenna will impart drag and that drag will cause a torque that will try to overturn the spacecraft. Even if you're in geosynchronous orbit, even though the effect is lesser, you'll feel a solar torque on this antenna and these effects might be small but somebody needs to think about them and somebody needs to account for them. And those are the system thinkers.

System thinkers consider the whole instead of the parts. They'll consider relationships rather than components. Rather than being focused on analysis I would argue that systems thinkers really perform synthesis, they look at multiple analyses and understand how they work together as part of a larger analysis.

Systems engineers are very concerned about causal loops in the system that can appear in otherwise very disconnected subsystems. The relationship between a communications antenna and the guidance navigation and control subsystem that I mentioned earlier, could be one example of this type of causal loop. And systems thinkers look at networks instead of hierarchies.

So the antenna, valve example showed how one component affects multiple subsystems. Let's look at a slightly different angle and look at another example of a systems thinking problem. Now back when I worked on the International Space Station, it actually looked like this. It was 20 years ago. So the Space Station was much smaller and because it was smaller, we had the luxury of being able to fly multiple attitudes on ISS in those early phases. So that of course, leads to the question what attitude should we fly?

Now to solve that problem we had to consider many different things. First and foremost, of course is crew safety. So how does this factor in? Well there is a desire to minimize the exposure of the pressurized volume cross section to the velocity vector to minimize the chance of an orbital debris strike striking the inhabited volume. So that's one driver. There is also a desire to minimize drag in order to minimize propellant use through both orbital decay and momentum accumulation. Now of course, we have to stay power positive, the angle of the solar rays to the sun needs to be optimized or everything will fall apart.

The attitude has to be certainly acceptable. We have to make sure that components hanging off the body don't get too hot or too cold. We have to make sure that the metal doesn't contract or expand, if the gradients are too large that could be a problem. And of course, we have to maintain communications. So we have to point the communications antennas at either the TDRSS communication satellite network or ground stations during ground passes. Now all of these trade-offs are interconnected. This type of problem can't be solved by a specialist, it requires systems engineers who are systems thinkers who can incorporate all of the input from the specialists.

Now systems thinking in my opinion, is the most critical attribute of systems engineering and it's being stifled by our current and document intensive processes. Now these processes have of course evolved over time. And there's a very, very good reason for them. Many of the processes followed around the world trace their roots back to accidents and failure investigations, where it was discovered that the process was lacking or we were missing interface documentation. And so we kept adding the results of these misses to the process to end up with what we have today. And it can be quite heavy weight this process and again, I say for good reason. And this is especially true of organizations that have been around a long time and have a long history and might have some painful lessons. And around this process and this documentation we've also defined systems engineering constructs like gated processes and change control boards, that all do good things and they're necessary, but they're increasingly challenging to get right.

And systems engineering escapes still exist today. You might remember the Starliner mission NASA commercial crew mission from December, 2019. This was the Boeing mission, not the SpaceX one. The mission failed to meet its objectives because the Starliner capsule ended up in an incorrect orbit and it was unable to dock with the Space Station. This cost the program over $400 million and over a one year schedule slip. So the impact is huge.

Now most of the press and the media represent what happened as a software error. The root cause was traced back to an incorrect mission time onboard the Starliner. It was actually 11 hours off of what it should have been and of course the spacecraft mode and phase logic obeyed the mission time, configured the spacecraft into the incorrect mission phase, configured the thrusters for a high bandwidth control that quickly depleted fuel to the point that the rendezvous with Space Station was no longer possible.

Now the mission elapsed timer may have a home in software but I would argue that characterizing this as a software issue is false. And in fact, when I scoured all of the different news reports and articles I found one in which the program manager admitted that there was a missing requirement to initialize the mission elapsed timer from the Atlas 5 launch vehicle at the correct time. Now if this is true, it's clearly a systems engineering error.

Further the error could have been discovered if there had been an end-to-end test of the full launch countdown sequence and ascent run together. In fact, the program chose to run a launch countdown test that ended at T minus 0 and a separate test for the ascent phase that began at T minus 0. So because these tests were decoupled, the mismatch in the timer didn't carry over from one sequence to the other.

Now I'd argue that it's a great success and a great tribute to the team at the mission control center in Houston that Starliner was recovered safely. And in fact that brings us to the flip side of incidents like this one, systems engineers have a long history of using systems thinking to solve really, really hard problems. In 1970, this was before my time, the number one oxygen tank of the Apollo 13 service module ruptured due to an explosion.

What happened next I would highly recommend watching the movie Apollo 13 if you have not already, back in the late 1990s one of my lab partners worked on the Apollo program. He claims the movie is actually fairly accurate, but long story short, the engineers did some very creative things and Eugene Kranz, who was the lead flight director on that mission shown on the picture there, he held the opinion that the early days of Mission Control created some of the finest systems engineers in the world and it's hard to argue with. And one key thing to point out is that these engineers had to access all of these different engineering silos that existed and they had to gather the knowledge required to put together the big picture.

Now that brings up the point that systems thinking requires access to knowledge in order to synthesize that knowledge and be able to solve problems. In my experience I feel that there are two types of organizational culture when it comes to sharing knowledge, tribal knowledge culture hides and hoards information and information sharing culture exposes and shares information.

Which type of culture do you think you're in? Sometimes it's really hard to tell. I have been in some very heavily tribal knowledge cultures where it's actually taken me quite a bit of time to even recognize that I was in one. And many of those teams are actually very, very highly regarded for their expertise. But if you ask yourself a few different questions it might become clearer which one you're in. You might be in a tribal knowledge culture if it's difficult or it would be difficult for somebody else to do your job.

So I noticed that I was in a tribal knowledge culture when I started training somebody in our systems and I quickly realized how many undocumented tricks we had, how difficult the data was to get to until you acquired a knowledge of where it is, and then you have all these little inside things to your team like, oh yeah, every second Sunday you have to skip step nine in the procedure, it doesn't quite work right, everybody knows that. Well, everybody inside the team knows that, but others don't. You might also ask yourself how many people know how to do a specific task? If you have a fuel/girl or a solar array deployment dynamics guy who are the only ones that understand how to run the engineering tooling to do those tasks, you might be in a tribal knowledge culture.

And this type of culture is really, really easy to develop, even unintentionally because it has this promise of ultimately false rewards but rewards nonetheless, if you become an expert within a tribal knowledge culture, there is a perception that you're very highly skilled because you're doing things that are really difficult because a lot of times they're undocumented or they take that special knowledge that only your group has.

Now in contrast, if you're in an information sharing culture this type of culture rewards simplicity instead of complexity. Plainly, what that means is things that really should be easy are easy. That's a warning sign of a tribal knowledge culture, if you go into a culture and you expect to do a task that's easy, like plotting some data, that turns out to be really hard because the tools are really complicated. It doesn't have to be that way. In a different culture that task could be easy.

Information sharing culture also rewards team output over individual output. That actually enables individuals to learn each other's tasks because they can help the team be more productive and it results in knowledge sharing. It also rewards transparency instead of doing things that are pure magic, which really just means that things are clearly documented and the documentation is accessible. And as a result of all of this it makes the work of the team accessible to outsiders.

And of course one of those outsiders to a team like this might be systems engineering. Now the information sharing culture it takes some effort to maintain at the expense of the individual because it does take some careful thought, it takes some effort put into documentation but in the end the bit benefits the whole. And systems thinking is much, much easier in an information sharing culture.

So what makes a good systems engineering organization? Well I would argue it's one that values systems thinking, understands that the processes that are in place, the documentation that exists, it doesn't exist just for its own sake. It exists to enable systems thinking and to provide systems engineers with the data they need to perform it. And a good systems engineering organization also promotes a culture in which systems thinking can thrive.

So let's keep systems thinking in mind as we continue to take a look at model-based systems engineering. The industry is undergoing an evolution, now back some time in the 1960s, we recognized as an industry that the hardware was too complex to control without software and that started the transition from a hardware-centric industry towards a software-centric industry that I would argue lasted some decades.

In the late 1990s, there was a running joke at NASA that the Space Shuttle had just enough software to run all the hardware the Space Shuttle of course was a design of the '70s, whereas the International Space Station, a product of the '90s, had just enough hardware to run all the software. And honestly organizations are still coping with how to deal with software centrism because there are still a lot of folks from the older days that grew up with a very hardware-centric mentality.

Now more recently we started seeing a shift to a more model-centric industry. The reason for that it's kind of the same as it was before, the systems of today, including all of the software now living in those systems is simply becoming too complex to understand without models. The models are a way to explore systems before they're built and they're also a way to provide abstraction from the system to make it easier to understand and they can provide different views that promote understanding.

And when I'm talking about models they're all different kinds of models, they can be very simple, something as simple as a state chart can be a model on the left side, you see the code representing the same functionality, on the right side, you see a state chart model. Personally I find the state chart much, much easier to digest and understand in terms of system behavior than I do a full reading of the code that accomplishes the same function.

Now model-based systems engineering is the most widely adopted modeling construct among systems engineers. So it's basically a way to model systems engineering. And model-based system engineering it lives in the digital world, it is a subset, it's a part of digital engineering and it's the piece of the digital transformation that people are talking about.

Now the goal of model-based system engineering is to address some of the disconnects that can occur in document-based systems engineering. So in the old way. I would maybe find something that needed to be changed in the design, I would change the document. I'd go to a change control board, the chief engineer would say, does your change impact anything else? I'd say well no it doesn't and if I had a good reputation, if the chief engineer believed me, the change would go through. If I made a mistake in the past, maybe not.

So model-based systems engineering we see a transition from paper documents being the primary artifact to a digital model being the primary artifact. And one of the key differences is that the digital model includes built into it relationships between components and dependencies, between components and between requirements. So one obvious benefit is it's very, very powerful for impact analysis. Having this digital model right off the bat can greatly improve the agility of an organization.

Let me explain a little bit more why. In the old way, of this very complicated, difficult to manage document intensive process, there is a great resistance to changing a design even in the presence of defects and even before flight because there is a very, very real worry of making a catastrophic enhancement. In other words, introducing a new defect that may be even worse than the one that was just discovered and having it be an escape, not finding it.

I've certainly over my career seen many instances where it was considered lower risk to document and work around a design defect whose impacts were known and that could be bounded. And of course, as soon as that becomes the approach then it's no longer a defect it become something known as a feature and certainly those of you that have done mission operations understand that spacecraft have a lot of features.

So in model-based systems engineering, having this formal way to capture relationships between requirements and architecture, including, by the way, both the functional or behavioral view of the system and the physical components that make up the system, is very powerful. I have personally experienced it to be particularly effective in requirements engineering using tools like use case analysis and activity diagrams to decompose a system from the concept of operations and the high level requirements, down to low level requirements from functional decomposition that are allocated to functions that can then be allocated to hardware and software.

I'll give you an example of a high level requirement might be to image a location on the ground from a satellite. When you start thinking about the different tasks involved in this you recognize the functional decomposition of the imaging may involve moving the spacecraft, swinging the spacecraft around, pointing the camera to the Earth, activating the camera, turning it on, collecting the data, down-linking the data. So there's all these different functions that get allocated to different subsystems. And if all this information can be captured and connected digitally and linked together, it does a much more thorough job of capturing dependencies in the system than the paper-based documentation.

Now earlier this year I attended a European Space Agency workshop whose topic was model-based system engineering and there were a number of interesting papers presented at this workshop. But one thing I found interesting was the wrap up and conclusion at the end where the organizers of the workshop quantified the technical and organizational benefits that were presented in these papers at the workshop. They found that improved design was the most noted technical benefit, which of course is great, that's exactly what model-based system engineering is trying to do. And as the organizational benefit they listed better process control. Now that to me is not very surprising, because it's much easier to manage the process when it doesn't include so many disparate documents and when it's easier to understand the dependencies within that process.

Now despite all these great things that I've talked about, model-based system engineering today is still, it's not perfect. This is the result of a survey that MathWorks gave to our customers a couple of years ago where we asked them how happy they were with their current tool choices for modeling system architecture. Shockingly only 8% were happy, 44% were outright sad with the rest coming in somewhere in the middle.

So we asked them a few more questions, why is this the case? So one thing we heard is that there tends to be a steep learning curve when it comes to model-based system engineering. SysML, or systems modeling language, which is the most commonly used de facto language of MBSE, is very rich but it's also quite complicated as an architecture modeling language. It's based on UML, which has a software heritage, it inherits some of its complexity. So it can be difficult for new people to learn and get spun up on.

But on the flip side, once systems engineers learn how to practice SysML and be effective with SysML, there's also a temptation to get quite deep into the language. It's a very complicated language. I remember sitting in probably too many meetings where we were doing use case analysis arguing whether the right relationship between two use cases was the extend relationship or the include relationship. And if I step back from that now and look back at it whichever one we chose probably did not have any impact on the design. So it is possible to spend-- to overengineer the model so to speak.

Actually I have a cautionary tale about a satellite development program that did exactly this. It was kind of earlier on when model-based system engineering was just emerging, and the system engineering team got very, very intricate with the model. In fact, unfortunately to the point that they fell behind their schedule because they were spending too much time on that model. As a consequence, the subsystems engineers started their design work without even having the proper requirements from the system engineering team. So the effect was the exact opposite of what model-based system engineering is trying to accomplish.

Now model-based system engineering does a really great job with relationships but one thing we heard from our customers is that we sometimes struggle how to analyze other aspects of the design, for example performance. And consequently, it's common to jump outside of model-based system engineering tools to perform things like trade studies and other analyses.

But by far the biggest pain that our customers experienced with their MBSE tools was transitioning between architecture and design. Most model-based system engineering models are static and they don't simulate, and this by the way ties in why it can be difficult to analyze them. So the connection between these descriptive MBSE models and the dynamic subsystem level models with teams that are using model-based design to do their subsystem modeling, the integration between those is a challenge.

So in the previous slide, I mentioned an example where the systems engineering and design were disconnected due to a schedule conflict but you can actually also go the opposite way. There is a different program that I'm aware of that used a model-based system engineering tool not only to define requirements and architecture but they actually even defined their algorithms in the system engineering tool. And they had a fully defined set of algorithms that they were able to hand over in SysML to the software engineering team who would then translate them into software code.

Now at some point, the algorithm designers realized that before their designs were actually implemented in code, way too much time would pass and they just weren't willing to wait that long to have the algorithms be executed and tested through execution. So they actually ended up not redesigning, but re-implementing all of those algorithms from SysML into Stateflow, one of our MathWorks tools, that is dynamically executable and they ended up building a fully simulatable model of those algorithms that they could test on in parallel while the software engineering team was writing code and that ended up benefiting them because they actually found errors. But what's unfortunate about that tale is that they basically duplicated effort by writing the algorithms in both SysML and an executable tool instead of just choosing the one that ultimately better suited their needs.

So to summarize this section about model-based systems engineering, model-based system engineering can definitely benefit your systems engineering organization but there is more to a complete digital systems engineering solution. So let's take a look at what I call fully realized digital engineering.

So the goal of digital engineering in this context is to enable the use of connected models throughout the program lifecycle to digitally represent the system in the virtual world. And now we have this new concept of a digital thread which runs through time across the lifecycle and through the system across all the different systems and components. If this concept of digital engineering is applied consistently, the natural end outcome is that you end up with something called a digital twin. A digital twin is the system in digital form but one that's so close to reality that it can be used in place of the real thing to analyze and assess the system. So this clearly goes further than architecture and requirements modeling, it's beyond what a SysML model can do.

Now of course when we talk about digital engineering in the space community we often hear people say, well, of course, we do digital engineering, we've been doing this for decades. We understand that unlike a car you can't just take a satellite to the test track and work on it in order to do full V&V, we have to do simulation and modeling. And that's true but the way that we've done things in the past tends to be missing that digital thread.

Very often there is a substantial amount of modeling and simulation isolated to the subsystem silos earlier in the design phase and the more complete system simulations that are used for example for mission operations training as one example, they only come together typically after the design is already complete. And at that point it's of course too late to change. And also the ownership of these models and simulations that we traditionally have it's usually at the subsystem and domain engineering level. It's more rarely at the systems engineering level.

So some questions that an organization might ask themselves to see if they're fully digitally adopted might include questions like, do you have a model management process? Do you actually manage the models or do you manage the paper artifacts and the results of the analysis [? of those ?] Do you hold reviews in a model-based environment or a paper-based environment? Do you have a program in place to train your workforce in digital engineering tools? And how early in the program lifecycle can you perform integrated system simulation? I've seen very, very few organizations do this really well. And I'm always very, very impressed when I see it, integrated system simulation in the early design phases.

Another interesting question is, do you do digital acquisition? In other words, if you are procuring hardware or software or even a full system from a vendor what do you expect to receive? Do you expect to receive the hardware? In digital acquisition you would expect to receive the entire technology stack. So that includes not only the hardware, but also all of the models, all of the software and all of the data that went into the design. In other words, everything that composes or comprises a digital twin.

So the way we currently developed our systems tends to look like this. We design, we build, and then we test. Now, there's a new model-based paradigm emerging which looks more like model, analyze, build. And there is a potential for a very fundamental shift here in the way we do business.

Personally I believe that this adoption or transition will likely be more incremental but the bottom line is that as we shift into this model centric digital engineering approach there will be less time and effort invested in test and integration simply because there will be less defects to find. The majority of the defects that are currently found in test will be discovered in the model and analyze phases which will dominate the lifecycle. And test and integration by the way very roughly, today consumes about half or more, sometimes less but usually more of a program's budget. So this could have very significant impacts.

Now digital engineering is getting a lot of attention, the US Department of Defense has a digital engineering strategy. It emphasizes five main goals. I'm going to highlight the two of them that kind of tie-in with the themes that I'm raising in this presentation the most. One of them is the formalization of the development, integration and use of models. So that's the model centrism. And the other one that I've touched on quite a bit is transformation of the culture and the workforce. So those are two points that are recognized by the DoD.

The European Space Agency has also defined digital engineering as a key competence domain. They're very active in this area through workshops and information sharing. And NASA actually has gone so far that they've included the objective to advance digital engineering into the Human Landing System program requirements. So of course, that's not a shell but it is written in there that the HLS program has this objective to advance towards digital engineering. Now notice this doesn't prescribe how to do it. That responsibility is given to industry to figure it out.

Now of course at MathWorks one of our mission statements is to enable engineers to do their best work. And we believe that this digital engineering transformation can help engineers do their best work. So we're watching all this evolution very carefully and very much trying to stay with it and ahead of it.

Now going back to model-based systems engineering, as we saw before, it's quite good at requirements and architecture, whereas at the subsystem level many subsystem teams are using model-based design as their digital engineering implementation. Let's take a little bit more a close look of how do they connect what's in between? There's this whole area I feel of digital systems engineering that's unexploited that could provide tremendous value and opportunity for systems thinking for systems engineers.

So let's put this in the context of a program lifecycle or a design V that utilizes a model-based environment. So the way model-based engineering for the most part tends to look today is we have systems engineering doing model-based systems engineering in the requirements and architecture domain. We have subsystems engineers working on their subsystem design and implementation using model-based design. And later in the lifecycle we have system integration and test and system operations where digital twin might be a new name but for a long time we've had some concept of a system level representation of our system.

Sometimes this was known as the iron bird, where you start to put together hardware and software as it becomes available. So you can do fuller levels of integration. And certainly by the time you're training your operators and rehearsing your mission procedures, you have some type of a system model simulator. So I'm using the name digital twin to encompass all of that in this context.

Now even when we employ these model-based techniques we more often than not end up with three different and disconnected types of models. And that's where we have an opportunity for change. If we are able to connect those models through the digital thread, now we also have an opportunity to combine those models into one environment and the combination of those models has its natural owner as systems engineer.

So I envision a future in which systems engineers become the owners not only of model-based system engineering models but also of integrated, executable, analyzable all models across all life cycle phases. And of course at different phases of the lifecycle there'll be different levels of fidelity of the modeling, across the lifecycle there'll be different levels of modeling. So this is not a simple construct, but it definitely has the potential to expand the ability of systems engineers to do systems engineering and apply systems thinking in a digital environment.

So let's explore this a little bit further through an example. Let's say we want to develop an imaging satellite system, because I get to define what it is I'm going to have the [? ConOps ?] be that the MathWorks headquarters needs to be imaged once a day. So there's our high level requirement. Now we can probably start sketching out our architecture, except of course we're wanting to do this through digital engineering. So instead of using my piece of paper, I'm going to open up my MATLAB and open up an architecture model using Simulink.

So here's my architecture model. I have my high level requirement right here, map to the system, I can decompose requirements to the space and ground segments and allocate them. I can dig in a little bit deeper and define my subsystems and I can allocate requirements and child requirements to those subsystems. I can define interfaces between components and define how they connect in as much detail as I want.

And of course up until now this has been pretty basic model-based systems engineering but if you watched carefully you saw that I just pushed something called a run button and now you'll see the satellite is actually simulating my architecture flying around in space. Now how was I able to do that? Well it's simply because in this construct if you dig deep enough, you will actually encounter inside the architecture model a digitally linked executable model that allows the architecture model to execute and simulate.

Now because this is all sitting on top of MATLAB I can also take advantage of this and use those simulation results to do some analysis. So here you see a script that takes a look at the simulation that we just flew and verifies that it does indeed meet the mission requirement of flying over headquarters. And there's no reason to constrain myself to one subsystem, I can actually use the same simulation results as well as attributes and stereotypes that are defined within the systems engineering model of the architecture to run other static analysis, in this case to figure out how to size my battery and my solar array. And this analysis like I said, it pulls in the amount of time that the satellite spent in the sunlight from the dynamic simulation result and it pulls in the battery cell characteristics and the photovoltaic array cell characteristics from the architecture model and integrates them.

So here we have a very simple and very quickly shown digital thread between requirements, architecture, data parameters, time domain simulation and analysis. It's an executable analyzable model that bridges the gap between architecture and detailed design and at the same time provides a single source of truth for everything. I think that something like this is the future of systems engineering and of course I didn't show it to you now, but I recognize that the modeling ecosystem that's used on production programs is incredibly diverse and interoperability has to be considered. So these things do need to work together so that the best tool is used for the best job.

Now as I mentioned before, it's important to enable the workforce to be productive in the digital world. So a digital transformation strategy should always include training and investments into the workforce. I wouldn't buy my daughter, 10 years from now, a car and expect her to figure it out. I would sign her up for a driving class and make sure she gets licensed before she can go anywhere.

So likewise, there shouldn't be an expectation that a company can just purchase software and things will magically happen. That's not true. We have to invest in the people being able to use that correctly. And ultimately the measures and the metrics of success, they are people measures, their productivity, outcome, teams meeting milestones. And of course cultural changes are also very, very important to ensure the free flow of information and getting away from that tribal knowledge culture I talked about earlier and this requires leadership support. Tribal knowledge culture will resist putting all of its data into one easily accessible model.

So to summarize, the vision of fully realized digital engineering expands upon model-based system engineering, includes it but it expands upon it, and empowers systems thinking by giving system engineers more of the tools that they need to do a broader part of their job than just the requirements and the architecture, but also to look at system level analyses, putting different executable subsystem models together to make sure that they work correctly as a system.

So let's take a look at how to incorporate AI into all of this. I'm going to start this section by defining some terms. If any of you saw my talk last year, some of this might be familiar but it never hurts to review. So let's start by asking, why is it that machine learning and AI and deep learning are so popular today, why are they rising up right now? And that's simply because the three magic ingredients of machine learning, which is computing power, data, and algorithms, are really coming together at this point in time.

And honestly, the algorithms are the ones that have really been mature for the longest, the machine learning algorithms have been around for decades. But when people first started using them they were using them on insufficient data and using insufficient computing power for the results to be practical. So it kind of fizzled for a while until now. We already heard some of the panelists talk about the tremendous amount of data that's being generated in the world today from space systems alone, never mind the other systems.

So what's machine learning? Let's start with some definitions. So first artificial intelligence or AI the definition is really quite simple, it's the capability of a machine to imitate human behavior. I feel like in space industry more than any other it's very tightly coupled to the concept of autonomy, the ability of vehicles to perform increasingly complicated tasks without human intervention. Autonomy of course has been a key attribute of spacecraft for decades but the algorithms that enable autonomy have traditionally been designed using traditional or manual means rather than through machine learning. And certainly for safety critical algorithms that remains true today.

So let's take a look at a couple of examples that honestly could be called AI. So this is a shot of the next generation of human transport to International Space Station, fully autonomous from launch to docking. The astronauts never have to touch the controls unless something goes wrong.

I think this is a great example of how the space industry can actually pragmatically realize autonomy more quickly than some other industries, for example, automotive. And that's because for us the problem in this case is much, much more constrained. There are no pedestrians that will jump out in front of you when you're docking to Space Station. There are no stop signs obscured by trees. It's a very constrained problem and as a result, you can do this with traditional computer vision techniques, machine and deep learning aren't mandatory but they certainly have the potential to improve the efficiency of the algorithm development.

So this image is from NASA's Opportunity Mars Rover. This Rover has software that enables it to select its own target in this case, the yellow triangle, for scientific observation without ground interaction. This is an earlier example of AI that's based on sensed image data and this type of capability is becoming more and more important as we have more and more missions that operate further from Earth, where the distances to light time make ground control impractical or impossible. This type of technology continues to fly on the later generation NASA Rovers and they're making it better and better all the time.

Now machine learning is distinct from the manual methods I talked about earlier in that it uses computers to learn directly from data without having to use first principles. So I always like to think of this as machine learning is, it's a lot like how humans learn as children. We identify the world around us, we look at it, we form correlations, we identify patterns. And I like to use the example of learning how to catch a baseball or a cricket ball, I can learn how to catch a cricket ball without having any understanding of physics and mathematics or Newton's laws. So machine learning is analogous to that, it does not need to know the physics and math.

Deep learning is a subset of machine learning. It's usually associated with neural networks that have multiple layers of processing, the more layers they are the deeper the learning. Now in deep learning each layer transforms its input data in a nonlinear way and as a result that data as it travels through the layers becomes more and more abstract. And because this abstraction happens in ways that aren't intuitive to the human mind, neural networks earned the reputation of being kind of a black box algorithm. And of course our industry hates black boxes, we like white boxes even though we claim to do black box testing.

But why is deep learning so popular right now? It's very simply accuracy. About five years ago now deep learning algorithms surpassed human accuracy in some applications and that's why industries like automotive are pursuing deep learning primarily for applications like autonomous driving and even the space industry. We're seeing deep learning becoming a go to technique.

So if we examine the concept of machine learning a slightly different way, there are basically two ways to get a computer to do what you want, there is the traditional approach, where we write a program that processes data. So you put the program into the computer. And then you put data into the computer, it runs through the program and you get a desired output.

Now machine learning, you flip it around. You actually feed the data and the desired output to the computer and the computer essentially writes the program for you. Now usually we don't call it a program, we call it a machine learning model. And of course, you also want to deploy your machine learning model, could be on your ground system, could be a standalone app, could be on an embedded device onboard the spacecraft, and the deployed algorithm will now receive new data and generate the output to be used by the production system.

Now, you may have heard, I hear this all the time in media and press, that machine learning systems are continuously learning and getting smarter. What that means in practice is actually that there is continued training that continues in parallel on a different system than the deployment system and the algorithm and the production system is periodically upgraded to take advantage of improvements based on better and better and ever increasing training data set.

So broadly speaking, there are three categories of machine learning that can be used for different types of applications. On the left, we have unsupervised learning. So this is actually a type of machine learning algorithm that can be trained without any output data. An example of where this is useful is for example to find hidden correlations or patterns in the data. This could be useful for things like anomaly detection. If you have large quantities of let's say, satellite telemetry data with lots of hidden correlations that you may not actively analyze, applying unsupervised learning to that data might actually help you discover something anomalous that's happening that you might not otherwise be able to see.

I would say most machine learning applications in the space industry today are in the supervised learning category. So that's the kind that uses both input and output data. There's two broad categories of supervised learning called classification or regression. The only difference between the two is in classification you define a category the output belongs to. So for example, if you're pointing a camera at something you might want to know, is this a satellite or is it an asteroid, those are two different classifications. Regression is what you'd use if the output is a quantity of data, for example a predicted future temperature in degrees Kelvin.

Reinforcement learning is kind of the new kid on the block at least when it comes to the space industry. Controls engineers in particular are very excited about it. This type of machine learning uses a trial and error method to accomplish a task where success is rewarded and failure is not. So actually I think this type of learning is probably closest to the cricket ball analogy where if I learn to catch a ball with my dad, if I catch the ball successfully, he cheers me, if I don't it hits me in the face. So there's my reward cost function and I learn from that.

So what's all this talk about AI in a systems engineering talk and why even mention it in the context of systems engineering? Well simply, at some point your AI model will need to be integrated into your system. The machine learning algorithm development falls within the domain of data science and arguably you could say that the application to a domain for example, if you take a machine learning algorithm, then you learn how to do computer vision or perception with it, falls into a subsystem domain engineering but the application of the AI within the system most definitely falls into the systems engineering domain. And we're moving towards a world where, as a systems engineer AI will become just kind of another tool in the toolbox, another piece of the puzzle that can be used to construct the system around a set of requirements.

So let's get a little bit more specific about the types of things that machine learning can do aboard a spacecraft. I like to present this in the context of the data, information, knowledge, wisdom, pyramid. Sometimes just called the wisdom hierarchy. Now if you go and watch a movie about AI you will usually encounter some type of a robot or computer being that has some kind of preprogrammed wisdom right on top of this pyramid and then accumulates all kinds of knowledge and ultimately turns evil and then tries to do bad things to humans.

The types of machine and AI applications that I'm talking about don't live in that wisdom layer. I think the sweet spot right now is probably closer to information tier, sensing state estimation, what type of things am I looking at, even though I think that it's slowly creeping more into the knowledge layer or the how should I do that type of thing. But I definitely don't envision the why should we do this mission piece of it ever falling into the AI domain. I think that's strictly for us humans.

So let me show you a couple of generic examples of the types of algorithmic applications I'm talking about. So imagine you're tasked with developing an algorithm to classify different human states, walking, sitting, standing, based on inertial measurement unit data. Now given the audience that I'm talking to, I know that you could do it but I suspect that it would probably be hard and it would probably take a lot of work to get right. Some of my colleagues used deep learning with MATLAB to develop a machine learning algorithm that does this classification for you, you see it running in this video, it actually works pretty well. So you can maybe envision an adaptation of this use case involving estimating states of a spacecraft or perhaps some other system under observation.

So these videos show MATLAB doing object detection and tracking and segmentation. So these are the types of algorithms that could be very relevant for rendezvous and proximity operations or terrain relative navigation for things like a lunar landing or Mars landing and these algorithms are also not easy to develop using traditional methods like we currently do. We can do it but it's kind of the domain of the privileged few right now.

So here's an example of a robot learning how to walk using MATLAB's Reinforcement Learning Toolbox. Some of my colleagues put this together and you can see at first things don't go so well but gradually over time the walking improves as successful attempts are rewarded. And after a while, just like a child would, the robot actually learns to walk fairly decently again, without having any knowledge of physics.

So let's get to our industry applications. So machine learning has been, the first place that it was deployed has been on the ground segment and this has been going on for some time now. One prominent use case is telemetry outlier detection. I see this spearheaded by large satellite fleet operators around the world. So there are establishments out there that might be flying a couple of dozen satellites with a fairly lightly staffed mission control room. And companies like those have adopted machine learning algorithms to help call attention to things going wrong.

They're not trying to take any independent or autonomous action yet based on of these detections, but they're at least able to call a human's attention to a potential problem, in many cases before the operator themselves would notice anything is wrong, even if they're there staring at the data. And this is-- they say that it takes-- and this was probably a couple of years ago that I heard this, but able to train these algorithms by collecting about six months of data, nominal data, from when the satellite is first operational and after that first six months they have enough data to train their algorithms and they can start implementing this scheme.

Now this similar type of application will also apply to ground test data analysis. Many of the large programs going on in the world today are actually generating so much test data that the test data itself is becoming unmanageable to analyze just by the humans, the test engineers. So AI can play a role there. So it's not just the data from space, it's also data generated on ground that this can be helpful with.

And of course geospatial analytics is a major use case. We get tons and tons of satellite imagery down to earth every day. There are lots of different applications for it from say, agriculture and determining whether a field is doing well, to business use cases like counting the cars in a parking lot or a shopping mall before a holiday to predict the economy.

Now more recently, we've also seen machine learning algorithms deployed in space. Kind of the classic machine learning problem that the community has entertained for a while is the cloud detection problem. Basically what happens with these earth imaging satellites if they're using optical sensing certainly, is they're continuously collecting images or their tasked and they're collecting images and then they downlink those images to the ground. But in some cases, half of your images could be obscured by cloud cover. And those images are not useful.

So you're basically wasting a substantial part of your valuable downlink bandwidth to downlink useless images or useless pieces of your image. So wouldn't it be nice if there was an algorithm onboard the spacecraft that could detect these clouds and either discard the images that are heavily clouded or apply maybe a higher level of compression to the portions of the image that it thinks that cloud was detected in or apply whatever other fancy techniques to conserve the bandwidth.

Well a European satellite called PhiSat-1 has now solved this problem on orbit successfully. It launched recently, some of the first papers from this mission are just starting to come out now. This picture is actually not a PhiSat-1 picture, I wasn't able to get one. This is a NASA picture where I put some MATLAB cloud detection on top and as a result, while PhiSat is purely a technology demonstrator spacecraft, there is already work going on at Thales Alenia Space to apply a similar methodology in the hyperspectral domain using an implementation called CHIMES that is slated to fly on a future Sentinel satellite. So that could be a real production use case of machine learning in space.

Now there's also a technology demonstrator from NASA that flew recently. I think this was in September of last year. There's a CubeSat called Seeker that has a convolutional neural network onboard that's trained to detect and localize a Northrop Grumman spacecraft called Cygnus using images from a camera.

This was also a technology demonstrator, it was deployed in September of 2019. It looks like the visual navigation algorithm. So the neural network performed fairly well. Unfortunately, there was a loss of vehicle control on this mission that is most likely related to an unrelated failure of a propulsion controller FPGA connector. So while the mission itself was not fully successful, the results from the CNN were quite positive.

Now, if we take a look at some more future looking use cases, lunar landing could certainly be a potential use case for machine learning and there is a tremendous amount of interest in lunar landing right now from all over the world. The lunar surface contains hazards that can pose a risk to the spacecraft as it lands, slopes, craters, rocks, that can damage the spacecraft. So onboard hazard detection and avoidance is needed to ensure a safe autonomous landing.

Back in the 1960s, the onboard hazard detection avoidance was called Neil Armstrong and just used his joystick to guide the spacecraft around a boulder field but of course for unmanned spacecraft that's not an option. And the next generation of crewed spacecraft also have an autonomous landing requirement.

Now there are many techniques of course for designing HD algorithms, you can use image processing computer vision but let's take a look at what it would take to use deep learning. So one of the tricky things about landing on the moon and performing this hazard detection is that while we generally have pretty good data coverage of the lunar surface, the resolution of the imagery that we have is not sufficient to guarantee the identification of all the hazards that could damage a spacecraft upon landing, they're just-- they're hazards that are too small to see with the cameras and the images we have today that could still damage the spacecraft.

So if you want to simulate a landing or if you want to train an algorithm you have to have additional scene generation to simulate the lunar scene even if it's not fully real. There are tools that exist for doing this. The one I'm showing here is a European tool called PANGU, and they were kind enough to give us a video of a simulated lunar landing trajectory near the South Pole of the moon, it looks something like this.

So then what we did just as a proof of concept to show that it can be done is we used MATLAB to train a model to detect craters and applied it on top. And now you end up with something that looks kind of like this. Now is crater detection the right application for deep learning for lunar landing? Maybe not craters are actually pretty simple to detect using other means but I think this kind of gets the concept across of types of things that can be done.

Now just like with the other modeling ecosystem, there are a lot of different AI models to start from that are available. There is a common standard called ONNX that's supported by many of these, so that helps with the exchange of these algorithms that can be used as a starting point.

Now this slide I'm acknowledging my employer MathWorks for being named a leader by Gartner and for data science and machine learning platforms but the reason I'm doing that is because actually some of the reasoning ties in with the broader topic of my talk, which is that the MathWorks difference is understanding that success is more than just having an effective model or an algorithm, it's ultimately based on your ability to incorporate deep learning and integrate it into your system and design workflow and I think that ties in very nicely with the theme that I'm trying to paint here with this talk.

Now I mentioned the key enablers of machine learning, hardware, data and algorithms. Each of those comes with their own set of challenges for space. We recognize that radiation qualified hardware is not exactly cutting edge compared to the terrestrial counterpart. Data availability is an issue, we have a lot of data from Earth, we have data from moon like I talked about that might be augmented, we have some data from Mars but beyond that in deep space, we just don't have the data. And last but not least, we don't know how to understand. We don't understand how to verify machine learning algorithms. They don't fit the traditional V&V white box, full traceability of code model.

Now, even just very recently the first challenge, the hardware challenge, it's getting better. Recently Xilinx released a radiation hardened version of the Kintex UltraScale that people are starting to look at and prototype on. And there's also the European BRAVE FPGA, which has three different versions and the most performant version is also something I think people will start looking at deploying these types of algorithms on.

So like I mentioned earlier, when it comes to the data availability I think Earth and moon are probably the right places to start and that's where we see a lot of the efforts going on. And as for the verification, validation piece, I think that's still for the industry to solve. There are some suggestions that we might be able to offer. We may not know how to verify a machine learning algorithm necessarily or a deep learning algorithm, but we do know how to test it. So that's one approach is testing through probability based requirements.

There is precedent for that. We often do Monte Carlo analyses, we have high level requirements for docking, landings, et cetera, that are probability based. They are typically verified by some combination of analysis and test, so that is a potential option we could look at.

And of course the statistical approaches can definitely be improved, purely testing the algorithm is probably not enough for many of the more critical missions and there's also a precedent for sanity checking the output of a component. Now traditionally sanity checks have been put in place to protect against hardware failures. So this would be a slight push to the paradigm where now we put sanity checks in place to test against machine learning software failures or undesirable outputs but the nice thing is that the sanity checking logic that you can implement can be a fully deterministic algorithm that can be verified using the traditional means. So in a way you've bounded the problem.

And if sanity checking is not enough. There is another design approach that can be resorted to that's common, which is designing in dissimilar redundancy. So in this case, you might design an algorithm that works in parallel with a machine learning model, an algorithm that's designed using traditional means using the same inputs and writing similar outputs to the machine learning model.

Now it may not be as accurate as the machine learning model because otherwise there would be no reason to have a machine learning model but it might be good enough to serve as a monitor to ensure that the output of the AI is sensible. So this is a slightly more costly but an additional level of robustness you could add to the system. So just summarizing those three conversation starter approaches for testing the algorithms, black box testing to probabilistic statistical requirements and adding sanity checking and dissimilar redundancy to the design in order to achieve certification.

So ultimately the decision to adopt machine learning on a program has to be considered at the system level, and this again is where systems engineers will come in, based on the requirements of the program do the trade studies show that machine learning is the right option and the requirements can't be satisfied using other means or other constraints can't be satisfied including cost and schedule. If so, it's going to be systems engineering's place to figure out where AI belongs in the architecture and how to integrate it into the rest of the system and what is considered sufficient test or qualification of the algorithm. And that of course, has to do with risk management and will depend on how mission critical the particular functionality of the AI is.

So in summary, systems engineering is essential to both successful adoption of digital engineering and AI in space. I argue that systems thinking is more important than ever today with the complexity of our systems and with the potential enablers we have for systems thinking.

We discussed that fully realized digital engineering includes and expands upon model-based system engineering to include other forms of digital systems engineering as well, connected through a digital thread. And I made an argument for why systems engineers should prepare for AI and what their role might be in its incorporation.