28 January 2022
Evolvability by design for the SAFIR-Med Command & Control Centre 

A couple of years ago, Airbus posted an image of the wiring of their A380. They were having troubles getting everything to work, and 'appealed to the public' to search for the problem. While obviously a joke, I have to admit that for a second, my engineering mind was scanning the absolute rats nest to look for the problem. Given that the A380 has been flying for a couple of years, we can assume it was fixed eventually. Similar issues quite often occur in software. A thousand good and well considered decisions, which in the end result in an immovable wall of code, full of annoying ripple effects. At NSX - Normalized Systems, we call these combinatorial effects. In a stand-alone or a small application, these development impacts can be contained, but when put into a larger application or in an application landscape, changing anything in the software becomes a burden. Moreover, this problem will get worse when performance is a factor. This is why, when I was tasked with engineering the Command-and-Control center for Helicus in the SAFIR-Med project, we wanted to take every little advantage of the Normalized Systems Theory has to offer in order to obtain evolvability, predictability and transparency.

nsx image.jfif

Not only does the Command-and-Control center handle the resource management for every conceivable resource (Orders, Flights, Drones, Cargo containers, Drone Cargo Ports and so on), it also monitors the ongoing (multi drone) flights in real-time, and is able to offer a multitude of possible mitigations. On top of that, the system will need to be adapted many times in the future in a continuous way and within predictable budgets. This will be the case, as the C2C moves from today’s demonstration flights with 5 different drones to full commercial 200+ (multi) drone fleet operations in a consistently reliable way. To make matters truly challenging, the C2C needs to integrate with a multitude of other aviation and healthcare customer systems.  A.o. these currently include five autopilots, three  U-space Service Providers (USSP), two embedded hospital systems, (and a partridge in a pear tree), all evolving independently. That is typically what Normalized Systems Theory is about: managing modularity under continuous change and innovation. To do this, we fully leverage the Normalized Systems cookbook. Firstly, the NS tooling, coding robots, expand a full Java stack. This method guarantees many scalabilities from the outset, both in terms of performance as well as functionality. Next, a scalable architecture for the real-time system components is deployed. For this, a message broker is used as core (MQTT in the case of the Helicus C2C). This allows to launch asynchronous messaging between agnostic parts of the system. Finally, a separate adapter (called a Ground Control Adapter) is used per each separate connected autopilot. This adapter takes the diverse interfaces (ranging from raw UDP and TCP sockets to REST interfaces) and translates them to our own generic standardized format. This allows us to continuously scale up the number of integrations with no additional impact.  

Of course, the real test of any system is to take to the air.  The C2C was therefore remotely connected and interfaces were tested where an Antwerp based C2C consecutively remotely controlled partner drones in the Antwerp harbour, cross border in The Netherlands as well as in Germany.  And true to form, the NS system performed very well. During several live trials, it felt like you could hit the system with a hammer and it would still work. Of course, with every test additional improvement came to light that can be implement without risking the system as a whole, thanks to the normalized (= compartmentalized) nature of the system.  

If you really want to dive into the several aspects of how we put NS theory into practice, do not hesitate to consult out academy page: https://normalizedsystems.org/academy/ 

Written by ir. Christophe van Hove