Swisscom AG, or more specifically its Swisscom Health division, is a leader in networked solutions in the rapidly growing eHealth market. Its customers include renowned Swiss healthcare institutions. Swisscom Health is also the technology provider for XAD Stammgemeinschaft, Switzerland’s number one EHR service provider.
Background
As the technology provider for XAD Stammgemeinschaft, Swisscom Health develops and operates the technical platform for EHR functionalities offered in Switzerland. This includes all interfaces (APIs) for deep integration of the institutions connected to the EHR as well as portals for patients and healthcare providers.
As part of the modernisation of the Swisscom data centre, the task was to upgrade the entire EHR platform to an up-to-date and sustainable technology stack (Cloud Native Stack). The security of the platform was paramount.
Where Novaloop came in
Novaloop was tasked with modernising the entire system and deployment architecture of the EHR platform, taking a leading role in the project as a whole. In the course of this modernisation, focus was placed on ensuring that the process could be taken over autonomously by the development team at each stage, from the development through to the rollout of every component (self-service, DevSecOps).
Goals
The modernisation of the platform was carried out with an eye to the following goals:
- Fast feedback loop: Potential infrastructural problems are recognised early on in the development process and the configuration gap between production and dev/test environments is small
- Simplified, developer-centred processes to establish high data and network security
- More efficient commissioning of software components (self-service)
- Automatic provisioning of infrastructure and traceability (Infrastructure as Code)
- Cloud-agnostic technology stack
Technologies
To implement this, Novaloop used state-of-the-art technologies from the cloud-native ecosystem. The following list is not final and focusses mainly on the platform’s configuration and deployment aspects. Technologies for the implementation of other operative aspects such as logging, monitoring and backup were omitted intentionally.
Orchestration
Google’s open-source solution, Kubernetes, was used to orchestrate containers. Out of the box, the Kubernetes installations are not sufficient to meet the high security requirements for the operation of an eHealth solution.
Consistent application of the principle of least privilege for access by individuals and systems is essential. The process was carried out using standard tools provided by Kubernetes:
Continuous Delivery
For continuous deployment and environment-specific configurations of infrastructure and software artefacts, Novaloop worked according to GitOps principles. With GitOps, all configurations necessary for deployment are saved in one or multiple Git repositories. Here are some of the advantages this provides:
- All configurations are always stored in a versioned and traceable state.
- Use of developer-centred approval processes such as pull requests can also be used for infrastructure- or environment-specific configurations.
- The system’s (target) state is always stored in a traceable and versioned state in Git. If there’s a gap between the actual state in the system and the target state in Git, an alarm is raised or automatic deployment takes place.
The Kustomize and Helm tools are used to compose the Kubernetes deployment manifesto. To improve readability, Kustomize was used to compose the software artefacts for the workload. Helm was used exclusively for infrastructure components such as HashiCorp Vault. As a general rule, many open-source software providers already offer their configuration in the form of Helm charts.
Service Mesh
The eHealth industry is subject to a particularly strict network security requirements, which make the roll-out less efficient. Non-functional aspects such as transport encryption and the certificate management associated with this are far from trivial tasks for the development team. In previously used solutions, transport encryption and certificate management had to be implemented and managed directly in the application code. There was no central management of transport encryption, so every development team needed this knowledge at its disposal.
The remedy for this is service meshes, which make the previously decentralised configurations for each team obsolete and can be managed centrally by a system/platform team. Not only does this boost efficiency, it also allows potential security problems to be mitigated on a central level. One familiar issue is the validation of the chain of trust in the application stack being disabled for practical reasons. As a consequence, the application approves all endpoints, even those which aren’t trustworthy. In a service mesh, these connections can be monitored centrally and safely enforced.
Security works according to the “onion principle”. That’s to say: it needs to have several layers. This also applies to transport encryption within the platform. Components may only come into contact with each other if this is required by a specific use case in the solution. The various layers (bottom-up) are:
If one layer is compromised, the next layer beyond that one steps in to provide protection. This security concept is known as the “zero-trust model”.
Password and certificate management
Proper management of passwords and certificates is not a trivial matter. The processes for the management of sensitive information ideally don’t differ at all from stage to stage (DEV, TEST, PROD). The approach to day-2 processes regarding passwords and certificates can be established early on by the development teams and then applied consistently thereafter.
HashiCorp Vault provides a secure key-value store for storage of sensitive data. Passwords are generated automatically in an environment’s Init deployment and stored in the key-value store. The logic for password distribution is outlined declaratively in a YAML definition (GitOps). The passwords are provided via Init container in the form of an encrypted in-memory volume of the various application containers.
HashiCorp Vault PKI is used to issue internal X.509 certificates. Initial issuance and rotation of certificates takes place via API calls and is fully automated. This is an important requirement for the introduction of short-lived certificates. Therefore, cumbersome and error-prone revocation processes (CRL, OCSP) can be abandoned for internal traffic. The validity period of a certificate can be limited to as little as 15 minutes.
Testimonial
We worked with Novaloop for the design and development of our cloud-native platform. We are very satisfied with the implementation of the task and the project results. Our new platform has increased the autonomy of our development teams, massively cut deployment times and ultimately increased efficiency in delivery.
Federico Marmori, Head of Development & Operations Health at Swisscom
Leave a Reply