European Space Agency: Data Factory, the Single Source of Truth

A transparent database for all

With the Data Factory, the European Space Agency (ESA) has created a single source of truth that makes its data and project situation transparent, its processes more efficient and decisions more sustainable.

Please adjust your cookie settings to play the video.

Challenge

Initial request was to migrate existing applications to modern technologies – to the cloud

Complete new development using current technologies in addition to migration

Analysis and migration of large data sets from a heterogeneous landscape

Standardization of the data for a common „language“

Solution

Entire project is based on Azure technologies

Creation of a central Data Factory using Microsoft Azure Cloud

Azure Synapse for the database and Azure Web Apps for the frontend

Integration of Power BI dashboards with Azure Web Apps

Benefit

Creation of a single, reliable source of truth through the Data Factory

Establishment of a „common language“ across all directorates and projects by identifying the object model and ensuring transparent data labeling for stakeholders

Faster project decisions and improved performance with real-time Power BI dashboards

Foundation for further automation and AI projects

What stood out particularly positively for us in this project was the professionalism and speed with which novaCapta was able to execute the project, as well as the speed at which we can now access our data.

Bernhard Isemann Senior Data Manager & Head of ESA Data Factory European Space Agency

Our Director General expressed the desire to see and measure our organization's performance on a dashboard at any given time, in order to navigate the ESA effectively. We have successfully achieved this in collaboration with novaCapta and the Data Factory, to the satisfaction of all stakeholders.

Bernhard Isemann Senior Data Manager & Head of ESA Data Factory European Space Agency

Consolidating data and ensuring quality – a central data platform in use Glossar

The European Space Agency (ESA) intends to take a significant step towards the digitalization of its data by replacing its exis ting on-premise databases. To find the right partner for this en deavor, the ESA turned to Microsoft, as it was immediately clear that the technologies should be centered around the Microsoft ecosystem. Out of ten service providers from across Europe, the ESA selected novaCapta as the most promising partner. Initial ly, the plan was to simply migrate the existing data to the cloud to make the current database more future-oriented. However, after thorough consideration, ESA and novaCapta decided on a more comprehensive approach: alongside the introduction of the Azure Cloud and the associated data migration, the Data Factory was created.

„The approach and expertise of novaCapta convinced us. We jointly decided to abandon the original plan of a ‚simple‘ mi gration and opted to completely overhaul our data structure. Our goal was to enhance our performance and accelerate our processes. This was reflected in novaCapta‘s concept,“ explains Bernhard Isemann, Senior Data Manager & Head of ESA Data Factory, ESA.

Understanding Data, Objects, and Business Processes

The first step involved identifying how many different data warehouses the ESA had across Europe and analyzing this he terogeneous landscape. To better understand and analyze the processes, data, and objects, and to define the stakeholders‘ requirements, two-day workshops were held. Understanding the project approach and the user journey – what data is nee ded – was essential not only for defining a common language in the Data Factory, but also for engaging all relevant stakehol ders and establishing a transparent process.

Single Point of Truth with the Data Factory

With a solid understanding of the data, the new hybrid struc ture (on-premises data servers and Azure Cloud) was imple mented. Then, using Azure Synapse – an analytics service that combines data warehousing with big data analytics – the data was retrieved and processed using Power BI. Naming the data in a traceable manner was a key aspect to establishing a unified language across all directorates and projects. Next, the first real-time dashboards were created with Power BI, using commercial data such as financial and contract figu res. These dashboards were integrated into Azure Web Apps through Power BI embedded and made available to all relevant stakeholders. The advantage of Power Platform components like Power BI is that the ESA can independently create reports in the long term, deploy them through the established mecha nism, and keep the dashboard alive without relying on external service providers.

“With the Data Factory, we no longer need to question where the data comes from or whether the figures are correct. It gives us a single source of truth on which we can directly proceed to project planning and discussions. This saves us time and ensu res that all project participants share the same understanding,” explains Bernhard Isemann. The overall system, consisting of databases, Microsoft Azure, and Power BI, is called the ESA Data Factory. With this future oriented data factory, the ESA can now take further steps to wards artificial intelligence, enhancing its performance and the scope of data analysis. In the long term, dynamic dashboards could be created that, with AI integration, operate independ ently, interpreting and answering prompts. Further projects around the Data Factory are already in planning, including pro fessionalizing the Factory’s services and conducting security hardening.

Download Case Study European Space Agency als PDF (DE)

Download Case Study European Space Agency als PDF (EN)

Company Profile: European Space Agency (ESA)

The European Space Agency (ESA) is Europe’s gateway to space. Its mission is to shape the development of European space capability and ensure that invest ments in space benefit Europe’s citizens and the world. By coor dinating the financial and intel lectual resources of its members, ESA conducts programs and activities that far exceed the ca pabilities of any single European nation.

https://www.esa.int/