Incremental Alignment of Heterogeneous and Dynamic Data for Decision Support

Overview & Synopsis

In the era of Big Data, where data volumes are massive and increasingly complex, the need for information remains crucial for every organisation. Indeed, whether it involves data collected from the immediate environment (clients, suppliers, competitors, etc.) or data generated internally, this information represents a capital resource that organisations must exploit to remain efficient. For this reason, decision-makers need to make the best decisions at the right time. In this context, decision support systems aim to help them analyse all this data to facilitate and guide their strategic choices. From an information system perspective, two approaches can be identified. The first is data integration (data warehouses, silos, etc.), which aims to consolidate all the data required for decision-making into a single homogeneous space. The second focuses on storing raw data (data lakes) in a space that is inherently heterogeneous. Regardless of the chosen approach, one of the major challenges remains data alignment. Alignment establishes correspondences between the data available within the organisation to better integrate, catalogue, and more generally exploit them for decision support. In the dynamic environments in which organisations operate, a dual challenge impacts the performance of data alignment approaches. Indeed, in these environments, the data used by the organisation can be both heterogeneous (diversity in the data and its structures) and dynamic (evolution over time in data content and structures, along with the emergence of new data sources). Numerous data alignment approaches have been proposed in the literature. However, to our knowledge, these studies focus exclusively on the challenge of data heterogeneity. They fail to address the issue of data dynamics. To address this, the objective of this thesis is to consider data dynamics to improve their alignment in dynamic environments. The issue of data dynamics has, however, been explored in other fields beyond data alignment. Among these are data stream management and dynamic networks. Various solutions have emerged to address these challenges. For instance, dynamic graphs allow the representation and maintenance of evolving relationships between entities. They provide an effective approach to modelling constantly changing systems. At the same time, stream processing offers mechanisms for processing data in real time, eliminating the need for global recalculations. More recently, dynamic embedding techniques have stood out. They capture complex relationships while enabling incremental updates. These approaches address the growing demands of rapidly evolving data environments. These works inspired us to design and develop the incremental data alignment approach we propose, named 'IDAGEmb'. This original contribution in the context of decision support is based on dynamic graph embeddings. It takes into account the evolving nature of data by progressively updating alignments while minimising the resources required for their computation, thus avoiding global recalculations. The findings from our experiments underscore the relevance of incremental alignment for decision support in highly dynamic environments. We propose a concrete solution aimed at improving data alignment computation and, consequently, enhancing data analysis efficiency and the overall decision-making process.