This second part of the streaming ETL series will discuss the typical ETL challenges data lineage, error handling and data quality with a special focus on the specificities to be considered in a streaming approach. Relating batch reporting requirements to streaming ETL is also tackled. The series is concluded with an overview on testing streaming ETL systems and a summary.
The new central data system for the banking and insurance sector by SAP is FSDP, the Financial Services Data Platform. When establishing FSDP, financial service companies are faced with the challenge of integrating this new platform into their overall architecture. This article gives an introduction to solving this ETL problem with event streaming using Apache Kafka to build a reliable and scalable real-time data integration.
We started our FPSL-journey on this blog with a post that gave a brief and general overview of SAP S/4HANA for financial products subledger as a product (What is SAP FPSL?). A second post shed light on the product scope
One main characteristic of SAP FPSL is the clear distinction between financial instruments on the one hand and insurance contracts on the other hand. This, however, does not mean that insurances for instance are limited to the functionality of the subledger accounting for insurance contracts. Insurances that possess a portfolio of financial instruments or reinsurances that are active in the context of capital markets instruments can in fact use both the subledger accounting solution for insurance contracts and financial instruments.
From a technical perspective, SAP FPSL is deployed as an add-on to SAP S/4 HANA. This, however, does not necessarily mean that S/4 HANA is a prerequisite for FPSL.