Modern data stack to leverage the potential of data
Situation
- Introduction of a modern data stack based on Snowflake (data warehouse), adverity (ELT) and dbt (data modeling)
- Implementation of two initial use cases for Analyzing customer behavior in the online shop and the Evaluation of product performance
Complication
- In a preliminary project among other things, challenges relating to the existing data landscape were identified and a new architecture was proposed
- In addition, use cases to activate the potential of a new data infrastructure were prioritized
- The internal teams also did not yet have the necessary skills to use this type of infrastructure
Solution
- Implementation of various use cases with the aim of analyzing customer behavior in the shop and evaluating product data relating to inventory and sales
- Introduction of a modern data stack by connecting various data sources in order to be able to combine the currently distributed data and evaluate it in a more targeted manner
- Provision of data for reports, dashboards and further analyses
Added value
- Easier and central access to data from various data sources
- Gain new insights by connecting multiple data sources
- Presentation of new findings in dashboards
A Glimpse into the Data Future of Babymarkt
In our preliminary project (see https://www.datainstitute.io/case-studies/babymarkt) We first examined the challenges of Babymarkt's existing data architecture (among others). The result showed that the current data infrastructure was unable to exploit the full potential of Babymarkt's data treasures. It was clear to all parties involved that data would play a particularly important role in maintaining and expanding the market position of babymarkt as No. 1 in Europe for baby and children's products. In particular, the existing infrastructure reached its limits due to increasing requirements in connection with further and more detailed analyses of customers and products. Against this background, we developed a proposal for a new architecture that is heavily based on the modular and best-of-breed approach of the modern data stack and provides the basis for a sustainable Optimizing data infrastructure forms.
In the end, the decision was made in favour of implementing a variant of our recommended setup (see chart) which, on the one hand, resolves the technological deficiencies of the old setup and, on the other hand, the high level of acceptance of the already existing setup visualization tools exhausts. This should be supplemented by implementing strategically relevant use cases to fully activate the potential of the new data infrastructure. Babymarkt attached great importance to constantly expanding its own expertise through close sparring between us and internal colleagues. This collaboration not only enabled sustainable transfer of knowledge, but also supported the successful integration of the new architecture into existing structures.
At the heart of this new technical solution is a Snowflake data warehouse in the AWS cloud, which is complemented by adverity for data integration and dbt for data modeling. Of course, we have also implemented a layer architecture (comparable to “Bronze-Silver-Gold”) and a separate production and development environment. As a scheduling tool, we finally chose GitHub Actions due to the ease of use and the still fairly simple dependencies within the system. In this way, colleagues from babymarkt do not have to learn another complex tool and can instead concentrate fully on data modelling via dbt.
Two initial use cases were implemented to activate the new system: the analysis of customer behavior in the online shop and the evaluation of product performance. Both use cases are aimed specifically at expanding analytical capabilities and achieving one's own business goals. The Modern Data Stack also provides centralized and simplified access to data from various sources. In addition, new insights can be gained through best-practice modelling of this data and data can be visualized in Microstrategy in a simplified way. These measures not only promote data-based decisions, but also create the basis for long-term optimization of business processes and strategies.
However, the implementation of the project also presented specific challenges. One of the key difficulties was the limited internal capacity available to set up and activate the new data architecture. In addition, legal frameworks, in particular with regard to the linking and use of tracking data, had to be taken into account. However, through close cooperation and coordination with colleagues from babymarkt, we were able to overcome these hurdles together and successfully implement the project. Babymarkt now has a scalable modern data infrastructure - the next use cases are already being planned.
Questions about the project?
Roxane Stelzel and Mike Kamysz are the contact for this project.
The contact to our client can be organized on request.
Stay up to date
Follow us on LinkedIn.
Data news for pros
Want to know more? Then subscribe to our newsletter! Regular news from the data world about new developments, tools, best practices and events!
Stay up to date
Follow us on LinkedIn.