Legacy data migration with a custom data factory

Snapshot

In partnership with Databricks, Blueprint designed and built a centralized storage architecture that decreased hardware costs while enabling advanced Data Analytics, 360° Customer views, and improved regulatory compliance.

Our work

The Problem

Our client, a global leader in the Entertainment/Restaurant (Eatertainment) industry, had a vital need to analyze a variety of very large data sets, including marketing, operations, and privacy law compliance data due to explosive growth. These objectives were blocked by out-of-date hardware and architecture, the extreme volume of the data itself, and the fact that their four main information types (Game Transactions, In-store Location Data, Point-of-Sale (POS) Data, and Mobile Data) were of varying formats stored in different locations.

The Blueprint Way

Once it was established that all four data sources were vital to achieving the business goals, the focus shifted to planning how to aggregate, store, and make that data available for analysis. Blueprint utilized a data-acquisition pattern called a Metadata Driven Azure Data Factory to move 15 terabytes of data from SQL servers to an Azure Data Lake and then implemented Databricks Delta Lake to form a “Lake House” that married customer-level data from Gaming, POS, and Mobile sources. This data is now queryable for the first time with Databricks. In addition, Blueprint engineers created Azure Event Hubs to stream data that contains Personally Identifiable Information (PII) and split that data, sending PII data to a database with PII-masking triggers, and sending non-PII data into the Data Lake, ensuring that compliance with global privacy regulations is achievable.

Impact

Let's build your future.

Share with your network

Share on twitter
Share on facebook
Share on linkedin
Share on email

You may also enjoy

Case study

Smart data integration drives sustainable growth

Cloud-based data integration makes an upstream oil and gas provider’s data seamlessly accessible

Case study

Explosive growth experienced through a modernized workplace

Building a nonprofit technology infrastructure for seamless collaboration