Our client, a global leader in the Entertainment/Restaurant (Eatertainment) industry, had a vital need to analyze a variety of very large data sets, including marketing, operations, and privacy law compliance data due to explosive growth. These objectives were blocked by out-of-date hardware and architecture, the extreme volume of the data itself, and the fact that their four main information types (Game Transactions, In-store Location Data, Point-of-Sale (POS) Data, and Mobile Data) were of varying formats stored in different locations.
The Blueprint Way
Once it was established that all four data sources were vital to achieving the business goals, the focus shifted to planning how to aggregate, store, and make that data available for analysis. Blueprint utilized a data-acquisition pattern called a Metadata Driven Azure Data Factory to move 15 terabytes of data from SQL servers to an Azure Data Lake and then implemented Databricks Delta Lake to form a “Lake House” that married customer-level data from Gaming, POS, and Mobile sources. This data is now queryable for the first time with Databricks. In addition, Blueprint engineers created Azure Event Hubs to stream data that contains Personally Identifiable Information (PII) and split that data, sending PII data to a database with PII-masking triggers, and sending non-PII data into the Data Lake, ensuring that compliance with global privacy regulations is achievable.