Bobby Huang
Author Bobby Huang

Real Time Data Access – Fast and Simple

September 04 2019 | Conduit

Conduit was designed to help organizations overcome the technical challenges of data accessibility, so they can begin to unlock the value of the data they are paying to store.

Blueprint Technologies was founded in 2013 as a management consulting firm specializing in building custom analytics tools and services. As we take a closer look at recent client engagements, a few data management trends have become more pronounced:

  • Companies are capturing and storing data at a tremendous rate, resulting in skyrocketing cloud storage and on-prem costs. This has been evident in our own engagements and many research articles have covered this very topic: Forbes    Cisco Study   Markets & Markets Study
  • Compute capabilities have become more complicated as data sets sit across multiple environments, in multiple source languages with varying security and access requirements:
"And through all the cloud complication, you could think about consumer data sitting in one repository, customer data sitting in another repository, ingress and egress of data as data scientists start looking at the data to mine it, to get some actual insights. There are a lot of complications and every egress and ingress costs associated with it." - Javier Polit, Chief Information Officer Procter & Gamble
  • Despite the growing costs of storage and complexities with accessibility, business leaders continue to push for solutions that unlock value from disparate datasets, especially tools that enable real time reporting. This should come as no surprise given the amount of tech press coverage “big data” has received the last few years. You can read more about it here:  Forbes    McKinsey    NJIT Infographic

Unlocking true value is difficult

Data accessibility is complex and challenging:

  • Not all data types are easy to connect to and analyze.
  • Data sits in different environments and can’t be brought together.
  • ETLs are time consuming and very expensive.
  • Real-time refreshing capabilities are limited.
  • Source data structure wasn’t originally set up for high velocity of reads.
  • Managing permissions can be cumbersome as more teams want access to more data.
  • And the list goes on…

These complexities are why Conduit exists.

Conduit was designed to help organizations overcome the technical challenges of data accessibility, so they can begin to unlock the value of the data they are paying to store.

How to unlock the value of data

Once Conduit is part of an organization’s data management journey, companies can start to harness their current data assets and extract their value.

The technology works like this: Once a data set is identified, find the source type in Conduit’s catalogue of connectors and create a connector. This is simple, just input the data location url plus credentials, select caching timeframe preference as well as which groups should have access and you’re ready to go! It shouldn’t take more than 10 minutes. Conduit then creates a Spark connector which can be accessed via any visualization tool (tableau, power BI) or to any API. This allows for real time direct access to the data set. Source data read concerns are minimized because the volumes of reads can be controlled by how often the data is cached. It’s simple, fast and easy.

Due to Conduit’s unique capability to translate data, it enables what we call hybrid multi-cloud joins. This is where different data types sitting in different cloud environments can be brought into one visualization platform and joined together to appear as one compatible data set. For example, a csv customer ID list sitting in Azure blob storage and a customer purchase history log in a PostgreSQL table in AWS can all be brought into Tableau and joined by primary key. Companies that store data in different cloud platforms no longer need to rely on moving data around for connectivity and analysis, ultimately saving time and resources.

Conduit also provides centralized security with Active Directory. This feature allows permissions to be managed and the data group and user group level rather than one user, one data set; greatly reducing level of effort in managing data security.

Now is the time to act

The growth in digital data volume isn’t slowing down anytime soon. As IoT and the use of digital devices continue to proliferate, enterprises will be sitting on massive amounts of data. This growth in volume is predicted to outpace data center capacity, resulting in higher storage costs. Enterprises will be under pressure to make all that data more accessible and justify the resulting costs.  

Conduit empowers enterprises to unlock the full value of their data assets by enabling direct real-time access to source data that existing visualization platforms cannot natively connect to.

Blueprint centers every opportunity on the principle that with the right vision, time and resources, anything can be accomplished. They call this the “Art of the Possible.” Conduit is one tool that makes achieving possibilities a lot easier. 

Get Conduit today

Get a Conduit demo or install today in your Azure environment. Unleash the power of your data now.