Microsoft OneLake – the foundation of Fabric

One Lake

The Data Lake is the underlying technology where all your Fabric data is stored

OneLake is at the heart of Microsoft’s new data product – Microsoft Fabric
The “Data Lake” is a key concept in modern data management and Business Intelligence.
A “Lake House” is a section of OneLake which has been allocated to work on a particular project.

Microsoft OneLake forms Fabric’s foundation because:

  1. All the data in Fabric is stored in the Data lake – “One Lake”
  2. There is only ever one “One Lake” per Fabric Tenant and organization.  So it is clear we all work from one source of the truth, and chaotic proliferation of multiple copies of data is prevented.
  3. It can consume any type of data, structured databases and unstructured files, and start to relate them together in a structured manner for reporting.
  4. OneLake provides a very fast Azure based platform for all data processing, most importantly BI, Analytics and reporting, and also for running your own Machine learning models.
  5. It allows you to control security access at a very granular level and these security controls will then propagate to every application that references this single lake of data.  Azure’s security model working alongside Microsoft Purview mitigates strongly against data loss of any kind.
  6. OneLake prevents the massive proliferation of data copies, and separate silos of data, that has increasingly plagued organizations since big data became a critical part of modern business.
  7. When OneLake links in data from external data sources, it is able to create a shortcut to the data but not import it.  It can still access this data extremely fast.  This allows you to more easily consumer, relate, and report on data form many sources, leave the original data in its native location, and still produce intelligence from it at speed.
  8. It can mitigate against data sovereignty issues (such as those defined by GDPR), by giving reporting access against data without the data having to be copied from its origin.

Microsoft OneLake Architecture

The whole architecture is designed to reduce the headaches that have been seen across all data departments for years, and to help organizations prepare for a future where great BI, Machine learning and AI is a core part of business.  None of those work until your underlying data is in good shape, and

Schematic of Microsoft Fabric

Legacy Data Architectures

Why not continue with data architectures as they were?  There are a number of data challenges which appear to be similar across almost all IT departments and data teams.  These commonly hamper organization’s ability to be productive, or to move towards meaningful use of AI and machine learning.  One Lake is a significant step towards resolving these and facing the future in good shape, and its is the data foundation of Microsoft Fabric.

Now our data is on a firm foundation, let’s look at the tools within Fabric to get it ready for reporting.