In today's data-driven world, effective data management is crucial for businesses looking to stay ahead of the competition. Microsoft Fabric emerges as a powerful solution designed to simplify and optimise data management. This comprehensive suite offers a range of tools that cover everything from data integration and storage to sophisticated analytics and visualisation. Whether you are a seasoned data professional or a business leader looking to enhance your data strategy, Microsoft Fabric provides the capabilities needed to turn data into actionable insights.
A lot has been said lately about Microsoft's new tool, Microsoft Fabric. But what is Fabric anyway? The truth is that, just as the tool is multifaceted, there is also more than one possible answer to this question. Although I'll provide the technical answer later, a developer (which is my case) would answer by saying that it's a practical and versatile tool that speeds up development, while a manager would say that it's an excellent resource for those looking to improve business efficiency by reducing costs.
Microsoft Fabric aims to democratise access to information and the means of analysing it. The platform aims to change the paradigm of how data is worked to generate information, thus allowing everyone to have access to it, not just data analysis professionals.
In this article, I aim to deepen the reader's understanding of the heart of the Microsoft Fabric structure, which is OneLake, and also to give a general understanding of the tools contained in the platform.
WHAT IS MICROSOFT FABRIC ANYWAY?
The technical answer is that Microsoft Fabric is a data analysis platform from Microsoft and Azure, where we have various tools for different workloads in a single solution focussed on a central data lake called OneLake. The platform meets all the most common needs in a cloud data environment, from data storage and migration, through business intelligence and data science to real-time data analysis, but still manages to remain cohesive while combining various tools and technologies in one place.
Microsoft Fabric is a platform that aims to facilitate and speed up data development. Enabling organisations to consolidate data from multiple sources in a single environment provides flexibility that allows data professionals to focus more on the end product and less on the technologies needed to develop it. In more practical terms, it is no longer necessary to spend hours and hours discussing how many and which licences will be required to carry out a project.
Figure 1. Diagram of different experiences all accessing the same OneLake data storage. (Source: What is Microsoft Fabric?)
ONELAKE
As said before, Microsoft Fabric is a platform focused on a central data lake called OneLake. This central lake on an organisational scale, or "OneDrive for data", as Microsoft itself often refers to it, is where all the data will be centralised. This allows for a change in approach, making it no longer necessary for each department to manage its own data lake. Instead, all the data is concentrated in OneLake, where the required warehouses and lakehouses will be created. This centralisation makes it easier to manage the data itself and allows for centralised data governance.
OneLake is built on second-generation Azure Data Lake Storage and supports any file format; whether structured or unstructured, it accepts it all. Within the same organisation (or a tenant, in Fabric terminology), it is possible to create individual workspaces, each with its access and data management policies, which gives the flexibility to meet the specific needs of each sector within the company.
THE TOOLS
Any platform that aims to fulfil these many different types of tasks will also need a complex and robust set of tools that enable it to do so, and indeed, Microsoft Fabric has a vast, eclectic and yet cohesive set of tools.
This set of tools gives users agility and practicality in their work while OneLake maintains the governance and security of the data and the platform.
Here’s a comprehensive overview of the tools available on the Microsoft Fabric platform today and their primary functions:
Data Factory
It is a data orchestration platform that allows you to create, schedule, and manage ETL (Extract, Transform and Load) or ELT (Extract, Load and Transform) workflows.
It allows you to move, transform and integrate data from various sources to destinations such as Azure SQL Data Warehouse or Power BI.
It is a powerful tool for creating data pipelines and automating ingestion and transformation processes with over 200 available connectors.
Synapse Data Engineering
One of the core experiences of Microsoft Fabric plays two main roles: creating lakehouses and notebooks.
Lakehouses are logical lakes contained within OneLake.
Notebooks enable data processing using Apache Spark
Synapse Data Science
Is a machine-learning platform.
It offers tools for training, deploying and managing machine learning models.
Data scientists can use Synapse Data Science to create and experiment with predictive models.
Synapse Data Warehouse
Is a fully managed and scalable data warehouse service.
It allows you to store large volumes of data and run complex queries in parallel.
It is ideal for analysing data and business reports.
The data contained within the logical warehouses created with Synapse Data Warehouse is stored in OneLake using Delta tables.
Synapse Real-Time Analytics
Is a real-time streaming processing tool.
It allows you to process and analyse streaming data from various sources, such as IoT sensors and application logs.
Power BI
It is a data visualisation tool that lets you create interactive dashboards, reports, and graphs.
It integrates well with various data sources, including Data Factory and Synapse Data Warehouse.
With Power BI, you can explore data, create attractive visualisations and share insights.
Data Activator
Is a tool for creating and managing data pipelines.
It allows you to orchestrate data ingestion, transformation and loading workflows.
It's an excellent choice for automating large-scale data processes.
In conclusion, Microsoft Fabric has emerged as a robust and integrated platform that offers a variety of tools designed to facilitate development and collaboration in IT environments.
Among these tools, OneLake stands out for its bold concept and ability to integrate and analyse data, allowing organisations to extract valuable insights efficiently and quickly. The platform has been designed to be adaptable and scalable, both of which are essential characteristics in today's constantly evolving technological landscape. Microsoft Fabric offers several promising solutions and its applicability and potential impact on organisations is quite vast.
In my opinion, it is a platform that has in its DNA the continuous quest to improve and revolutionise industry standards as we know them today.
______
by Bruno Gomes
@ Passio Consulting
コメント