I am excited to share that I have been got a chance to explore about mirroring in MS fabric, here I am writing this blog to share my views on using mirrors in Microsoft Fabric based on my personal experience. This is a great chance for me to delve into the intricacies of Microsoft Fabric and highlight the benefits and challenges of using mirrors within this framework. I will be discussing various aspects such as setup, configuration, and practical applications.
What is Mirroring in Fabric?
Mirroring in Fabric is a low-cost, low-latency data replication technology that unifies data from several systems into a single analytics platform. Your current data estate, including information from Snowflake, Azure Cosmos DB, and Azure SQL Database, may be continually replicated onto Fabric’s OneLake.
Now that you have access to the most recent data available in One Lake’s queryable format, you can leverage all of Fabric’s capabilities, including data engineering, notebook execution, Spark analytics, visualizing with Power BI Reports, and more.
With Fabric, users help from a highly integrated, comprehensive, and user-friendly solution that streamlines their analytics requirements thanks to mirroring. Created to foster transparency and cooperation between Microsoft and third-party technology solutions capable of interpreting the open-source Delta Lake table format, Mirroring is a low-cost, low-latency turnkey solution that lets you make a copy of your data in OneLake for use in any kind of analysis.
Why Use Mirroring in Fabric?
Mission-critical operational or analytical data is often stored in silos in many businesses today. Complex ETL (Extract Transform Load) pipelines, business procedures, and decision silos are now necessary for accessing and handling this data, resulting in:
Constrained access to constantly changing significant data, conflict between technology, procedure, and people, prolonged delays in establishing vital data pipelines, lack of independence to utilize required resources for examination and sharing ideas, insufficient basis for people to collaborate on data, and absence of standard open data formats for various analytical scenarios such as BI, AI, Integration, Engineering, and Apps.
Enable Mirroring
Using the setting included in the Power BI admin site, Power BI administrators may choose to allow or disable Mirroring for certain security groups or the whole company.
Once you select the Dataware housing in MS fabric, you can see options to create mirrored DB like Azure SQL DB, Cosmos DB, Mirrored Azure SQL Managed, snowflake as highlighted in.
Eg: I have chosen Azure cosmos DB as below:
Below is the detailed link where you can configure the Azure Cosmos DB from Microsoft Learn site.
Mirrored databases are an item in Fabric Synapse Data Warehousing distinct from the Warehouse and SQL analytics endpoint.
The object in the mirrored database. Data replication into OneLake and conversion to Parquet in a format suitable for analytics are managed by mirroring. Data engineering, data science, and other downstream scenarios are made possible by this.
SQL analytics endpoint is produced automatically.
The default semantic model is produced automatically.
Note: Mirroring for Azure Cosmos DB is currently in preview. Production workloads aren’t supported during preview. Currently, only Azure Cosmos DB for NoSQL accounts are supported.
Benefits of Mirroring in MS Fabric
High Availability
Mirroring in MS Fabric provides high availability by maintaining a synchronized copy of the data in a secondary location, which can be used in case the primary location fails, ensuring continuous availability of the data.
Disaster Recovery
Mirroring in MS Fabric provides disaster recovery capabilities by maintaining a secondary copy of the data in a separate location, which can be used to recover the data in case of a disaster at the primary location.
Read Scaling
Mirroring in MS Fabric provides read scaling capabilities by allowing read operations to be performed on the secondary copy of the data, which reduces the load on the primary location and improves the overall performance of the system.Mirroring enables you to create read replicas for scaling out read workloads. This allows you to distribute read traffic across multiple replicas, improving the performance of your applications.
Limitations in Microsoft Fabric mirrored databases from Azure Cosmos DB (Preview)
When continuous backup is enabled on Azure Cosmos DB accounts, you cannot disable the analytical store functionality.
On an Azure Cosmos DB account that has previously deactivated the analytical store capability for a container, you cannot setup continuous backup.
Among these restrictions include the inability to stop continuous backup after it has been turned on, as well as the absence of support for multi-region write accounts. See Azure Cosmos DB continuous backup limits for further details.
On a single Azure Cosmos DB account, you may enable both the analytical store and continuous backup functionalities.
You must have either the admin or member role in your workspace in order to enable mirroring for an Azure Cosmos DB account.
Mirroring is fully disabled when replication is stopped.
Restarting replication seeds every target warehouse table anew. Mirroring is essentially restarted by this action.
Backup
Backing up data involves making a copy of the data and storing it in a separate location for safekeeping. It provides disaster recovery capabilities and helps to mitigate the risk of data loss.
While security features like Row-level security (RLS), Object level security (OLS), and others ensure that you can limit access to critical information, sharing makes access control and administration easier. Secure and democratic decision-making inside your company is another benefit of sharing.
By sharing, users may make a mirrored database accessible to other users or a group of users, but they keep the workspace and its contents private. Access to the default semantic model and SQL analytics endpoint are also provided when someone shares a mirrored database.
Conclusion:
High Availability
Mirroring in MS Fabric provides high availability for your application, ensuring that it is always up and running, even in the case of hardware or software failures.
Disaster Recovery
Mirroring in MS Fabric provides disaster recovery capabilities, allowing you to recover your data and applications in the event of a catastrophic failure.
Read Scaling
Mirroring in MS Fabric provides read scaling capabilities, allowing you to distribute read traffic across multiple replicas of your data, improving performance and reducing latency.
Happy reading!