Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
[This article is prerelease documentation and is subject to change.]
Orchestrate multimodal AI insights (preview) lets you use a structured framework to generate AI enrichments from your data in the silver lakehouse. It integrates the AI-generated enrichments into your healthcare data estate in the enrichment store through a prebuilt pipeline. You can deploy and configure this capability after deploying healthcare data solutions to your Fabric workspace and the healthcare data foundations capability. This article outlines the deployment process and explains how to set up the sample data.
Orchestrate multimodal AI insights (preview) is an optional capability under healthcare data solutions in Microsoft Fabric. You have the flexibility to decide whether or not to use it, depending on your specific needs or scenarios.
Prerequisites
Install the foundational notebooks and pipelines in Deploy healthcare data foundations.
Run the appropriate data transformation pipeline for the data and model you're using:
- For the Text Analytics for health model, refer to Deploy healthcare data foundations and Ingest clinical data.
- For the MedImageInsight and MedImageParse models, refer to Deploy and configure DICOM data transformation and Use DICOM data transformation.
Deploy orchestrate multimodal AI insights (preview)
You can deploy the capability and the sample data using the setup module explained in Healthcare data solutions: Deploy healthcare data foundations. Alternatively, you can also deploy the sample data later using the steps in Deploy sample data.
If you didn't use the setup module to deploy the capability and want to use the capability tile instead, follow these steps:
Go to the healthcare data solutions home page on Fabric.
Select the orchestrate multimodal AI insights (preview) tile.
On the capability page, select Deploy to workspace.
To connect the data linked to an Azure Key Vault resource associated with your healthcare data solutions environment, enter the name of the key vault resource. This value should reference the key vault service deployed with the Healthcare data solutions in Microsoft Fabric Azure Marketplace offer. Hence, make sure you first complete the steps in Deploy Azure Marketplace offer and set up resources. You can skip providing the parameter values for Fhir Server Uri and Export Start Time. You can also skip the steps related to the FHIR service setup (steps 6 to 9) in the guidance.
The capability uses this key vault to store secrets in the prebuilt use cases.
The deployment can take a few minutes to complete. Don't close the tab or the browser while deployment is in progress. While you wait, you can work in another tab.
After the deployment completes, you can see a notification on the message bar.
Select Manage capability from the message bar to go to the Capability management page.
Here, you can view, configure, and manage the artifacts deployed with the capability.
Artifacts
The capability installs the following artifacts in your healthcare data solutions environment:
| Artifact | Type | Description |
|---|---|---|
| healthcare#_msft_ai_enrichments_metadatastore | Lakehouse | Stores the metadata for AI orchestration to effectively capture and manage enrichment definitions, view definitions, and contextual information. |
| healthcare#_msft_ai_enrichments_bronze_ingestion | Notebook | Converts the AI model output in the Process folder to the respective delta tables in the bronze lakehouse. |
| healthcare#_msft_ai_enrichments_silver_ingestion | Notebook | Converts the AI model output results in the respective delta tables to the appropriate data model in the silver lakehouse. |
| healthcare#_msft_ai_enrichments_ingestion | Data pipeline | Sequentially runs the healthcare#_msft_ai_enrichments_bronze_ingestion and healthcare#_msft_ai_enrichments_silver_ingestion notebooks to transform the AI output from a raw state in the bronze lakehouse to a transformed state in the silver lakehouse. |
| healthcare#_msft_ai_enrichments_ta4h_execution healthcare#_msft_ai_enrichments_medimage_insight_execution healthcare#_msft_ai_enrichments_medimage_parse_execution |
Notebooks | Each model execution notebook defines the enrichment view and enrichment definition based on the model configuration and input mappings. It also specifies the model processor and transformer. The model processor calls the model API, and the transformer produces the standardized output while saving the output in the bronze lakehouse in the Ingest folder. These model execution notebooks use either the Text Analytics for health, MedImageInsight, or MedImageParse healthcare models. |