Storage account file event is not triggered for Azure Databricks on SFTP upload

Hi Dimitry Nechaev,

Thank you for reaching out on Microsoft Q&A forum

To understand the issue briefly, could you please share the screen shot of container settings & full overview shot as well?

Thanks

@Priya ranjan Jena which settings of the container? there are encription, sas tokens in settings etc.

as of the account, here they are

I'm not sure how they can help with file events from SFTP uploads

Hi Dimitry Nechaev

Thanks for providing scree shot.

By reproducing the issue in our end ,we find some possibilities that can be checked below.

1.Hierarchical Namespace (HNS) must be enabled for SFTP to be supported in Azure Blob Storage. HNS enables the directory and sub-directory structure that SFTP clients expect when connecting to Azure Storage. Without HNS, SFTP functionality cannot be enabled.

Note:You cannot modify the Hierarchical Namespace (HNS) setting on an Azure storage account after it has been created. The HNS setting is a one-time configuration option during the account creation process. Once a storage account is created with or without HNS enabled, that setting cannot be changed, please find the scree shot for reference.

So, you can re-create a new storage account with HNS enabled in that or if you want to enable the hierarchical namespace feature with existing storage account, then you can upgrade Azure Blob Storage with Azure Data Lake Storage capabilities using below reference link, post upgrade you should able to see the below settings under SFTP

https://learn.microsoft.com/en-us/azure/storage/blobs/upgrade-to-data-lake-storage-gen2-how-to?tabs=azure-portal

2.Make sure that your Azure Blob Storage is configured to send events when new blobs are created via SFTP. For SFTP uploads, specific events like Microsoft.Storage.BlobCreated need to be correctly set up. During uploading via SFTP, two events may be generated:

a)SftpCreate for the initial empty blob created when opening the file.

b)SftpCommit when the upload completes.

3.Additionally you can check logs and monitoring that might detail why the trigger isn't firing,which can help in pinpointing the issue resides in event delivery or azures function calling your databricks job.

If you find this comment helpful, Please “up-vote” for the information provided , this can be beneficial to community members.

Kindly let us know if you have any additional questions.

Thanks

hi @Priya Ranjan Jena @Priya ranjan Jena

Thanks for your response.

1. Hierarchical Namespace (HNS) must be enabled for SFTP to be supported in Azure Blob Storage

This is surely enabled just because I cannot have SFTP (nor use in Databricks) if not enabled. So that's done.

2 .Make sure that your Azure Blob Storage is configured to send events when new blobs are created via SFTP.

HOW ? WHERE ??? Please help to navigate.

SftpCommit

I will try adding this into the events monitored in the event grid to see if this help.