11,795 questions with Azure Data Factory tags

Sort by: Updated
1 answer

How to fix ERROR [HY000] [Microsoft][Xero] (1019) Unable to complete RSA Signature: error:1E08010C:DECODER routines::unsupported ERROR [HY000] [Microsoft][Xero] (1019) Unable to complete RSA Signature: error:1E08010C:DECODER routines::unsupported

Hi All, I am using Azure Default Xero connector + OAuth2.0 flow. I have supplied the required information Client ID Client Secret Tenant Id REfresh Token If I click on TEst Connection , I get below mentioned error. Can you please help ? ERROR…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-29T04:09:20.3166667+00:00
Bhagat, Anil 0 Reputation points
edited a comment 2025-08-29T18:07:05.36+00:00
Kalyani Kondavaradala 1,015 Reputation points Microsoft External Staff Moderator
0 answers

Timestamp column from Parquet file (created with Pandas) becomes NULL values in Data Flow

Hello, I'm facing an issue where a timestamp column in a Parquet file is consistently read as NULL within an Azure Data Flow. Critically, the Data Flow does not throw any errors, making the problem difficult to debug. Here is a summary of my environment…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-26T04:25:25.26+00:00
hh 0 Reputation points
commented 2025-08-29T18:01:13.38+00:00
Smaran Thoomu 29,500 Reputation points Microsoft External Staff Moderator
2 answers

How to identify the user who manually executed an Azure Data Factory pipeline?

Someone manually triggered a pipeline in Azure Data Factory and I need to identify which user executed it. However, I’m not seeing any caller ID or user information in the run history under the Monitor tab in ADF Studio. Is there a way to retrieve the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-28T10:12:43.9466667+00:00
EWDService 0 Reputation points
commented 2025-08-29T17:42:34.34+00:00
Swapnesh Panchal 5 Reputation points Microsoft External Staff Moderator
1 answer

Connect Dataverse to Sql Sever using Azure data factory

How do we achieve parallelism while loading data from Dataverse to Sql Server and wise versa. if we set degree for parallelism 4 while copy data from Dataverse to Sql Server in ADF, will it launch 4 copy activities to process the data ? if we use…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-29T11:20:57.89+00:00
Mareedu, Tulasiram (Cognizant) 0 Reputation points
answered 2025-08-29T16:24:36.27+00:00
Vinodh247 37,216 Reputation points MVP Volunteer Moderator
1 answer One of the answers was accepted by the question author.

Data Flow Sink with "Continue on Error" and Output to separate file not getting created

Hello, I have a Sink destination linked to Azure SQL DB in a data flow. Where I wanted to capture database rejected records into Azure BLOB file. I have tried choose to Continue on error with Transaction commit Single/Batch and selected Output to…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-22T18:53:48.97+00:00
Karthik K Koneru 21 Reputation points
accepted 2025-08-29T16:21:13.4166667+00:00
Karthik K Koneru 21 Reputation points
1 answer

ADF-Dataflow Table sync records not processed after 12 hrs

I have a pipeline which uses a dataflow with SQL CDC feature enabled. I am syncing tables between one Azure SQL to another Azure SQL DB. Since yesterday the pipeline is failing after running for 12 hrs. A particular table dataflow activity is getting…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-26T11:34:13.23+00:00
Aaron Wong 40 Reputation points
commented 2025-08-29T16:18:17.47+00:00
Smaran Thoomu 29,500 Reputation points Microsoft External Staff Moderator
1 answer One of the answers was accepted by the question author.

parallelism in ADF

Hi, how parallelism works in ADF..Please explain with scenario

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2024-11-19T14:09:40.22+00:00
Vineet S 1,390 Reputation points
commented 2025-08-29T11:31:33.4233333+00:00
Mareedu, Tulasiram (Cognizant) 0 Reputation points
2 answers

Update Oracle Connection version 2.0 from 1.0

This is connected to another question I have. https://learn.microsoft.com/en-us/answers/questions/5490043/adf-oracle-connection-deprecated-version-1-0 Since the version 1 will be decommissioned soon. We need to update our Oracle connections to version 2.…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-22T14:59:35.27+00:00
Colinn Calaguas 70 Reputation points
commented 2025-08-29T11:21:15.1966667+00:00
Colinn Calaguas 70 Reputation points
2 answers

ADF can not parse Parquet files

I have a pipeline that consist copy data activity like following; I installed OpenJDK on all servers where we use our Integration Runtime. But still, I get error like the following; { "errorCode": "2200", …

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,432 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2022-11-24T08:21:06.88+00:00
Melih Kavraz 1 Reputation point
commented 2025-08-28T22:38:13.99+00:00
Najeebullah Hussaini 0 Reputation points
2 answers One of the answers was accepted by the question author.

The ADF Databricks job Activity doesn't seem to pass parameters

I have created an ADF pipeline that uses the databricks job activity. The linked service works well. When I run the pipeline, the databricks job fails - the ADF component fails, and I see a failed job run in databricks too. The problem seems to be the…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-05-14T12:15:22.5966667+00:00
Debie, Hanneke 35 Reputation points
commented 2025-08-28T18:30:01.7333333+00:00
Erick Salazar 0 Reputation points
1 answer

Azure Data Factory - Save output of stored procedure activity to table

I have scoured online looking for a good answer but I have not found anything concrete, so posting here. When using the Stored Procedure activity, is it possible to send the results to an Azure SQL Database table? It is 2025, so I certainly hope so. But…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-26T20:52:34.03+00:00
Scott Klein 5 Reputation points
commented 2025-08-28T16:10:56.2766667+00:00
Abhisek Mishra 85 Reputation points Microsoft External Staff Moderator
1 answer

Azure Data factory use copy activity to get Jira issues via REST API

Hi, I'm using Azure data factory to get Jira issues via HTTP REST API. Linked service works fine. Base URL: https://my-organization.atlassian.net/ Relative URL:…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-28T00:45:58.6366667+00:00
Bo Xiao 66 Reputation points
commented 2025-08-28T14:08:48.76+00:00
Smaran Thoomu 29,500 Reputation points Microsoft External Staff Moderator
0 answers

Can we map the program and customer (relate) using ADF pipeline?

Can we map the program to customer using ADF pipeline? like how we used to create customer, contacts in ADF. when I tried to map the customer with program using ADF got the below error " Operation on target Dynammics customer program map…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-28T03:51:48.3133333+00:00
Sharifuinnisa H (INFOSYS LIMITED) 0 Reputation points Microsoft External Staff
commented 2025-08-28T13:15:20.74+00:00
Smaran Thoomu 29,500 Reputation points Microsoft External Staff Moderator
1 answer

How do I call an API that redirects on the backend with a Copy Activity in Azure Data Factory without an authentication error?

I'm running into issues in Azure Data Factory (ADF) with an API I am working with. I am unfortunately not at liberty to share the API URL or much data on the call I am making, but I can say that I have been testing this API call in Postman and through…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2024-05-16T14:25:53.8566667+00:00
Westerfield, Jared T 5 Reputation points
commented 2025-08-28T13:03:50.18+00:00
Tom Legge 0 Reputation points
1 answer

snowflake scan from purview

when scanning snowflake public account endpoint from the purview I am getting and error "Error: (3805) Connector Exception: Configuration file conf/ModelBridgeList.xml is invalid". Same error self hosted and Microsoft integration runtime. It…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-27T16:58:12.71+00:00
prasantc 976 Reputation points
answered 2025-08-28T11:26:21.7033333+00:00
Venkat Reddy Navari 5,815 Reputation points Microsoft External Staff Moderator
0 answers

The ADF Xero connector has started failing since a recent Xero API update

A recent Xero API update has caused our ADF Xero connector to fail across all pipelines, do we have a resoltuion date?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-26T09:23:39.61+00:00
Dataloop 0 Reputation points
commented 2025-08-28T07:00:05.65+00:00
Venkat Reddy Navari 5,815 Reputation points Microsoft External Staff Moderator
1 answer One of the answers was accepted by the question author.

Get data from log analytics with system assigned managed identity

Hi, I have set up a connection to log analytics in ADF. I grant access to the Log analytics resource for data factory. In a web activity it works fine and I see data in the activity, but in a copy activity I get empty results: I am using the same…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-27T09:38:05.8666667+00:00
Ronald Mussche 56 Reputation points
accepted 2025-08-28T06:46:51.3166667+00:00
Ronald Mussche 56 Reputation points
1 answer

Error pipeline step Extraction from SAP COEP Table

Hi i have a pipeline for extracting SAP History data of COEP table with a filter year GE 2024, however i'm getting an error like this. Failure happened on 'Source' side.…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-27T21:39:38.71+00:00
Hernando Jesus Urbina Perez 0 Reputation points
answered 2025-08-28T02:58:48.13+00:00
Suwarna S Kale 3,956 Reputation points
1 answer

Azure Data factory V2 for each loop run twice in pipeline with couple of seconds delay

Hi,We had one incident in our production environment of Azure data factory v2 instance, where for-each iterator loop run twice in one single run of pipeline with some couple of seconds delay. In foreach loop iterator, we are doing some data load steps…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2025-08-28T00:53:42.11+00:00
Kartheek Vasantha 0 Reputation points
answered 2025-08-28T02:56:10.9766667+00:00
Suwarna S Kale 3,956 Reputation points
1 answer

Azure Data factory V2 for each loop run twice in pipeline with couple of seconds delay

Hi, We had one incident in our production environment of Azure data factory v2 instance, where for-each iterator loop run twice in one single run of pipeline with some couple of seconds delay. In foreach loop iterator, we are doing some data load…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,795 questions
asked 2020-09-29T09:18:44.867+00:00
Mansing 6 Reputation points
answered 2025-08-28T00:49:15.4333333+00:00
Kartheek Vasantha 0 Reputation points