Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
A data pipeline run occurs when a data pipeline is executed. This means that the activities in your data pipeline ran and were executed to completion. For example, running a data pipeline with a Copy data activity performs that action and copy your data. Each data pipeline run has its own unique pipeline run ID.
You can start pipeline runs in three ways:
On-demand runs: Select Run in the pipeline editor to trigger an immediate run. You'll need to save any changes before the pipeline starts.
Scheduled runs: Set up automatic runs based on time and frequency. When you create a schedule, you specify start and end dates, frequency, and time zone.
Event-based runs: Use event triggers to start your pipeline when specific events occur, such as new files arriving in a data lake or changes in a database.
On-demand data pipeline run
To manually trigger a data pipeline run, select Run found in the top banner of the Home tab.
You can also select Schedule in the top banner of the Home tab and select Run now to trigger an immediate run.
You are prompted to save your changes before triggering the pipeline run. Select Save and run to continue.
After your changes are saved, your pipeline will run. You can view the progress of the run in the Output tab found at the bottom of the canvas.
Once an activity completes in a run, a green check mark appears in the corner of the activity.
Once the entire pipeline executes and the output status updates to Succeeded, you have a successful pipeline run!
Scheduled data pipeline runs
When you schedule a data pipeline run, you can set multiple, specific schedules for each pipeline, so your data is prepared and available when you need it.
Select Schedule, found in the top banner of the Home tab, and then select Add Schedule to view your options. By default, your data pipeline isn't set on a schedule.
On the Schedule configuration page, you can specify a schedule frequency, start and end dates and times, and time zone.
Tip
When scheduling a pipeline, you must set both a start and end date. There's no option for an open-ended schedule. To keep a pipeline running long-term, set the end date far in the future (for example, 01/01/2099 12:00 AM). You can update or stop the schedule at any time.
Once configured, select Save to set your schedule.
You can add up to 20 schedules for a single pipeline by selecting Add Schedule again after saving your first schedule. Each schedule can have different frequencies and start and end times.
Manage scheduled runs
You can manage your scheduled runs by selecting Schedule in the top banner of the Home tab. From there, you can edit existing schedules, or enable or disable schedules using the toggle switch.
To delete a schedule, select the Edit icon (pencil) next to the schedule you want to delete. In the Edit Schedule pane, select Delete schedule at the bottom of the pane.
