Machine learning in Azure: How do I publish a pipeline to the workspace once I've already built it in Python using the SDK?

Question

I don't know where else to ask this question so would appreciate any help or feedback. I've been reading the SDK documentation for azure machine learning service (in particular azureml.core). There's a class called Pipeline that has methdods validate() and publish(). Here are the docs for this:
https://docs.microsoft.com/en-us/python/api/azureml-pipeline-core/azureml.pipeline.core.pipeline.pipeline?view=azure-ml-py
When I call validate(), everything validates and I call publish but it seems to only create an API endpoint in the workspace, it doesn't register my pipeline under Pipelines and there's obviously nothing in the designer.
My question: I want to publish my pipeline so I just have to launch from the workspace with one click. I've built it already using the SDK (Python code). I don't want to work with an API. Is there any way to do this or would I have to rebuild the entire pipeline using the designer (drag and drop)?

Anders Swanson · Answer

Totally empathize with your confusion. Our team has been working with Azure ML pipelines for quite some time but PublishedPipelines still confused me initially because:

what the SDK calls a PublishedPipeline is called as a Pipeline Endpoint in the Studio UI, and
it is semi-related to Dataset and Model's .register() method, but fundamentally different.

TL;DR: all Pipeline.publish() does is create an endpoint that you can use to:

schedule and version Pipelines, and
re-run the pipeline from other services via a REST API call (e.g. via Azure Data Factory).

You can see PublishedPipelines in the Studio UI in two places:

Pipelines page :: Pipeline Endpoints tab
Endpoints page :: Pipeline Endpoints tab

Machine learning in Azure: How do I publish a pipeline to the workspace once I've already built it in Python using the SDK?

One Answer

Add your own answers!

Ask a Question