Azure ML deployment is too complicated for inference

Yes — Azure ML deployment can feel overly complex for simple inference, especially when all you want is: “take a model, expose an endpoint, and send requests.”

Common pain points are:

• too many concepts: workspace, compute, environments, registries, endpoints, deployments
• YAML and CLI steps for things that should be one or two commands
• slow iteration when testing inference code
• more setup than lightweight use cases need

If your goal is just inference, simpler options often work better:

• Azure Container Apps: good for a small API around a model
• Azure Functions: if inference is lightweight and request volume is modest
• App Service / Container Apps with FastAPI or Flask: straightforward custom serving
• Managed online endpoints in Azure ML: still useful if you need ML governance, but not the simplest
• Docker + any container host: often the most direct path

A practical rule

Use Azure ML when you need:

• model versioning
• experiment tracking
• managed deployment lifecycle
• ML ops governance
• scaling and monitoring in one system

Use something simpler when you need:

• just an HTTP inference API
• fast development and easy debugging
• minimal platform overhead

My recommendation

If inference is the main goal and you don’t need full ML platform features, I’d consider:

1. package the model in a small FastAPI app
2. containerize it
3. deploy to Azure Container Apps or App Service

If you want, I can help you with one of these:

• a simpler Azure deployment pattern for inference
• a FastAPI + Docker example
• a comparison of Azure ML vs Container Apps for model serving