Lesson 3 of 5
Serve the Model with a Local HTTP API
Create a small web service that loads the registered model and returns predictions when it receives a POST request at `http://localhost:5000/predict`.
7 minMLOps
After this lesson, you will be able to create a local web service that loads a registered MLflow model and returns predictions through an HTTP API.
Set Up the Serving Environment
You will use Flask, a lightweight web framework for Python, to build the API. Install Flask with a version bound to ensure compatibility.
pip install "flask>=2,<3"Create a new file called serve.py in the same directory as your previous scripts. This script will load the registered model and define the prediction endpoint.
Load the Registered Model
Start by importing the necessary modules and loading the model from the MLflow registry. Use the model name iris_classifier and version 1, which you registered in the previous lesson.
import mlflow
from flask import Flask, request, jsonify
model = mlflow.sklearn.load_model("models:/iris_classifier/1")
app = Flask(__name__)The mlflow.sklearn.load_model function loads the model in the same way as in the previous lesson. Flask creates the web application object.
Define the Prediction Endpoint
Add a route that accepts POST requests at /predict. The client will send input features as JSON, and the server will return predictions as JSON.
@app.route("/predict", methods=["POST"])
def predict():
data = request.get_json()
features = data["features"]
predictions = model.predict(features)
return jsonify({"predictions": predictions.tolist()})The request.get_json() method parses the incoming JSON body. The body must contain a key features with a list of lists, where each inner list is one sample. The model's predict method returns a NumPy array, which we convert to a Python list with tolist() so it can be serialised to JSON.
Run the Service
Add a main block to start the Flask development server on port 5000.
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)Run the script from the command line:
python serve.pyYou should see output indicating that the server is running on http://0.0.0.0:5000. Keep this terminal open.
Test the API
Open a new terminal and send a POST request using curl. The iris dataset has four features: sepal length, sepal width, petal length, and petal width. For example, send a sample with values [5.1, 3.5, 1.4, 0.2].
curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"features": [[5.1, 3.5, 1.4, 0.2]]}'The server responds with a JSON object containing the predictions. For this sample, the model should predict class 0 (setosa). The exact output may vary slightly depending on the trained model, but it will be a list of integers.
You can send multiple samples at once by including more inner lists in the features array.
Handle Errors Gracefully
A robust API should return a clear error if the request is malformed. Add a check for missing or invalid data.
@app.route("/predict", methods=["POST"])
def predict():
data = request.get_json()
if not data or "features" not in data:
return jsonify({"error": "Missing 'features' key in JSON body"}), 400
features = data["features"]
try:
predictions = model.predict(features)
except Exception as e:
return jsonify({"error": str(e)}), 400
return jsonify({"predictions": predictions.tolist()})This returns a 400 status code with an error message if the request is invalid. The try block catches errors from the model, such as wrong feature dimensions.
Stop the server with Ctrl+C, update serve.py with the error handling code, and restart it. Test an invalid request:
curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"wrong_key": []}'You should receive a 400 response with the error message.
Complete serve.py
Here is the full serve.py file after all changes:
import mlflow
from flask import Flask, request, jsonify
model = mlflow.sklearn.load_model("models:/iris_classifier/1")
app = Flask(__name__)
@app.route("/predict", methods=["POST"])
def predict():
data = request.get_json()
if not data or "features" not in data:
return jsonify({"error": "Missing 'features' key in JSON body"}), 400
features = data["features"]
try:
predictions = model.predict(features)
except Exception as e:
return jsonify({"error": str(e)}), 400
return jsonify({"predictions": predictions.tolist()})
if __name__ == "__main__":
app.run(host="0.0.0.0", port=5000)This service is now ready to serve predictions from your registered model.
Key takeaways
- Flask provides a simple way to expose a model as an HTTP API.
- Load the model once at startup using
mlflow.sklearn.load_model. - The
/predictendpoint expects a JSON body with afeatureslist and returns a JSON object with predictions. - Always validate input and handle errors to return meaningful HTTP status codes.
- Run the service with
python serve.pyand test it usingcurl.
Try it
Stop the service from earlier with Ctrl+C if it is still running, then create a new script serve_v2.py that loads the latest version of the model you registered under a different name in the previous lesson's exercise (e.g., iris_classifier_v2). Use the same Flask setup, but change the endpoint to /predict_v2. Run the service and send a POST request with two iris samples of your choice. Verify that the response contains two predictions. A good result is the service running without errors and returning a JSON object with a predictions list of length 2.
Official documentation
Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.