Skip to content

Lesson 3 of 5

Serve the Model with a Local HTTP API

Create a small web service that loads the registered model and returns predictions when it receives a POST request at `http://localhost:5000/predict`.

7 minMLOps

After this lesson, you will be able to create a local web service that loads a registered MLflow model and returns predictions through an HTTP API.

Set Up the Serving Environment

You will use Flask, a lightweight web framework for Python, to build the API. Install Flask with a version bound to ensure compatibility.

pip install "flask>=2,<3"

Create a new file called serve.py in the same directory as your previous scripts. This script will load the registered model and define the prediction endpoint.

Load the Registered Model

Start by importing the necessary modules and loading the model from the MLflow registry. Use the model name iris_classifier and version 1, which you registered in the previous lesson.

import mlflow
from flask import Flask, request, jsonify

model = mlflow.sklearn.load_model("models:/iris_classifier/1")
app = Flask(__name__)

The mlflow.sklearn.load_model function loads the model in the same way as in the previous lesson. Flask creates the web application object.

Define the Prediction Endpoint

Add a route that accepts POST requests at /predict. The client will send input features as JSON, and the server will return predictions as JSON.

@app.route("/predict", methods=["POST"])
def predict():
    data = request.get_json()
    features = data["features"]
    predictions = model.predict(features)
    return jsonify({"predictions": predictions.tolist()})

The request.get_json() method parses the incoming JSON body. The body must contain a key features with a list of lists, where each inner list is one sample. The model's predict method returns a NumPy array, which we convert to a Python list with tolist() so it can be serialised to JSON.

Run the Service

Add a main block to start the Flask development server on port 5000.

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

Run the script from the command line:

python serve.py

You should see output indicating that the server is running on http://0.0.0.0:5000. Keep this terminal open.

Test the API

Open a new terminal and send a POST request using curl. The iris dataset has four features: sepal length, sepal width, petal length, and petal width. For example, send a sample with values [5.1, 3.5, 1.4, 0.2].

curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"features": [[5.1, 3.5, 1.4, 0.2]]}'

The server responds with a JSON object containing the predictions. For this sample, the model should predict class 0 (setosa). The exact output may vary slightly depending on the trained model, but it will be a list of integers.

You can send multiple samples at once by including more inner lists in the features array.

Handle Errors Gracefully

A robust API should return a clear error if the request is malformed. Add a check for missing or invalid data.

@app.route("/predict", methods=["POST"])
def predict():
    data = request.get_json()
    if not data or "features" not in data:
        return jsonify({"error": "Missing 'features' key in JSON body"}), 400
    features = data["features"]
    try:
        predictions = model.predict(features)
    except Exception as e:
        return jsonify({"error": str(e)}), 400
    return jsonify({"predictions": predictions.tolist()})

This returns a 400 status code with an error message if the request is invalid. The try block catches errors from the model, such as wrong feature dimensions.

Stop the server with Ctrl+C, update serve.py with the error handling code, and restart it. Test an invalid request:

curl -X POST http://localhost:5000/predict -H "Content-Type: application/json" -d '{"wrong_key": []}'

You should receive a 400 response with the error message.

Complete serve.py

Here is the full serve.py file after all changes:

import mlflow
from flask import Flask, request, jsonify

model = mlflow.sklearn.load_model("models:/iris_classifier/1")
app = Flask(__name__)

@app.route("/predict", methods=["POST"])
def predict():
    data = request.get_json()
    if not data or "features" not in data:
        return jsonify({"error": "Missing 'features' key in JSON body"}), 400
    features = data["features"]
    try:
        predictions = model.predict(features)
    except Exception as e:
        return jsonify({"error": str(e)}), 400
    return jsonify({"predictions": predictions.tolist()})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

This service is now ready to serve predictions from your registered model.

Key takeaways

  • Flask provides a simple way to expose a model as an HTTP API.
  • Load the model once at startup using mlflow.sklearn.load_model.
  • The /predict endpoint expects a JSON body with a features list and returns a JSON object with predictions.
  • Always validate input and handle errors to return meaningful HTTP status codes.
  • Run the service with python serve.py and test it using curl.

Try it

Stop the service from earlier with Ctrl+C if it is still running, then create a new script serve_v2.py that loads the latest version of the model you registered under a different name in the previous lesson's exercise (e.g., iris_classifier_v2). Use the same Flask setup, but change the endpoint to /predict_v2. Run the service and send a POST request with two iris samples of your choice. Verify that the response contains two predictions. A good result is the service running without errors and returning a JSON object with a predictions list of length 2.

Official documentation

Ready to use this in a job?

Rolebound applies to matching roles for you.

Start now

Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.

    Cookies on Rolebound

    We use necessary cookies to sign you in and keep the site working — those cannot be turned off. We would also like to set analytics and advertising cookies so we can see which pages help people get interviews and which adverts brought them here. Nothing optional is set unless you say yes, and you can change your mind at any time from “Cookie settings” in the footer. Read our Privacy Policy.

    Serve the Model with a Local HTTP API | Track, Serve, and Monitor a Model After the Notebook | Rolebound