Skip to content

Lesson 3 of 6

Find the best line by minimising error

You derive and code the least squares formulas for slope and intercept, then verify that the fitted line has the lowest MSE among all lines.

5 minMachine Learning

After this lesson, you will be able to find the slope and intercept that give the lowest mean squared error for a straight-line model, using the least squares formulas. You will code these formulas in Python and check that the fitted line has a lower MSE than any other line you choose.

The best line is the one with the smallest MSE

You have a model that predicts exam score from hours studied using a straight line. You also have a way to measure how wrong a line is: the mean squared error, or MSE. Now you want the line that is least wrong overall. That line is the one with the smallest possible MSE for your data.

For a straight line, there is a direct way to find the best slope and intercept. It comes from a method called least squares. The idea is to choose the slope and intercept that make the sum of the squared errors as small as possible. The formulas below give exactly those values.

The least squares formulas

Let hours be the list of x-values and scores the list of y-values. Let n be the number of data points. The best slope, often called m, is:

m = (n * sum(x*y) - sum(x) * sum(y)) / (n * sum(x*x) - sum(x)**2)

The best intercept, often called b, is:

b = (sum(y) - m * sum(x)) / n

Here sum(x*y) means the sum of each x multiplied by its matching y. sum(x*x) means the sum of each x squared. The other sums are the usual totals.

These formulas come from setting the derivative of the MSE to zero. You do not need to know calculus to use them. You only need to compute the sums and plug them in.

Code the formulas

Create a new file called least_squares.py. In it, write a function that takes the hours and scores lists and returns the best slope and intercept.

# least_squares.py
def least_squares(hours, scores):
    n = len(hours)
    sum_x = sum(hours)
    sum_y = sum(scores)
    sum_xy = 0
    sum_xx = 0
    for i in range(n):
        sum_xy += hours[i] * scores[i]
        sum_xx += hours[i] ** 2
    slope = (n * sum_xy - sum_x * sum_y) / (n * sum_xx - sum_x ** 2)
    intercept = (sum_y - slope * sum_x) / n
    return slope, intercept

The function first computes the sums. Then it applies the two formulas. It returns the slope and intercept as a pair.

Now use this function to find the best line for your data. Create a script called fit_line.py.

# fit_line.py
from data import hours, scores
from least_squares import least_squares

slope, intercept = least_squares(hours, scores)
print('Best slope:', slope)
print('Best intercept:', intercept)

Run the script. You will see the slope and intercept that minimise the MSE for this dataset.

Check that the fitted line has the lowest MSE

The formulas guarantee the lowest MSE, but you can check by comparing the fitted line with other lines. Import the mean_squared_error function from mse.py and compute the MSE for the fitted line. Then compute the MSE for a few other lines and see that none is lower.

Add this to fit_line.py after the print statements:

from mse import mean_squared_error

mse_best = mean_squared_error(hours, scores, slope, intercept)
print('MSE for best line:', mse_best)

mse_other1 = mean_squared_error(hours, scores, 10, 20)
print('MSE for slope 10, intercept 20:', mse_other1)

mse_other2 = mean_squared_error(hours, scores, 5, 40)
print('MSE for slope 5, intercept 40:', mse_other2)

Run the script again. The MSE for the best line should be smaller than the MSE for the other lines. If you try any other slope and intercept, the MSE will be larger.

Why the least squares line is the best

The MSE is a measure of how far the predictions are from the actual scores. The least squares formulas choose the slope and intercept that make the total squared distance as small as possible. No other straight line can do better on the same data.

This does not mean the line is perfect. The data may not follow a straight line exactly. But among all straight lines, this one has the smallest MSE.

You now have a complete way to fit a line to data. In the next lesson, you will learn why fitting on all the data is not enough to judge how well the model will work on new data.

Key takeaways

  • The least squares formulas give the slope and intercept that minimise the mean squared error for a straight-line model.
  • The best slope is (n * sum(x*y) - sum(x) * sum(y)) / (n * sum(x*x) - sum(x)**2).
  • The best intercept is (sum(y) - slope * sum(x)) / n.
  • You can code these formulas in a function that takes the data lists and returns the slope and intercept.
  • The fitted line has a lower MSE than any other line you choose for the same data.

Try it

Write a new Python script called check_best.py. In it, import the hours and scores lists from data.py, the least_squares function from least_squares.py, and the mean_squared_error function from mse.py. Use least_squares to get the best slope and intercept. Then compute the MSE for the best line and for five other lines with slopes and intercepts of your choice. Print the slope, intercept, and MSE for each line. Run the script. Check that the best line has the lowest MSE among all six lines. A good result is a script that runs without errors and clearly shows the best line has the smallest MSE.

Official documentation

Ready to use this in a job?

Rolebound applies to matching roles for you.

Start now

Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.

    Cookies on Rolebound

    We use necessary cookies to sign you in and keep the site working — those cannot be turned off. We would also like to set analytics and advertising cookies so we can see which pages help people get interviews and which adverts brought them here. Nothing optional is set unless you say yes, and you can change your mind at any time from “Cookie settings” in the footer. Read our Privacy Policy.