Linear regression

Linear regression is a supervised learning model used to predict a continuous output (dependent) variable based on one or more input (independent) variables.

1. Simple Linear Regression

Simple linear regression is the most basic form, using only a single independent variable. This model assumes that the relationship between the input variable and the output variable can be described by a straight line.

Linear Regression Equation

The model's equation is:

Where: - is the dependent variable (the predicted value). - is the independent variable. - is the intercept, the value of when . - is the slope, representing the average change in for a one-unit increase in .

2. Multiple Linear Regression

When there is more than one independent variable, the model is called multiple linear regression.

Multiple Linear Regression Equation

This model takes the form:

Where: - is the dependent variable. - are the independent variables. - is the intercept. - are the regression coefficients corresponding to each independent variable, representing the average change in for a one-unit increase in the corresponding independent variable, while all other independent variables are held constant.

3. Cost Function

The goal of linear regression is to find the coefficients that make the regression line best fit the observed data. To do this, we use a cost function to measure the error between the predicted values and the actual values.

Squared Error Cost Function

The most common cost function is the Mean Squared Error (MSE):

Where: - is the number of training examples. - is the predicted value for the example. - is the actual value for the example. - represents the vector of all coefficients .

The objective is to find the values of that minimize the cost function .

4. Solutions for the Linear Regression Model

There are two main approaches to finding the optimal coefficients:

a) Normal Equation

This method uses linear algebra to find the exact solution to the problem.

Where: - is the vector of optimal estimated coefficients. - is the design matrix containing the input variables. - is the vector of the output variables.

b) Gradient Descent

This method is an iterative algorithm for finding the minimum of the cost function. It updates the coefficients in the direction opposite to the derivative of the cost function.

Where: - is the coefficient. - is the learning rate, which controls the size of the update step.