Multiple linear regression (1)

In multiple regression, each coefficient is interpreted as the estimated change in y corresponding to a one unit change in a variable, when all other variables are held constant. For example: 

So in this example, dollar 9000 is an estimate of the expected increase in sales y, corresponding to a dollar 1000 increase in capital investment x1, when marketing expenditures x2 are held constant.

New considerations:

• Adding more independent variables to a multiple regression procedure dose not mean the regression will be "better" or offer better predictions; in fact, it can make things worse, i.e., OVERFITTING.

• The addition of more independent variables creates more relationships among them. So not only are the independent variables potentially related to the dependent variable, they are also potentially related to each other. When this happens, it is called MULTICOLLINEARITY.

• The ideal is for all of the independent variables to be correlated with the dependent variable but NOT with each other.

Multiple linear regression lingo

Low bias? Low variance?

-- It depends

-- Typically unbiased is better

-- Sometimes the variance is so out of control. You sacrifice a little bias to fix the variance

• Ridge regression
• Lasso