An important operation: DOT PRODUCT

• An important operation: DOT PRODUCT

The dot product is a single number, a SCALAR that you form by multiplying two vectors

      V          W                         

 

           

         = 1*2 + 0*8+2*(-6)+5*1+(-2)*0 =-5

 

Dot product is a single number that represents the relationship between these two vectors. It underlies nearly all of the important computations in signal processing. In statistics, the dot product is the basis for the CORRELATION COEFFICIENT, and also underlines covariance for COVARIANCE  matrices.

• Linear least-squares models

 

 

Things are the same when we have a matrix A, not a vector 

 

matlab code x= inv(A'*A)*A'*b   OR  x= (A'*A) \ A'*b 
In statistics  we have 

 

 

where X (or A) is design matrix, and  BETA (or x) is weights, Y (or b) is actual data we collect.

 

• The magic  

1. It's square ( A=A transpose)
2. It's symmetric

   

 

The diagonals contain the variances and the off diagonal contain covariances.

Variances: 

Covariance:   

 

-1<R(correlation)<1 ( 0.8<|R|<1 strong correlation; 0.5<|R|<0.8, moderate)

 is more convenient to explain. Now, when someone says "The statistically significant R^2 was 0.9" you can think to yourself.."very good! the relationship between the two variables explain 90% of the variation in the data!" And when someone else says "The statistically significant R^2 was 0.01" you can think to yourself "Dag, who cares if that relationship is significant, if only accounts for 1% of the variation in the data. Something else must explain the remaining 99%".

Standard Error is the average distance of the data points from the regression line, in dependent variable unites. It gives us a measure of how tightly around the regression line our data points are.