In order to understand statistics, you have to do the calculations yourself!
Warnings such as these are often given in statistics courses and for good reason too! Doing the work yourself really cements the understanding of statistics.
Yet when it comes to econometrics, the main take aways from the workshops are primarily in terms of the syntax of yet another computer program.
In this series of post titled: handcoded, I show how many workhorses of the econometrician’s toolbox can be implemented in a very simple manner.
This manual implementation greatly helps in keeping an understanding of what actually happens when we call e.g.
MCMC.. with a few parameters.
The Linear Model
Then using the Ordinary Least Squares approach to solving a model, we start with the following equation of the OLS model for a univariate regression.
This can be solver for the following (hat denotes the estimator, bar denotes the mean):
We start by loading a basic data set.
Inspect the data set.
Assign our variables to objects (in the global environment)
We can now estimate the slope parameter:
Using the slope parameter we can now compute the intercept.
Lets check this using the built in command.
The matrix model
In matrix form we can specify our general equation as:
From which we can derive our estimator:
The matrix estimation
Use the built in command.
Now we estimate our beta ourselves, the function used to invert is called
Now lets estimate with an intercept
To hand code this, we need to add a vector of ones (
Note that the single
1 that we are binding to the vector
X will be repeated until it is the same length.
We also have to discuss the above
^-1 is not correct syntax for a matrix inversion. In the above case it would still work correctly because our
X matrix is in fact a vector. If we pre-multiply this vector with the transpose of itself, we obtain a scalar.
However, for matrices wider than one column this is not the case.
^-1 will invert every individual number in the matrix, rather than the matrix as a whole.
We want to obtain to obtain the inverse of the matrix, because this will allow us to pre-multiply on both sides, eliminating
XI on the Right-Hand Side (RHS).
We therefore use a different tool, the
solve() function from the
This function implements the QR decomposition,
which is an efficient way of deriving an inverse of a matrix.
Now we can use this matrix to estimate a model with an intercept.
Note that this is programmatically exactly the same the way that the
lm() function does this.
We can suppress the automatic intercept and include our
XI variable and we will obtain the same results.
We have now constructed a univariate (univariate) model, however, from a programmatic point of view, the hurdles of multivariate modelling have already been overcome by estimating a model with an intercept (making
X a matrix).
It is therefore very easy to use the same method in a case with two independent variables.
we start by binding the two independent variables together (with vector of
1s, since we want an intercept).
Now we estimate our model
And that’s all! The leap from univariate to multivariate modelling was truly very small.
EDIT: in tomorrow’s post we use the method we developed here to create an easy to use function.