In this short post I want to quickly demonstrate how the most basic neural network (no hidden layer) gives us the same results as the linear model.

First we need data

We can now specify a model.

Let’s start by estimating a linear model.

Now lets use the built in nnet package to estimate a neural network without a hidden layer.

Let look at the results next to each other.

We can also estimate this model using the neuralnet package (this package needs to be installed first using install.packages("neuralnet")). This package allows us to nicely visualise the results.

Of course it only becomes interesting if we include a hidden layer.

The attention R package, describing how to implement from scratch the attention mechanism - which forms the basis of transformers - in the R language is now ...

EDIT 2022-06-24: this code is now available (with helper functions) in the R package attention, which is on CRAN. You can install it simply using:
install.pa...