1 The MVAR Model
The MVAR model in Khan et al. (2018) is defined as x, where e
(i) xare the residuals, A is a matrix of weights, ut are inputs, and xt are data vectors at time t on trial i. Equivalently ∆x, where ∆ and A˜ = A−I.
Let us use this formulation and drop the tilde for convenience. To make things conceptually simpler we will restrict our discussion in this document to the model ∆x, i.e. one with ut = 0. Given a complete dataset and similarly for ∆X, E, this can be written as a matrix equation ∆X = AX + E.
2 Fitting an MVAR model to Data
Given a data matrix X, an unregularised MVAR model is fit by least squares, i.e. the estimated weights are Aˆ = argminAkE(A)k2F = argmin. In Matlab, the unregularised model is fit by simply writing A = ∆X/X, where the forwardslash command means “mrdivide”, which is a type of least squares algorithm.
To perform a regularised model fit, we instead solve Aˆ = argmin, where f(A) is a regularisation function and λ is a hyperparameter to be optimised. Common choices are f(A) = kAk2F = Pij A2ij or f(A) = kAkF = Pij |Aij| (which are L2 and L1 regularisation respectively). Custom functions in languages such as Matlab can solve these linear estimation problems.
3 Simulating Synthetic Data from an MVAR Model
Once we have an MVAR a model and we know the parameters A, data can be generated from the model by drawing residuals et at each time step and iteratively using the equation ∆xt = Axt−1+et to produce data xt. Residuals should usually be drawn from a standard normal distribution. Simulating from a regularised MVAR model is the same as simulating from the unregularised one – regularisation only determines the estimated A.
To simulate synthetic data in this way, we first need to have a weight matrix A. We can do this in two ways – 1) we can fit the MVAR to real data and then generate synthetic data using the inferred Aˆ, which is called forecasting or prediction 2) we can construct a fully synthetic MVAR model by choosing by hand a particular matrix A and simulating from that.
4 Project Goals
The goal of the project is to test the performance of regularisation methods on MVAR models. This will be done by constructing a variety of fully synthetic “ground truth” MVAR models with known weight matrices A, simulating data from them, then fitting an MVAR model to the simulated data using different forms of regularisation. The quality of model fit will then be assessed by comparing the ground truth parameters A to the inferred parameters Aˆ, and also by generating forecasting data from the fitted MVAR model and comparing it to the ground truth data using cross-validation.