Time Series Analysis Summary
Tokelo Khalema 2008060978 BSc. Actuarial Science University of the Free State Bloemfontein November 1, 2012

Time Series Analysis
A time-series is a stochastic process {Xt : t = 1, . . . , T } with a continous state space and discrete time domain. It arises naturally as an ordered series of values observed over time. Examples include daily closing prices of a stock index recorded over several years, say, the flow rate of the River Nile, road casualties in Great Britain over the years 1969-84, etc. Stationary time-series are particularly easy to analyse. A series is stationary if its mean and variance are constant over time. Special aids are available to help determine whether or not a series is stationary. Particularly notable in this regard are the autocorrelation function (ACF) and the partial autocorrelation function (PACF). These are plots of the sample autocorrelation and partial autocorrelation coefficients at various time lags, respectively. If the ACF decays gradually to zero, then the series is non-stationary. If on the other hand the ACF and PACF decay rapidly to zero, then the series is stationary. A series being non-stationary can be brought about by, among others, a trend, irregular fluctuations, or seasonal variation. Non-constant variance, or as commonly called, heteroscedasticity can be eliminated by using a variance-stabilising transformation. A number of ways exist that eliminate a trend. Two of which are, to subtract a regression line and to difference the series. The latter means creating the series ∆Xt = Xt − Xt−1 . In order to remove seasonality, the period must first be determined by creating a periodogram, and then differencing according to the period. As an example, a half-yearly periodic component from time-series Xt can be removed by taking the difference, ∆6 Xt = Xt − Xt−6 . Stationary time-series models include, Moving Average (MA), Autoregressive (AR), and Autoregressive Moving Average (ARMA) processes. Autoregressive Integrated Moving Average (ARIMA) processes are used to model nonstationary time-series. The ACf and PACF are also used for model identification. Say the autocorrelations tail off to zero while the partial autocorrelations cut off after p lags, then an appropriate model is an AR(p) model. But if the partial autocorrelations tail off to zero and the autocorrelations cut off after q lags, then we have an MA(q) process. An ARMA process can also be identified using the ACF and PACF. Say the autocorrelations tail off after q − p lags and q ≥ p, while the partial autocorrelations tail off after p − q lags and p ≥ q, then we have an ARMA(p, q) process. A white-noise process is a sequence of uncorrelated random variables Xt from a fixed distribution with mean zero, and finite constant variance. It is stationary, from its definition. When a model is fitted to a time-series, it is required that the residuals be white noise. The Ljung-Box-Pierce test is usually used to test for white noise. The Box-Jenkins approach to time-series analysis is the most commonly advocated time-series analysis methodology. The steps of this approach are as follows: 1. Draw the graph 1

2. Stabilise the variance 3. Test for stationarity and difference if necessary 4. Estimate the parameters 5. Estimate the coefficients 6. Carry out the diagnostic checks 7. Then use the model for forecasting if it passed the diagnostics These steps are summarised in figure 1 below. Postulate a general class of models Identify the model Reject or update fitted model

Estimate parameters

no

Are residuals white noise?

yes Use model for prediction Figure 1: Flow-chart to illustrate the general Box-Jenkins approach to timeseries analysis. Many standard packages include toolboxes for time-series analysis. These include among others, R, Matlab, and SAS.

2

LakeHuron

576

577

578

579

580

581

582

1880

1900

1920 Time

1940

1960

Figure 2: Time-series plot of the level of Lake Huron between years 1875 and 1972.

An illustrative Example
To demontrate how a series might be analysed in practice, we consider one sufficiently comprehensive example and carry out the analysis in R. The R builtin data set LakeHuron consists of measurements of the level of Lake Huron from 1875 to 1972. Figure 2 is a plot of this time series. > data(LakeHuron) > plot(LakeHuron) To view the behaviour of the mean and variance we issue the following command. > boxplot(LakeHuron~factor(c(gl(4,20),rep(5,18)))) From figure 3, the mean does not look very constant while the variance looks reasonably stable. From the acf in figure 4 below we see autocorrelations that decay very slowly with many of them rather significant. This suggests that the series is non-stationary. > par(mfrow=c(1,2)) > acf(LakeHuron,lag.max=12);pacf(LakeHuron,lag.max=12) Since we have a non-stationary series, we have to difference it to attain stationarity. This is further suggested by the Dickey-Fuller unit root test that 3

576

577

578

579

580

581

582

1

2

3

4

5

Figure 3: Side-by-side boxplots to visualise heteroscedasticity and non-constant mean.

Series LakeHuron
1.0

Series LakeHuron

0.8

0.6

Partial ACF 0 2 4 6 Lag 8 12

ACF

0.4

0.2

0.0

−0.2

−0.2 2

0.0

0.2

0.4

0.6

0.8

4

6 Lag

8 10

Figure 4: ACF and PACF plots of the Lake Huron time-series.

4

Differenced Time Series
2 dts −2 1880 −1 0 1

1900

1920 Time

1940

1960

Figure 5: Time-series plot of the differenced series. accepts differencing. Since we have a flat series we use the Dickey-Fuller test with ”none”. > ur.df(LakeHuron,"none",0) ############################################################### # Augmented Dickey-Fuller Test Unit Root / Cointegration Test # ############################################################### The value of the test statistic is: -0.0634 We proceed to difference the series and plot it. > dts <- diff(LakeHuron); p <- par(mfrow=c(1,1)) > plot(dts,main="Differenced Time Series") The differenced series above looks stationary. The unit root test also rejects H0 : φ0 = 1. Since once again we have a flat series we use the Dickey-Fuller test with ”none”. > ur.df(dts,"none",0)

############################################################### # Augmented Dickey-Fuller Test Unit Root / Cointegration Test # ############################################################### 5

−2

−1

0

1

2

1

2

3

4

5

Figure 6: Side-by-side boxplots to visualise heteroscedasticity and non-constant mean in the differenced series.

The value of the test statistic is: -8.716 This also suggests that d = 1 in the model ARIMA(p, d, q). Next we investigate the behaviour of the mean and variance. > boxplot(dts~factor(c(gl(4,20),rep(5,17)))) Now both the mean and variance look reasonably stable. This suggests that the series is stationary. A look at the acf and pacf in figure 7 below further supports that the series is stationary. All autocorrelations and partial autocorrelations lie within the confidence interval but for at most two of them that just barely touch the limits of the interval. It would be resonable to fit a simpler model, viz. an ARMA(1, 2) (since there’s suggestive evidence that q > p) model and then move on to a more complex model if we feel rather unsated with the quality of the fit. Model selection criteria exist that could be used to compare several candidate models. These include the AIC, BIC, and HCQ criteria. > acf(dts);pacf(dts) > m <- arima(LakeHuron,c(1,1,2)) > m 6

Series dts
1.0 0.2

Series dts

0.8

0.6

Partial ACF 0 5 10 Lag 15

ACF

0.2

0.4

0.0

−0.2

−0.2

−0.1

0.0

0.1

5

10 Lag

15

Figure 7: ACF and PACF plots of the differenced series. Series: LakeHuron ARIMA(1,1,2) Coefficients: ar1 ma1 0.6475 -0.5837 s.e. 0.1292 0.1393

ma2 -0.3279 0.1082

sigma^2 estimated as 0.4816: log likelihood=-102.56 AIC=213.12 AICc=213.56 BIC=223.42 From the above output the fitted model is Xt = 0.6475Xt−1 + t − 0.5837

t−1

− 0.3279

t−2 .

We assess the goodness of fit of the above model by examining the residuals. To do this we perform the Ljung-Box-Pierce test which conclusively accepts white noise: > Box.test(m$residuals,12) Box-Pierce test data: m$residuals X-squared = 5.0503, df = 12, p-value = 0.9563 7

Histogram of Residuals
30

Normal Q−Q Plot q 1.5

q q q q q q q q q q

25

20

Sample Quantiles

1.0

−0.5

q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q q qq q q q q qq q

Frequency

15

10

−2

−1

0

1

2

−1.5

0

−1.0

5

0.0 −2

0.5

0

1

2

Residuals

Theoretical Quantiles

Figure 8: A histogram and a normal quantile-quantile plot of the model residuals. For further diagnostic checks on the model, a histogram and normal quantilequantile plot of the residuals are created in figure 8 below, which do not show much deviation from normality: > hist(m$residuals,main="Histogram of Residuals") > qqnorm(m$residuals);qqline(m$residuals,col=2) Since none of the diagnostic checks considered above gives us any reason to doubt the quality of the fit, we accept the model and proceed to predict future values with the fitted model. Say we want to predict 10 years ahead. Then we issue the following command: > predict(m,n.ahead=10) $pred Time Series: Start = 1973 End = 1982 Frequency = 1 [1] 579.5848 579.2943 579.1061 578.9843 578.9054 578.8543 578.8212 578.7998 [9] 578.7859 578.7769 $se Time Series: 8

Start = 1973 End = 1982 Frequency = 1 [1] 0.6939623 1.0132292 1.1478702 1.2191141 1.2622767 1.2915184 1.3133202 [8] 1.3309101 1.3460033 1.3595592 This concludes the all the steps of the Box-Jenkins approach to time-series analysis. The moment a time series model has been accepted, it is ready to be put to use. Besides forecasting, as demonstrated above, other purposes of time series analysis are: • Description of data • Construction of a model which fits the data • Deciding whether the process is out of control, requiring action • For vector time series, investigating connections between two or more observed processes with the aim of using values of some of the processes to predict those of the others There are more complicated models than those discussed above. Commonly employed for modelling financial time series that exhibit periods of swings followed by periods of relative calm are the so-called Autoregressive Conditional Heteroscedasticity (ARCH) models. These models assume the variance of the current error term, t to be a function of the actual sizes of error terms of the previous time periods. The garch() function is available in R’s tseries package and is used for fitting GARCH models. A GARCH model, or a Generalised Autoregressive Conditional Heteroscedasticity model arises if an ARMA model is assumed for the error variance.

9

Bibliography
[1] R Development Core Team (2012). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URLhttp://www.R-project.org/.

10