• unlimited access with print and download
    $ 37 00
  • read full document, no print or download, expires after 72 hours
    $ 4 99
More info
Unlimited access including download and printing, plus availability for reading and annotating in your in your Udini library.
  • Access to this article in your Udini library for 72 hours from purchase.
  • The article will not be available for download or print.
  • Upgrade to the full version of this document at a reduced price.
  • Your trial access payment is credited when purchasing the full version.
Buy
Continue searching

Maximum likelihood estimation of an unknown change-point in the parameters of a multivariate Gaussian series with applications to environmental monitoring

Dissertation
Author: Pengyu Liu
Abstract:
The computable expressions for the asymptotic distribution of the change-point maximum likelihood estimator (mle) were derived when a change occurred in the mean and covariance matrix at an unknown point of a sequence of independently distributed multivariate Gaussian series. The derivation was based on ladder heights of Gaussian random walks hitting the half-line. We then demonstrated that change in a single parameter or change-point analysis in a univariate series can be derived as special cases. A simulation study was carried out to investigate the robustness of the asymptotic distribution to departure from normality, the sample size, location of change-point and amount of change under the multivariate and univariate case. The comparison of the asymptotic mle with Cobb's conditional MLE and Bayesian estimation method using non-informative prior and conjugate prior was also carried out in the simulation study. The asymptotic distribution of the change-point mle was used to compute the confidence interval of the change-point of the stream flows at Northern Quebec Labrador Region and zonal annual mean temperature deviations.

TABLE OF CONTENTS

Page

ACKNOWLEDGMENT ................................................................................................... iii ABSTRACT ....................................................................................................................... iv LIST OF TABLES .............................................................................................................. x LIST OF FIGURES .......................................................................................................... xiv CHAPTER 1 INTRODUCTION ................................................................................................. 1 2 LITERATURE REVIEW ...................................................................................... 5 2.1 Change-point Detection for Mean and/or Covariance ................................ 9 2.1.1 Change in both mean and covariance/variance ........................................ 9 2.1.2 Change in mean only ............................................................................... 17 2.1.3 Change in covariance/variance only ....................................................... 21 2.2 Change-point Estimation Setup ................................................................... 25 3 INFERENCE FOR CHANGE-POINT IN THE MEAN ONLY OF A GAUSSIAN SERIES ...................................................................................................................... 28 3.1 Multivariate Case .......................................................................................... 28 3.2 Univariate Case ............................................................................................. 40

vii

4 INFERENCE FOR CHANGE-POINT IN MEAN AND COVARIANCE OF A GAUSSIAN SERIES .................................................................................................... 45 4.1 MLE of a Change-point in Mean and Covariance of a Multivariate Gaussian Series .................................................................................................... 45 4.1.1 Asymptotic distribution of change-point MLE ......................................... 46 4.1.2 Distribution of linear combination of chi-square distribution ................ 61 4.1.3 Algorithmic procedure to compute the change-point mle ....................... 68 4.2 Special Cases .................................................................................................. 72 4.2.1 Mean and Variance of a Univariate Gaussian Series ............................. 72 4.2.2 Covariance Only of a Multivariate Gaussian Series ............................... 76 4.3 Bayesian Method for Estimating Change-point in Mean and/or Covariance of a Multivariate Gaussian Series .................................................. 79 4.3.1 Conjugate Prior ....................................................................................... 80 4.3.2 Non-informative Prior ............................................................................. 90 4.4 Conditional MLE Method for Estimating Change-point in Mean and/or Covariance of a Multivariate Gaussian Series ...................................... 92 5 SIMULATION STUDIES TO ASSESS ROBUSTNESS ............................................ 95 5.1 Simulation Setup ........................................................................................... 95 5.2 Multivariate Simulations ............................................................................ 102

viii

5.3 Univariate Simulations ............................................................................... 134 6 APPLICATION TO ENVIRONMENTAL MONITORING .................................... 164 6.1 River Stream Flows in the Northern Québec Labrador Region ............. 164 6.1.1 Multivariate change-point model setup ................................................. 171 6.1.2 Detection of an unknown Change-Point in River Stream Flows ........... 173 6.1.3 Asymptotic Distribution of the Change Point MLE for River Stream Flows ............................................................................................................... 181 6.2 Change-point Analysis of Zonal Temperature Deviations ...................... 184 6.2.1 Dataset description ................................................................................ 184 6.2.2 Change-point Analysis at South Polar................................................... 190 6.2.2.1 Change-point Detection .................................................................... 192 6.2.2.2 Bivariate change-point analysis for layer 3 and 4 ............................. 196 6.2.2.3 Univariate change-point analysis for layer 1 .................................... 207 6.2.2.4 Univariate change-point analysis for layer 2 .................................... 212 6.2.3 Change-point Analysis at North Polar .................................................. 217 6.2.4 Discussion about Polar Temperature Deviations .................................. 228 BIBLIOGRAPHY .......................................................................................................... 232 APPENDIX .................................................................................................................... 241

ix

A. Average Spring stream flows during 1957-1995 in the Northern Québec Labrador region ...................................................................................................... 242 B. Annual mean temperature deviation for South Polar ..................................... 244 C. Annual mean temperature deviation for North Polar ..................................... 246

x

LIST OF TABLES

Table 3.1. Asymptotic probabilities Pr⁡(ξ

∞ = ±?), where ? = 0,1,2,… for the maximum likelihood estimate of the change-point in the case of normal distribution. .............................................................................................................. 39 Table 4.1. Probability of linear combination of chi-squared distribution using (ii) Imhof‘s (1961) estimation; (iii) Davies‘ (1973) method; (iv) Imhof‘s (1961) exact formula using R integration; (v) Saddlepoint approximation.. ....................... 66 Table 5.1 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/50 and = 1.5 for bivariate series. ........................................................... 103 Table 5.2 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/50 and = 2 for bivariate series. .............................................................. 104 Table 5.3 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/30 and = 1.5 for bivariate series. ........................................................... 105 Table 5.4 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/30 and = 2 for bivariate series. .............................................................. 106 Table 5.5 Square root of mean squared error of the change-point mle when ?/𝜏 = 50/25 and = 1.5 for bivariate series............................................................... 107 Table 5.6 Square root of mean squared error of the change-point mle when ?/𝜏 = 50/25 and = 2 for bivariate series. ................................................................. 108 Table 5.7 Bias of the change-point mle when ?/𝜏 = 100/50 and = 1.5 for bivariate series. ....................................................................................................... 109

xi

Table 5.8 Bias of the change-point mle when ?/𝜏 = 100/50 and = 2 for bivariate series. ....................................................................................................... 110 Table 5.9 Bias of the change-point mle when ?/𝜏 = 100/30 and = 1.5 for bivariate series. ....................................................................................................... 111 Table 5.10 Bias of the change-point mle when ?/𝜏 = 100/30 and = 2 for bivariate series. ....................................................................................................... 112 Table 5.11 Bias of the change-point mle when ?/𝜏 = 50/25 and = 1.5 for bivariate series. ....................................................................................................... 113 Table 5.12 Bias of mean squared error of the change-point mle when ?/𝜏 = 50/25 and = 2 for bivariate series. ............................................................................. 114 Table 5.13 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/50 and = 1.5 for univariate series. .............................................. 135 Table 5.14 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/50 and = 2 for univariate series. ................................................. 136 Table 5.15 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/30 and = 1.5 for univariate series. .............................................. 137 Table 5.16 Square root of mean squared error of the change-point mle when ?/𝜏 = 100/30 and = 2 for univariate series. ................................................. 138 Table 5.17 Square root of mean squared error of the change-point mle when ?/𝜏 = 50/25 and = 1.5 for univariate series. ................................................ 139 Table 5.18 Square root of mean squared error of the change-point mle when ?/𝜏 = 50/25 and = 2 for univariate series. ................................................... 140

xii

Table 5.19 Bias of the change-point mle when ?/𝜏 = 100/50 and = 1.5 for univariate series. ..................................................................................................... 141 Table 5.20 Bias of the change-point mle when ?/𝜏 = 100/50 and = 2 for univariate series. ..................................................................................................... 142 Table 5.21 Bias of the change-point mle when ?/𝜏 = 100/30 and = 1.5 for univariate series. ..................................................................................................... 143 Table 5.22 Bias of the change-point mle when ?/𝜏 = 100/30 and = 2 for univariate series. ..................................................................................................... 144 Table 5.23 Bias of the change-point mle when ?/𝜏 = 50/25 and = 1.5 for univariate series. ..................................................................................................... 145 Table 5.24 Bias of the change-point mle when ?/𝜏 = 50/25 and = 2 for univariate series. ..................................................................................................... 146 Table 6.1. Asymptotic distribution of 𝜉

∞ under case (i), (ii) and (iii) for the change-point mle of the six rivers from the Northern Québec Labrador region.. .. 183 Table 6.2. Change-point detection of South Polar annual mean temperature deviations during 1958 – 2008 for mean and/or covariance (variance), mean only and covariance (variance) only. .................................................................................... 194 Table 6.3 Cross correlations of the residuals at layers 3 and 4 for South Polar annual mean temperature deviations during 1958 – 2008. ................................................ 201 Table 6.4 Computed probabilities for 𝜉

∞ using Maximum Likelihood, Cobb‘s conditional mle, and Bayesian methods using conjugate and non-informative priors for South Polar annual mean temperature deviations during 1958 – 2008. . 204

xiii

Table 6.5 Computed cumulative probabilities for 𝜉

∞ using Maximum Likelihood, Cobb‘s conditional mle, and Bayesian methods using conjugate and non-informative priors for South Polar annual mean temperature deviations during 1958 – 2008. ............................................................................................... 205 Table 6.6 Computed probabilities and cumulative probabilities for 𝜉

∞ at South Polar during 1958 – 2008 at layer 1 (surface). ................................................................ 211 Table 6.7 Computed probabilities and cumulative probabilities for 𝜉

∞ at South Polar during 1958 – 2008 at layer 2 (850 – 100 mb). ...................................................... 216 Table 6.8. Change-point detection of North Polar annual mean temperature deviations during 1958 – 2008 for mean and/or covariance (variance), mean only and covariance (variance) only. .................................................................................... 219 Table 6.9 Cross correlations for the residuals at layers 1 and 4 for North Polar annul mean temperature deviations during 1958 – 2008. ................................................ 224 Table 6.10 Computed probabilities of 𝜉

∞ using Maximum Likelihood, Cobb‘s conditional mle, and Bayesian methods using conjugate and non-informative priors for North Polar annual mean temperature deviations at layers 1 (surface) and 4 (100 – 50 mb). .............................................................................................. 226 Table 6.11 Computed cumulative probabilities for 𝜉

∞ using Maximum Likelihood, Cobb‘s conditional mle, and Bayesian methods using conjugate and non-informative priors for North Polar annual mean temperature deviations at layers 1 (surface) and layer 4 (100 – 50 mb) during 1958 – 2008. ........................ 227

xiv

LIST OF FIGURES

Figure 5.1 Comparison of the kk, ke, ek, and ee estimation methods for MLE and Cobb‘s method when ?/𝜏 = 100/50 for bivariate series. .................................. 118 Figure 5.2 Comparison of the kk, ke, ek, and ee estimation method for MLE and Cobb‘s method when ?/𝜏 = 100/30 for bivariate series. .................................. 119 Figure 5.3 Comparison of the kk, ke, ek, and ee estimation method for MLE and Cobb‘s method when ?/𝜏 = 50/25 for bivariate series. .................................... 120 Figure 5.4 The effect of sample size and change-point position to the MLE estimation method for bivariate series. .................................................................................... 121 Figure 5.5 The effect of sample size and change-point position to the Cobb‘s estimation method for bivariate series. .................................................................. 122 Figure 5.6 The effect of sample size and change-point position to the Bayesian‘s estimation method for bivariate series. .................................................................. 123 Figure 5.7 Comparison of estimation methods when the MLE and Cobb used ‗kk‘ for parameter estimates for bivariate series. ................................................................ 124 Figure 5.8 Comparison of estimation methods when the MLE and Cobb used ‗ke‘ for parameter estimates for bivariate series. ................................................................ 125 Figure 5.9 Comparison of estimation methods when the MLE and Cobb used ‗ek‘ for parameter estimates for bivariate series. ................................................................ 126 Figure 5.10 Comparison of estimation methods when the MLE and Cobb used ‗ee‘ for parameter estimates for bivariate series. ................................................................ 127

xv

Figure 5.11 Effect of the degrees of freedom when the series follow multivariate t-distribution using MLE method for bivariate series. ........................................... 128 Figure 5.12 Effect of the degrees of freedom when the series follow multivariate t-distribution using Cobb‘s method for bivariate series. ........................................ 129 Figure 5.13 Effect of the degrees of freedom when the series follow multivariate t-distribution using Bayesian method for bivariate series. ..................................... 130 Figure 5.14 Comparison of estimation methods when the series follow multivariate t-distribution with df=5 for bivariate series. .......................................................... 131 Figure 5.15 Comparison of estimation methods when the series follow multivariate t-distribution with df=10 for bivariate series. ........................................................ 132 Figure 5.16 Comparison of estimation methods when the series follow multivariate t-distribution with df=20 for bivariate series. ........................................................ 133 Figure 5.17 Comparison of the kk, ke, ek, and ee estimation method for MLE and Cobb‘s method when ?/𝜏 = 100/50 for univariate series. ................................ 147 Figure 5.18 Comparison of the kk, ke, ek, and ee estimation method for MLE and Cobb‘s method when ?/𝜏 = 100/30 for univariate series. ................................ 148 Figure 5.19 Comparison of the kk, ke, ek, and ee estimation method for MLE and Cobb‘s method when ?/𝜏 = 50/25 for univariate series. .................................. 149 Figure 5.20 The effect of sample size and change-point position to the MLE estimation method for univariate series. ................................................................ 150 Figure 5.21 The effect of sample size and change-point position to the Cobb‘s estimation method for univariate series. ................................................................ 151

xvi

Figure 5.22 The effect of sample size and change-point position to the Bayesian‘s estimation method for univariate series. ................................................................ 152 Figure 5.23 Comparison of estimation methods when the MLE and Cobb used ‗kk‘ for parameter estimates for univariate series. .............................................................. 153 Figure 5.24 Comparison of estimation methods when the MLE and Cobb used ‗ke‘ for parameter estimates for univariate series. .............................................................. 154 Figure 5.25 Comparison of estimation methods when the MLE and Cobb used ‗ek‘ for parameter estimates for univariate series. .............................................................. 155 Figure 5.26 Comparison of estimation methods when the MLE and Cobb used ‗ee‘ for parameter estimates for univariate series. .............................................................. 156 Figure 5.27 Effect of the degrees of freedom when the series follow univariate t-distribution using MLE method for univariate series. ......................................... 157 Figure 5.28 Effect of the degrees of freedom when the series follow univariate t-distribution using Cobb‘s method for univariate series. ...................................... 158 Figure 5.29 Effect of the degrees of freedom when the series follow univariate t-distribution using Bayesian method for univariate series. ................................... 159 Figure 5.30 Comparison of estimation methods when the series follow univariate t-distribution with df=5 for univariate series. ........................................................ 160 Figure 5.31 Comparison of estimation methods when the series follow univariate t-distribution with df=10 for univariate series. ...................................................... 161 Figure 5.32 Comparison of estimation methods when the series follow univariate t-distribution with df=20 for univariate series. ...................................................... 162

xvii

Figure 6.1. Average Spring flows of six rivers: (a) Romaine, (b) Churchill Falls, (c) Manicougan, (d) Outardes, (e) Sainte-Marguerite, (f) À la Baleine during 1957-1995 from the Northern Québec Labrador region. ....................................... 167 Figure 6.2: Twice log-likelihood ratio for a given change-point for the six rivers from the Northern Québec Labrador region.. ........................................................ 175 Figure 6.3: Plot of auto correlations for residuals from six rivers, (a) Romaine, (b) Churchill Falls, (c) Manicougan, (d) Outardes, (e) Sainte-Marguerite, (f) À la Baleine. .................................................................................................................. 179 Figure 6.4. Layers of atmosphere for Angell‘s (2009) radiosonde temperature data. .. 185 Figure 6.5. South Polar annual mean temperature deviations during 1958 – 2008. ..... 187 Figure 6.6. North Polar annual mean temperature deviations during 1958 – 2008. ..... 188 Figure 6.8. Twice the log likelihood ratio statistics for South Polar annual mean temperature deviations during 1958 – 2008 at layer 3 (850 – 300 mb) and layer 4 (100 – 50 mb). ........................................................................................................ 197 Figure 6.9. Autocorrelation and partial autocorrelation plots of residuals with 95% significant limits for South Polar annual mean temperature deviations during 1958 – 2008 at layer 3 (300 – 100 mb) and layer 4 (100 – 50 mb)........................ 200 Figure 6.10. Twice the log likelihood ratio statistics for South Polar annual mean temperature deviations during 1958 – 2008 at layer 1 (surface). ........................... 208 Figure 6.11. Autocorrelation and partial autocorrelation plots of residuals with 95% significant limits for South Polar annual mean temperature deviations during 1958 – 2008 at layer 1 (surface). ............................................................................ 209

xviii

Figure 6.12. Twice the log likelihood ratio statistics for South Polar annual mean temperature deviations during 1958 – 2008 at layer 2 (850 – 300 mb). ................ 213 Figure 6.13. Autocorrelation and partial autocorrelation plots of residuals with 95% significant limits for South Polar annual mean temperature deviations during 1958 – 2008 at layer 2 (850 – 100 mb). ................................................................. 214 Figure 6.14. Twice log likelihood ratio statistics for North Polar annual mean temperature deviations during 1958 – 2008 at layer 1 (surface) and layer 4 (100 – 50 mb). ................................................................................................................ 221 Figure 6.15. Autocorrelation and partial autocorrelation plots of residuals with 95% significant limits for North Polar annual mean temperature deviations during 1958 – 2008 at layer 1 (surface) and layer 4 (100 – 50 mb). ................................. 223

xix

Dedication

This dissertation/thesis is dedicated to my mother and father

who provided both emotional and financial support

1

1 INTRODUCTION Classic change-point methods involve two fundamental inferential problems, detection and estimation. Under the maximum likelihood based approach, the detection part is addressed through likelihood ratio statistics and their asymptotic sampling distributions. The estimation part started with the point estimate of the change-point from the detection part. Even though asymptotic distributions of change detection statistics are non-standard, much progress has been made in this regard, at least for the case of detecting a single unknown change-point in a time series. The specific scenarios include changes in the parameters of univariate and multivariate exponential families, multiple linear regression models, autoregressive models, and even long range dependent time series models. Chapter 2 gave a comprehensive review about the change-point analysis using maximum likelihood method and other alternative methods. The maximum likelihood ratio statistics for change-point detection is also derived for the estimation problems in our study. Tackling the estimation problem, we derive in this study the computable expressions for the distribution of the change-point mle when a change occurs in the mean and/or variance/covariance of a univariate or multivariate Gaussian series. The derived asymptotic distribution is quite elegant and can be computed in a simple and straightforward manner. For the Gaussian case, Fotopoulos et al. (2009) demonstrates that the second suggested approximation in Jandhyala and Fotopoulos (1999) is the exact solution to the estimation of change-point mle. In Chapter 3, the asymptotic distribution was derived for the change-point mle for change in mean only

2

in multivariate Gaussian series. Chapter 4 derived the case for change in both mean and covariance. As the estimation requires computing the distribution of a linear combination of non-central chi-square random variables, Chapter 4 also discussed this issue for presenting the algorithmic procedure for estimating change-point mle. It should be noted that the parameters of the distribution before and after the change-point are assumed known. However, this should not pose difficulties, since Hinkley (1970) has shown that the asymptotic distribution of the change-point mle when the parameters are unknown is equivalent to that when the parameters are known. From a practical point of view this asymptotic equivalence result is extremely important. In practice, apart from the change-point being unknown, the parameters before and after the change-point also invariably remain unknown. The problem of deriving the distribution of the change-point mle when the parameters are unknown is the one that practitioners would be most interested as opposed to the distribution of the change-point mle for the case when the parameters are known. There is no apriori reason to believe that the distributions of the change-point mle for the known and unknown cases be asymptotically equivalent. It is in this sense that the asymptotic equivalence result of Hinkley (1972) plays a key role for practitioners. One only needs to examine whether this asymptotic property holds well for reasonable sample sizes, and for this we carried out a simulation study in Chapter 5, where the asymptotic distributions are computed under different combination of sample size, location of change-point, dimension of the observations, and the choice of estimating parameters before and after the change-point.

3

Since the solution derived in the paper assumes Gaussianity, we also explored the robustness of this computable expression when the series deviates from Gaussianity. If the derived result is indeed robust to such departures, then it can be applied more widely than merely Gaussian processes. While a simulation study covering a wide class of non-Gaussian families of distributions may be of interest for practitioners, in this study, a limited robustness study is pursued by performing large scale simulations wherein the error terms follows the univariate or multivariate t-distribution. The degrees of freedom were changed from being small to large, so that we are able to observe how the asymptotic distributions behave as the underlying distribution approaches the Gaussianity. The simulation for univariate and multivariate case are carried out in Chapter 5. Hinkley‘s (1972) approach to deriving distribution of the change-point mle is perceived as the unconditional approach in the literature. Against this, Cobb (1978) proposed a conditional approach to the distribution of the change-point mle, wherein the distribution of the mle is derived by conditioning upon sufficient information on either side of the unknown change-point. It is relevant to compare the conditional and unconditional distributions in terms of their performance, including robustness properties. Thus Cobb‘s conditional distribution is also included in the simulation study. As pointed out by Cobb (1978), since the conditional distribution of the change-point mle can also be interpreted as the Bayesian posterior for the change-point under a uniform prior on the unknown change-point, the comparisons between the two distributions have a broader appeal than what might appear at first

4

glance. The simulation study for Cobb‘s method in Chapter 5 also includes the cases for known and unknown parameters. In Chapter 6, we apply the methodology derived in Chapter 3 and 4 to multivariate analysis of hydrological and the climatology data. The hydrological data, previously analyzed with Bayesian method using conjugate priors by Perreault et al. (2000), represents the average spring stream flows of six rivers during 1957-1995 in the northern Québec Labrador region. The multivariate change-point analysis shows that a significant increase in mean stream flow has occurred 1984. The climatology data, which was provided by Angell (2009), represents the mean annual air temperature for surface and upper layers (850 – 300, 300 – 100 and 100 – 50 mb) from 1958 to 2008 at north and south polar. The analysis showed that at the south polar, a cooling effect has occurred at the lower stratosphere at 1981, where the change is in both mean and covariance matrix, and a warming effect at the surface temperature at 1976, where only change in mean has happened. At the north polar, a cooling effect at the lower stratosphere and a warming effect at the surface occurred at 1988, and the change has happened in mean only.

5

2 LITERATURE REVIEW Maximum likelihood estimation of an unknown change-point first begins with obtaining the mle as a point estimate. Interval estimates of any desired level, which are preferred over point estimates can be constructed around the mle provided distribution theory for the mle is available. Hawkins (1977) and James et Al. (1987) studied change-point detection in the series following independent univariate normal distribution with possible change of mean. Kim and Siegnumd, D. (1989) and Chu and White (1992) developed the detection for change in simple linear regression for slope and intercept. Worsley (1988) and Henderson (1990) used likelihood ratio test for the change in hazard ratio. Worsley and Srivastava (1986) tested change in mean in multivariate normal series. As for real life applications of change-point detectioin, see Braun and Müller (1998) for application of change point methods in DNA segmentation and bioinformatics; Fearnhead (2006), Ruggieri et al (2009) for applications in geology; Perreault et al (2000a, 2000b) for applications in hydrology; Jarušková (1996) for applications in meteorology; Fealy and Sweeney (2005), DeGaetano (2006) for applications in climatology; Kaplan and Shishkin (2000), Lebarbier (2005) for applications in signal processing; Andrews (1993), Hansen (2000) for applications in econometrics; and Lai (1995), Wu et al. (2005), Zou et al. (2009) for applications in statistical process control. However, distribution theory for a change-point mle can be analytically intractable, particularly when no smoothness conditions are assumed regarding the amount of change. Convincing arguments have not yet been made in the literature regarding the appropriateness of imposing

6

smoothness conditions on the amount of change. As a consequence of its intractability, only a few computationally useful results for the distribution of the change-point mle under abrupt change have been developed. For univariate models, the distribution theory and computational procedures have been derived by Jandhyala and Fotopoulos (1999) for change in mean only of a normal distribution, Jandhyala et al (2002) for change in variance only of a normal distribution and Jandhyala and Fotopoulos (2007) for estimating simultaneous change in both mean and variance of the univariate normal distribution. Earlier Jandhyala et al (1999) computed asymptotic distribution of the change-point mle for Weibull models and applied it to estimate change in minimum temperatures at Uppsala, Sweden. For multivariate models, Jandhyala et al (2008) derived the estimation for change in mean vector only of a multivariate normal distribution. However, distribution theory for change-point MLEs in the other parameters (covariance matrix only, or both mean and covariance matrix) of multivariate models, Gaussian or otherwise, has not yet been derived in the literature. Similarly, the methodology has not been developed for estimating changes in the parameters of regression models, and thus one cannot yet handle changes in polynomial trends under the MLE approach. Note that, as mentioned previously, the detection part has been developed for all these situations and it is the distribution theory of the change-point MLE that poses greater analytical difficulties. In this sense, this project makes an important progress by considering the problem of estimating change in both the mean vector and the covariance matrix of a multivariate

7

normal distribution by the MLE method, and then by applying it to the analysis of zonal temperature deviations from surface to lower stratosphere layer. In contrast, advances in the Bayesian approach to change-point methodology have been occurring at a faster pace. Ever since Markov Chain Monte Carlo (MCMC) methods were seen as a tool for overcoming the computational complexities in Bayesian analysis, there has been rapid progress in the overall development of this important methodological tool, and advances in Bayesian change-point analysis have not lagged behind. The main advantage of the Bayesian approach to the change-problem is that both detection and estimation parts of the problem are solved simultaneously once posterior distribution of the unknown change-point is made available, mainly because all inferences about the unknown change-point are made from the posterior distribution. Consequently, with recent advances in the methodology, the Bayesian approach to change-point analysis is able to provide inferential methods ranging from simple to complex situations, some of which include change in mean and/or variance of the univariate normal distribution (Perreault et al 1999, Perreault et al 2000a, 2000b), change in the mean vector of a multivariate normal distribution (Perreault et al 2000), change in mean and/or covariance of a multivariate Guassian series (Son and Kim 2005), single change in the parameters of a multiple linear regression model (Seidou et al 2007), nonlinear change (Schleip et al. 2009), and also the more complex case of estimating multiple change-points (Barry and Hartigan 1993, Fearnhead, 2005, 2006; Seidou and Ouarda 2007). Carlin et al.

8

(1992) proposed hierarchical Bayesian change-points model using the Gibbs sampler with application to changing regressions, Poisson process and Markov chains. Clearly, developments in the mle methodology under abrupt changes lag behind its Bayesian counterpart. As tools for statistical modeling and analysis, it is desirable that both methods be available for practitioners. As such, data analysis will benefit from having a choice of competing methods for any given scenario and there is no need to curtail advances in either of the two approaches. It is entirely possible that one of the methods may be more suitable for the analysis of a particular data series, and seen from this perspective, it is difficult to argue against further advancements in the mle methodology for change-point analysis. Asymptotic distribution theory for the change-point mle in the abrupt case was first initiated by Hinkley (1970, 1971, 1972). While Hinkley (1970) derived the asymptotic theory for the change-point mle in a general set-up, the distribution was not in a computable form primarily due to the technical difficulties in nature. It turned out that Hinkley (1970) computed the distribution for change in the mean of a normal distribution only through certain approximations. While Hu and Rukhin (1995) provided a lower bound for the probability of the mle being in error of capturing the true change-point, Jandhyala and Fotopoulos (1999) and Fotopoulos and Jandhyala (2001) derived upper and lower bounds and also suggested two approximations for the asymptotic distribution of the change-point mle. Similarly, Borovkov (1999) also provided only upper and lower bounds for the distribution of the

Full document contains 267 pages
Abstract: The computable expressions for the asymptotic distribution of the change-point maximum likelihood estimator (mle) were derived when a change occurred in the mean and covariance matrix at an unknown point of a sequence of independently distributed multivariate Gaussian series. The derivation was based on ladder heights of Gaussian random walks hitting the half-line. We then demonstrated that change in a single parameter or change-point analysis in a univariate series can be derived as special cases. A simulation study was carried out to investigate the robustness of the asymptotic distribution to departure from normality, the sample size, location of change-point and amount of change under the multivariate and univariate case. The comparison of the asymptotic mle with Cobb's conditional MLE and Bayesian estimation method using non-informative prior and conjugate prior was also carried out in the simulation study. The asymptotic distribution of the change-point mle was used to compute the confidence interval of the change-point of the stream flows at Northern Quebec Labrador Region and zonal annual mean temperature deviations.