Long memory models

Printer-friendly version

Processes where we know that ancient history is still relevent for the future predictions, even if we know the recent history.

In my own mental map this is near-synonymous with stateful models
where we ignore the state, which I suppose is a kind of coarse graining.
If we aren’t concerned with a process but i.i.d. occurrences we might look at our
hidden variables differently.

This particular approach up being a popular simplification, because hidden states can be computationally difficult
to infer as well as having possibly high
sample complexity.
Maybe for other reasons too?

But in this formulation, we have a Markov process but
because we do not observe the whole state it looks non-Markov.
This is reasonably consistent with reality, where we believe
the current state of reality determines the future,
but we don’t know the whole current state.
Related: hidden variable quantum mechanics.

Note “long memory” is considered as a model for with time series, but clearly spatial random fields, or random fields indexed by any number of dimensions, with or without causality constraints, can have this property.

Main questions for me:

  1. Can we use the “memory length” of a system to infer the number of hidden states for some class of interesting systems, or vice versa?
  1. which classes?
  1. Can we infer the memory length alone as a parameter of interest in some classes? (need making precise). Information criteria don’t do this model order selection consistently.


Beran, J. (1992) Statistical Methods for Data with Long-Range Dependence. Statistical Science, 7(4), 404–416.
Beran, J. (2010) Long-range dependence. Wiley Interdisciplinary Reviews: Computational Statistics, 2(1), 26–35. DOI.
Berkes, I., Horváth, L., Kokoszka, P., & Shao, Q.-M. (2006) On discriminating between long-range dependence and changes in mean. The Annals of Statistics, 34(3), 1140–1165. DOI.
Breidt, F. J., Crato, N., & de Lima, P. (1998) The detection and estimation of long memory in stochastic volatility. Journal of Econometrics, 83(1–2), 325–348.
Csörgö, S., & Mielniczuk, J. (1999) Random-design regression under long-range dependent errors. Bernoulli, 5(2), 209–224. DOI.
Diebold, F. X., & Inoue, A. (2001) Long memory and regime switching. Journal of Econometrics, 105(1), 131–159. DOI.
Doukhan, P., Oppenheim, G., & Taqqu, M. S.(2003) Theory and applications of long-range dependence. . Birkhauser
Farmer, J. D., Gerig, A., Lillo, F., & Mike, S. (2007) Market efficiency and the long-memory of supply and demand: is price impact variable and permanent or fixed and temporary?. Quantitative Finance, 6(2), 107–112. DOI.
Giraitis, L., & Surgailis, D. (1999) Central limit theorem for the empirical process of a linear sequence with long memory. Journal of Statistical Planning and Inference, 80(1–2), 81–93.
Gneiting, T. (2000) Power-law correlations, related models for long-range dependence and their simulation. Journal of Applied Probability, 37(4), 1104–1109.
Granger, C. W. J., & Joyeux, R. (1980) An Introduction to Long-Memory Time Series Models and Fractional Differencing. Journal of Time Series Analysis, 1(1), 15–29. DOI.
Horváth, L. (2001) Change-point detection in long-memory processes. Journal of Multivariate Analysis, 78(2), 218–234.
Künsch, H. R.(1986) Discrimination between monotonic trends and long-range dependence. Journal of Applied Probability, 23(4), 1025–1030.
Lahiri, S. N.(1993) On the moving block bootstrap under long range dependence. Statistics & Probability Letters, 18(5), 405–413. DOI.
Robinson, P. M.(2003) Time series with long memory. . Oxford Univ Pr
Saichev, A. I., & Sornette, D. (2010) Generation-by-generation dissection of the response function in long memory epidemic processes. The European Physical Journal B, 75(3), 343–355. DOI.

See original: The Living Thing / Notebooks Long memory models