Luckiness Normalized Maximum Likelihood-based Change Detection for High-dimensional Graphical Models with Missing Data

This study focuses on detecting dependency changes in multivariate time series. This is a practically important issue because, for example, changes in the relationships between several stocks may lead to early warning of critical economic events. The corresponding change detection methods should be feasible for addressing high-dimensional cases online. We use the Gaussian graphical model (GGM) to represent the conditional dependencies in multivariate and reduce our issue to change detection in GGM and consider the cases with the size of the graphical model significantly larger than the sample size as high-dimensional. Although there are several studies on GGM change detection, adequately controlling the sparsity of the model to handle high-dimensional data remains a significant challenge. To address this problem, we introduce new change statistics based on the luckiness normalized maximum likelihood (LNML) code length and minimum description length (MDL) adapted for sparse modeling. We demonstrate how to efficiently calculate the LNML-based change statistics and provide a theoretical guarantee in terms of Type I and Type II error probabilities. We expand the methodology to include cases in which missing values exist. We compare the effectiveness of our methodology with that of conventional methods through experiments using synthetic and real-world datasets.

Authors

Institutions

Publication Details

Published
2024-12-15
DOI
https://doi.org/10.1109/bigdata62323.2024.10825444
Primary Topic
Statistical Methods and Inference
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Luckiness Normalized Maximum Likelihood-based Change Detection for High-dimensional Graphical Models with Missing Data

Kenji Yamanishi, Linchuan Xu, Zhongyi Que
Statistical Methods and Inference
article

Luckiness Normalized Maximum Likelihood-based Change Detection for High-dimensional Graphical Models with Missing Data

Kenji Yamanishi, Linchuan Xu, Zhongyi Que
article en

Abstract

This study focuses on detecting dependency changes in multivariate time series. This is a practically important issue because, for example, changes in the relationships between several stocks may lead to early warning of critical economic events. The corresponding change detection methods should be feasible for addressing high-dimensional cases online. We use the Gaussian graphical model (GGM) to represent the conditional dependencies in multivariate and reduce our issue to change detection in GGM and consider the cases with the size of the graphical model significantly larger than the sample size as high-dimensional. Although there are several studies on GGM change detection, adequately controlling the sparsity of the model to handle high-dimensional data remains a significant challenge. To address this problem, we introduce new change statistics based on the luckiness normalized maximum likelihood (LNML) code length and minimum description length (MDL) adapted for sparse modeling. We demonstrate how to efficiently calculate the LNML-based change statistics and provide a theoretical guarantee in terms of Type I and Type II error probabilities. We expand the methodology to include cases in which missing values exist. We compare the effectiveness of our methodology with that of conventional methods through experiments using synthetic and real-world datasets.

The University of Tokyo (JP)
Climate action
Openalex Percentile: Top 74%
Statistical Methods and Inference
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Luckiness Normalized Maximum Likelihood-based Change Detection for High-dimensional Graphical Models with Missing Data — Kenji Yamanishi, Linchuan Xu, et al. · (2024) | TGRS Research Map | TGRS