Self-Evolving Neural ODEs with Dwell-Time-Constrained Structural Compression

We propose an adaptive latent-width framework for continuous-time neural networks that dynamically regulates the number of active latent coordinates during training. The proposed architecture integrates a factor-weighted encoder derived from a Tucker decomposition with Liquid Time-Constant (LTC) dynamics formulated as a Neural Ordinary Differential Equation (Neural ODE). To facilitate structured adaptation of the latent representation, we introduce a spectral-energy probe that monitors the contribution of latent components, together with an importance-based binary masking mechanism governed by a minimum dwell-time constraint to prevent excessively frequent structural updates. The framework is evaluated on three distinct benchmarks: MNIST, CIFAR-100, and a single-subject functional Magnetic Resonance Imaging (fMRI) dataset. Its performance is assessed through predictive accuracy, active-state compression, representational similarity, spectral-energy diagnostics, sensitivity bounds, and checkpoint-level Jacobian spectral analysis. On MNIST, the final experimental run achieves a test accuracy of 97.51%, with only 104 of 256 latent coordinates active, corresponding to a 59.38% reduction in active latent dimensionality. Across five random seeds, the model achieves an average accuracy of 97.47 ± 0.17%. On CIFAR-100, the selected checkpoint attains 48.02% test accuracy and a macro-F1 score of 0.4766, while retaining 109 active ODE coordinates. On the single-subject fMRI benchmark, a six-fold leave-one-run-out evaluation yields an accuracy of 43.63 ± 6.86%, with a separate temporal-interval analysis identifying a performance peak at 4 seconds. The theoretical analysis, provided in the appendix, establishes boundedness, Lipschitz sensitivity, an energy-balance identity, a perturbation bound under latent-coordinate collapse, structural properties of the width-adaptation schedule, and a sufficient condition for local contraction. Despite these results, the observed positive eigenvalues in checkpoint Jacobian spectra, the moderate performance on the fMRI benchmark, and the absence of direct runtime measurements for masked implementations highlight important limitations. In particular, the framework does not establish global stability or demonstrate practical inference-time acceleration. Overall, this work presents an integrated approach to adaptive structural evolution in continuous-time neural networks, combining latent representation control, continuous-time dynamics, and importance-driven dimensional adaptation within a unified learning framework.

Authors

Institutions

Publication Details

Journal
Zenodo (CERN European Organization for Nuclear Research)
Published
2026-10-31
DOI
https://doi.org/10.5281/zenodo.23835584
Primary Topic
Neural Networks and Applications
Type
preprint
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
preprint

Self-Evolving Neural ODEs with Dwell-Time-Constrained Structural Compression

Gurpreet Singh, Dr.Jasvir Singh
Zenodo (CERN European Organization for Nuclear Research)
Neural Networks and Applications
preprint

Self-Evolving Neural ODEs with Dwell-Time-Constrained Structural Compression

Gurpreet Singh, Dr.Jasvir Singh
preprint en

Abstract

We propose an adaptive latent-width framework for continuous-time neural networks that dynamically regulates the number of active latent coordinates during training. The proposed architecture integrates a factor-weighted encoder derived from a Tucker decomposition with Liquid Time-Constant (LTC) dynamics formulated as a Neural Ordinary Differential Equation (Neural ODE). To facilitate structured adaptation of the latent representation, we introduce a spectral-energy probe that monitors the contribution of latent components, together with an importance-based binary masking mechanism governed by a minimum dwell-time constraint to prevent excessively frequent structural updates. The framework is evaluated on three distinct benchmarks: MNIST, CIFAR-100, and a single-subject functional Magnetic Resonance Imaging (fMRI) dataset. Its performance is assessed through predictive accuracy, active-state compression, representational similarity, spectral-energy diagnostics, sensitivity bounds, and checkpoint-level Jacobian spectral analysis. On MNIST, the final experimental run achieves a test accuracy of 97.51%, with only 104 of 256 latent coordinates active, corresponding to a 59.38% reduction in active latent dimensionality. Across five random seeds, the model achieves an average accuracy of 97.47 ± 0.17%. On CIFAR-100, the selected checkpoint attains 48.02% test accuracy and a macro-F1 score of 0.4766, while retaining 109 active ODE coordinates. On the single-subject fMRI benchmark, a six-fold leave-one-run-out evaluation yields an accuracy of 43.63 ± 6.86%, with a separate temporal-interval analysis identifying a performance peak at 4 seconds. The theoretical analysis, provided in the appendix, establishes boundedness, Lipschitz sensitivity, an energy-balance identity, a perturbation bound under latent-coordinate collapse, structural properties of the width-adaptation schedule, and a sufficient condition for local contraction. Despite these results, the observed positive eigenvalues in checkpoint Jacobian spectra, the moderate performance on the fMRI benchmark, and the absence of direct runtime measurements for masked implementations highlight important limitations. In particular, the framework does not establish global stability or demonstrate practical inference-time acceleration. Overall, this work presents an integrated approach to adaptive structural evolution in continuous-time neural networks, combining latent representation control, continuous-time dynamics, and importance-driven dimensional adaptation within a unified learning framework.

Zenodo (CERN European Organization for Nuclear Research)
Punjabi University (IN)
Neural Networks and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon