Hill neural networks.

We present a novel neural network architecture with strong innate sparsity properties that also allows explaining the model decisions by identifying the bases for selecting and rejecting different decision alternatives. The elementary building block of the proposed architecture is a Hill neuron with antagonistic activation outputs, one for activation and the other for repression. Both outputs are relayed to the neurons of the next layer modulated by distinct weights and included jointly in the summation to provide the input to the next layer neurons, and pitted against each other during training that promotes one of the two by increasing its weight and reducing that of the other in a way that minimizes the training error. The activation and repression functions are themselves parameterized by an activation threshold and a Hill coefficient that are also included in the training updates, shaping them actively and adaptively by training. Post-training evaluation of Hill coefficients and connection weights allowed substantial reduction in model size while preserving the recognition performance of the original model without re-training. Experiment results on MNIST datasets of handwritten digits and letters show that the recognition performance of conventional neural network models of the same size featuring single-output artificial neurons can be matched using a small fraction of hidden layer neurons and connections: On the MNIST dataset of handwritten digits, the reduced model possessed %42.75 of the hidden layer neurons and %15.82 of the connections of a conventional artificial neural network. On the EMNIST dataset of handwritten letters, the percentages were %83.38 and %31.65, respectively. Dual saliency maps of activation and repression associated with the output layer neurons along with the important hidden layer neurons highlighted the image features that were instrumental in favoring the selected class and rejecting the others. These results demonstrate the potential of the proposed Hill neural networks as suitable candidates for sparse, interpretable and explainable learning systems.

Authors

Institutions

Publication Details

Journal
PubMed
Published
2026-11-12
DOI
https://doi.org/10.1016/j.neunet.2026.109869
Primary Topic
Neural Networks and Applications
Type
article
Field-Weighted Citation Impact
0.00
Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Hill neural networks.

Bilge Karaçalı
PubMed
Neural Networks and Applications
article

Hill neural networks.

Bilge Karaçalı
article en

Abstract

We present a novel neural network architecture with strong innate sparsity properties that also allows explaining the model decisions by identifying the bases for selecting and rejecting different decision alternatives. The elementary building block of the proposed architecture is a Hill neuron with antagonistic activation outputs, one for activation and the other for repression. Both outputs are relayed to the neurons of the next layer modulated by distinct weights and included jointly in the summation to provide the input to the next layer neurons, and pitted against each other during training that promotes one of the two by increasing its weight and reducing that of the other in a way that minimizes the training error. The activation and repression functions are themselves parameterized by an activation threshold and a Hill coefficient that are also included in the training updates, shaping them actively and adaptively by training. Post-training evaluation of Hill coefficients and connection weights allowed substantial reduction in model size while preserving the recognition performance of the original model without re-training. Experiment results on MNIST datasets of handwritten digits and letters show that the recognition performance of conventional neural network models of the same size featuring single-output artificial neurons can be matched using a small fraction of hidden layer neurons and connections: On the MNIST dataset of handwritten digits, the reduced model possessed %42.75 of the hidden layer neurons and %15.82 of the connections of a conventional artificial neural network. On the EMNIST dataset of handwritten letters, the percentages were %83.38 and %31.65, respectively. Dual saliency maps of activation and repression associated with the output layer neurons along with the important hidden layer neurons highlighted the image features that were instrumental in favoring the selected class and rejecting the others. These results demonstrate the potential of the proposed Hill neural networks as suitable candidates for sparse, interpretable and explainable learning systems.

PubMedVol. 206(Pt C)
Izmir Institute of Technology (TR)
Neural Networks and Applications
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon