Empirical Study of Rectifier and Dropout in Feedforward Neural Networks
In this paper, we analyze the unique sparsity properties of dropout, rectified activation functions, and other activation functions through simple experiments. We explore how the distinctive membership functions of each contribute to these sparsity characteristics.
Authors
- Sunhwan Lim (ORCID: https://orcid.org/0009-0009-0101-0925)
- Sungpil Woo (ORCID: https://orcid.org/0000-0002-2561-1356)
- Dae Young Kim (ORCID: https://orcid.org/0000-0001-5540-5134)
Institutions
- Electronics and Telecommunications Research Institute (KR)
Publication Details
- Published
- 2024-10-16
- DOI
- https://doi.org/10.1109/ictc62082.2024.10826989
- Primary Topic
- Neural Networks and Applications
- Type
- article
- Field-Weighted Citation Impact
- 0.00
Funders
- National Research Foundation of Korea