Visualize and Paint GAN Activations

We investigate how generated structures of GANs correlate with their activations in hidden layers, with the purpose of better understanding the inner workings of those models and being able to paint structures with unconditionally trained GANs. This gives us more control over the generated images, allowing to generate them from a semantic segmentation map while not requiring such a segmentation in the training data. To this end we introduce the concept of tileable features, allowing us to identify activations that work well for painting.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>The code is available at: https://github.com/rherdt185/visualize-gan-activations

Authors

Institutions

Publication Details

Published
2024-09-22
DOI
https://doi.org/10.1109/mlsp58920.2024.10734757
Citations
2
Primary Topic
Cell Image Analysis Techniques
Type
article
Field-Weighted Citation Impact
1.32

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
article

Visualize and Paint GAN Activations

Peter Maaß, Rudolf Herdt
2 citations
Cell Image Analysis Techniques
1.32
article

Visualize and Paint GAN Activations

Peter Maaß, Rudolf Herdt
article en
2 citations

Abstract

We investigate how generated structures of GANs correlate with their activations in hidden layers, with the purpose of better understanding the inner workings of those models and being able to paint structures with unconditionally trained GANs. This gives us more control over the generated images, allowing to generate them from a semantic segmentation map while not requiring such a segmentation in the training data. To this end we introduce the concept of tileable features, allowing us to identify activations that work well for painting.<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup><sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup>The code is available at: https://github.com/rherdt185/visualize-gan-activations

University of Bremen (DE)
Deutsche Forschungsgemeinschaft
Openalex Percentile: Top 14%
Cell Image Analysis Techniques
1.32
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.