This repository contains the code used for the experiments presented in the paper: "Hypericons for interpretability: decoding abstract concepts in visual data".
The repository is organized into three main sections:
-
The ARTstract Dataset v0.1: This section includes a novel dataset of cultural images tagged with Abstract Concepts. It represents the first iteration of the dataset and serves as the foundation for the experiments conducted in the paper.
-
Baselines for ACD: Here, you will find the baseline models implemented using finetuned CNNs (Convolutional Neural Networks). These models are specifically designed for the task of Abstract Concept Detection (ACD) in Cultural Images. They serve as a reference point for evaluating the performance of the proposed approaches.
-
Explainability Experiments: This section focuses on the experiments conducted to explore visual interpretability in deep networks. It includes both traditional methods, such as GRAD-CAM, as well as the novel SD-AM (Stable Diffusion-denoised Activation Maximization) method proposed in the paper. These experiments aim to provide insights into non-traditional approaches for visual explainability when dealing with abstract concepts.
Please refer to the individual directories for more detailed information and instructions on how to use the code and datasets.
If you are using the data or methods provided in this repository, we kindly request that you cite the following paper:
@misc{martinez2023hypericons,
title = {Hypericons for interpretability: decoding abstract concepts in visual data},
author = {Martinez Pandiani, Delfina Sol and Lazzari, Nicolas and van Erp, Marieke and Presutti, Valentina},
year = {2023},
howpublished = {\url{https://doi.org/10.1007/s42803-023-00077-8}},
journal = {International Journal of Digital Humanities},
license = {CC BY 4.0}
}
This citation acknowledges the original authors and provides the necessary information to reference their work. Thank you for respecting and acknowledging their contribution!
Feel free to explore the contents of this repository and use the provided code and datasets for further research and experimentation.