acceptodds
Withdrawn from ICLR 2027

SPELUNKA: A Framework for Finding Unanticipated Associations in Vision-Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language Models (VLMs) are the backbone of many state-of-the-art multimodal systems, but often encode spurious and harmful associations from their training data. While prior work focuses on removing known biases from VLMs, methods for discovering unanticipated associations for retrieval tasks remain underexplored. We introduce \methodname, an automated, inference-time framework for finding unanticipated distributional biases for retrieval tasks in VLMs without pre-specified labeled subgroups, inference-time human supervision, or model fine-tuning. Given a concept of interest, \methodname generates captions for related images, extracts relevant entities from captions, and selects semantically meaningful associations. A frequency-based detection rule then identifies spurious associations by measuring over- or under-representation of these attributes in a VLM's top-ranked outputs relative to their full dataset distribution. We evaluate \methodname across four datasets, and find that it 1) successfully recovers both synthetic and naturally occurring spurious correlations, 2) achieves an average AUROC of 0.93 across four datasets in identifying images strongly associated with a query, and 3) outperforms baselines in finding features that are strongly associated with a given query.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-language, fairness, interpretability, spurious correlation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_djBTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SPELUNKA discovers attributes that are over- or underrepresented in VLM retrieval results. It generates image captions, extracts and groups entities, and compares their frequencies in top-ranked images against the full dataset. The method requires no predefined subgroup labels or model finetuning. Experiments include attribute recovery, cross-dataset transfer, retrieval prediction, and debiasing examples. The paper reports an average AUROC of 0.93 for predicting top-5% retrieval membership.

Critical strengths
  1. The method addresses a practical auditing problem. Discovering retrieval associations without predefined subgroup labels is useful. The pipeline produces readable descriptions rather than unnamed clusters. Figure 3 also shows how a discovered association can guide an existing debiasing method.

  2. The main text provides several complementary forms of evidence. These include quantitative comparisons, label recovery across captioners, cross-dataset consistency, and visual case studies. Table 2 shows large gains at 16 entities. For example, SPELUNKA achieves 0.93 AUROC on jewelry, compared with 0.80 for the strongest reported baseline. These results support its ability to summarize retrieval behavior.

Critical weaknesses
  1. The evaluation does not sufficiently distinguish spurious associations from legitimate retrieval features. Section 3.2 treats frequency shifts as spurious associations. However, a feature can be overrepresented because it genuinely helps identify the query. The AUROC evaluation in Section 5.3 rewards any feature that predicts retrieval membership. It therefore does not establish accurate spurious-feature discovery. Similarly, Figure 6 shows that background entities occur in Waterbirds captions, but does not demonstrate that the audited model relies on them. Section 6 acknowledges part of this limitation, yet the main contribution claims remain stronger than the evidence.
  2. The caption-bias guarantee requires a stronger assumption than “better than chance.” The derivation in Section 3.3 uses the same entity-specific true-positive and false-positive rates for the top-ranked subset and the full dataset. This requires caption errors to remain stable under retrieval selection. If a captioner detects an entity more easily in high-ranked images, the observed frequency shift can be distorted or even reversed. Figure 5 provides useful examples, but does not test this assumption. The guarantee is valid under the stated noise model, not for arbitrary biased captioners.
Questions for authors
  1. Can you evaluate discovery precision and recall using known spurious and genuinely relevant attributes? For Waterbirds, can you show that the detected backgrounds affect retrieval scores while bird identity is controlled?

  2. Do captioner true-positive and false-positive rates differ between top-ranked and remaining images? Please state the required noise assumptions explicitly and clarify how they support Appendix E’s guarantees.

  3. Were entity selection and frequency-based ranking performed using only the training half before the random-forest evaluation? Were baseline features selected under the same restriction?

  4. Bias-to-Text already discovers interpretable bias keywords from image captions, although it uses classification errors. What does SPELUNKA add beyond adapting this approach to retrieval groups? Can you compare against a retrieval adaptation using the same captions and feature budget?

  5. How does the bootstrap handle the overlap between the top-ranked subset and the full dataset? Is retrieval repeated within each bootstrap sample? Do the discoveries remain reliable after correcting for testing many entities?

2 comments
AuthorsOpenReview ↗

Rebuttal Part 1.

We thank the reviewer for their response and provide the following point-by-point rebuttal.

W1. For the core experiments, including the mentioned section 3.2 and 5.3, we use a query that is representative of the class of images. For example, in Figure 4, we are using the query “person” on a dataset of people. For the core experiments, we are using a query that is representative of the class of images, i.e. in Figure 4, we are using the query “person” on a dataset of people. Because every image in the dataset could satisfy the query, the images that are retrieved display model-specific associations to the query. Normal semantic entities that are common across all images for that class (e.g. the entity "screen" for the class "TV") are unlikely to emerge as under or over-represented in the top-k images. The goal of Figure 6 is to demonstrate the validity of the entities extracted by SPELUNKA, i.e. SPELUNKA can find entities that are already known to be spurious from prior work. Other results such as Table 1 and Figure 20 demonstrate how the entities extracted from SPELUNKA reflect biases in the model.

We additionally establish the spurious nature of these features through a counterfactual experiment via image editing. The hypothesis being that adding an entity in an image that SPELUNKA identifies should change the distribution in the direction of the identified bias. This would mean that SPELUNKA identified a causal feature.

We created a pool of 1000 portraits using stable diffusion, filling in a randomized prompt that varied age, ethnicity, gender, expression, clothing, setting, and lighting. We ran SPELUNKA over the pool using the query “person” and withdrew the entities that were significant in both FairFace and the pool. For each of the overlapping entities, we used Qwen Image Edit to create 20 additive pairs (a pair with the entity and a do nothing null pair). We then measured the image’s rank with the entity in the pool minus the rank of the entity without it.

Out of the model/entity pairs that were significant, 7/8 had a shift that matched the pool’s bias sign.

Additionally, while phone related entities (phone, cell phone) did not show sign agreement with FairFace, the other entities all had sign agreement (serious expression, smiling, Asian man, young woman).

ModelEntityFairFacePoolSign MatchAdd Pairs Rank ShiftShift Match Pool
clip-vit-base-patch16phone0.082-0.013FALSE-208TRUE
clip-vit-base-patch32serious expression0.060.354TRUE178TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Ksmiling-0.087-0.108TRUE-40TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Kphone0.062-0.013FALSE-343TRUE
siglip-base-patch16-224Cell phone0.086-0.1FALSE18FALSE
siglip-base-patch16-224phone0.062-0.013FALSE-97TRUE
aimv2-large-patch14-224-litAsian man0.1680.143TRUE125TRUE
aimv2-large-patch14-224-lityoung woman-0.053-0.115TRUE80TRUE

W2. We will make it more clear in the main text that we assume that \alpha and \Beta are the same in D and in the top-k, but we show that it is not an unreasonable assumption. Empirically, in Figure 5 and Figure 12 (in the Supplement), we demonstrate broad sign agreement of labels and pseudolabels across multiple datasets.

AuthorsOpenReview ↗

Rebuttal Part 2.

Q1. We find that SPELUNKA is able to properly extract water-related entities and land-related entities from images with those backgrounds. We randomly sample 2000 images from the test set of which 1002 have a water background and 998 have a land background. We search for a match of words that are water-related or land-related (either in singular or plural form) in SPELUNKA-generated entities. We find that 17 water-related words cover 92.3% of all images labeled water and 22 land-related terms cover 92.4% of all images labeled land. Restricting to the 5 water-related words (i.e. water, ocean, lake, river, pond) still covers 84.3% of all images labeled water and 5 land-related terms (i.e. forest, tree, grass, field, ground) cover 83.5% of all land labeled images. Note that these 5 terms were chosen to relate directly to water or land, but they are not necessarily the most frequently occurring entities from SPELUNKA. This shows that SPELUNKA can correctly recover background labels and could detect background-related biases in the Waterbirds dataset.

Q2. See response to W2.

Q3. Entity selection and frequency-based ranking was performed only using the training half for the random-forest evaluation. Baseline features had the same restriction. We will make this more clear in the main text.

Q4. We believe our paper is very different from Bias-to-Text. Beyond their objective of discovering error slices, they use an existing algorithm for keyword selection which creates opacity in bias selection. Further, they do not show statistical significance or prevalence of their found biases and can not show that these biases are of the model (i.e. no validation to other models). Because their biases focus on error slice discovery, they generate coarse descriptions.

We find that bias-to-text cannot be fairly adapted into a retrieval setting in a straight-forward way. Beyond keyword extraction from misclassified images, bias-to-text also uses similarity scores between those keywords and the misclassified images to generate the top word candidates with downstream goals being related to fairness in the classification task. Thus, bias-to-text is too enmeshed in classification to generate a fair comparison that is true to the original method.

Q5. For each bootstrap, the dataset is resampled with replacement. Then, retrieval occurs over the set of entities to find the magnitude of each potential spurious correlation. These magnitudes create 95% CIs.

Reviewer_F8aUOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SPELUNKA, an automated framework for identifying previously unknown spurious associations in VLMs, particularly for image retrieval tasks. Unlike existing approaches that require predefined bias attributes, labeled subgroups, or model fine-tuning, SPELUNKA identifies potentially spurious associations without these requirements. The framework comprises of two stages. First, an image-captioning model generates descriptions of dataset images, and an LLM extracts semantically meaningful entities, such as objects, attributes, and backgrounds. Then, SPELUNKA compares the frequency of these entities in the top-k retrieved images against their frequency in the complete dataset to identify overrepresented or underrepresented attributes. The authors additionally provide a theoretical analysis showing that the direction of the measured frequency shift is preserved in expectation under a particular noisy-pseudolabel model. Experiments across four datasets (Fashion Product, FairFace, Dollar Street, and EmoSet) and multiple VLM architectures demonstrate that SPELUNKA can identify meaningful associations, recover known synthetic and naturally occurring biases, and outperform adapted versions of Domino, Distill, and RaVL.

Critical strengths

Strengths -

  1. Introduction is well written and I felt the whole paper was easy to follow.
Critical weaknesses

The paper has two critical weaknesses -

  1. Insufficient validation of whether identified associations represent actual model biases - I find that the paper does not convincingly distinguish genuine model-induced spurious correlations from underlying dataset associations. In Section 3.2, the authors identify biases by comparing entity frequencies between top-k retrieved images and the entire dataset. However, an overrepresented attribute may simply be semantically relevant to the query rather than reflecting a spurious correlation. Although Section 5.1 demonstrates that some associations persist across datasets, this does not establish that they originate from model biases. I believe controlled counterfactual experiments are necessary to demonstrate whether the identified attributes causally influence retrieval predictions.

  2. Inadequate evaluation metric and limited methodological novelty - I am not convinced that the experimental evaluation sufficiently supports the paper's central claims. In Section 5.3 and Table 2, the authors report an average AUROC of 0.93 by predicting whether images belong to the top 5% of retrieval results. However, this measures retrieval-membership predictability rather than the accuracy of spurious-correlation detection. Furthermore, the methodology in Section 3.2 primarily combines existing captioning models, LLM-based entity extraction, clustering, and frequency comparisons. I would like to see direct bias-detection metrics and comparisons with simpler frequency-based baselines to better establish the effectiveness and novelty of the proposed framework.

Questions for authors

See strengths.

1 comment

We thank the reviewer for their response and provide the following point-by-point rebuttal.

W1. For the core experiments, we are using a query that is representative of the class of images, i.e. in Figure 4, we are using the query “person” on a dataset of people. For the core experiments, we are using a query that is representative of the class of images, i.e. in Figure 4, we are using the query “person” on a dataset of people. Because every image in the dataset could satisfy the query, the images that are retrieved display model-specific associations to the query. Normal semantic entities that are common across all images for that class (e.g. the entity "screen" for the class "TV") are unlikely to emerge as under or over-represented in the top-k images.

We also conduct the following counterfactual experiment with SPELUNKA using image editing. The hypothesis being that adding an entity in an image that SPELUNKA identifies should change the distribution in the direction of the identified bias. This would mean that SPELUNKA identified a causal feature.

We created a pool of 1000 portraits using stable diffusion, filling in a randomized prompt that varied age, ethnicity, gender, expression, clothing, setting, and lighting. We ran SPELUNKA over the pool using the query “person” and withdrew the entities that were significant in both FairFace and the pool. For each of the overlapping entities, we used Qwen Image Edit to create 20 additive pairs (a pair with the entity and a do nothing null pair). We then measured the image’s rank with the entity in the pool minus the rank of the entity without it.

Out of the model/entity pairs that were significant, 7/8 had a shift that matched the pool’s bias sign.

Additionally, while phone related entities (phone, cell phone) did not show sign agreement with FairFace, the other entities all had sign agreement (serious expression, smiling, Asian man, young woman).

ModelEntityFairFacePoolSign MatchAdd Pairs Rank ShiftShift Match Pool
clip-vit-base-patch16phone0.082-0.013FALSE-208TRUE
clip-vit-base-patch32serious expression0.060.354TRUE178TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Ksmiling-0.087-0.108TRUE-40TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Kphone0.062-0.013FALSE-343TRUE
siglip-base-patch16-224Cell phone0.086-0.1FALSE18FALSE
siglip-base-patch16-224phone0.062-0.013FALSE-97TRUE
aimv2-large-patch14-224-litAsian man0.1680.143TRUE125TRUE
aimv2-large-patch14-224-lityoung woman-0.053-0.115TRUE80TRUE

W2. As discussed in W1, because we use a query that is representative of the class, all features identified are model-specific associations with the query. Further, experimentally, we first demonstrate this is true through finding associations in one datasets of people (FairFace) and finding sign agreement of these associations in an independent dataset (UTKFace) in Table 1. We also provide additional validation that these are spurious through Figure 6, finding associations in Waterbirds, and in a synthetically inserted bias in Figure 20 in the Supplement. We propose SPELUNKA as a novel pipeline that solves an open problem of finding unanticipated associations in unlabeled datasets without the need for a classification task. We provide both mathematical and experimental analyses to validate this pipeline.

Reviewer_QP7COpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SPELUNKA. It finds hidden biases in VLM retrieval. It captions images, extracts entities, and compares entity frequencies in the top-k results against the full dataset, which needs no labels, no classification task, and no fine-tuning. The authors also show that a biased captioner still preserves the direction of bias.

Critical strengths
  • The method extends bias discovery from classification to retrieval. Prior methods like Domino and RaVL need error slices. Retrieval has no clear accuracy metric. SPELUNKA uses a simple frequency-shift rule instead. Its outputs are readable words like “smiling” or “black-and-white”. These can go straight into projection debiasing.
  • The paper raises a key question: can a biased captioner audit another model? The answer is clean. The observed shift equals (α−β)(\alpha − \beta) times the true shift. So the sign holds if the captioner beats chance. This argument could help other pseudo-label auditing work.
  • SPELUNKA surpasses Domino and RaVL on unlabeled datasets. Biases found on FairFace also transfer to UTKFace. The grayscale and Waterbirds tests add further support.
Critical weaknesses
  • Frequency shift could be not same as spurious correlation. δ\delta only measures association. It cannot separate core features from spurious ones. It also cannot separate model bias from co-occurrence in the data. The grayscale intervention is convincing. Could the authors do this more broadly? For example, use image editing to change only the expression or the background, then check whether the rank changes. B2T[1] is cited but not compared. It could be adapted to retrieval as a baseline.
  • The noise assumption may break. The theory assumes α\alpha and β\beta are the same in D and in the top-k. But BLIP and CLIP share much of their training data. Salient objects are easier to caption and easier to retrieve. Captioners also tend to name race mainly for non-white faces. If noise depends on rank, the sign guarantee fails. Could the authors estimate TPR and FPR separately for the top-k and for the full set? Also, the top-k is a selected subset, not an i.i.d. sample.

Reference
[1] Kim et al. Discovering and Mitigating Visual Biases through Keyword Explanation. CVPR 2024

Questions for authors

Please see Critical Weaknesses part.

1 comment

We thank the reviewer for their response and provide the following point-by-point rebuttal.

W1. We conduct the following counterfactual experiment with SPELUNKA using image editing. The hypothesis being that adding an entity in an image that SPELUNKA identifies should change the distribution in the direction of the identified bias. This would mean that SPELUNKA identified a causal feature.

We created a pool of 1000 portraits using stable diffusion, filling in a randomized prompt that varied age, ethnicity, gender, expression, clothing, setting, and lighting. We ran SPELUNKA over the pool using the query “person” and withdrew the entities that were significant in both FairFace and the pool. For each of the overlapping entities, we used Qwen Image Edit to create 20 additive pairs (a pair with the entity and a do nothing null pair). We then measured the image’s rank with the entity in the pool minus the rank of the entity without it.

Out of the model/entity pairs that were significant, 7/8 had a shift that matched the pool’s bias sign.

Additionally, while phone related entities (phone, cell phone) did not show sign agreement with FairFace, the other entities all had sign agreement (serious expression, smiling, Asian man, young woman).

ModelEntityFairFacePoolSign MatchAdd Pairs Rank ShiftShift Match Pool
clip-vit-base-patch16phone0.082-0.013FALSE-208TRUE
clip-vit-base-patch32serious expression0.060.354TRUE178TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Ksmiling-0.087-0.108TRUE-40TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Kphone0.062-0.013FALSE-343TRUE
siglip-base-patch16-224Cell phone0.086-0.1FALSE18FALSE
siglip-base-patch16-224phone0.062-0.013FALSE-97TRUE
aimv2-large-patch14-224-litAsian man0.1680.143TRUE125TRUE
aimv2-large-patch14-224-lityoung woman-0.053-0.115TRUE80TRUE

We find that bias-to-text cannot be fairly adapted into a retrieval setting in a straight-forward way. Beyond keyword extraction from misclassified images, B2T also uses similarity scores between those keywords and the misclassified images to generate the top word candidates with downstream goals being related to fairness in the classification task. Thus, BT2 is too enmeshed in classification to generate a fair comparison that is true to the original method.

W2. We will make it more clear in the main text that we assume that \alpha and \Beta are the same in D and in the top-k, but we show that it is not an unreasonable assumption. Empirically, in Figure 5 and Figure 12 (in the Supplement), we demonstrate broad sign agreement of labels and pseudolabels across multiple datasets.

Reviewer_cANwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SPELUNKA, a framework for automatically identifying unanticipated associations in vision-language models (VLMs) without predefined bias attributes or manually labeled data. The method first generates image captions and extracts semantic attributes using language models. It then compares the frequency of these attributes in the top-ranked retrieval results against their frequency in the full dataset to identify over- or under-represented attributes. The authors evaluate the framework on multiple datasets, demonstrating its ability to recover known associations and identify attributes that are strongly predictive of retrieval rankings.

Critical strengths
  1. Practical relevance: The paper addresses an important problem of identifying unanticipated associations in VLMs, particularly in retrieval tasks where ground-truth labels may be unavailable.
  2. Simple and interpretable approach: The proposed framework is straightforward to implement and does not require predefined bias attributes, manual annotations, or model fine-tuning. It also produces human-interpretable attributes that can be inspected for potential biases.
  3. Extensive experimental evaluation: The authors evaluate the method across multiple datasets and VLM backbones, including experiments on known spurious correlations, robustness to different captioning models, and comparisons with existing methods.
Critical weaknesses
  1. Unclear problem definition: The paper does not clearly distinguish spurious correlations from normal semantic associations. The proposed method identifies attributes that are over- or under-represented in retrieval results, but such frequency differences do not necessarily indicate bias. Therefore, it is unclear whether the method actually solves the problem claimed by the authors.
  2. Limited methodological novelty: The proposed framework mainly combines existing image captioning, entity extraction, clustering, and frequency comparison techniques. While this combination is practical, the individual components are relatively simple, and the paper does not sufficiently explain why this approach represents a significant methodological contribution.
  3. Misalignment between evaluation and research objectives: The main quantitative evaluation uses the discovered attributes to predict whether images appear in the top-ranked retrieval results. However, high AUROC only demonstrates that these attributes are predictive of retrieval rankings, not that they represent spurious correlations. Therefore, the experimental results do not fully support the paper's central claims.
Questions for authors
  1. How do the authors distinguish spurious correlations from legitimate semantic associations? Since SPELUNKA identifies attributes based on frequency differences between top-ranked retrieval results and the full dataset, how can it determine whether an identified association is actually spurious?
  2. The proposed framework combines existing techniques, including image captioning, entity extraction, clustering, and frequency comparison. What is the key technical novelty of SPELUNKA beyond integrating these components?
  3. The main quantitative evaluation measures how well the discovered attributes predict whether an image appears in the top-ranked retrieval results. How does this metric demonstrate the ability to identify spurious correlations, not just simply identifying attributes correlated with retrieval rankings?
1 comment

We thank the reviewer for their response and provide a point-by-point rebuttal.

W1. For the core experiments, we are using a query that is representative of the class of images, i.e. in Figure 4, we are using the query “person” on a dataset of people. Because every image in the dataset could satisfy the query, the images that are retrieved display model-specific associations to the query. Normal semantic entities that are common across all images for that class (e.g. the entity "screen" for the class "TV") are unlikely to emerge as under or over-represented in the top-k images.

W2. Our proposed pipeline (how we combine existing image captioning and clustering components) is the technical novelty of our work. We prove that this pipeline works through mathematical and experimental results.

W3. As discussed in W1, because we use a query that is representative of the class, all features identified are model-specific associations to the query. Further, experimentally, we first demonstrate this is true through finding associations in one datasets of people (FairFace) and finding sign agreement of these associations in an independent dataset (UTKFace) in Table 1. We provide additional validation that these are spurious through Figure 6, finding known associations in Waterbirds, and in a synthetically inserted bias in Figure 20 in the Supplement.

Further, we conduct the following counterfactual experiment with SPELUNKA using image editing. The hypothesis being that adding an entity in an image that SPELUNKA identifies should change the distribution in the direction of the identified bias. This would mean that SPELUNKA identified a causal feature.

We created a pool of 1000 portraits using stable diffusion, filling in a randomized prompt that varied age, ethnicity, gender, expression, clothing, setting, and lighting. We ran SPELUNKA over the pool using the query “person” and withdrew the entities that were significant in both FairFace and the pool. For each of the overlapping entities, we used Qwen Image Edit to create 20 additive pairs (a pair with the entity and a do nothing null pair). We then measured the image’s rank with the entity in the pool minus the rank of the entity without it.

Out of the model/entity pairs that were significant, 7/8 had a shift that matched the pool’s bias sign.

Additionally, while phone related entities (phone, cell phone) did not show sign agreement with FairFace, the other entities all had sign agreement (serious expression, smiling, Asian man, young woman).

ModelEntityFairFacePoolSign MatchAdd Pairs Rank ShiftShift Match Pool
clip-vit-base-patch16phone0.082-0.013FALSE-208TRUE
clip-vit-base-patch32serious expression0.060.354TRUE178TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Ksmiling-0.087-0.108TRUE-40TRUE
CLIP-ViT-L-14-laion2B-s32B-b82Kphone0.062-0.013FALSE-343TRUE
siglip-base-patch16-224Cell phone0.086-0.1FALSE18FALSE
siglip-base-patch16-224phone0.062-0.013FALSE-97TRUE
aimv2-large-patch14-224-litAsian man0.1680.143TRUE125TRUE
aimv2-large-patch14-224-lityoung woman-0.053-0.115TRUE80TRUE

Q1. See W1 for why the semantic associations found by SPELUNKA are often spurious. However, we do note in our limitations that SPELUNKA does not assess harmfulness or contextual relevance. We leave that as a human-in-the-loop intervention where users who are trying to find biases in their dataset can choose which ones to correct for, which is why SPELUNKA does not automatically remove identified associations.

Q2. We propose SPELUNKA as a novel pipeline that solves an open problem of finding unanticipated associations in unlabeled datasets without the need for a classification task. We provide both mathematical and experimental analyses to validate this pipeline.

Q3. As in W1, we emphasize that because we are using a query that applies to the full dataset (i.e. “person” on FairFace), anything that shows up under SPELUNKA is a model-specific association to the query. I.e. “smiling” should not make a “person” more of a “person.” We have a synthetic experiment, an experiment holding entities across datasets, and the provided counterfactual experiment to further underscore how the found entities are spurious.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.