acceptodds
Withdrawn from ICLR 2027

SOFIA: Self-adaptive Optimization for Federated learning with Inpainting-based Anomaly-aware pretraining for robust defect detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual defect detection is critical for quality control in modern manufacturing, where data are often distributed across multiple production sites and subject to strict privacy constraints. In industrial settings such as pharmaceutical production, these challenges are further amplified by limited defect samples and strict data-sharing restrictions, making centralized training impractical while standard federated learning struggles with heterogeneous data distributions and limited defect samples. We introduce SOFIA, a privacy-preserving federated defect detection framework that jointly addresses these challenges by combining client-adaptive hyperparameter optimization based on Horse Self-adaptive Optimization with object-oriented inpainting-based pretraining. The former adapts local training to client-specific data characteristics, while the latter generates semantically localized synthetic defects to improve representation learning under limited data. Experiments on pharmaceutical and metal surface defect detection benchmarks show that SOFIA improves detection performance over centralized and federated baselines. Ablation results further indicate that the two mechanisms play complementary roles: adaptive optimization improves federated training, while inpainting-based pretraining provides its largest benefit when integrated with the federated optimization framework. We further study how SOFIA behaves as the number of participating clients grows, finding consistent gains in performance and stability as more clients participate, suggesting favorable scalability as federation size increases. These findings highlight the importance of jointly addressing optimization and representation learning for scalable and reliable visual defect detection in distributed, privacy-constrained industrial settings.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Federated Learning, Swarm Optimization, Inpainting, Visual Defect detection, Deep Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_QdR3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SOFIA, a federated framework for industrial defect detection that combines client-specific hyperparameter optimization with object-oriented inpainting-based pretraining. Horse Self-adaptive Optimization (HSO), a simplified variant of the Wild Horse Optimizer, searches for local training configurations. Object-Oriented Inpainting (OOI) uses GradCAM++ masks and MI-GAN to generate localized synthetic anomalies from defect-free images. These components are integrated with a YOLO-World detector and federated model aggregation. Experiments are conducted on the Pill and NEU-DET datasets. The paper reports comparisons with detection baselines, component ablations, and analyses involving different communication rounds and client counts. Its main claim is that jointly adapting local optimization and improving representations through synthetic anomalies benefits federated defect detection.

Critical strengths
  1. The paper addresses a practically relevant combination of challenges. Distributed industrial data, limited defect examples, and differences between clients provide a reasonable motivation for jointly considering local optimization and representation learning. The framework offers a concrete way to investigate their combination in federated defect detection.
  2. The ablation results provide encouraging evidence for the combined framework. In Table 3, adding OOI to FL+HSO increases overall mAP50 from 0.842 to 0.872, with improvements for both Normal and Damage classes. OOI does not improve the separately trained local baseline in the same table. This difference is an interesting empirical finding that merits further investigation.
Critical weaknesses
  1. The central pretraining-to-detection pipeline is insufficiently specified. Section 3 describes MI-GAN reconstruction, adversarial learning, and knowledge distillation, and subsequently states that the pretrained weights are used in the detection phase. However, it does not clearly explain which network is pretrained, how the generated images are annotated for detector training, or which parameters are transferred into YOLO-World. The GradCAM++ reference model and mask-construction procedure also need clarification. Similarly, HSO’s neighborhood operator and candidate-selection procedure are not sufficiently specified. Appendix A reports changes to the final training configuration, including changes to batch size and learning rate on NEU-DET, but the rule connecting these adjustments to the HSO-selected configuration remains unclear. These omissions make it difficult to reproduce the proposed method and identify what produces the reported improvement.
  2. The experimental controls do not sufficiently establish the added value of the proposed components. Table 3 lacks an FL+OOI configuration, so it cannot determine whether the observed benefit requires HSO specifically or follows from combining OOI with federated training more generally. Figure 4 compares FL and FL+HSO, but does not establish HSO’s advantage over a simpler search method under an equal evaluation budget. Likewise, the contribution of object-oriented masking is not isolated from that of synthetic-image pretraining. Baseline interpretation is also problematic. The “CL” baseline in Section 4.4 consists of independently trained local models rather than a model trained on pooled data. Moreover, a client receiving federated updates can benefit from other clients’ data, so using one-third of the images locally does not establish a reduction in total training-data requirements. These issues weaken the attribution and scope of the paper’s main claims.
Questions for authors
  1. Please specify the complete OOI training and transfer procedure. Which model and target are used for GradCAM++? How are masks constructed? Is MI-GAN frozen or further trained? How are synthetic images annotated, which network receives pretraining, and how is its checkpoint used to initialize YOLO-World?
  2. Please clarify the executable HSO protocol and subsequent configuration adjustments. What are the population size, search iterations, neighborhood operator, candidate-selection rule, and per-client evaluation budget? Which data split is used to calculate fitness? For NEU-DET, why are the batch sizes and learning rate changed between Tables 6 and 7, and are equivalent adjustments applied to the baselines?
  3. Can you provide focused controls for the claimed component contributions? In particular, FL+OOI would help assess whether HSO is necessary for the reported interaction. A search baseline with the same evaluation budget and a random-mask baseline using the same generator and pretraining budget would help establish the contributions of HSO and object-oriented masking. Please report variability across independent runs for the principal comparisons.
  4. Please clarify model evaluation and training-data accounting. Are the reported client results obtained from the aggregated global model or locally updated models, and are they evaluated on the same test set? Were the Figure 2a baselines rerun under comparable conditions? How does the “one-third of the training data” claim account for information obtained through federated aggregation?
  5. Please clarify the heterogeneity and client-scaling protocols. How are client distributions constructed, and what measurable differences exist between them? When the number of clients increases, is the total dataset held fixed? What observations and independent repetitions underlie the confidence intervals, and how should these intervals be distinguished from the variation across clients reported in Table 9?
Reviewer_Uq4VOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a recipe for federated defect detection, combining client-specific hyperparameter search, inpainting-based synthetic anomalies, and a YOLO-World detector trained with FedAvg. The application is reasonable, and the combination may be useful in practice. However, my overall impression is that the paper reads more like a technical application report than a research contribution with a clear new insight. The proposed search procedure appears to be a fairly standard gradient-free search, and the motivation for combining the components is stronger than the evidence explaining why they work together. The experiments report some encouraging gains, but the baseline choices, incomplete controls, and unclear training details make those gains difficult to interpret. As presented, I am not convinced that the paper establishes much beyond a potentially useful recipe for the evaluated settings.

Critical strengths
  • The practical motivation is reasonable and it could be worthwhile application of FL in industry.
  • The combined approach shows some positive results.
Critical weaknesses
  • I do not see a clear methodological contribution. The paper combines existing tools, and HSO reads like standard stochastic hyperparameter search. There is no comparison showing why this search procedure is needed.
  • The combination produces little new insight. The authors report that the components work better together, but do not explain why or rule out simpler alternatives.The most interesting observation is that synthetic-anomaly pretraining helps FL+HSO but does not help local-only training. However, the paper largely restates this result as evidence that the components are complementary. It does not investigate the mechanism or establish when this interaction should occur.
  • The experiments are not convincing. The “centralized” baseline is actually local-only training, comparisons use different detector architectures, and equally budgeted tuning controls are missing. The gains are difficult to attribute to the proposed method.
  • Even the recipe is unclear. The pretraining and weight-transfer procedure is ambiguous, and final training departs from the searched settings. This makes the method difficult to reproduce and assess.

Overall, I find the contribution insufficient. The paper does not establish a meaningful methodological advance, and the experimental work is not rigorous enough to make it a convincing application study.

Questions for authors

I have no additional questions. My concerns are about the limited contribution and insufficient experimental support. Addressing them would require substantial additional work rather than clarification in the rebuttal.

Reviewer_YjPKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

SOFIA combines client-specific hyperparameter search, object-oriented inpainting, and defect detection in a federated setting. HSO searches local training configurations. OOI uses attention-derived masks and inpainting to produce localized synthetic anomalies for pretraining. The resulting training procedure is evaluated on pharmaceutical and steel-surface defect images. The paper reports detection comparisons, component ablations, and analyses of communication rounds and client counts. Its main claim is that adapting optimization and improving representations together helps when clients have different data distributions and limited defect data.

Critical strengths

The reported Pill ablation provides a promising result and shows a useful boundary of the proposed combination.

Table 3 reports overall mAP50 of 0.872 for the complete method, compared with 0.842 for FL+HSO. It also retains the result that adding OOI to local training reduces the score from 0.843 to 0.839. Reporting this negative result helps readers see that the component is not uniformly beneficial. Figure 4 additionally includes CL, FL, and FL+HSO. These experiments provide a starting point for investigating when the combined procedure helps.

Critical weaknesses

1、The “centralized learning” baseline trains independent local models, leaving no comparison with centralized training on pooled data. The “one-third of the training data” claim also needs clarification because Algorithm 1 aggregates clients, while Tables 6–7 list 420 training images per client. Finally, the six-class NEU-DET description conflicts with the damaged/normal labels in Figure 14, leaving comparability with published six-class results uncertain. 2、The experiments divide data evenly across clients but do not clearly establish controlled differences in client distributions or systematically vary the number of real defect labels, limiting support for the two challenges motivating SOFIA. HSO lacks comparisons with random search and relevant federated hyperparameter optimization methods under the same computational budget, including FedEx and FedPop . Table 3 also omits FL+OOI, so the existing ablations do not establish whether OOI’s improvement depends on HSO.

Questions for authors

1、What class mapping, fixed training/validation/test splits, and source of defect-free images were used for NEU-DET? Please explain the damaged/normal labels in Figure 14, identify the model producing 75.4% in Figure 2a, and confirm whether the compared results use the same task and evaluation protocol. 2、Have you evaluated a pooled-data centralized model with the same backbone, initialization, and training budget? Does “one-third of the training data” describe each client’s local allocation or the total unique images used across search, pretraining, and final training. 3、Could you compare HSO with random search and a relevant federated hyperparameter optimization method under the same total training budget, and add the missing FL+OOI ablation? Please also evaluate the gains under explicitly controlled client-distribution differences and several defect-label budgets, since these experiments would directly test the claimed benefits.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.