acceptodds
Under review as a conference paper at ICLR 2027

Sin-SMFL: An Ablation Study of Modulation, Class Balancing, and Gain Calibration in Detection Losses

Abstract

Modulated classification losses (e.g., Focal Loss) and class-frequency reweighting schemes have each been studied for object detection, but their composition and the practical hyperparameter interactions it introduces remain underexplored. We conducted a study of sine-modulated classification loss combined with effective-number class balancing (Sin-MFL) on YOLOv8 using the COCO 2017 object detection dataset, comprising 27.6 GB of data with 118K training images, 5K validation images. We compared the six configurations across different class using three random seeds with 15-epoch training to understand the effect of each design choice. We find that, first, combining sine modulation with class balancing under-performs the detector's own unmodified default loss. This gap occurs because sine modulation reduces the magnitude of the classification loss compared with binary cross-entropy (BCE). As a result, a gain tuned for BCE gives too little weight to the modulated loss. Second, once this gain is corrected, sine modulation is a robust improvement over classical power-law (Focal Loss) modulation, improving 77 of 80 COCO classes when balancing or the gain are well fixed. Third, a learned loss balancer automatically fixes most of the loss-weighting problem without hyperparameter search, but manual tuning still performs better for some individual classes. Fourth, effective-number class balancing mainly improves performance for the 10–15 rarest classes, but it does not reduce the overall performance gap between different class frequencies. We report these findings, including the negative and mixed results, as a case study showing that loss-function modifications do not always combine effectively under realistic, limited training settings.

Primary area:
optimization
Keywords:
object detection, loss function design, focal loss, class imbalance, ablation study, YOLO

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4YQEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors proposed a sine-modulated BCE loss for object detection. They used YOLO8 and COCO and trained the model for 15 epochs from scratch and compared different loss functions like BCE, Focal Loss, Cost-sensitive losses like effective number of samples, and their proposed sine-modulated loss. The analysis shows that the sine-modulated loss favours the medium-difficulty samples and it produces low gradients for too easy and too hard examples. The authors conducted ablations and showed that the baseline loss is best for the 15-runs and the sine-modulated loss second best outperforming the Focal loss.

Critical strengths

The sine-modulated loss looks interesting and it is something that have never seen before for the object detection task. I like the analysis of the gradient and it shows that this design favours the medium-difficulty samples.

Critical weaknesses

The current experiments and analysis is insufficient. In detail, the experiments are run for 15 epochs not the full 300, and the results are suboptimal ~40 AP in COCO. There is little breadth, only one model and one dataset thus it is not easy to say whether the sine-modulated loss works in more general applications.

Questions for authors

I would like to give some suggestion to the authors to improve the quality of the paper. First, I would like to see comprehensive ablation at 300 epochs because the current study only uses 15 and it is too little training epochs to draw some valuable conclusions. Also, the authors could consider using this loss to 3 datasets in total e.g. they could use LVIS dataset [a1] that fits better with the rare-category narrative and the V3DET [a2] or OpenImages [a3]. Next, regarding the model, I think the authors could consider adding more models in the survey. There are many sources of imbalance in detection no class imbalance only, and there have been many architectures that tackle different imbalance problems [a4]. For example, I would like to see whether this loss can benefit Detr architectures, particularly their sampling through the Hungarian loss. Also, do we need this loss in RPN networks? Next, what is the cost of the sine-modulated function, is it slow during training or stable? Going beyond detection, there are sota papers like Deit3 that use BCE to train for image classification, so, does this modulating loss work for classification either balanced or imbalanced datasets like ImageNet-1K and/or ImageNet-LT, iNaturalist18 and Places-LT [a5]. Regarding the eval-metris, COCO mAP could be problematic as shown here [a6] so please consider giving more eval-metrics and visual and qualitative explanations of the loss and the behavior and the representation learned

[a1] Gupta et. al. 'LVIS: A Dataset for Large Vocabulary Instance Segmentation; in CVPR19 [a2] Wang et. al. 'V3Det: Vast Vocabulary Visual Detection Dataset' in ICCV2023 [a3] Kuznetsova et. al. 'The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale' in IJCV 2020 [a4] Oksuz et. al. 'Imbalance Problems in Object Detection: A Review' tPAMI [a5] Liu et. al. 'Large-Scale Long-Tailed Recognition in an Open World' in CVPR2019 [a6] Dave et. al. 'Evaluating Large-Vocabulary Object Detectors: The Devil is in the Details' in Arxiv.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 4YQE

Thank you for your review. My reply to each point is below. All changes are in the revised PDF.

Summary, point 1: "trained from scratch". This is not correct, and the paper was not clear. The main runs start from the COCO-pretrained yolov8s.pt, which scores 0.4409 mAP50-95 before any training. This also explains the AP of about 40. We now say this in Section 4. From scratch, 50 epochs: BCE 0.3671, Focal 0.3500, Sin-SMFL 0.3470 (Section 5.7).

Summary, point 2: "sine-modulated loss second best, outperforming Focal". After we tuned the gain of every loss, sine and Focal are equal. Best results at 15 epochs (gains 0.25 to 4; Table 3): BCE 0.4121, Sin-SMFL 0.4048, Focal 0.4042. The difference between Sin-SMFL and Focal is 0.0006, and the run-to-run difference is up to 0.004. Per class, 42 better and 38 worse (p = 0.74). We corrected the abstract and the introduction.

W1. Only 15 epochs, not 300. AP is only about 40. We ran 50 epochs: BCE 0.4312, Sin-SMFL 0.4278, Focal 0.4272 (Table 5). The order and the gaps are the same as at 15 epochs. We did not run 300 epochs, and we cannot say what happens at 300. This is in the Limitations.

W2. Only one model and one dataset. We added YOLOv8n and YOLOv8m (15 epochs, one run each, BCE / Sin-SMFL / Focal): 0.3422 / 0.3305 / 0.3321 and 0.4560 / 0.4471 / 0.4472. BCE is best for both. We also added LVIS (see Q2). All models are still YOLOv8.

Q1. Ablation at 300 epochs. Not done. See W1. We have 50-epoch results only.

Q2. More datasets: LVIS, V3Det, OpenImages. We added LVIS (from scratch, 50 epochs, official LVIS evaluator, one run; Section 5.8, Table 6):

  • AP: BCE 0.1419, Focal 0.0951, Sin-SMFL 0.0890.
  • Rare-class AP: 0.0019, 0.0141, 0.0136. Frequent-class AP: 0.2610, 0.1619, 0.1543.
  • Focal and Sin-SMFL are about 7 times better than BCE on rare classes, worse on frequent classes, and equal to each other.
  • By number of training instances: for 11 to 20 instances, Focal 0.0354 and Sin-SMFL 0.0356 against BCE 0.0182. For 501 or more, BCE 0.257 against 0.157 and 0.151.
  • Classes with 1 or 2 training instances score 0 for all three losses. V3Det and OpenImages: not done. We list them as a limitation.

Q3. More models: DETR with Hungarian matching, RPN-based detectors, other kinds of imbalance. Not done. We only tested YOLOv8 (three sizes) and class imbalance. The paper makes no claim for other detectors. This is in the Limitations.

Q4. Cost of the sine function: slow, or stable in training? We did not measure it, so we give no number. This is in the Limitations.

Q5. Does the loss work for classification: ImageNet-1K, ImageNet-LT, iNaturalist18, Places-LT? Not tested. This is in the Limitations.

Q6. More metrics than COCO mAP, and visual or qualitative explanations. Metrics: done. We added per-class AP, AP by class frequency group, the official LVIS AP, APr, APc, APf, and intervals over classes for the rare-class comparison (Sections 5.6 and 5.8). Qualitative figures and the full protocol of Dave et al.: not done. Strength: sine favours medium-difficulty examples. We tested the strength of the modulation (15 epochs, gain 2). Weaker modulation scores higher: sine n = 1 / 2 / 3 gives 0.4100 / 0.4048 / 0.3959. Focal gamma 1 / 2 / 3 gives 0.4076 / 0.4042 / 0.3951. So we do not see an accuracy gain from the sine shape.

On seeds: the training code fixes its data-loader seed, so runs with different nominal seeds are not independent. We now report one run per configuration with a run-to-run difference of up to 0.004, and no standard deviations or p-values (Section 4).

Do these answers cover your concerns?

Reviewer_4YQEOpenReview ↗

Thank you for your answers

My concerns are not addressed yet. The LVIS results are too low to be informative, in the future please see EFL CVPR2022 [a1] for best practices of YOLO training in LVIS. The COCO finetuning is not convincing, because the COCO pretrained model used a sigmoid loss, not a sine-modulated, there could be a misalignment during the finetuning, training-from-scratch is a more convincing experiment. Regarding the breadth, I think there should be more experiments with DETR and some discussion beyond object detection makes the paper more complete. I will keep my score.

[a1] Li et. al. Equalized Focal Loss for Dense Long-Tailed Object Detection, CVPR2022

AuthorsOpenReview ↗

Thankyou for the review (Reviewer 4YQE )

Thank you for the further comments. This paper is an ablation study: it explains what each design choice contributes, and it does not aim to prove Sin-SMFL best. Below are all the results that bear on your three points.

1. LVIS and EFL. We now cite EFL and state that our LVIS numbers (YOLOv8s, from scratch, 50 epochs) are for comparing the three losses with each other; an EFL-style setup is future work. On rare categories, Sin-SMFL and Focal reach about 7 times BCE's AP and match each other:

LVIS v1 val, from scratch, 50 epochsAPAPr (rare)APcAPf
BCE0.14190.00190.09030.2610
Focal0.09510.01410.06710.1619
Sin-SMFL0.08900.01360.06020.1543

2. Fine-tuning vs scratch. We trained from scratch as you suggest, and also fine-tuned for 15, 50 and 100 epochs and on two more model sizes:

COCO val2017, mAP50-95BCEFocalSin-SMFL
YOLOv8s fine-tune, 15 epochs0.41210.40420.4048
YOLOv8s fine-tune, 50 epochs0.43120.42720.4278
YOLOv8s fine-tune, 100 epochs0.43280.43220.4314
YOLOv8s from scratch, 50 epochs0.36710.35000.3470
YOLOv8n fine-tune, 15 epochs0.34220.33210.3305
YOLOv8m fine-tune, 15 epochs0.45600.44720.4471

Sin-SMFL is within 0.003 of Focal in every setting. With longer fine-tuning the gap to BCE narrows from 0.0073 to 0.0014 (100 epochs, all three within 0.0014). The from-scratch gains were taken from the fine-tuning sweep and not retuned.

3. Beyond detection. We added CIFAR-100-LT (ResNet-32, 200 epochs, Appendix F). Without class weights BCE, Focal and the sine loss tie (38.3, 37.9, 38.2). With class weights, Sin-SMFL is ahead of Focal at every setting (+1.4 points at beta 0.999, +2.8 at beta 0.99; within seed spread):

CIFAR-100-LT top-1 (%), 3 seedsAllManyMediumFew
BCE (no class weights)38.3365.9936.108.66
Focal (beta 0.999)24.6448.5919.702.46
Sin-SMFL (beta 0.999)26.0250.7021.232.82
Focal (beta 0.99)29.2255.5024.783.73
Sin-SMFL (beta 0.99)31.9959.6427.994.40
Focal (no class weights)37.8965.9035.498.01
Sine (no class weights)38.1765.1936.338.80

This is a stand-in, not ImageNet-LT, and we make no claim for DETR.

Why we believe the paper is ready.

  • It answers a practical question with controlled evidence: in this detector, gain calibration moves mAP by up to 0.005 (Focal at 100 epochs: 0.4258 to 0.4322), more than the choice of loss shape (under 0.0015 once each loss is tuned).
  • The design is rigorous: a factorial ablation, paired per-class comparisons, a measured noise band, the official LVIS evaluator, and every number traced to a logged run.
  • Breadth now covers COCO at 15, 50 and 100 epochs, from scratch, three model sizes, LVIS and CIFAR-100-LT.
  • Your review and the others led us to match every claim to the evidence.

For these reasons we believe the paper is ready as an ablation study, and we respectfully ask you to reconsider your score.

Thank you for the detailed review. It led to the from-scratch comparison, the 100-epoch runs and the narrower, better-supported claims.

Reviewer_uzwnOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies sine modulation, effective-number class weighting, and classification loss gain in YOLOv8. It compares six configurations on COCO 2017 using YOLOv8s, with 15 training epochs and three seeds for the aggregate results. Sine modulation performs better than power modulation at the same default classification gain. Increasing the gain improves Sin-SMFL substantially, but the original BCE baseline still performs best. Class weighting mainly benefits a small group of rare classes.

Critical strengths

The paper reports the negative results clearly. Its comparisons show that changing the classification gain has a large effect on the outcome, and the main results are repeated across three seeds. This is a useful practical observation for implementing and comparing detection losses.

Critical weaknesses
  1. The contribution is limited, and the experiments are too narrow to establish a useful general finding. The method makes small changes to familiar loss weighting choices. Effective-number class weighting and uncertainty-based loss balancing are existing methods. The main new component is the sine modulation function, and much of the performance recovery comes from changing a scalar classification gain. I find the methodological novelty limited. The evaluation also uses a well-established setup: YOLOv8s on COCO, with only 15 epochs of training. There are no results on another detector or dataset. For an empirical study, the paper needs to show how far its findings generalize. LVIS would be relevant to the class imbalance question, and Objects365 would provide a useful test on another detection dataset. The current experiments do not establish whether the conclusions extend beyond this short training run and detector configuration. The comparison with Focal Loss is also incomplete. The paper emphasizes gain calibration but does not report a comparable gain sweep for the Focal baseline. Without that comparison, it is unclear how much of the reported advantage comes from the sine function itself. The tuned Sin-SMFL configuration also remains below the original BCE baseline.

  2. The figure quality needs substantial improvement. Figures 1 and 4 contain text that is too small to read comfortably at normal page size. The legend in Figure 6 is crowded and difficult to distinguish. These figures carry the study design and several main comparisons, so their readability directly affects the reader’s ability to assess the results. The presentation looks insufficiently polished. These are basic issues that should have been fixed before submission.

Questions for authors
  1. Clarification: Did you tune the classification gain for the Focal baseline? If so, does sine modulation still perform better when both methods receive the same tuning budget?

  2. Clarification: Were the models trained from scratch or initialized from pretrained weights? Please specify the checkpoint and pretraining dataset, since this matters for interpreting the 15-epoch results.

  3. Additional validation: Do you have evidence that the main findings hold with longer training, another detector, or datasets such as LVIS and Objects365? Broader validation would materially affect my assessment of the empirical contribution.

2 comments
AuthorsOpenReview ↗

Response to Reviewer uzwn

Thank you for your review. My reply to each point is below. All changes are in the revised PDF.

W1a. The contribution is limited. Effective-number weighting and uncertainty-based balancing already exist. Much of the recovery comes from one scalar gain. We agree. The paper now presents this as a study of gain calibration, not a new method. It says three things (Sections 5.3 and 5.4):

  • A single classification gain explains most of the difference between the losses.
  • Effective-number weights are almost constant on COCO (about 0.88 for every frequent class), so they act like a fixed gain.
  • BCE with a tuned gain is the strongest baseline we found.

W1b. Only YOLOv8s on COCO, 15 epochs. No other detector or dataset. LVIS and Objects365 would help. We added these (one run each; Sections 5.7 and 5.8):

  • 50 epochs, BCE / Sin-SMFL / Focal: 0.4312 / 0.4278 / 0.4272.
  • YOLOv8n: 0.3422 / 0.3305 / 0.3321. YOLOv8m: 0.4560 / 0.4471 / 0.4472.
  • LVIS, from scratch, 50 epochs, official LVIS evaluator: AP 0.1419 / 0.0890 / 0.0951. Rare-class AP 0.0019 / 0.0136 / 0.0141. Frequent-class AP 0.2610 / 0.1543 / 0.1619.
  • On LVIS, Focal and Sin-SMFL are about 7 times better than BCE on rare classes, worse on frequent classes, and equal to each other. They beat BCE below about 20 training instances and lose above it. Classes with 1 or 2 training instances score 0 for all three.
  • We trained LVIS from scratch because its images overlap with COCO, which the pretrained weights have already seen.
  • BCE is first and the other two are within 0.003 of each other in every setting. Not done: Objects365, other detector families (all models are YOLOv8), 300 epochs. These are listed in the Limitations.

W1c. No gain sweep for the Focal baseline. It is unclear how much of the advantage comes from sine. We ran the sweep (gains 0.25 to 4, 15 epochs; Table 3). Best results: BCE 0.4121 (gain 1), Sin-SMFL 0.4048 (gain 2), Focal 0.4042 (gain 4). Sine does not beat Focal once each has its own gain. The difference is 0.0006, and the run-to-run difference is up to 0.004. Per class, Sin-SMFL is better on 42 classes and worse on 38 (p = 0.74). The lead of sine at low gain (+0.019 at gain 0.5) comes from the sine loss being larger. We removed the claim that sine improves on Focal. Focal's best gain is the largest we tried, so a larger gain may help Focal a little.

W1d. Tuned Sin-SMFL is still below BCE. Yes. BCE is higher by about 0.007 (0.4121 vs 0.4048), and by 0.003 at 50 epochs. The paper now says this in the abstract and in Section 5.4.

W2. Text in Figures 1 and 4 is too small. The legend in Figure 6 is crowded. We redrew them. Figure 1 is now a plain table of the six configurations with larger text. Figure 4 has larger text and clearly different line colors. Figure 6 is now a bar chart of paired per-class results and has no legend.

Q1. Did you tune the Focal gain? Does sine still win with the same tuning? In the first version, no. Now yes, and sine does not win. See W1c.

Q2. Scratch or pretrained? Which checkpoint and pretraining data? Pretrained. All main runs start from the Ultralytics yolov8s.pt, trained on COCO. It scores 0.4409 mAP50-95 before any training, higher than every fine-tuned run. We now say this in Section 4. From scratch, 50 epochs: BCE 0.3671, Focal 0.3500, Sin-SMFL 0.3470 (Section 5.7).

Q3. Do the findings hold with longer training, another detector, LVIS, Objects365? Longer training (50 epochs), two more model sizes and LVIS: yes, same order of losses (see W1b). Another detector family and Objects365: not tested. We make no claim beyond YOLOv8.

Also, on seeds: the training code fixes its data-loader seed, so runs with different nominal seeds are not independent. We now report one run per configuration with a run-to-run difference of up to 0.004, and no standard deviations or p-values (Section 4).

Do these answers cover your concerns?

Thanks for your corrections and addidional experiments.

Overall, I think this is an experimental report rather than a paper. Considering both the contribution and the results of the paper, I believe its overall contribution to the detection community is limited.

Therefore, I will keep my rating.

Reviewer_wmF3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors train YOLOv8s on COCO for 15 epochs in six settings. They vary how much easy examples and rare classes count. They also vary how much the model should focus on identifying objects relative to locating them. The standard loss averaged 0.408 mAP50–95 across three runs. Sin-SMFL scored 0.376 at the default classification weight. Raising that weight from 0.5 to 2.0 brought the score to 0.402. Learning all three loss weights gave 0.394. The paper also reports gains for some rare classes, although the average across all classes did not improve when class weighting was added.

Critical strengths
  1. Keeping the standard loss in the comparison gives the gain from tuning a clear meaning. Increasing one weight closed 0.026 of the original 0.032 gap, leaving a gap of 0.006 after 15 epochs.

  2. In one matched run, adding class weights helped 26 classes, hurt 51 and left three unchanged. Those counts give a clearer picture of who benefits than the average score alone.

Critical weaknesses
  1. At the default weight, sine beat power for 77 of 80 classes in one matched run. But the sine formula also gives most examples more weight than the power formula does. Without giving power the same chance to tune its weight, it is hard to tell how much of the lead comes from the shape of the sine curve.

  2. The five rarest classes gained 0.0224 on average in that run. One class, toaster, had only nine validation instances and gained 0.130. Using the rounded numbers in Table 4 and the text, the other four averaged about −0.0045. The claim about rare classes depends heavily on which classes are counted.

Questions for authors
  1. Which run supports the statement in the abstract that sine beats power after the classification weight is adjusted? If Table 2(d) is the only comparison, could you try the same weight choices for power and sine? That would show whether the lead survives tuning.

  2. Do the rare-class gains also appear in the other two runs? If their predictions were saved, could you report the averages for the same rare classes, both with and without toaster? This would show how much the result depends on one class or one run.

  3. Did the 15 epochs start from pretrained weights or from scratch? Which YOLOv8 version, learning-rate schedule and data augmentation settings were used? These details would help readers repeat the short-budget comparison.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Reviewer wmF3

Thank you for your review. My reply to each point is below. All changes are in the revised PDF.

W1. Sine beat power for 77 of 80 classes, but sine gives more weight to most examples. Power was not tuned the same way. You are right. We ran the same gain sweep (0.25, 0.5, 1, 2, 4) for BCE, Focal and Sin-SMFL (15 epochs, mAP50-95; Section 5.4, Table 3).

  • Sin-SMFL minus Focal at gain 0.5 / 1 / 2 / 4 is +0.019 / +0.010 / +0.004 / -0.001.
  • Best results: Focal 0.4042 (gain 4), Sin-SMFL 0.4048 (gain 2). The difference is 0.0006. Run-to-run difference is up to 0.004.
  • Per class, Sin-SMFL is better on 42 classes and worse on 38 (p = 0.74).
  • The sine loss is about 2.5 times larger than the power loss on easy examples (when n = gamma = 2), so a larger share of the lead at low gain comes from loss size.
  • Focal's best gain is the largest one we tried, so a larger gain may help Focal a little. We removed the claim "sine beats Focal" from the abstract and the introduction.

W2. Toaster (9 validation instances) gained 0.130. The other four of the five rarest classes averaged about -0.0045. The rare-class claim depends on which classes are counted. You are right. The gain for the rarest classes comes mostly from toaster. We rewrote Section 5.6.

  • On our final results (Sin-SMFL, gain 0.5, with vs without class weights, 15 epochs), the 15 rarest classes split 6 better and 9 worse. The mean gain for the 5 rarest (+0.047) comes from toaster (+0.153).
  • With each loss at its best gain, on the 27 rarest classes, Sin-SMFL minus Focal is +0.0039 at 15 epochs (95% interval over classes +0.0003 to +0.0075) and +0.0010 at 50 epochs (-0.0031 to +0.0051).
  • BCE is as good as both other losses on this group (Table 4).
  • On LVIS, Focal and Sin-SMFL are about 7 times better than BCE on rare classes (APr 0.0141 and 0.0136 vs 0.0019) and equal to each other.
  • We now say in the paper that the rare-class gain on COCO is small and mixed.

Q1. Which run supports "sine beats power after the weight is adjusted"? Can you try the same weights for both? No run supports it. We tried the same weights for both (answer to W1 above and Table 3). With the weight chosen for each loss, they are equal. The claim is removed.

Q2. Do the rare-class gains appear in the other two runs? Report the averages with and without toaster. The other two seeds are not independent runs. The training code fixes its data-loader seed, and several configurations give the same result as seed 0, also per class. So we cannot give two independent repeats, and the paper now says this (Section 4). Instead we report the 27 rarest classes (at most 5,131 training instances), with and without toaster (Table 4):

mAP50-95 over the 27 rarest classes (15 epochs with toaster / 15 epochs without / 50 epochs with / 50 epochs without):

  • BCE: 0.4902 / 0.4889 / 0.5082 / 0.5075
  • Focal: 0.4820 / 0.4834 / 0.5066 / 0.5069
  • Sin-SMFL: 0.4859 / 0.4875 / 0.5076 / 0.5071

Removing toaster changes the averages by at most 0.002. The intervals above are over classes for one run, so they do not include run-to-run difference.

Q3. Pretrained or from scratch? Which version, learning-rate schedule and augmentation? All main runs start from the COCO-pretrained Ultralytics yolov8s.pt. It scores 0.4409 mAP50-95 before any training, which is higher than every fine-tuned run. We should have said this in the paper, and now do (Section 4). Settings: 640-pixel images, batch 16, AdamW, learning rate 0.001, 3 warmup epochs, default Ultralytics augmentation. Each run's full settings are in its args.yaml. We also ran from scratch for 50 epochs: BCE 0.3671, Focal 0.3500, Sin-SMFL 0.3470 (Section 5.7).

Do these answers cover your concerns?

AuthorsOpenReview ↗

Thank you, Reviewers 4YQE, wmF3 and uzwn: summary of the results and contributions of our study

Summary for all reviewers: what Sin-SMFL and this study offer

Thank you all for the reviews. This paper is an ablation study. It explains what each design choice contributes (modulation, class weighting, gain). It does not set out to prove that Sin-SMFL is the best loss, and we ask that it be read that way. Our claim: Sin-SMFL is a usable alternative to Focal Loss, including on rare categories.

COCO val2017, YOLOv8s, mAP50-95 (each loss at its best gain)

EpochsBCEFocalSin-SMFL
150.41210.40420.4048
500.43120.42720.4278
1000.43280.43220.4314

100-epoch gain sweep, COCO val2017, YOLOv8s, mAP50-95 (one run each)

Lossgain 1gain 2gain 4
BCE0.43280.4319not run
Focal0.42580.43130.4322
Sin-SMFL0.42890.43140.4268

LVIS v1 val, from scratch, 50 epochs

LossAPAPr (rare)APcAPf
BCE0.14190.00190.09030.2610
Focal0.09510.01410.06710.1619
Sin-SMFL0.08900.01360.06020.1543

CIFAR-100-LT top-1 (%), 200 epochs, mean of 3 seeds

Loss (class weighting)AllManyMediumFew
BCE (none)38.3365.9936.108.66
Focal (beta 0.999)24.6448.5919.702.46
Sin-SMFL (beta 0.999)26.0250.7021.232.82
Focal (beta 0.99)29.2255.5024.783.73
Sin-SMFL (beta 0.99)31.9959.6427.994.40
Focal (none)37.8965.9035.498.01
Sine (none)38.1765.1936.338.80

Sin-SMFL settings (15 epochs, gain 2)

SettingValuemAP50-95
Sine exponent n1 / 2 / 30.4100 / 0.4048 / 0.3959
Balancing beta0.99 / 0.999 / 0.99990.4021 / 0.4048 / 0.3951
Class balancingoff / on0.4015 / 0.4048

What Sin-SMFL offers

  • On par with Focal. At each loss's tuned gain, within the noise (about 0.004), at 15 and 100 epochs.
  • Close to BCE with longer training. Within 0.0014 at 100 epochs.
  • Better than Focal at the default gain. 0.378 vs 0.362 (15 epochs).
  • Rare categories. On LVIS, about 7 times BCE's rare AP, matching Focal.
  • Classification. On CIFAR-100-LT, ahead of Focal in every weighted setting (+1.4 to 2.8 points, suggestive); with no class weights all losses tie.
  • Stable settings, bounded gradient, drop-in (replaces only the classification term).

What the study offers

  • Factorial design separating modulation, class weighting and gain, with paired per-class comparisons.
  • Gain calibration explained. The sine loss is about (pi/2)^2 larger than the power loss on easy examples, so the right gain differs by loss; a learned balancer recovers most of the loss at the default gain.
  • Wide checks. 50 and 100 epochs, from scratch, three model sizes, LVIS, CIFAR-100-LT.

Scope: YOLOv8-style detectors; most settings are single runs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.