acceptodds
Under review as a conference paper at ICLR 2027

OmniCens: Self-Improving Distillation for Multimodal Classification of Millions of Profiles

Abstract

Classifying millions of social-media profiles from images, text, and contextual evidence is valuable for audience measurement but expensive when every prediction depends on a frontier model. We propose OmniCens, a self-improving distillation framework for reducing this dependence through local open-source multimodal models. Its central principle is that an expensive teacher response can serve both the current classification and subsequent student learning. We formulate the objective as minimizing amortized classification cost under precision, recall, and coverage constraints, accounting for teacher inference, calibration, student updates, and local execution. A confidence-based acquisition rule directs teacher supervision toward uncertain profiles, and repeated distillation targets recurring weaknesses while retaining the complete input evidence and attribute schema. The evaluation protocol isolates the value of continued learning and selective supervision through comparisons with fixed distillation, non-learning cascades, and random acquisition under matched budgets. Independent annotations assess predictive quality separately from teacher agreement, while held-out profiles and communities assess generalization. The key question is when reductions in teacher dependence recover the cost of adaptation without degrading classification quality. OmniCens connects this trade-off to a practical objective: making detailed audience characterization and creator–brand matching feasible for researchers and organizations with limited computational resources.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal classification, knowledge distillation, active learning, vision-language models, selective prediction, efficient inference, open-source models

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_JqH1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

OmniCens, an anonymous ICLR 2027 submission, proposes cutting the cost of classifying social-media audience members into a fixed 1,000-field categorical profile. A frontier teacher (Grok 4.7) labels profiles, and each label also trains a small local student that would eventually handle confident cases on its own, an idea the authors concede is already established by Cache & Distil and Online Cascade Learning. The student pairs frozen CLIP and BGE encoders with a 14k-parameter head that adjusts class-frequency priors. In a pilot of only ten people over two rounds, measured purely as agreement with Grok since no human labels exist, it never clearly beats a majority-class baseline (e.g., 60.99% vs. 58.73%). The title promises several components: million-profile scale, uncertainty-based acquisition, calibrated routing, matched-budget baselines, and independent evaluation, but the authors have not implemented them, and they conclude the student is not yet fit to replace the teacher.

Critical strengths

None

Critical weaknesses

Critical weaknesses

  1. The central claim is untested. The paper is about reducing teacher cost through selective, self-improving distillation, yet none of the components that would show this were run. There is no uncertainty-based acquisition, no calibrated router, and no matched-budget comparison against random acquisition, fixed distillation, or a non-learning cascade. No teacher calls were ever avoided, so there is no measured cost saving. The paper describes a proposal and presents a pilot of the data pipeline.
  2. There is no ground truth. All metrics are agreement with Grok. The paper’s objective is defined by precision and recall against independent labels, which don’t exist. Whether the teacher is accurate, and whether the student is a good copy of an accurate system or a bad copy of a flawed one, is unknown.
  3. There’s no novelty. The authors acknowledge that the core loop already exists (Cache & Distil, Online Cascade Learning) and that they claim no new distillation or aggregation method. The remaining contributions, keeping complete inputs, reconstructing negative labels, freezing predictions, are good engineering practice, not research contributions at ICLR’s bar. There’s also no comparison to those prior methods.

Other weaknesses

  1. The student doesn’t work. Across both prospective rounds, the learned models mostly trail a predictor that ignores the input and outputs each field’s most common value. Retraining on new data made things worse in round one and gave +0.5 points in round two, still below the baseline. A distillation paper whose student can’t beat class priors has no positive result to report.
  2. Unreliable teacher labels. The ground-truth labels come from Grok’s consumer chat interface rather than its API. The authors uploaded each profile’s images, text, and the full labeling rules into long chat sessions that processed many profiles in sequence (22M tokens for 18 profiles).
  3. Unusual and ad hoc experimental details. Several details suggest a non-standard, improvised setup rather than a controlled experiment. Most notably, the authors admit they cannot verify whether Grok actually looked at the images: they only observe “input-read operations” displayed by the chat interface, which they note “does not prove model attention.” Other oddities point the same way: experiments run on a laptop with CPU usage throttled to 20% of one core and thermal backoff, cost discussions mixing tokens, CPU seconds, and prepaid cash, and repeated attention to trivia such as float64 byte counts and logit discrepancies of 10−1610^{-16}. Meanwhile, basic choices, like using the API or an established vision model resolution, go undefended.
Questions for authors

None

Reviewer_7ApoOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This manuscript proposes OmniCens, a workflow that distills a frontier VLM teacher (Grok 4.7) into a small local head on frozen CLIP/BGE features, to classify profiles into about 1,000 categorical fields. The evidence is 4 development profiles and prospective rounds on 6 and 4 people. The students never beat the class-prior majority, adaptation lowers the agreement , and continuation gains +0.50pp but stays below the majority. Uncertainty acquisition, matched-budget arms, independent annotation and held-out-community evaluation are not executed.

Critical strengths
  1. Honest: Negative results are reported honestly, documenting the pitfalls of pseudo-label distillation.

  2. Transparency: the Scope paragraph and Secs. 5-6 separate what is implemented from what is not, and all metrics are teacher agreement, not accuracy.

  3. The controls are reasonable: shuffled-semantics, class-majority, always-UNKNOWN.

Critical weaknesses
  1. Overclaim. The title says 'millions of profiles', but it seems evidence covers 10 people.

  2. If I haven't missed any information, defining components (routing, three arms, independent annotation) are not executed.

  3. 'Self-improving' actually degrades: warm 56.67% < cold 57.08% < majority 58.73% (Table 2), and +0.50pp continuation stays below the majority (Table 3).

  4. Metrics are teacher agreement only, and the teacher condition changes mid-study (coverage 8.45%).

  5. Reproducibility. The Reproducibility Statement cites an 'anonymous NumPy supplement' but it should be provided. However, I can't find the code in the Supplementary Material.

Questions for authors
  1. Why does adaptation fall below the class-prior majority: overfitting from 20 updates on 6 people, the field-averaged weighting in Eq. 3, or teacher label noise? Sec. 5.5 admits priors and residual are confounded, and an isolating ablation seems cheap.

  2. The random-acquisition arm needs no new teacher labels, so why was it not run?

  3. At 151.983s wall time for 2 profiles, feature extraction alone is about 880 core-days for a million profiles. How is the title's scale feasible?

  4. Please provide the Numpy supplement code.

Reviewer_a4xCOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how teacher-generated labels can reduce the cost of classifying social-media profiles from images and text. OmniCens uses frozen image and text encoders with a small trainable classifier, combining class priors with a learned residual to predict profile attributes. Teacher responses are reused for later training. The paper reports reconstruction of historical labels and a small prospective pilot comparing adapted students with fixed students and majority-class predictors.

Critical strengths

The authors candidly report negative results.

Critical weaknesses

The central contribution remains unvalidated. The abstract describes confidence-based acquisition, independent annotations, and matched-budget comparisons, but the body states that these are unfinished. The cost accounting and implementation diagnostics do not show lower total cost at matched quality and coverage. The presentation also mixes proposed components, earlier implementations, and corrected experiments, making it difficult to identify the final evaluated system and what it establishes.

Questions for authors

Which completed results support a reliable benefit from continued adaptation beyond the fixed-student and majority baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.