acceptodds
Under review as a conference paper at ICLR 2027

From the Zoo to the BULLPEN: Sample-Efficient LLM Profiling via Multi-Channel Model Embeddings

Abstract

Choosing among the hundreds of thousands of available large language models (LLMs) requires a thorough understanding of each candidate model’s properties. In practice, each property is measured by its own benchmark suite, so profiling a model on many properties means paying for many separate evaluations. To address this, model embeddings compress each model into a fixed-length vector from which many properties can be predicted. Existing methods, such as EmbedLLM, build these vectors from correctness patterns and question features, and often need large probe batteries to do so. However, many behaviors that are important when choosing a model, such as its verbosity, safety and style, show up more in how it answers than in whether it is correct. We introduce BULLPEN (Behavioral Understanding of Large Language models via Profile ENcodings), a framework that extends EmbedLLM in three ways. First, BULLPEN organizes the downstream tasks that prior work uses and adds 20 task benchmarks we built on behavior, style and provenance. Second, it is the first framework to read a model’s own answer text jointly with its correctness and the question text. Third, it encodes models from a few hundred questions instead of the full 24,118-question pool. Across 260 open-weight models, our answer-text encoder scores 0.48 ± 0.10 on the new tasks against 0.35 ± 0.10 for the best prior encoder. Our encoders that read answer text keep at least 91% of their full-pool skill from only 400 questions (1.3% of the inference compute of the full question pool), while a correctness-only baseline keeps 83% with the same 400 questions. Because these embeddings need only model outputs, they support behavioral auditing and low-cost profiling, and could extend to API-only models. Code and data: https://github.com/wcquesera/bullpen.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
large language models, representation learning, model embeddings, LLM evaluation, efficient evaluation, model routing

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TCb6OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

BULLPEN builds fixed-size model embeddings from answer text, correctness bits, and question text. The embeddings predict model behavior, style, provenance, capability, and preference. The method mainly uses BGE answer embeddings, PCA/PLS-SVD fusion, and simple prediction heads, with IRT-based probe selection. The study covers 260 open weight models on various tasks. It uses five-folds holdout and reports that 400 questions retain about 91% of full-pool skill.

Critical strengths

The problem is practical and important because profiling a growing model pool is expensive. The data and task coverage are broad, spanning new behavior and new provenance tasks, also established capability&preference tasks. The paper also admits that correctness-based encoders remain stronger on competence tasks, which makes the conclusions more balanced.

Critical weaknesses

The methodological novelty is a little bit limited; this is closer to a broad benchmark and representation study. The fused encoder does not significantly beat the strongest prior method across all tasks. The data is limited to single-turn QA tasks. It remains to be demonstrated whether this method is sufficiently effective for models in multi-turn, tool-use, or agentic tasks.

Questions for authors

For each new task, please state whether its labels use similar questions/answers as the probe.

Does the 400-question efficiency hold under different temperatures, system prompts, languages, multi-turn settings, agentic tasks or closed API models?

If the answer-text representation performs similarly to existing fingerprint methods, does BULLPEN’s main increment come from its task suite, IRT selection, and evaluation protocol rather than a new embedding mechanism?

3 comments

We thank the reviewer for a balanced and encouraging review, and for noting that the paper reports where correctness-based encoders remain stronger. We ran 8 of our 24 follow-up analyses in response, with numbers in the table at the end of the general response. They clarify what is new beyond the task suite, why the fused code ties the strongest prior method, which labels overlap the probe, and how robust the embedding is.

You noted that the work is closer to a benchmark study, and asked whether the gain comes from the suite rather than the embedding. We agree that the submission reads as a benchmark study, and the camera-ready version will reframe it around the second latent, response style, within an IRT framework. The gain comes from the answers. With tasks and protocol fixed, answer text beats correctness bits on the 20 new tasks at every width (row 6), and the choice of questions matters little (row 13). On behavior, style and safety it also beats the fingerprinting encoders LLMmap and LLM-DNA (row 10). On lineage alone, LLM-DNA ties our method, and we will say so.

On the fused encoder, you are right that it only ties the strongest prior method over all 46 tasks (row 9). Competence tasks dominate that average, and correctness already measures ability well. Where style matters, answer text alone beats JE-IRT and four other prior encoders on the 11 behavior, style and safety tasks (row 10). The fused code keeps most of this style, but full answer text still does better, most clearly on refusal (row 8). We therefore present the fused code as the best compact representation and answer text as the fuller measure of style.

We agree that the data is limited to single-turn QA, and the camera-ready version will restrict its claims accordingly. Multi-turn, tool-use, agentic and non-English settings remain future work. Our closed-model results show why harder settings matter. These models nearly saturate single-turn correctness, so correctness bits group them together while their answers still separate them by style.

You asked us to "state whether [each new task's] labels use similar questions/answers as the probe." We checked every task and will add a per-task table. No label uses the questions or answers that the encoders read, except answer divergence. Labels computed from answers, such as length, hedging and reward-model scores, use separate questions. Capability labels come from held-out questions of the same benchmarks, so they share format but not items. The rest come from public sources such as model cards or from a separate refusal probe. Answer divergence compares a model's answers with other models' answers, so it must read the stored answers. We keep it out of the answer-text controls.

On robustness, we regenerated answers for 19 models under two system prompts, and for 18 of them at temperature 0.7. Temperature does not move a model's position, but a system prompt does. At temperature 0.7, answer text re-identifies 17 of 18 models, against 11 for correctness bits. Under a neutral system prompt it re-identifies 13 of 19, and under a concise one 10 of 19, while correctness bits do better there (row 15). Reference and query answers must therefore share a system prompt, which the camera-ready version will state as a limitation. For closed API models, heads trained only on open models rank their hedging and answer length almost perfectly (row 17).

We thank the reviewer again for the encouraging assessment. These changes make the paper easier to read and its claims more precise.

The rebuttal addressed my questions. While the method’s novelty is limited, I find the empirical results valuable and maintain my rating.

Thank you again for taking the time to review our work. We appreciate your input and contributions to its development.

Reviewer_UrdQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

This paper proposes BULLPEN, a framework for constructing LLM embeddings using three information channels:

Correctness Bits: Whether a model answers each evaluation question correctly.

Question Text: The semantic information contained in the evaluation questions.

Answer Text: The complete responses generated by the model.

The framework employs existing techniques, including pretrained BGE text embeddings, PCA, PLS-SVD, and linear prediction heads, to construct fixed-dimensional representations of language models.

The experiments cover 260 open-weight models and 46 downstream tasks, including 20 newly constructed prediction tasks. The main claims are that incorporating answer text improves the prediction of model behaviors and that approximately 400 probe questions can capture most of the predictive performance obtained from substantially larger evaluation sets.

Critical strengths
  1. A potentially useful research direction.

Low-cost LLM evaluation and model selection are legitimate research problems. When choosing among numerous candidate models, estimating model capabilities and behavioral properties from a relatively small number of evaluations could reduce computational costs.

  1. Additional information from answer text.

Incorporating complete model responses provides information that binary correctness indicators cannot directly capture, including response length, linguistic style, hedging, and refusal behavior. The experiments demonstrate that answer-text representations can improve predictions for several behavioral and stylistic properties.

  1. Broad experimental coverage.

The paper compares BULLPEN against several existing approaches, including EmbedLLM, LOCUS, JE-IRT, LLMmap, and LLM DNA. It also includes channel ablations, response-swapping controls, sample-efficiency experiments, and held-out-model evaluations. These experiments provide useful empirical observations about the information captured by different representation channels.

Critical weaknesses

W1. Insufficient motivation and unclear practical significance

Although efficient LLM evaluation is a meaningful research direction, the paper does not sufficiently establish why its particular formulation of general-purpose model profiling addresses an important or pressing practical problem.

A substantial portion of the proposed downstream tasks involves predicting properties such as model license, parameter count, release date, context length, download count, and model family. For open-weight models, most of these attributes can be obtained directly from Hugging Face model cards or configuration files.

It is therefore unclear why practitioners would need to execute hundreds of inference queries to predict information that is already directly accessible.

Furthermore, statistical correlations between model outputs and external metadata do not imply that these metadata attributes are intrinsic or uniquely identifiable from model behavior. For example, download counts and licenses can change without any change to the model's parameters or output distribution.

The paper also combines two fundamentally different categories of prediction targets: behavioral or capability-related properties that may genuinely require evaluation, and external metadata that can be retrieved directly.


W2. Insufficient justification for the information content of a few hundred probe questions

A central claim of the paper is that a few hundred probe questions are sufficient to construct informative model profiles. However, the paper does not adequately investigate whether such limited observations can support reliable, multidimensional, fine-grained capability assessment.

For example, two language models may perform almost identically on the selected 400 general-purpose questions while exhibiting substantially different capabilities in a specialized mathematical domain, such as combinatorics, algebra, or geometric proof.

If the selected questions do not capture these differences, no subsequent embedding or dimensionality-reduction method can reliably distinguish the models based solely on those observations.

From an information-theoretic perspective, for a fixed training prior (D), if the model representation (Z) is constructed from observed responses (X), then the data processing inequality gives:

I(Y; Z | D) ≤ I(Y; X | D)

where (Y) is the target model property.

Encoding and dimensionality reduction cannot create information that is absent from the original observations. Predictions about unobserved capabilities must therefore rely on correlations learned from previously evaluated models, and these correlations may not generalize to new architectures or specialized fine-tuned models.

Although 400 questions might be sufficient for estimating broad aggregate performance, the effective number of observations relevant to any particular fine-grained capability may be extremely small.

The paper does not provide a theoretical characterization of this limitation or sufficiently systematic experiments on fine-grained capabilities, domain-specialized models, or models with atypical capability distributions.


W3. Misalignment between probe selection and the proposed profiling objectives

BULLPEN selects probe questions using Fisher information derived from a two-parameter Item Response Theory (IRT) model.

This procedure primarily identifies questions that discriminate between models with different levels of general ability.

However, distinguishing generally strong and weak models is fundamentally different from identifying specialized capability profiles.

For instance, questions that effectively separate strong and weak mathematical models may not distinguish models with different strengths in algebra, combinatorics, or geometry.

More importantly, questions selected using correctness-based discrimination are not necessarily informative about stylistic preferences, refusal behavior, safety properties, or model provenance.

The paper does not sufficiently justify why a single probe set optimized primarily for general ability discrimination should be appropriate for all 46 downstream tasks.


W4. Potential overinterpretation of sample-efficiency results

The paper emphasizes that BULLPEN retains approximately 91% of its full-pool performance using only 400 questions instead of 24,118.

However, this number represents the ratio between the predictive skill obtained using 400 questions and that obtained using the full question pool. It does not represent 91% prediction accuracy, reliability, or confidence in the resulting model profile.

If the full-pool representation performs poorly on a particular target, retaining 91% of that performance does not establish practical usefulness.

Furthermore, the comparison between 400 and 24,118 questions is primarily conducted on the 104 models that completed the full question set, using a less restrictive evaluation protocol in which the encoder has access to all evaluated models and only the prediction heads are evaluated out of fold.

In contrast, the main strict evaluation involving all 260 models primarily uses the shared 900-question pool.

These differences in evaluation conditions are important and should not be overlooked when interpreting the sample-efficiency claims.


W5. Limited methodological novelty

Several central ideas explored in this paper have already been investigated in previous work.

LLMmap, LLM DNA, and Idiosyncrasies in Large Language Models have examined how model-generated text can reveal model identity, behavioral patterns, or relationships between models.

Similarly, EmbedLLM, LOCUS, and IRT-based approaches have investigated compact model representations derived from limited evaluation data for capability prediction and model selection.

The main methodological contribution of BULLPEN is to combine correctness indicators, question embeddings, and answer embeddings within a unified profiling framework.

Its implementation relies primarily on established techniques, including pretrained BGE encoders, averaging of response embeddings, PCA, PLS-SVD, and ridge or logistic regression.

The strongest-performing answer-only method is essentially the normalized average of pretrained embeddings of model responses.

Methodological simplicity is not inherently problematic. However, when the algorithmic novelty is limited, a paper should establish a particularly important problem or demonstrate substantial practical improvements over existing alternatives.

Questions for authors

Please see weaknesses.

3 comments

We thank the reviewer for a rigorous review that helped sharpen the core details of the paper. In response, we ran 8 of our 24 follow-up analyses, with numbers in the general response's table.

W1 (practical significance). We agree that metadata does not motivate the paper. The camera-ready version will focus the headline on behavior.

  • The practical problem: Profiling a model today means running a separate benchmark for each property. With BULLPEN, a model answers a few hundred questions once, and its embedding encodes its benchmark scores, preference ratings and routing as well as how often it hedges or refuses. Models of near-equal accuracy can differ in hedging, refusal and answer length, and no card or accuracy leaderboard records these. Across 11 such tasks, answer text beats correctness bits (row 2), and the same heads place closed models by behavior (row 17).
  • Metadata: You are right that licenses and downloads change while weights stay fixed, and that practitioners read them from the card. The camera-ready tables will report these ten tasks separately from behavior and capability, as checks rather than use cases. They matter mainly when a card is wrong: among 19 models whose cards name the wrong parent, answers identify the true family for 8, compared with 2 for card text (row 3).

W2 (fine-grained capability). Partly. Beyond overall math accuracy, a few hundred questions carry skill for about half of 13 math sub-domains, and less for unseen organizations (row 14). Models with nearly equal math accuracy can be told apart, as the distance between their correctness bits tracks their sub-domain gap. We agree with your inequality: no encoder can recover what the probe does not observe. Our pooled answer embedding loses most of this signal, so sub-domain skill behaves as part of ability. With whole organizations held out, bits from all 1,181 questions predict 7 of 13 sub-domains, against 11 with random folds, so part of the signal relies on a model from the same organization being in training. Transfer to specialized fine-tunes and atypical models remains untested.

W3 (probe selection). Fisher selection targets ability, as you say, and the gain on style information does not depend on the selector. We added a selector built for style and compared it with Fisher, random and stratified selection at 400 questions. For answer text all four tie, since each answer carries style whatever the question (row 13). Fisher selection also does not help the math sub-domains you mention, keeping 6 of 13 against 8 for random questions (row 14).

W4 (sample efficiency). You are right that 91% was a ratio, so the camera-ready version will lead with absolute scores. We now score each of the 260 models with an encoder that never saw it. Answer text scores .41 from 25 questions and .43 from 900, against .30 and .37 for correctness bits. To your point about weak targets, answer text beats chance on 40 of 43 tasks at 900 questions and on all 40 at 400. The 91% ratio stays as a labelled secondary result.

W5 (novelty). We agree that the encoder is simple. Our contribution lies in the IRT framing for black-box profiling of models, the new tasks that test it, applying this to a large model zoo, and a controlled comparison.

  1. Prior work. EmbedLLM, LOCUS, and JE-IRT estimate ability from correctness. EmbedLLM and LOCUS profile 112 models on 10 benchmarks, tested mostly on competence, routing, and preference. LLMmap (42 models), LLM-DNA, and Idiosyncrasies read answer text but test only identity and lineage. We profile 260 models on 46 tasks. Our framing adds response style as a second latent and measures it from answers. Mean pooling suits this because under the factor model, the average answer embedding carries both latents.
  2. New tasks. Our 20 new tasks span six classes of properties; prior fingerprinting work was tested only on identity and lineage. Nine tasks measure behavior and style, such as hedging, refusal, and how much a rating owes to answer style.
  3. A controlled comparison. We retrained eight prior encoders in every fold on the same models and 900 questions, using the same frozen text encoder wherever they read text. Every encoder faces models it never saw and a random floor of its own width. Audits confirm competitors are not handicapped.
  4. Practical improvement. Under this protocol, mean answer text beats LLMmap, LLM-DNA, LOCUS, JE-IRT, and RouteLLM-MF on behavior, style, and safety (row 10). On the 20 new tasks it beats correctness bits at every width (row 6). On behavior, style and safety, 25 answers beat 900 bits (row 12).

We therefore view the simplicity as a finding: pooling answer embeddings carries style that more complex encoders miss.

In short, the fused code from one set of answers ties the best prior encoder over all 46 tasks (row 9), and answer text adds behavior that prior encoders miss (rows 2, 10).

Your critique made our claims more focused and our evidence stronger.

Thank you for the thoughtful, transparent, and rigorous rebuttal. I particularly appreciate your intellectual honesty in running the additional analyse and candidly reporting where the empirical evidence aligns with the limitations raised. Your planned revisions, especially decoupling the Hugging Face metadata tasks into sanity checks and leading with absolute skill scores under strict model holdouts, substantially improve the clarity and soundness of the paper.

That said, while the paper is now a solid, well-controlled empirical study on black-box model profiling, my core concern regarding methodological novelty (W5) remains. Relying primarily on normalized mean-pooled pretrained embeddings, PLS-SVD, and linear prediction heads—alongside the inherent information-theoretic and cross-organization generalization limits for fine-grained capabilities (W2)—still falls short of the bar for ICLR. If half-points were available, I would place this work at a 2.5 after your strong rebuttal, as it is not quite at the 3 level for ICLR. Consequently, I will maintain my current rating of 2, though I think this is a very solid piece of empirical work that would be a strong fit for another venue with a greater focus on benchmarks and evaluation.

Dear Reviewer UrdQ,

Thank you sincerely for your careful reading of our rebuttal and for your generous reply. We are grateful that you found the additional analyses rigorous, and your suggestions to separate the metadata tasks, to lead with absolute scores on held-out models, and to state the limits on fine-grained capability will all be reflected in the revised paper.

We appreciate your candid view on methodological novelty, and we would like to respectfully offer a different perspective on the role that simplicity has played in this line of work. In the research on model representations that BULLPEN builds on, simple estimators have been the norm. EmbedLLM¹ (ICLR 2025) learns one embedding per model with what its authors describe as matrix factorization, combining each model's vector with a sentence embedding of the question through an element-wise product and a linear classifier trained to predict correctness. LLM DNA² (ICLR 2026) requires no training at all, since it embeds a model's responses to a fixed set of prompts with an off-the-shelf sentence encoder, concatenates these embeddings and applies a random Gaussian projection. In both cases, the value of the work lay in what the representation captures, and we see the contribution of BULLPEN in the same way. These representations describe a model either by its ability, estimated from correctness as in EmbedLLM and the IRT-based encoders³,⁴, or by its lineage, read from its outputs as in LLM DNA. BULLPEN is the first framework to read answer text jointly with correctness, and it models both ability and response style, a second latent that correctness cannot identify. On behavior, style and safety, the answer encoder outperforms five retrained prior encoders, including LLM DNA, and on unseen models, answers to 25 questions predict these properties better than correctness on all 900 questions. As the central question of the paper is what a representation of a language model should contain, we believe the work fits well within ICLR's focus on representation learning.

On fine-grained capability, we fully agree with the principle that no encoder can recover what its probe set does not observe. At the same time, our analysis suggests that a few hundred questions already capture a meaningful share of fine-grained skill. With 400 random questions, BULLPEN's correctness channel predicts about half of 13 math sub-domains, and models with nearly equal math accuracy remain distinguishable, since the distance between their correctness patterns tracks their gap across those sub-domains. This signal weakens for organizations unseen in training, and we will state that limit clearly in the paper.

We hope this perspective is helpful, and we thank you again for your thoughtful engagement across both rounds. Your review has made this a substantially better paper.

References

¹ Zhuang et al. EmbedLLM: Learning Compact Representations of Large Language Models. ICLR 2025 (Spotlight).

² Wu et al. LLM DNA: Tracing Model Evolution via Functional Representations. ICLR 2026 (Oral).

³ Yao et al. A Geometric Lens on LLM Abilities through Joint Embedding Item Response Theory (JE-IRT). TMLR 2026.

⁴ Chen et al. Learning Compact Representations of LLM Abilities via Item Response Theory. arXiv:2510.00844, 2025.

Reviewer_E9wgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that existing model embeddings rely mainly on question text and correctness signals while ignoring model responses. BULLPEN incorporates answer text, uses PLS-SVD to combine multiple channels, and applies IRT to select a compact probe set. The paper also introduces 20 tasks covering model behavior, style, and provenance. Across 260 open-weight models, answer-based embeddings perform strongly on these tasks, and a few hundred questions recover much of the performance obtained from the full question pool.

Critical strengths

The paper extends model-embedding evaluation beyond competence to behavior, style, and provenance. The new tasks and collected responses could be useful resources for future work.

The experiments include model-held-out folds, random-embedding controls, answer-swapping tests, and retrained baselines.

Averaging answer-text embeddings is simple but effective. This provides a useful baseline for future work on model representations.

Critical weaknesses

The tasks are only loosely connected to the stated motivation. Many tasks predict public metadata (e.g., architecture, size, license) or properties that can be measured directly from the responses. The paper does not show when an embedding is preferable to these simpler options. Where prediction is actually needed, such as preference, selection, and routing, the benefit of answer text is small and inconsistent.

The value of the proposed fusion method is unclear. The question channel is a fixed linear function of the correctness bits (Eq. 1). PLS-SVD keeps only the answer-text directions that covary with correctness, which may suppress the very information that motivates adding answers. Consistent with this, the joint encoders do not clearly outperform the best single-channel encoder.

The main comparison is confounded by embedding dimension. The answer-only encoder uses 768 dimensions, versus 32 for the joint encoders and the correctness-only baseline, and a random-embedding floor does not fully control for this difference. At 32 dimensions, its advantage on externally labeled tasks shrinks considerably.

Rapid saturation may only show that an averaged embedding stabilizes quickly, not that answers are more informative than correctness bits. The headline result also relies on the fast protocol, in which the encoder is fit on all models

Questions for authors

In what practical setting does the proposed embedding outperform directly reading model metadata or estimating the target property from the same responses? API-only, missing-metadata, or mislabeled-model settings would be particularly relevant.

Do the main conclusions hold when all embeddings have the same dimensionality? How do the results change after removing template tokens, controlling response length, and excluding reasoning traces?

Under a simple latent-factor model, when should answer text contain useful information beyond correctness? Why is cross-covariance an appropriate fusion objective for preserving this complementary information?

1 comment

We thank the reviewer for a careful and technically sharp review that identified important gaps. The tasks were loosely tied to the motivation, the fusion objective was not justified, and the main comparison mixed embedding widths. We agree with these points and ran 14 of our 24 follow-up analyses in response. The camera-ready version will build the paper around a second latent, response style. Each paragraph below reports what we found, with numbers in the table at the end of the general response.

  1. You noted that many tasks predict public metadata. We agree that metadata alone does not justify the embedding, so the camera-ready version will report these ten tasks separately. On the 11 behavior, style and safety tasks, which no card records, answer text beats correctness bits by a wide margin (row 2). The metadata tasks still show that answers recover facts of record, which matters when a card is wrong. For preference, selection and routing, the gain is small because these targets depend mostly on ability. Answers mainly add signal where models of equal ability differ, such as in how often they hedge or refuse.

  2. You asked when the embedding beats reading metadata or estimating the target from the same responses, and named three settings. We tested each of them and found that answers help most for mislabeled models, for API-only models, and for behavior that no computed statistic captures. For mislabeled models, among 19 whose card names the wrong parent, answers point to the true family for 8, against 2 for the card text (row 3). For API-only models, heads trained on open models rank the hedging and answer length of nine closed models almost perfectly (row 17). We cannot check facts such as their size. In the case of missing metadata, answers recover the family no better than the rest of the card or the bits (.18 vs .15 and .17–.21). Against estimates from the same responses, the embedding wins only on targets that are not computed statistics, such as reward-model scores (rows 4–5).

  3. On why cross-covariance suits fusion, the fused code keeps most of the answer information, but not all of it. At matched width it beats both correctness bits and answer text alone in every seed (row 7). At 32 dimensions it matches the 768-d answer embedding over all tasks (.48 vs .46). Our latent-factor model explains the remainder. Cross-covariance keeps style that correlates with ability, so full answer text still does better on style unrelated to ability, most clearly on refusal (row 8). We also tested adding back the part of the answer text that bits cannot predict. It gains on answer length but loses on math, so the code's overall score does not improve (row 16). We agree that the question channel adds nothing beyond the bits (Eq. 1). We therefore present PLS-SVD as the best compact code and answer text as the fuller measure of style.

  4. You also noted that the main comparison mixes 768 and 32 dimensions. We refit every encoder at 32, 64 and 128 dimensions, each against a random floor of the same width. On the 20 new tasks, answer text beats correctness bits at every width (row 6). As you observed, the gap is small at 32 dimensions, and over all 46 tasks the two tie there. The fused code, in contrast, wins at every width (row 7).

  5. Fast saturation, you suggested, may only reflect an average that settles early, and the headline encoder saw every model. We re-ran the curves with each model scored by an encoder that never saw it. Answers to 25 questions still beat correctness bits from all 900 on behavior, style and safety (row 12). An average that settles early but carries little could not do this, so each answer carries much information about style.

  6. Finally, we ran the controls you asked for. Templates, length and reasoning traces do not explain the advantage of answer text (row 11). Removing templates or reasoning traces barely changes the score. Cutting each answer to the median length keeps most of the margin, and length statistics alone do no better than correctness bits.

We thank the reviewer again. These analyses made our claims more precise and better supported.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response to all reviewers

We sincerely thank all three reviewers for their careful reading and constructive critiques, which prompted 24 follow-up analyses. Profiling a model today means running a separate benchmark for each property. With BULLPEN, a model answers a few hundred questions once, and its embedding then encodes/predicts its benchmark scores, preference ratings+routing as well as other novel behavioral signals. We now explain what answer text adds beyond correctness with a latent-factor model. All numbers are in the table at the end.

Framing: information beyond correctness bits. We describe each model mm by two hidden quantities: its ability θ(m)\theta(m) and its response style s(m)s(m), such as length, hedging, refusal, formatting and lineage. Following item response theory, a correctness bit 1[θ(m)⊤w(q)>c(q)]\mathbf{1}[\theta(m)^\top w(q) > c(q)] depends on ability alone, where w(q)w(q) is the ability question qq tests and c(q)c(q) its difficulty. The embedding of mm's answer to question qq reflects both:

e(m,q)=μ(q)+Λ(q)θ(m)+Γ(q)s(m)+ε.e(m,q) = \mu(q) + \Lambda(q)\theta(m) + \Gamma(q)s(m) + \varepsilon.

Here μ(q)\mu(q) is the answer all models share, Λ(q)\Lambda(q) and Γ(q)\Gamma(q) set how ability and style shift it, and ε\varepsilon is noise. Correctness bits therefore see style only where it correlates with ability, and in our data they predict only 18% of the reliable differences between models' answers (row 1).

How the framing answers the shared concerns

  1. Does answer text add anything beyond correctness? It does on behavior, style and safety, as described by the framing. On these 11 tasks, answer text beats correctness bits (row 2). Statistics such as length are better computed directly, but on other targets the embedding adds to them (rows 4–5).
  2. Does PLS-SVD fusion discard the answer information? It keeps most of it, but not all. At matched width the fused code beats both single channels (row 7). It keeps style that correlates with ability, but full answer text still does better on style, most clearly on refusal (row 8).
  3. Is the gain due to the task suite? No. Tasks and protocol are the same for every encoder. Answer text still beats correctness bits and the fingerprinting encoders LLMmap and LLM-DNA on behavior, style and safety (rows 2, 10).
  4. Is fast saturation just an average that settles early? No. On models the encoder never saw, answer text from 25 questions beats correctness bits from all 900 on behavior, style and safety (row 12). For answer text, which questions are chosen matters little (row 13).

Closed API models. As Reviewers E9wg and TCb6 asked, we placed nine closed Claude and GPT models among the 260 open models without refitting. All nine beat every open model on accuracy, and heads trained only on open models rank their hedging and length almost perfectly (row 17).

Quick takeaway. From one set of answers, the fused code ties the best prior encoder over all 46 tasks (row 9), and answer text beats every prior encoder on behavior, style and safety (row 10).

We sincerely thank the reviewers again. Their questions gave the paper a clearer central idea and more precise claims.

Key numbers. Scores run from 0 (chance) to 1 (perfect), with 95% intervals. "Style tasks" are the 11 behavior, style and safety tasks.

#What we foundValue
1Share of reliable answer differences that bits predict18%
2Text (768-d) minus bits (32-d), style tasks+.34 [.25, .44]
3Wrong-parent cards, true family found by answers vs card text8 vs 2 of 19
4Text minus each label's own defining formula (length, hedging, refusal)−.11, −.12, −.15
5Text minus statistics, six other targets+.20 [.10, .30]; +.08 without Arena
6Text minus bits at 32, 64, 128 dims, 20 new tasks (all tasks, 32 dims)+.065, +.135, +.138 (−.023)
7Fused code minus bits and minus text, 32 dims, all tasks+.102, +.125; 3 of 3 seeds
8Text (768-d) minus fused code, style tasks; refusal+.09 [.03, .15]; .45 vs .27–.28
9Fused code vs JE-IRT, all 46 tasks.50 vs .52, tied
10Text (768d) minus LLMmap (384), LLM-DNA (128), LOCUS, JE-IRT (256), RouteLLM-MF, style task+.25, +.17, +.28, +.20, +.23
11Change without templates or traces; margin kept at median lengthunder .01; 78%
12Unseen models, text at 25 minus bits at 900 questions, style tasks (all tasks)+.27 [.15, .39] (.41 vs .37)
13Style selector minus Fisher, 400 questions, text encoders+.001 [−.012, .013]
14Math sub-domains predicted by bits: 400 random, 400 Fisher, all; held out by organization8, 6, 7 of 13
15Re-identified, text vs bits: temperature 0.7; neutral prompt; concise prompt17 vs 11 of 18; 13 vs 17; 10 vs 12 of 19
16Split code minus fused code: all tasks; length; mathtie at .468; +.07; −.12
17Closed models: accuracy vs best open; rank of hedging and length.86–.93 vs .81; Spearman .97
AuthorsOpenReview ↗

Interactive companion site for the revised results

To help the reviewers and ACs assess our work and its revisions, we have put up an anonymous companion site:

https://wcquesera.github.io/bullpen/

We built it so readers can try the downstream uses we describe. The results shown are empirical demonstrations of efficient model profiling:

  1. a new model is placed in the embedding space from its answers to a small question set, and
  2. its nearest neighbors and predicted properties (e.g., leaderboard score, model family) can then inform model selection.

You can browse the embedding space, place a new model from its answers alone (uploads are processed in the browser), and see its nearest neighbors and predicted properties. Everything on the site comes from the released code and data and is only built on what's within the scope of the paper and the OpenReview responses.

We hope the site is useful to the readers.

Thank you again to the reviewers and ACs for their time and feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.