- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Doc2LoRA, a representation framework that addresses the issue of scientific paper embeddings serving search rather than generation, where points created by mixing papers cannot be interpreted or questioned. Specifically, the framework introduces conversational embeddings, where every paper representation expands into a LoRA adapter that turns the base LLM into a model open to natural-language questions about the corresponding idea. It also introduces a small invertible transform trained on citation pairs, serving the purpose of aligning the generation-trained space for retrieval while preserving decodability through its inverse. Evaluations against five baselines on cluster labeling suggest the decoded mixtures shift in step with the mixing weight and remain competitive for search, showing that a single embedding space shares the potential of finding existing papers and generating novel ideas.
Critical strengths
-
Novel definition of conversational embeddings. The paper defines scientific articles as a LoRA component inserted into LLMs that can be interacted with natural-language questions. Arithmetic operations such as averaging a cluster of papers or interpolating between two papers remain well defined, and the resulting points, which correspond to no existing paper, can still be decoded and interrogated. Prior representations do not satisfy this definition, as text encoders support search without decodability, vector-to-text methods reconstruct source text without guarantees for synthetic points, and concept-based vectors restrict interpretation to a predefined vocabulary.
-
Complete ablation and control studies. The experiments are sufficiently designed to rule out alternative explanations in addition to report headline scores. In the interpolation study, the in-context baseline is given both source texts and the exact mixing proportions, an informational advantage the other methods do not have, yet it remains near the midpoint while the proposed embeddings track the mixing weight, and the verbatim copy rate shows that the decoded blends are not assembled by copying the sources. Applying the same invertible transform to dedicated text encoders yields at most +0.009, confirming that the retrieval gain surfaces geometry already present in the Doc-to-LoRA space.
Critical weaknesses
-
Gap between conversational embedding and its contribution towards combinatorial novelty. Averaging and interpolation perform a uniform, untyped blending of whole documents, and the interpolation experiments only measure whether the decoded text tracks the mixing weight and how much wording it copies, which establishes that the blends are legible mixtures of the two sources. Combinatorial novelty, however, is defined by integration [1, 2]: selecting specific notions from the source works and articulating an integration logic that connects them, aligns their structures, and resolves incompatibilities between their assumptions, rather than merely placing their contents side by side [3]. Thus, there remains a gap that explicitly justifies the connection between how the mere arithmetic operations of conversational embeddings contribute to the reasoning in integration, which thereafter leads to combinatorial novelty.
-
Lack of human evaluation in quality judgement. All quality judgements in the paper are delegated to a panel of five LLM judges described by the pairwise win rates behind the cluster-labelling results. Therefore, the lack of human evaluation on the decoded labels or blended abstracts poses a threat to the reliability of the evaluation results.
References:
[1] Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013). Science 342,468-472(2013).DOI:10.1126/science.1240474
[2] Swanson, Don R. Undiscovered public knowledge. The Library Quarterly 56.2 (1986): 103-118.
[3] Shen, Y., de Sousa, D. X., Nascimento, R. M. A., Guo, H., & Zhu, X. (2026). IDRBench: Understanding the capability of large language models on interdisciplinary research. Forty-third International Conference on Machine Learning
Questions for authors
I would consider raising my score if the authors provide additional evidence to the weakness addressed above.