acceptodds
Under review as a conference paper at ICLR 2027

Structuring Neural Representations with Geometric Supervision

Abstract

Can mechanistic understanding of language models enable training with targeted supervision of internal representations? We leverage previously uncovered neural geometry to design training objectives that sharpen the representations of task-relevant variables. First, we fine-tune Llama-3.1-8B to answer questions such as “What month is four months after June?” by directly supervising representations of the input month (June = 6) and offset (4), which are encoded with Fourier harmonics in the residual stream. Second, we fine-tune the model to predict line breaks in text wrapped to a fixed width by directly supervising the number of characters since the preceding newline, which is represented by a low-dimensional helical structure. In both experiments, we achieve parity with standard fine-tuning using only internal supervision with no output-level task loss. Moreover, the success of geometric supervision can not be isolated to the targeted representations because patching in those representations is not sufficient to recover the performance gain. Together, these findings show how mechanistic understanding of neural geometry can inform training.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability, Geometry, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_tDAYOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether mechanistic knowledge of a model’s internal representations can be used to improve task performance through direct geometric supervision. Using Llama-3.1-8B, the authors study two tasks: month arithmetic, where relevant variables are represented by Fourier features, and linebreaking, where character count follows a helical structure. In both cases, a LoRA trained to align the internal representations with an idealized geometry improves performance without any output-level task loss.

Critical strengths

The paper presents an interesting idea that uses mechanistic knowledge of internal representations to construct a training objective rather than relying exclusively on output-level supervision. The results on month arithmetic show that this can work well.

Critical weaknesses

My biggest concern about the paper is that the evidence provided supporting the approach is preliminary. In fact, the evaluation is limited to two highly structured tasks on a single model. It is therefore unclear how broadly the approach applies, especially when the relevant variables or their geometric structure are not known in advance. The authors really need to expand their evaluations to include more tasks and especially a wider range of models to fully showcase the potential of this approach and how it stands compared to fthe standard ine-tuning.

Questions for authors
  • Have the authors explored this approach beyond the one Llama model considered?
  • What is the justification for these particular tasks and were there any exploration beyond them?
2 comments
AuthorsOpenReview ↗

Response to Reviewer tDAY

Thank you for your review, we’re glad you thought our work was interesting. Our argument is not that this is a robustly supported method for training models that should be used in place of standard fine-tuning. Instead, we wanted to provide a proof-of-concept for how understanding models could possibly lead to us improving them in future research. We focus on two tasks that were deeply investigated by prior research, and leveraging that prior work to test out our new idea about using neural geometry as a learning signal.

We’ve taken this opportunity to sharpen our title, abstract, and introduction to make it clear what our contributions are and the scope of our experimental work! We have posted an updated version of the paper in response to reviewer feedback, and discuss your specific questions here:


What is the justification for these particular tasks? It is unclear how broadly the approach applies, especially when the relevant variables or their structure is not known in advance.

Thank you for raising this point. The reason we chose these two tasks (months arithmetic, and in-context linebreaking) is because they have been deeply investigated in prior research, and are tasks where prior work provides a detailed account of relevant variables, their geometric structure, and that geometry’s algorithmic role. Scaling our approach to more tasks is very time-intensive, as it relies on detailed mechanistic accounts of causal variables on new tasks—the scope of an entire other research paper. We have updated the introduction and discussion to make it more clear that our method depends on descriptive accounts of neural geometry, and to clarify the focus of our paper and contributions.


Have the authors explored this approach beyond the one Llama model?

We focus on a single model (Llama-3.1-8B), as this is where prior interpretability research has provided a detailed mechanistic account of the month arithmetic task, and show results for the linebreaking task in the same model. Because our method relies on existing, detailed mechanistic descriptions of the geometry of task variables, extending it to other models that may or may not implement the same mechanism is not always straightforward. However, we have run preliminary analyses on two additional models (OLMo-3-7B and Gemma-2-9B) for the linebreaking task, where we find similar helix-like representations of the \texttt{character-count} variable. Applying the same kind of geometric fine-tuning to these models improves over the base model, but does not achieve as low a linebreaking error as standard LoRA fine-tuning.

  • For Gemma-2-9B, the base model achieves a linebreaking error of 20.8 characters, geometry LoRA: 9.4 characters and the task LoRA: 6.1 characters.
  • For OLMo-3-7B, base model achieves a linebreaking error of 28.1 characters, geometry LoRA: 17.0 characters, and task LoRA: 9.3 characters.

As with Llama-3.1-8B, most of the gap is in the low in-context-example regime: in Gemma-2-9B, the geometry LoRA matches the task LoRA once the model has seen about 8 in-context examples (4.5 vs. 4.6 characters), while in OLMo-3-7B a gap remains (10.6 vs. 6.7). These preliminary results report the best configuration from a small hyperparameter sweep for each model.

Reviewer_tDAYOpenReview ↗

Response to Rebuttal

Thank you for the rebuttal. I understand the premise of the authors but it is hard to judge the usefulness of such understanding given that the proof-of-concept is limited. In my opinion, the paper would really benefit from increasing the breadth of the tasks investigated and the family of models involved beyond the preliminary results, which will make it stronger. With that in mind, i retain my current recommendation.

Reviewer_spGPOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies: can known hidden-state geometry can be used to improve the model. The authors run two controlleded tasks on Llama-3.1-8B, using LoRA to push representations towards two ideal geometry structures, no output NLL loss.

Results show that, the better hidden state geometry can improve task result, comparable to normal task lora in month tasks. However directly patch the geometry back into the base model does not improve it, showing that the model supervised by the geometry loss does't only change the specific geometry.

geometry loss results in better geometry and better task results. only patch better geometry doesn't results in better task results. really interesting.

Critical strengths
  1. Interesting idea and solid experiments. Using interpretation methods to improve LLM.
  2. The learned gemotry only cannot improve the task results after patch it back, superising results. The author does good ablation.
Critical weaknesses

Good paper, I have no critical concerns. I am curious about what model really learns with the geometry LoRA. Which part of the parameters changes ( patching the geometry learned by lora does not improve the model) ?

Questions for authors

When patching the geometry, or applying the LoRA, what will the model ability changed on other OOD benchmark? e.g., IFBench, Math, etc.

1 comment
AuthorsOpenReview ↗

Response to Reviewer spGP

Thank you for your positive feedback and thoughtful questions. We’re glad you found our paper interesting and experiments solid. We have posted an updated version of the paper in response to reviewer feedback, and discuss your specific questions here:


What does the model learn with the geometry LoRA if patching the geometry does not improve the model? Where in the model do the parameters change?

Thank you for the question and your curiosity. We have included two new appendix sections with more details about our patching experiment setup and results (Appendix A.4 for the months task, and B.6 for the linebreaking task). In both tasks, we find that patching from the fine-tuned model into the base model within the supervision coordinates does not match fine-tuning performance. However, we find that patching within a low-dimensional subspace that encodes the difference between the base model and the fine-tuned model activations can recover much of the fine-tuning improvement. For the months task, this activation difference appears to be well approximated by the sum\texttt{sum} variable (recovering up to 87.4% of the fine-tuned performance), suggesting that the model has learned to represent the sum\texttt{sum} as a byproduct of learning to construct cleaner Fourier features, and this patching is done at layer 18. For the linebreaking task, the best patching results for the geometry LoRA happen at later layers (layers 14-20 achieve 9.3 while layers 25+ achieve closer to 8.3 linebreaking error), while the original edit layers were 0-5 and the supervision layer was layer 5.


When patching the geometry, or applying the LoRA, what will the model ability changed on other OOD benchmark? e.g., IFBench, Math, etc.

We evaluate the effect of LoRA training on OOD benchmarks for the months task in Appendix A.5 of the paper, and for the linebreaking task in Appendix B.7. We evaluated on GSM8K, MMLU, FineWeb, and WikiText and found that applying the geometry LoRA or task LoRA has a pretty minimal effect on OOD benchmarks (No change on FineWeb, +0.1 pts on MMLU, -0.5 pts on GSM8k). This could be due to the fact that our LoRAs are low rank and edit very few parameters. For the linebreaking task, the geometry LoRA has a larger off-target effect compared to the months task, reducing performance on other benchmarks by a small amount (-2.9 pts on MMLU, -2.7 pts on GSM8k), while the task LoRA has a smaller effect (-0.5 and +0.0 points).

Reviewer_R5eXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether previously identified representational geometry can provide useful training signals for language models. The authors train LoRAs on Llama-3.1-8B using internal objectives that align month-arithmetic representations with Fourier features and character-count representations with a helix, without output-level supervision. This improves both tasks, reaching performance comparable to task supervision on month arithmetic. Interestingly, patching the supervised Fourier coordinates into the base model does not recover the improvement. The contribution is primarily an application of existing mechanistic knowledge to training, rather than a new account of the underlying computation.

Critical strengths
  • A useful demonstration that mechanistic knowledge can inform training objectives (Sections 3.2, 4.2).
  • Substantial months-task ablations, including random-subspace and permuted-target controls (Appendix A.4.1) and alternative fine-tuning methods (Table 5).
  • The gap between successful training and unsuccessful activation patching is interesting (Section 3.3).
  • Favorable parameter efficiency on month arithmetic (Table 5), with limited changes on the evaluated external tasks (Table 4).
Critical weaknesses
  • The helix loss supervises both character count and Fourier coordinates (Section 4.2, Eq. 10). Without a count-only baseline, it is unclear how much the geometric structure itself contributes.
  • The patching results in Section 3.3 leave the source of the improvement unresolved. The supervised coordinates may need to change together with other representations; reverse patching would help clarify their role.
  • Better helix alignment does not imply better linebreaking performance (Figures 4 and 6), and alignment drops outside the training range (Figure 25). The relationship between the geometric metric and behavior needs clarification.
  • The strong PCA ablation result (Figure 5c) does not directly validate the supervised helix frame, whose ablation has little in-range effect (Figure 24).
  • The selection protocol for the sweeps in Appendices A.3–A.4 is unclear, particularly whether held-out-range accuracy was used to choose the final configuration.
Questions for authors
  1. In Section 4.2, can the authors compare the full helix loss with supervision of only the linear character-count coordinate in Eq. 10, using the same LoRA and training budget? This would test whether the Fourier components add value beyond count supervision.
  2. As a reverse test of Section 3.3, what happens when the supervised Fourier coordinates in the fine-tuned model are replaced with their base-model values? Does the gain survive? This would help distinguish whether those coordinates are necessary, despite being insufficient when patched alone into the base model.
  3. Figure 24 shows a large effect from PCA ablation but little in-range effect from helix-frame ablation. How much do these subspaces overlap, and how do the authors interpret this difference when motivating the supervision site in Section 4.2?
  4. Given the alignment/performance mismatch in Figures 4 and 6, do configurations with better helix alignment also tend to have lower linebreaking error? Please distinguish counts within the training range from the extended range in Figure 25.
  5. For the sweeps in Appendices A.3–A.4, what data and metric selected the final configuration? In particular, was accuracy on sums above 24 used to choose it, and was any evaluation range kept independent of model selection?
3 comments
AuthorsOpenReview ↗

Response to Reviewer R5eX (1/2)

Rebuttal for Reviewer R5eX: Thank you for your review, we’re glad you found our paper useful and easy to follow, and we appreciate the interest and careful suggestions. We have posted an updated version of the paper in response to reviewer feedback, and discuss your specific questions here:


Comparing helix with a count-only baseline on the linebreaking task

Thank you for the suggestion. Based on your feedback and that of Reviewer w1Xz, we also consider alternate baselines as training objectives for a LoRA on the linebreaking task: (1) direct regression of the character-count via a linear probe, (2) linear probes with evenly spaced values, (3) supervising only the Fourier coordinates of the helix, without the count coordinate, and (4) using the base model’s per-count centroids as targets. We train each LoRA using the same configuration as the geometry LoRA, only changing the readout or supervision target. In this setting, each of these baselines improves over the base model, with linebreaking errors of 7.9, 8.4, 8.2, and 9.0 characters, respectively, compared to the base model’s 13.5. However, none of them outperform the helix supervision which achieves a linebreaking error of 7.3 characters. The gains from the helix are smaller than in the months task, and supervising just the count variable comes close to the helix performance, though the helix structure contributes small additional gains not captured by these baselines. See Appendix B.3.2 for more details. We also adopted similar baselines for the months task and describe the results in Appendix A.3.3.


What about reverse patching? What is the source of the improvement if patching does not recover performance?

  • Thank you for suggesting the reverse test. We ran this experiment by replacing the supervised coordinates in each fine-tuned model with their base-model values. For the months task, we find this has almost no effect. Replacing the Fourier plane coordinates at layer 18 with the base model's values barely changes accuracy (73.3% to 72.8% for the geometry LoRA, and 71.9% to 71.3% for the task LoRA). For linebreaking, replacing the helix coordinates at layer 5 removes part of the geometry LoRA's improvement in linebreaking error (7.3 to 8.1 characters), while replacing a random 3-dimensional subspace has no effect (7.3). For the task LoRA, the effect is smaller (4.6 to 5.0). We take this as evidence that geometric supervision acts as a training signal that changes the model's computation more broadly, rather than the improvement living only in the supervised coordinates.
  • We have included two new appendix sections with more details about our patching experiment setup and results (Appendix A.4 for the months task, and B.6 for the linebreaking task). In both tasks, we find that patching from the fine-tuned model into the base model within the supervision coordinates does not match fine-tuning performance. However, we find that patching within a low-dimensional subspace that encodes the difference between the base model and the fine-tuned model activations can recover much of the fine-tuning improvement.
  • For the months task, this activation difference appears to be well approximated by the sum\texttt{sum} variable (recovering up to 87.4% of the fine-tuned performance), suggesting that the model has learned to represent the sum\texttt{sum} as a byproduct of learning to construct cleaner Fourier features.

How do you explain the different ablation results for the count representation (PCA vs Helix)? How much do these subspaces overlap?

  • Thank you for the question. We see the PCA ablation results, together with the other results presented in Section 4.1 as motivation that a helix is a reasonable target geometry for supervision in Llama-3.1-8B, as prior work (Gurnee et al.) found a similar helical count representation, but studied a closed-weight model. The PCA subspace and the helix frame we supervise are related but different subspaces (the principal angles between them are 48, 53, and 56 degrees). While it is true that ablating the PCA subspace increases the linebreaking error more than ablating the helix we supervise, we do not find that the PCA subspace is a better readout for geometric supervision.
  • To test this further, we also tried supervising within the PCA subspace, using the same loss and training budget (Appendix B.3.2). Supervising the ideal helix in the PCA subspace does not improve over the base model (13.2 vs. 13.5 characters of error), and supervising the base model's own centroids in the PCA subspace helps (9.6), but less than supervising the helix frame (7.3). We have updated Section 4.1 with language clarifying how the PCA results motivate the helix target, and have tried to adjust the language to better match the evidence provided and sharpen our focus on geometry as a learning signal, rather than over-emphasizing the importance of pca ablation experiment.

AuthorsOpenReview ↗

Response to Reviewer R5eX (2/2)

Do configurations with better helix alignment also tend to have lower linebreaking error?

Overall, we find that helix alignment is not necessarily correlated with having a lower linebreaking error. We do see that the Task LoRA and geometry LoRA are more aligned than the base model and they have better error. However, the geometry LoRA is unsurprisingly more aligned to the helix than the task LoRA, but has worse linebreaking error, so this trend does not hold for every comparison. To answer your question about Figure 25, the training range counts are widths m≤150m \leq 150, and the extended range is m∈[151,300]m \in [151,300]. Across 162 LoRA configurations that we trained, we do find that better helix alignment is correlated with a lower linebreaking error when examining just the training range (Spearman correlation ρ=−0.87\rho=-0.87 for widths m≤150m\leq150), but the correlation is weaker for the extended range (ρ=−0.47\rho=-0.47 for widths m∈[151,300]m\in[151,300]).


How were hyperparameters selected? Was any evaluation range kept independent of model selection?

Thanks for bringing this up, we’ve updated the appendix section A.3 and B.3.1, and clarified language in the main paper related to how we perform hyperparameter selection. Importantly, we use the same held-out evaluation process for both the geometry LoRA and the task LoRA when reporting the best generalization scores, so they are comparable. For the months task, we report in the main paper the LoRA with the best overall score based on their held-out generalization for simplicity, as the main focus of our paper is to show that geometry can be used as a learning signal. However, when using a validation split for hyperparameter selection on the months task (i.e. sums from [25,36]), the geometry LoRA and task LoRA achieve comparable performance (62.8±1.8%62.8\pm1.8\% vs. 62.8±6.5%62.8\pm6.5\%), and when comparing the same layer and rank configuration the geometry LoRA often outperforms the task LoRA in the held-out range. For the linebreaking task, the task and geometry LoRA use the same LoRA configuration by design and are thus comparable. For the geometry LoRA, we select hyperparameters with a validation set of documents similar to, but separate from the holdout set, there are some configurations we find after sweeping that give better test set results than the one we report.

I thank the authors for addressing my questions. The additional baselines resolve my main concern, and I will raise my rating accordingly.

Reviewer_w1XzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper explores whether shaping a model's internal representations can improve task performance without directly training on task outputs. The authors fine-tune Llama-3.1-8B with LoRA, aligning hidden activations to Fourier patterns for month arithmetic and a helix for character counting, while also controlling within-value variation. For month arithmetic, accuracy on sums up to 96 rises from 36.3% to 73.3% after training on sums up to 24, slightly above the task-loss LoRA baseline (71.9%). For line breaking, mean error drops from 10.7 to 7.4 characters, though task supervision does better at 4.5. The gap narrows with more in-context examples. Additional experiments examine what drives these improvements, but patching the learned Fourier coordinates into the base model does not reproduce the month arithmetic gains.

Critical strengths
  • This work is a practical use of mechanistic knowledge for training. On month arithmetic, a rank-1, single-layer geometry LoRA improves accuracy from 36.3% to 73.3% without using a task loss. This also beats the matched task LoRA at 54.3%. The improvements on both arithmetic and line breaking suggest the approach could be useful beyond a single task.

  • The controls help show what matters and where the method falls short. Random-subspace supervision reaches only 28.4% accuracy on months, while shuffled centroid assignments drop to 10.2%. The authors also test different layers, ranks, losses, seeds, and baselines. The negative patching result in 3.3 is particularly useful, as it shows that better representation geometry alone does not fully explain the performance gains.

Critical weaknesses
  • The experiments do not clearly show how much the idealized geometry contributes. The random-subspace and shuffled-assignment controls are useful, but both remove information known to be important. A stronger comparison would use direct regression of the input, offset, or count variables, or empirical centroid targets, with the same readout and LoRA setup. Without these baselines, it is hard to tell whether the gains come from the Fourier/helix structure itself or simply from supervising the right internal variables.

  • The mechanistic claims are stronger than the evidence supports. On line breaking, geometry supervision still trails task supervision (7.4 vs. 4.5 characters), with near parity only after at least five in-context examples. Figure 24 also shows that ablating the supervised helix frame has little effect within the training range, unlike the PCA count subspace. The patching experiment is only reported for months, with no equivalent test for line breaking. These results make it hard to claim that both learned geometries are causally responsible for the improvements.

Questions for authors

These questions could affect my assessment:

  1. Have you compared geometry supervision with direct supervision of the same variables or empirical-centroid targets, using the same LoRA setup and training budget? This would help isolate the benefit of the idealized geometry.

  2. How do you explain the different ablation results for the PCA count subspace and supervised helix frame in Figure 24? Was patching also tested for line breaking, as the abstract seems to suggest?

  3. How were the layers, ranks, periods, and other hyper parameters selected? Were the held-out month ranges and line breaking documents used only for final evaluation? This matters since several sweeps use the same range as the main result.

2 comments
AuthorsOpenReview ↗

Response to Reviewer w1Xz (1/2)

Thank you for your review, we appreciate your interest in our paper and helpful feedback. We have posted an updated version of the paper in response to reviewer feedback, and discuss your specific questions here:

How much does the idealized geometry contribute, compared to just supervising the right variables?

Thank you for the suggestion. For each task, we have now included results of several baselines that change either the readout or supervision target.

  • For the months task, we consider three alternate baselines as training objectives compared to the Fourier idealization we propose: (1) direct regression of variable values (the input and the offset) via linear probes, (2) linear probes with evenly spaced values per variable, and (3) using the base model’s centroids as targets instead of idealized circular coordinates. We find that each of these baselines does improve over the base model, achieving 45.9%, 60.5%, and 61.9% respectively, compared to the base model’s 36.3%. However none of them outperform using the idealized Fourier geometry as the supervision target, which achieves 73.3%. This shows that supervising the right variables can be helpful, but that the ideal structure also contributes additional gains. Details for this set of experiments have been added in Appendix A.3.3.

  • For the linebreaking task, we also consider similar alternate baselines as training objectives for a LoRA: (1) direct regression of the character-count via a linear probe, (2) linear probes with evenly spaced values, (3) supervising only the Fourier coordinates of the helix, without the count coordinate, and (4) using the base model’s per-count centroids as targets. We train each LoRA using the same configuration as the geometry LoRA, only changing the readout or supervision target. In this setting, each of these baselines improves over the base model, with linebreaking errors of 7.9, 8.4, 8.2, and 9.0 characters, respectively, compared to the base model’s 13.5. However, none of them outperform the helix supervision which achieves a linebreaking error of 7.3 characters. The gains from the helix are smaller than in the months task, and supervising just the count variable comes close to the helix performance, though the helix structure contributes small additional gains not captured by these baselines. See Appendix B.3.2 for more details.


Was patching also tested for linebreaking?

Yes, we performed a similar patching experiment for the linebreaking task and have updated the text in section 4.2 to describe the key results, with full experiment details included in Appendix B.6. In this setting, we find that patching from either fine-tuned model into the base model within the supervision coordinates has little effect on improving linebreaking error (dropping from 13.5 to 12.9 characters). However, patching within a low-dimensional subspace (k=64) at a later layer in the model can recover about 68% of the geometry LoRA’s improvement, and 100% of the task LoRA’s improvement. We also have included more details about the months patching result in Appendix A.4.


How do you explain the different ablation results for the count representation (PCA vs Helix)?

Thank you for the question. We see the PCA ablation results, together with the other results presented in Section 4.1 as motivation that a helix is a reasonable target geometry for supervision in Llama-3.1-8B, as prior work (Gurnee et al.) found a similar helical count representation, but studied a closed-weight model. The PCA subspace and the helix frame we supervise are related but different subspaces (the principal angles between them are 48, 53, and 56 degrees). While it is true that ablating the PCA subspace increases the linebreaking error more than ablating the helix we supervise, we do not find that the PCA subspace is a better readout for geometric supervision. To test this further, we also tried supervising within the PCA subspace, using the same loss and training budget (Appendix B.3.2). Supervising the ideal helix in the PCA subspace does not improve over the base model (13.2 vs. 13.5 characters of error), and supervising the base model's own centroids in the PCA subspace helps (9.6), but less than supervising the helix frame (7.3). We have updated Section 4.1 with language clarifying how the PCA results motivate the helix target, and have tried to adjust the language to better match the evidence provided and sharpen our focus on geometry as a learning signal, rather than over-emphasizing the importance of pca ablation experiment.


AuthorsOpenReview ↗

Response to Reviewer w1Xz (2/2)

How were layers, ranks, and other hyperparameters selected? Were held-out month ranges and linebreaking documents used only for final evaluation?

Thanks for bringing this up, we’ve updated the appendix section A.3 and B.3.1, and clarified language in the main paper related to hyperparameter selection. Importantly, we use the same held-out evaluation process for both the geometry LoRA and the task LoRA when reporting the best generalization scores, so they are comparable. For the months task, we do report the LoRA with the best overall score based on the held-out generalization for simplicity in the main paper, as the main focus of our paper is to show that geometry can be used as a learning signal. However, when using a validation split for hyperparameter selection on the months task, the geometry LoRA and task LoRA still achieve comparable performance (62.8±1.8%62.8\pm1.8\% vs. 62.8±6.5%62.8\pm6.5\%). For the linebreaking task, the task and geometry LoRA use the same LoRA configuration by design and are thus comparable. For the geometry LoRA, we select hyperparameters with a validation set, there are some configurations we find after sweeping that give better test set results than the one we report.


Claims are stronger for the linebreaking task than the evidence supports.

We have adjusted the language in the main text to use weaker language than “parity”. We now say the task LoRA and the geometry LoRA “converge to similar performance with more in-context examples”. However, we also emphasize that for this task, the most important ranges of inputs for evaluation are those requiring in-context learning, as line breaking is a fundamentally in-context task.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to All Reviewers

We would like to thank each of the reviewers for their helpful feedback and suggestions. We have made several improvements and additions to the paper based on reviewer comments. We provide a summary of the key changes we have made here, and respond to each individual reviewer in separate replies.

Changes made to the main paper:

  • We have changed the title of the paper to be “Neural Geometry as a Learning Signal”, as well as updated the abstract, introduction, related work, and discussion sections to make the paper’s focus and contributions more clear. (Reviewer tDAY)
  • We have included additional details about the patching experiments in Sections 3.3 and Section 4.2 of the main text, with new accompanying Appendix sections for the months task (A.4), and the linebreaking task (B.6) (Reviewer w1Xz, R5eX, spGP)
  • We have updated the linebreaking dataset to no longer split words in half when inserting linebreaks, and instead split at word boundaries. This adjusted some of the reported linebreaking error numbers slightly in Section 4. (e.g., the base model’s error went from 10.7 to 13.5 characters, the task LoRA achieved an error of 4.6 (up from 4.5) and the geometry LoRA achieved an error of 7.3 (down from 7.4), but the trends are the same.
  • We also corrected an indexing typo, linebreaking supervision is applied at layer 5, not layer 6.
  • We have included additional baselines for alternate supervision targets for both tasks, with details included in Appendix A.3.3 and Appendix B.3.2 (Reviewer w1Xz, R5eX)
  • Some Appendix figure numbers have changed (previous Figures 24 and 25 are now Figures 25 and 26 respectively.) (Reviewer w1Xz, R5eX)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.