acceptodds
Under review as a conference paper at ICLR 2027

Learning Not To Learn: When Evolving Norms Meet Those That Ought to Be Preserved

Abstract

Understanding social norms helps large language models (LLMs) provide culturally appropriate responses that can influence human judgments and values. As norms change, LLMs must revise outdated judgments while retaining knowledge that remains valid. Existing continual learning methods do not explicitly distinguish norms that require revision from those that remain valid, risking outdated judgments or unnecessary forgetting. We propose a continual learning framework guided by normative validity, including 1) a mechanism that distinguishes norms requiring revision from those remaining valid, and 2) an update constraint that protects the latter. We construct a dataset spanning cultures and three temporal stages from the International Social Survey Programme (ISSP) and General Social Survey (GSS). Each scenario includes its country and temporal context, together with a reference judgment and explanation. Comparisons across stages identify stable norms for protection while evolving norms and novel norms guide learning. When these objectives conflict, our update rule adjusts the learning direction to reduce interference. Across three stages, relative improvements in final stage macro F1 reach 13.29% on Qwen and 12.94% on Llama. Improvements in retaining previously learned norms that remain valid exceed those of the strongest baselines by 8.16% on Qwen and 23.02% on Llama. Ablations support the contributions of norm status distinctions and update constraints, while gains on unseen scenarios support generalization. By tying knowledge preservation to continued normative validity, our framework supports adaptation to social change while maintaining cultural continuity.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Continual Learning, Large Language Models, Dynamic Social Norms, Cultural Adaptation, Constrained Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kZXXOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper studies continual learning under evolving social norms, focusing on how LLMs can update outdated normative judgments while preserving knowledge that remains applicable. The authors construct a three-stage dataset from ISSP and GSS survey data, covering different countries, topics, and temporal contexts. They propose a norm-aware continual learning framework that distinguishes stable, evolving, and novel norms. Stable norms provide a constraint on parameter updates, while evolving and novel norms guide adaptation. The resulting update rule is derived using a local constrained optimisation objective and KKT conditions. Experiments on Qwen3-8B and Llama-3.1-8B-Instruct show improvements over several continual learning baselines, with additional evaluations of historical retention, unseen scenarios, and component ablations.

Critical strengths
  1. The paper addresses an interesting and meaningful problem in continual learning. Distinguishing knowledge that should be updated from knowledge that remains valid is particularly relevant when modelling social norms across different cultures and time periods. The use of longitudinal survey data provides a reasonable empirical starting point, and the distinction between stable, evolving, and novel norms gives the learning problem a clear structure.

  2. The proposed framework is straightforward and mathematically well motivated. The constrained update has a clear interpretation, and the KKT derivation is consistent with the stated local optimisation problem. The authors also provide comparisons against several continual learning baselines, results over ten random seeds, component ablations, and evaluations involving historical and unseen scenarios. These experiments provide useful initial evidence that norm-aware memory selection can improve adaptation and retention.

  3. The paper is well presented. The tables, figures, and overall layout are carefully designed and visually consistent, making the paper easy to follow. The distinctive formatting of citations and cross-references also improves readability and navigation. Compared with the other ICLR 2027 submissions I have reviewed, this paper stands out for its careful presentation and formatting.

Critical weaknesses

1. The validity of the constructed normative judgments needs stronger justification.

The paper derives normative judgments from aggregated survey attitudes, using predefined thresholds to assign appropriate, inappropriate, and context-dependent labels. However, an aggregate attitude towards a general statement does not necessarily determine whether a particular individual action is appropriate.

For example, Appendix C.2 and Table 4 describe an Australian couple who voluntarily postpone fertility treatment because they wish to marry before having children. The T3 reference judgment labels their decision inappropriate because contemporary attitudes no longer regard marriage as a requirement for parenthood. However, no longer requiring marriage does not imply that choosing to marry before having children is inappropriate. This raises concerns about whether the mapping from survey attitudes to generated scenarios always preserves the intended meaning of the original normative issue.

A related concern is that an evolving norm does not necessarily involve a change in the correct judgment. Appendix C.3 explicitly presents a norm that is classified as evolving because the strength of public opposition changes, even though the reference judgment remains appropriate across all three stages. Consequently, Evolving Macro-F1 does not necessarily measure whether the model has successfully revised an outdated judgment. The authors should report how many evolving norms actually involve changes in judgment labels and evaluate these cases separately.

Although the authors manually verified 30% of the generated data, the paper does not provide sufficient information about the verification criteria, annotator agreement, or the frequency of corrected or rejected examples. Given that the validity of these labels is central to the proposed learning objective, stronger evidence of data quality is necessary.

2. The technical contribution needs stronger comparisons against norm-aware continual learning baselines.

The proposed update rule is mathematically sound, but its gradient projection is closely related to existing methods such as GEM and A-GEM. In particular, when the permitted stable-loss increase is zero, the update has essentially the same projection form as A-GEM using a stable-norm reference gradient.

The main distinction therefore appears to be the selection of protected examples according to their continued normative validity. This is a reasonable idea, but the experiments do not establish whether the reported improvements require the proposed framework or can be obtained by applying the same norm-status information to existing methods.

For example, GEM, A-GEM, and Replay could be evaluated using only previously learned norms identified as stable. Such comparisons would help isolate the contribution of norm-aware memory selection from that of the gradient constraint. Without them, it is difficult to assess the extent of the technical advance over existing continual learning approaches.

Recent work on constrained continual learning and selective knowledge preservation is also relevant [1]–[3]. In particular, GEC [1] uses explicit epsilon constraints and gradient-guided updates, making it an important comparison for assessing the novelty of the proposed optimisation method.

3. The reported improvements are relatively small against the strongest baseline, and their statistical reliability is unclear.

The abstract highlights relative improvements of 13.29% on Qwen and 12.94% on Llama in final-stage Macro-F1. These figures are consistent with comparisons against Sequential LoRA. However, InsCL is the strongest baseline in Table 2, and the improvements over InsCL are considerably smaller: only 1.10 percentage points on Qwen and 1.22 percentage points on Llama for Current Macro-F1. The corresponding relative improvements are approximately 1.34% and 1.54%.

These differences are small compared with the reported variation across ten random seeds. For example, Qwen achieves 83.29 ± 2.81 with the proposed method, compared with 82.19 ± 1.01 for InsCL. The Stable-BWT improvements over InsCL are also only 0.39 and 0.61 percentage points for Qwen and Llama, respectively.

The paper reports means and standard deviations but provides no paired significance tests or confidence intervals for these comparisons. The results may still represent genuine improvements, but the current evidence does not establish their statistical reliability. The abstract should also make the comparison baseline clear when reporting large relative gains.

4. The evaluation does not sufficiently establish generalisation beyond the constructed scenarios and predefined norm states.

The main continual learning experiments use only two 8B models and three temporal stages. Although the appendix evaluates 46 models, these evaluations examine their existing normative judgment capabilities and do not demonstrate that the proposed continual learning method generalises across those models.

The unseen-scenario evaluation is also limited. Appendix C.1 clarifies that unseen scenarios may share the same countries and normative issues as the training data. Therefore, the results demonstrate transfer to new scenario descriptions within previously represented normative contexts, but do not establish generalisation to entirely unseen countries, topics, or temporal stages. The improvement over InsCL on the complete unseen set is also only 0.61 percentage points.

Furthermore, the method assumes that norm status is supplied in advance through survey comparisons. This assumption is explicitly acknowledged, but the framework's effectiveness when norm states are uncertain or incorrectly assigned remains untested. Since identifying what should be preserved is central to the method, sensitivity to these assignments is an important missing evaluation.

5. Important implementation details are missing, limiting reproducibility.

The paper describes the mathematical update rule, training sequence, evaluated models, and random seeds. However, it does not clearly report several essential training settings, including the actual constraint tolerance, learning rate, LoRA configuration, batch size, number of training epochs, and optimiser settings.

These choices can materially affect the gradient updates, the frequency of constraint violations, and the balance between adaptation and retention. The paper should also clarify how stable memory examples are selected and sampled during training, particularly when a previously stable norm becomes evolving at a later stage.

Without these details, it is difficult to reproduce the reported results.

6. The related work is incomplete, making the novelty of the paper difficult to assess.

The Related Work section overlooks several important studies published in 2025 and 2026. Sharma et al. [4] proposed TRACE (ICML 2026), which distinguishes preference information that should be inverted, suppressed, or retained as alignment requirements change. This is closely related to the central idea of the paper and deserves a direct comparison.

Other relevant studies include CALM (NeurIPS 2025) [5], CultureRL (AAAI 2026) [6], CultureManager (ACL 2026) [7], and COCOA (EMNLP 2025) [8], which investigate different aspects of dynamic cultural adaptation, cultural learning, and knowledge interference. The paper also omits recent work on survey-based cultural modelling [9], [10], gradient-constrained continual learning [1], and temporal or culturally grounded evaluation [11]–[15].

These studies do not necessarily use the same methods, but they are important for understanding the paper's contributions. The authors should discuss them and clarify what their approach adds to existing research.

7. The paper does not make full use of the nine-page limit.

The authors fill the remaining space with Discussion and Limitations, while relatively important experimental results and analyses remain in the appendix. In particular, roughly half of page 9 after the Conclusion could be used more effectively. Some key results should be moved into the main paper, while Discussion and Limitations could be moved to the appendix.

References

[1] S. Lai, C. Ma, F. Zhu, Z. Zhao, X. Lin, G. Meng, and Q. Zhang, "Gradient-guided epsilon constraint method for online continual learning," in Advances in Neural Information Processing Systems, vol. 38, 2025, pp. 137640–137669.

[2] L. Yang, Y. Liao, Y. Wang, and Y. Wang, "SLoRA: Balancing plasticity and forgetting in large language models for continual learning," in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2026, pp. 5437–5454.

[3] D. Jung, J. Lee, and H. Lim, "Towards scalable lifelong knowledge editing with selective knowledge suppression," in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2026, pp. 3215–3231.

[4] A. S. Sharma, D. Sanyal, M. Ray, V. Srivastava, S. Karande, and M. Mandal, "The realignment problem: When right becomes wrong in LLMs," in Proceedings of the 43rd International Conference on Machine Learning, vol. 306, 2026, pp. 110587–110617.

[5] L. Shen, X. Cai, Y. Long, I. Razzak, G. Chen, and S. Jameel, "CALM: Culturally self-aware language models," in Advances in Neural Information Processing Systems, vol. 38, 2025, pp. 130850–130883.

[6] W. Zhao, H. Li, Y. Zhao, H. Liu, B. Li, T. Liu, and B. Qin, "CultureRL: Internalizing cultural principles in large language models via norm-driven reinforcement learning," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 40, no. 44, 2026, pp. 38120–38128.

[7] B. Zhang, X. Zhao, J. Li, H. Chen, and Z. Chen, "Mind the gap in cultural alignment: Task-aware culture management for large language models," in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, 2026, pp. 16798–16813.

[8] C. Huang et al., "Governance in motion: Co-evolution of constitutions and AI models for scalable safety," in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 17187–17210.

[9] M. F. Adilazuarda, C. C. Liu, I. Gurevych, and A. F. Aji, "From surveys to narratives: Rethinking cultural value adaptation in LLMs," in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 18052–18079.

[10] J. Wang, Z. Zhao, T. Ni, and Z. Wei, "SocioBench: Modeling human behavior in sociological surveys with large language models," in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 26257–26289.

[11] K. Eriksson et al., "Everyday norms have become more permissive over time and vary across cultures," Communications Psychology, vol. 3, Art. no. 145, 2025.

[12] Y. Daryani, M. Bogen, and M. I. G. Daepp, "Accurate in space, unreliable in time: How LLMs represent national cultural change," arXiv preprint arXiv:2609.01902, 2026.

[13] M. Bhatia, S. Nayak, G. Kamath, M. Mosbach, K. Stańczak, V. Shwartz, and S. Reddy, "Value drifts: Tracing value alignment during LLM post-training," Transactions of the Association for Computational Linguistics, vol. 14, pp. 2033–2056, 2026.

[14] J. Oh, I. Cha, M. Saxon, H. Lim, S. Bhatt, and A. Oh, "Culture is everywhere: A call for intentionally cultural evaluation," in Findings of the Association for Computational Linguistics: EMNLP 2025, 2025, pp. 19156–19168.

[15] K. Eriksson, S. Karlsson, I. Vartanova, and P. Strimling, "Large language models outperform humans at estimating society's everyday norms," Communications AI & Computing, vol. 1, Art. no. 15, 2026.

Questions for authors
  1. How do the authors validate the mapping from aggregated survey attitudes to scenario-level normative judgments? In particular, could they explain the T3 marriage-before-parenthood example in Appendix C.2, and report the results of manual verification, including any label corrections or annotator disagreements?

  2. Among the norms classified as evolving, how many actually involve a change in the reference judgment between temporal stages? Could the authors report separate results for evolving norms with and without judgment-label changes?

  3. Could the authors compare the proposed method against GEM, A-GEM, and Replay using the same stable-norm information available to the proposed framework? This would clarify how much of the improvement comes from norm-aware example selection.

  4. Could the authors report paired statistical comparisons against InsCL across the ten random seeds? Please also clarify that the 13.29% and 12.94% improvements in the abstract are measured against Sequential LoRA, rather than the strongest baseline.

  5. How sensitive are the results to incorrect norm-status assignments and the predefined survey thresholds? Could the authors also provide the missing training hyperparameters and clarify the construction and sampling of stable-norm memory during sequential learning?

  6. How does the proposed framework differ from GEC [1], TRACE [4], CALM [5], CultureRL [6], and CultureManager [7]? What are its main technical contributions beyond these existing approaches?

Reviewer_DnfrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates a specific continual learning problem restricted to social norms. Since these are subject to change over time, the authors argue that LLMs should be updateable in this regard and consequently study how that could be achieved. A new dataset is compiled, comprising social norms at three different points in time and tested on a constrained continual learning method. The method performs comparably to the best baselines.

Critical strengths
  • I like the fundamental problem the paper motivates.
  • The paper is mostly well written and easy to understand.
Critical weaknesses

• Related work section is too selective. E.g. two methods from model editing are cited and compared to their own work, whereas there really are much more approaches available.

• Results are really close at the top and differences are not statistically significant, as far as I can tell.

• 4.1. It is not clear where the known methods end and where your contribution starts. It looks really similar to GEM. At the very least: Where do you deviate and why was that necessary? Currently, it seems the proposed method is rather an application of previous work than a fundamentally new approach.

• The dataset makes impermissible negations of norms. The example in appendix C.2 shows this immediately: First, trying for a baby before being married was frowned upon. Consequently, a couple marrying before having children is deemed appropriate. Somewhat later, that norm no longer holds. Now, the dataset assigns “inappropriate” to a couple that wants to marry first and only then try for children. This assignment is not correct, since the negation of prohibitive behavior does not simply mean that now one ought to do what was previously forbidden. Instead, it is okay to do it -- and just as okay not to do it. In the author’s logic we would get cases were legalizing drugs would require everyone to take them (even worse things can happen in their model: think legalizing abortion or assisted suicide…). For me, this is a deal breaker.

Questions for authors

Minor Comments:

  • It is not clear why the country grouping was chosen and why other groupings where not selected.
  • The shortest timespan in your dataset is 8 years. The current total LLM lifespan seems to be much shorter, so it is quite possible that far fewer norm evolutions occur for a specific model in a realistic setting. This might make the problem easier than what was studied here.
  • Figure 1 d) Unless you empirically show that LLMs would actually answer “maybe” in that case, I remain unconvinced by your claim. Many possible things could happen other than what you depicted.
  • Data from 1994, 2002 and potentially 2022, too, are possibly in the pretraining data of the LLMs. Is that a problem?
  • I am not sure I agree that norms that are superseded should simply be overwritten. Surely there are cases where more nuance would we appropriate?
Reviewer_7fZaOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how LLMs can adapt to evolving social norms while preserving previously learned norms that remain valid. The authors construct a cross-cultural dataset from ISSP and GSS surveys, covering three temporal stages, and categorize social norms as stable, evolving, or novel. Based on these categories, they propose a constrained continual learning framework that updates evolving and novel norms while protecting stable norms through gradient-based parameter updates. The authors evaluate their method on Qwen and Llama models against several continual learning baselines. Experimental results show improvements in adapting to evolving norms, retaining stable norms, and generalizing to unseen scenarios.

Critical strengths
  1. The paper introduces an interesting and meaningful continual learning problem that differs from traditional settings. Instead of simply preserving previously learned knowledge, the model needs to adapt to evolving social norms while retaining knowledge that remains valid.
  2. The authors construct a cross-cultural and temporal dataset based on real social surveys. The dataset provides a useful benchmark for studying how LLMs adapt to changing social norms.
  3. The authors conduct experiments on two LLMs and compare their method with several continual learning baselines. The ablation studies further support the effectiveness of the proposed framework.
Critical weaknesses
  1. The proposed method relies heavily on predefined norm status information. Although the paper aims to help LLMs adapt to evolving social norms, the method assumes that each norm has already been identified as stable, evolving, or novel based on external survey data or some predefined rules. However, such information may not always be available or accurate in real-world applications. The authors should further evaluate how incorrect or uncertain norm status assignments affect the performance of the proposed method.

  2. The current evaluation does not fully demonstrate whether the model can effectively adapt to changing social norms. The authors define evolving norms based on changes in survey attitudes, but these changes do not necessarily lead to different judgment labels. As illustrated in Appendix C, a norm can be classified as evolving even when its judgment remains unchanged across different stages. Therefore, a high Evolving Macro-F1 does not necessarily indicate that the model has successfully updated outdated judgments.

  3. In the proposed method, evolving and novel norms are used for model adaptation, while stable norms form the memory used to constrain parameter updates. However, it is unclear whether the existing continual learning baselines are provided with the same norm-aware data partition. This makes it difficult to determine whether the performance improvements mainly come from selecting which knowledge to learn and preserve, or from the constrained parameter update itself.

  4. The proposed constraint operates on the overall loss of stable norms rather than protecting individual norms or cultural contexts. Improvements in some stable norms may compensate for performance degradation in others, allowing the overall constraint to be satisfied even when certain norms are forgotten. This is particularly important because the paper focuses on preserving social norms across different cultural contexts, rather than general knowledge retention. The authors should further investigate whether the proposed constraint can effectively preserve stable norms across different countries, rather than relying only on aggregated performance metrics.

Questions for authors

Please see Critical Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.