acceptodds
Under review as a conference paper at ICLR 2027

LyriCo: Controllable Chinese Lyric Generation with Large Language Models

Abstract

Controllable Chinese lyric generation requires models to satisfy both content requirements and fine-grained structural constraints, including section layout, line count, line length, word-group lengths, and rhyme. We present LyriCo, a structured-text framework that encodes structural specifications as control tokens and fine-tunes decoder-only language models on annotated lyrics. Its data pipeline combines classifier-based genre annotation, theme labels extracted by a large language model, text-to-speech (TTS)-inspired Chinese word segmentation, and section annotations derived from music structure analysis and lyric alignment. We introduce two lightweight mechanisms to strengthen structural control: increased loss weights on control tokens and a line-local attention bias toward relevant control tokens. On a benchmark of 500 briefs spanning Pop and gufeng, ablation studies across seven configurations at the 1.5B parameter scale evaluate the contributions of both mechanisms. Compared with the standard supervised fine-tuning baseline, the combined configuration improves line-count accuracy from 93.40% to 96.80%, section accuracy from 95.19% to 98.80%, and word-format accuracy from 97.96% to 98.52%. With a 14B backbone, LyriCo achieves higher line-length, rhyme, and word-format accuracy than prompted GPT-5.5 and DeepSeek-V4-Pro on the same benchmark, while tying with GPT-5.5 for the highest line-count accuracy. These results show that emphasizing explicit control signals can improve compliance with structural constraints in Chinese lyric generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Lyric generation, Large language models, Structural constraints, Supervised fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_KjaxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LyriCo frames controllable Chinese lyric generation as supervised fine-tuning of decoder-only LMs on lyrics interleaved with control tokens. The tokens encode section layout, line count, per-line character count, per-line word-group (prosodic-phrase) lengths and rhyme class; theme and genre are given as text. Training data are ~190k Mandarin songs. They are annotated by a Pop/gufeng classifier, LLM-extracted theme labels, a TTS-prosody-based segmenter for word groups, and audio structure analysis aligned to LRC timestamps. Two mechanisms are added on top of standard SFT: an up-weighted loss on control tokens (λ), and a fixed, position-growing additive attention bias from content tokens to the current line's rhyme, length and word-format tokens (β). Evaluation uses 500 briefs (Pop/gufeng) with per-constraint accuracies. It includes seven 1.5B (λ, β) configurations, 14B and GLM-9B models compared with prompted GPT-5.5 and DeepSeek-V4-Pro, 506 ChipSong cases compared against published SongNet/ChipSong numbers, and a GPT-5.6 judge for theme and genre adherence.

Critical strengths
  1. The paper tackles a practically relevant and unusually fine-grained control problem: section layout, line count, Chinese-character count, internal group lengths, rhyme, genre, and theme in one representation. The 14B model achieves about 99% or better on every reported structural metric and is much more reliable than prompted API models on long word-group constraints. This is useful evidence that supervised adaptation can deliver precise structural compliance.
  2. The experimental scope and metric caveats are stated clearly. The paper distinguishes per-constraint from whole-brief success, acknowledges the near-ceiling content judge, and labels the ChipSong numbers as nonsample-matched published references. The annotation pipeline also reports explicit audit denominators. The TTS-inspired grouping is a promising representation, although singability itself is not validated.
Critical weaknesses
  1. Evidence for the two proposed mechanisms is not strong enough to separate their contribution from run and sampling variation. Standard SFT (L1A0) already reaches 95.19% section accuracy, 97.58% line-length accuracy, and 97.96% word-format accuracy. L3A3 adds 3.40 points in line count, 3.61 in section accuracy, 0.76 in line length, 0.80 in rhyme, and 0.56 in word format.
  • All ablation values appear to come from one training run per configuration and one stochastic sample per brief (temperature 0.9, top-p 0.95).
  • No training seeds, repeated generations, confidence intervals, or paired significance tests are reported.
  • The pattern is non-monotonic: L1A5 is not consistently better than L1A3, and L3A0 exceeds L3A3 on line-count accuracy.
  • The mechanisms are not isolated at 14B or on GLM-9B; those comparisons only show complete fine-tuning versus unfine-tuned backbones. The robust conclusion is that fine-tuning on the structured representation greatly improves compliance. The incremental value of loss weighting and attention bias remains uncertain, which limits methodological novelty.
  1. The evaluation does not establish overall lyric quality, strict joint success, or benchmark/data independence.
  • There is no human or reference-based evaluation of coherence, originality, artistic quality, or singability. The single GPT-5.6 content judge is near ceiling and measures only genre/theme consistency.
  • Appendix A.4 excludes parse failures from sample-level accuracies, scores only overlapping requested/generated lines, and exempts undefined "center sentences" from some checks. The paper does not report strict all-constraints success per brief with every failure counted.
  • Natural strong alternatives are missing: checker-and-resample or best-of-k prompting, grammar/constrained decoding, a 14B SFT-only ablation, and specialized systems retrained on the same data.
  • The 500-brief benchmark construction and its overlap with training songs, themes, or templates are not described. The ChipSong table uses different test sets, denominators, word-format aggregation, and rhyme classes, so bold cross-row maxima are not a controlled comparison.
  • The source, rights, and license terms for roughly 250k songs with lyrics, audio, and LRC data are unspecified. This affects reproducibility and raises a copyright/terms-of-use concern.
Questions for authors

Clarifications that could change my assessment:

  1. Can you repeat the Table 4 ablations across training seeds and repeated generations, report paired uncertainty, and test L3A3 versus L1A0? Does the configuration ordering persist?
  2. Please report strict per-brief success with parse failures and missing/extra lines counted as failures. What are "center sentences," and how many units does each system exclude from each metric?
  3. What do a 14B SFT-only model and checker-and-resample or constrained-decoding baselines achieve under matched compute and inputs?
  4. How were the 500 briefs constructed, and what overlap exists with training songs, themes, artists, or structural templates?
  5. What are the sources and license or permission terms for the lyrics, audio, and LRC files? What data, code, segmenter, prompts, outputs, and checkpoints will be released? Future-work suggestion: add blinded human evaluation of lyric quality and singability, and compare TTS-inspired grouping with length-matched or lexical alternatives.
3 comments

Thank you for identifying the evaluation details needed to assess the contribution. We re-scored existing outputs under a strict whole-lyric criterion, added paired uncertainty estimates, and report the ablations separately from the human comparison. The principal result: the complete configuration improves strict compliance over ordinary SFT by 5.6 percentage points on the same 500 briefs.

1. Strict whole-lyric compliance and matched ablations

L1A0 denotes ordinary SFT, L3A0 weighted loss only, and L3A3 weighted loss plus attention. We compare existing 1.5B outputs on all 500 briefs at checkpoint 6200 (the last commonly saved checkpoint, i.e., the same training progress) with repetition penalty 1.15.

A successful lyric must be completely parseable and satisfy the requested total line count, complete section-type sequence and section lengths, every line's Chinese-character count and rhyme class, and all word-group lengths. Missing or extra lines/groups, malformed output, and unclosed sections fail. Theme and perceived genre are separate content-quality dimensions.

ConfigurationStrict successesRate
L1A0 (ordinary SFT)130/50026.0%
L3A0 (weighted loss)170/50034.0%
L3A3 (weighted loss + attention)158/50031.6%

With 10,000 paired bootstrap resamples stratified by Pop/gufeng, weighted loss improves strict compliance over SFT by 8.0 points (95% interval: +2.8 to +13.4), and the complete configuration by 5.6 points (+0.6 to +10.6). The L3A3–L3A0 difference is −2.4 points ([−7.8, +3.2]). These intervals quantify variation across briefs for the saved models; no additional training or decoding was performed.

Adding attention to weighted loss raises all five per-constraint metrics: line count 95.60→96.40, section accuracy 98.22→98.80, line length 97.88→98.16, rhyme 94.78→94.92, and word format 98.10→98.46 (%). Because strict success requires all constraints simultaneously, per-constraint gains need not raise the whole-lyric pass rate.

Failure handling. The original parser extracts recognized content without requiring well-formed markers, hence its 0/500 sample-level errors. A complete-format check finds at least one violation (e.g., mismatched section start/end types) in 95/500 (19.0%), 17/500 (3.4%), and 21/500 (4.2%) outputs; all count as failures above. These measure format validity, not missing lyrics. The original per-metric excluded-unit counts have not been fully reconstructed; the strict evaluation includes every brief.

Center-sentence rule. This rule does not affect the 500-brief benchmark: no prompt supplies a center sentence, so the branch is never activated, and all generated lines are checked against their constraints. Appendix A.4 documents it for an optional LyriCo feature in which a user-supplied sentence must be retained; that requirement then takes priority over conflicting length, rhyme, or grouping requirements for that line. We will clarify this in Appendix A.4.

2. SFT and stronger inference baselines

L1A0 is the ordinary-SFT control with the same 1.5B backbone, representation, and training recipe; changing only the loss weight and attention coefficient isolates the added mechanisms. The 14B and GLM experiments evaluate the complete systems and have a different scope. A 14B SFT-only ablation and matched-compute comparisons with checker-based resampling, best-of-k selection, or constrained decoding are outside the current experiments; we will make the baseline scope explicit and discuss these methods in the revision.

3. Benchmark construction and overlap

The 500 briefs are programmatically generated with random sampling and coverage rules; the generator does not read training lyrics, model outputs, or scores. The set contains 250 Pop and 250 gufeng briefs, 34 themes, total line counts 10–36, 35 section-type/section-length combinations, 16 rhyme classes, and word-group lengths 1–7; complete prompts are unique and identical across L1A0, L3A0, and L3A3. As generated specifications rather than sampled songs, song- or artist-level splitting does not apply. Shared themes or structural patterns may still occur in training; the benchmark evaluates control coverage rather than a dedicated split of unseen combinations.

4. Reproducibility and release plan

We will release the LyriCo model weights, evaluation code, test prompts, and all generated test outputs from LyriCo and the directly evaluated baselines, allowing researchers to recompute the objective metrics and benchmark new systems.

The annotation pipeline uses open-source components for audio-level musical structure analysis and prosodic grouping, and an in-house classifier for Pop/gufeng labels; audio boundaries are mapped to lyric lines via LRC timestamps. We will explain each component's inputs, outputs, and role.

Thanks for the rebuttal that addresses several of my concerns, especially by adding strict whole-lyric results, explaining the benchmark construction, and evaluating forced control tokens.

However, the main concern about the contribution of the proposed mechanisms remains. In particular, L3A3 performs worse than L3A0 on strict whole-lyric success (31.6% vs. 34.0%), and the confidence interval includes zero. The lack of repeated training runs, a matched 14B SFT comparison, and clear information about data licensing also remains.

Therefore, I would keep my score unchanged.

Thank you for your follow-up. For attention’s contribution, our additional results provide complementary evidence on constraint coverage and lyric quality. Strict whole-lyric success treats any violation as a failure; the threshold analysis additionally distinguishes outputs satisfying a high proportion of the requirements, which is relevant to practical usability.

These comparisons use the matched 1.5B checkpoint (6200), with repetition penalty 1.15. The supplementary exploratory analysis below reports lyric-level pass rates across all 500 briefs. Each threshold requires correct overall structure and the specified proportion of lines to satisfy character-count, rhyme, and word-group constraints simultaneously:

Configuration≥90% compliant lines≥85% compliant lines≥80% compliant lines
Weighted loss only79.0%86.2%91.4%
Weighted loss + attention79.0%88.4%91.6%

Adding attention yields 11 additional lyrics meeting the 85% threshold, alongside higher observed scores on all five individual constraint metrics.

The LLM-as-judge assessment on 100 shared briefs also reports higher mean ratings across all four reported dimensions:

ConfigurationFluencyTheme relevanceStyleExpression
Weighted loss only2.703.913.662.46
Weighted loss + attention2.743.963.742.53

These results characterize attention’s observed effects on individual constraints, high-compliance outputs, and quality ratings. We will make this specific contribution explicit in the revised paper.

Reviewer_o3iSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes LyriCo, a framework for controllable Chinese lyric generation. It encodes theme, genre, section layout, line count, line length, word group lengths, and rhyme as control tokens, and finetunes LLM on about 190k annotated Mandarin songs. It uses control token loss weighting and a local attention bias. Experiments show improved accuracy.

Critical strengths

The paper formulates Chinese lyric generation as a joint control problem over content, genre, section layout, line count, line length, word group structure, and rhyme. This task setting is more complete than prior lyric generation work and is practically relevant to songwriting and song generation.

Data pipeline is large in scale, and the experiments include ablations, multiple backbones, and external ChipSong cases. The results provide clear evidence that the proposed control methods improve structural accuracy.

Critical weaknesses
  1. Although the task formulation is valuable, the method is mainly a combination of familiar techniques. Control tokens, loss weighting, and attention bias are existing techniques, and their combination does not provide enough conceptual or algorithmic novelty for me to support acceptance at ICLR.

  2. Several central data construction steps are named but not actually explained. The paper refers to “TTS inspired Chinese word segmentation”, “a Chinese predictor”, “an audio based model for music structure analysis”, and “lyric alignment”, but it does not specify what these models are or how they work. Data construction, as a focus of this work, should be described in detail, but the part of data construction is very vague. Also, the paper does not state any plans to release additional artifacts, such as the data construction code or the dataset, which limits reproducibility.

Some minor issues: the authors could improve the organization and clarity of the Introduction and data construction sections so that readers from different backgrounds can understand the problem. The current version is somewhat too terse and tends to list facts rather than provide concrete analysis.

Questions for authors
  1. What specific models are used in the data construction process, including those mentioned in the Weaknesses section? What are the exact steps of the TTS inspired Chinese word segmentation process?

  2. What is the main technical novelty beyond applying known control methods to Chinese lyric generation?

1 comment

Thank you for the assessment and for identifying where the presentation needs greater specificity. Our annotation pipeline combines open-source components with an in-house style classifier; we clarify the contribution, module roles, and release plan below.

1. Contribution relative to existing controllable generation methods

LyriCo builds on control-token representations, weighted objectives, and guided attention to address joint Chinese-lyric control: theme and style together with section layout, line count, per-line character counts, rhyme, and within-line prosodic groups. The contribution is the integrated representation and annotation pipeline, a line-local attention bias, and a controlled study of these mechanisms on decoder-only LLMs. We will make the connections to SongNet and ChipSong explicit in the Introduction and related work.

The 1.5B ablations hold backbone, data, and representation fixed, with ordinary SFT as the control. At the same checkpoint and repetition penalty, the complete configuration improves all five per-constraint metrics over SFT, and adding attention to weighted loss raises all five point estimates. We also report the stricter whole-lyric evaluation.

2. TTS-inspired grouping: model role and output

An open-source TTS component provides Chinese prosodic boundaries (prosodic words, prosodic phrases, intonation phrases), and the resulting contiguous character spans become the word-format annotations. These units need not be lexical words; their lengths and positions are what the generator controls. Table 1 documents the difference from Jieba: 87.12% of our groups contain three to six characters. The boundary audit shows exact agreement on 1,592/1,658 lines.

3. Style classification and theme annotation

The in-house style classifier assigns Pop or gufeng labels. Appendix A.1 describes its balanced seed set of 10,000 manually labeled songs (5,000 per style) and confidence threshold of 0.8. Theme annotation uses an LLM to describe situations, imagery, and emotions, derive a concise theme, and verify it against the complete lyric in a second pass. We will add the classifier architecture and training settings, and the theme-labeling model and prompts.

4. Musical structure and lyric alignment

An open-source music-structure model provides audio-level section boundaries and labels. Separately, boundaries are mapped to LRC line timestamps; a line crossing a boundary is assigned to the section covering most of its duration, and rule-based cleanup adjusts misalignments. The resulting section labels and line counts become training controls. We will state the cleanup rules and separate the pretrained model's outputs from our alignment and filtering steps.

5. Availability and presentation

We will release the LyriCo model weights, evaluation code, test prompts, and all generated test outputs from LyriCo and the directly evaluated baselines, allowing researchers to recompute the objective metrics and benchmark new systems.

We will revise the Introduction to clarify the joint-control task, the structured representation, the roles of the two control mechanisms, and the distinction from prior methods. We will reorganize the data and annotation section around each stage's inputs, processing, and outputs, with a worked example tracing a song to its final training representation.

Reviewer_gpPPOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Chinese lyric generation with content and structural constraints (LyriCo). Paper presents data producing pipeline with extracting genre annotation, theme labels, Chinese word segmentation and section annotations. The paper also proposes control token loss weighting and line-local attention bias mechanisms for structured lyrics training. The results show improved indvidual contraint accuracy compared unfine-tuned baseline.

Critical strengths
  • Data annotation pipeline is practical for model training. This enables model to be trained with structured lyrics with control tokens for section, line-count, word-group, and rhyme controls. This is a useful system-level contribution for Chinese lyric generation.
  • The LyriCo definitely improves the unfinetuned baseline models, and the 1.5B model ablations further shows the effect of the newly introduced mechanisms, by keeping the backbone, data representation, and training recipe fixed. The combined method improves accuracy compared to simple learning of structured output format (supervised fine-tuning, SFT).
Critical weaknesses

I think the novelty is limited and the additional value of attention bias is unclear. As the paper acknowledges, explicit structural control and multi-level length control already appear in prior work. The two proposed mechanisms seem incremental, so their additional benefit needs clear evidence. Though Table 4 shows L3A3 model performs the best for almost all metrics except SenNumA and WA-5, the evaluation does not fully estalish the quality and reliability of the generated lyrics. As control-token loss and line-local attention bias is critical aspect for structural alignment for generated lyrics, they might sacrifice the quality itself of the lyrics. Table 4 doesn't evaluate this, which is ablation study of the core contribution of the paper. The genre/theme evaluation does not measure naturalness or artistic quality.

Questions for authors

Questions

  • What are the parse failure (generation failure) rates?
  • How were the 500 evaluation briefs constructed and separated from the training data? Do they include structural combinations not seen during training?
  • Does calculation of PPL include control tokens or only lyric tokens?
  • Does the model really need to generate the control tokens itself? Since all structural contraints are specified in advance (brief S\mathcal{S}), this fully determines the control-token sequence. So the ground-truth control tokens can simply be insterted at each line boundary during inference. In that case, control-token accuracy would be 100% by construction, and even control-token then be unnecessary? (i.e., λ\lambda can be set to 0).

Suggestions

  • Is there a trade-off between structural accuracy and some other metrics like MA-Dn, MI-Dn, and PPL? It might be helpful if there is trade-off curve adjusting control-token loss weighting λ\lambda and attention bias weight β\beta each.
  • For the reproduction, paper needs to specify the values such as LnormL_{norm} and γmax\gamma_{max}.
7 comments

Thank you for your detailed questions. We clarify the attention mechanism, report the matched ablation results, and describe the evaluation protocols below. L1A0 denotes ordinary SFT, L3A0 weighted loss only, and L3A3 weighted loss plus attention.

1. Contribution and the additional value of attention

LyriCo combines a joint representation of section layouts, line lengths, rhyme, and prosodic word-group lengths with a line-local attention bias toward the relevant controls. Building on established control-token and weighted-supervision methods, our contribution is this formulation for Chinese lyric generation, its annotation pipeline, and the controlled evaluation of the added mechanisms.

At the last commonly saved checkpoint (6200) and repetition penalty 1.15, adding attention to weighted loss improves all five per-constraint metrics (percentages; whole-lyric results in Section 3):

ConfigurationLine countSection accuracyLine lengthRhymeWord format
L3A095.6098.2297.8894.7898.10
L3A396.4098.8098.1694.9298.46

2. Human comparison and LLM-as-judge ablation assessment

Human comparison. Across 100 prompts and five raters (criteria: narrative quality, theme alignment, expression), LyriCo-14B receives 125 wins, 26 ties, and 60 losses against GPT-5.5, and 115 wins, 38 ties, and 52 losses against DeepSeek-V4-Pro. These are judgment counts, not independent prompt counts, for the complete model.

LLM-as-judge ablation. We randomly sampled 100 briefs (50 Pop, 50 gufeng) from the 500-brief benchmark and rated the existing lyrics from L1A0, L3A0, and L3A3 (300 total; checkpoint 6200, penalty 1.15) with GPT-6 Astra (ultra reasoning), anonymized and in randomized order. We report four selected dimensions from this exploratory assessment (mean, 1–5 scale):

ConfigurationFluencyTheme relevanceStyleExpression
Ordinary SFT (L1A0)2.703.963.632.52
Weighted loss only (L3A0)2.703.913.662.46
Weighted loss + attention (L3A3)2.743.963.742.53

Scores are numerically close. The complete configuration has the highest observed fluency, style, and expression (+0.04, +0.11, +0.01 over SFT) and matches SFT on theme relevance.

3. Parsing errors and output-format validity

The original parser extracts lyrics despite unmatched lines or mismatched section markers, and reports 0/500 extraction errors for each configuration. Under a complete-format check, 95/500 (19.0%), 17/500 (3.4%), and 21/500 (4.2%) outputs from L1A0, L3A0, and L3A3 contain at least one format violation, mostly mismatched section start/end types (90, 15, and 17 outputs; may overlap with other issues). These are format violations, not cases in which no lyric was generated.

Strict whole-lyric compliance additionally requires every mechanical constraint to hold. With all 500 briefs included and format-invalid outputs counted as failures, strict success rates are 26.0%, 34.0%, and 31.6%. We will state these definitions explicitly.

4. Randomized construction and shared test requirements

The 500 briefs are generated programmatically with randomized constraint sampling and coverage-balancing rules (cycling through a theme pool, both genres, section templates, and line-count targets; sampling line lengths, rhyme classes, and groupings under validity constraints). The generator does not read training lyrics, model outputs, or scores. The set contains 250 Pop and 250 gufeng briefs, 34 themes, 10–36 total lines, 35 section-type/section-length pairs, 16 rhyme classes, and word-group lengths 1–7, with no duplicate prompts. L1A0, L3A0, and L3A3 receive exactly the same 500 prompts and requirements. The benchmark evaluates control across these combinations; randomized construction alone does not establish that every combination is absent from training.

5. What PPL measures

PPL is computed on the lyric text of 100 reference songs excluded from training, scoring only lyric-body tokens (control markers and special tokens excluded). We will make the preprocessing, scored-token selection, and aggregation explicit. PPL complements the human and LLM-as-judge quality assessments.

6. Inserting known controls and setting λ=0

Known control tokens can be forced at designated positions, e.g., with a logits processor. In our evaluated configuration, LyriCo instead learns to predict them jointly with lyric text through weighted supervision and generates them autoregressively, with the attention bias providing guidance. Either way, correct labels alone do not guarantee that the text satisfies the specified lengths, rhymes, or groupings, so our evaluation checks the generated text itself.

A dynamic forcing controller may add runtime overhead (position tracking, per-step enforcement) and integration work for batched or accelerated inference, depending on implementation; our approach does not require it. λ=0 with forced insertion is a separate variant not evaluated in the current ablation.

7. Parameter trade-offs and missing constants

Diversity scores of the seven configurations at checkpoint 6200 and penalty 1.15 (percentages):

ConfigurationMA-D1MA-D2MI-D1MI-D2
L1A080.7995.092.1234.69
L2A080.8795.152.1134.48
L3A081.0495.072.1134.38
L1A179.4493.932.0934.52
L1A380.0594.352.1034.13
L1A579.6494.262.1034.20
L3A380.8995.122.1234.51

Lexical diversity is similar across settings. We will report these alongside constraint accuracies; comparable PPL for the full grid at this checkpoint is not available. The attention constants are L_norm=25 and γ_max=1.5 in training and generation; we will add them to the specification.

Thank you for the detailed response and the additional results. I also appreciate MA/MI-Dn results. The LLM-as-judge ablation also sugests taht the two mechanisms do not noticeably hurt lyric quality.

  1. I still have concerns about the benefit of the attention bias. When attention bias is added to weighted loss (L3A0 to L3A3), each per-constraint metric improves by less than 1 point, where in stric whole-lyric result, the success rate decreases from 34.0% to 31.6%. So I still think the additional benefit of attention bias is not clear. I think the claim about attention bias should be weaker in the paper.

  2. You agreed that control tokens can be forced, but there is no experiment for this. In your response, format violations happen in 19% of ordinary SFT outputs, and these are error of control tokens themselves. If control tokens are inserted from the brief, these errors become zero by construction. Line count and section accuracy have the largest gains in Table 4, and they also mostly depend on the control tokens. So I think "ordinary SFT + forced control tokens" is a necessary baseline. I think it doesn't need new training, it only needs decoding with the existing L1A0 checkpoint.

Reviewer_a93ZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LyriCo, a system for generating Chinese lyrics under constraints such as song structure, line length, phrase length, and rhyme. The authors construct an annotated lyric dataset and propose control-token loss weighting and a line-local attention bias to improve constraint adherence. Their experiments on 500 briefs show the improvement of the proposed method regarding line-count accuracy, section accuracy and word-format accuracy compared to baseline methods.

Critical strengths
  1. The paper addresses a practical problem and considers several important lyric constraints together.
  2. The authors provide a substantial annotation pipeline and evaluate the system on different model backbones. The ablation study provides useful evidence that emphasizing control tokens can improve some structural metrics.
Critical weaknesses
  1. The novelty of the two mechanisms is limited. Weighted loss for special tokens and guided attention have been studied previously, for example in Octopus v2 and ChipSong. The paper should better distinguish its specific contribution from these approaches.
  2. The 14B results lack a comparison with the same backbone using standard fine-tuning. In addition, the ChipSong reference scores use different test cases and evaluation definitions, making direct comparison difficult.
  3. The paper does not report a human evaluation of generated lyric quality or naturalness. A subjective evaluation is desirable to measure the quality.
Questions for authors
  1. Can the authors compare the 14B model with standard fine-tuning and report variability across runs?
  2. Have the authors evaluated the fluency and naturalness of the generated lyrics with human judges?
3 comments

Thank you for recognizing the practical value of joint lyric control. Below, we clarify the contribution and provide matched SFT comparisons and human quality results.

1. Contribution relative to Octopus v2 and ChipSong

Our contribution is a framework for jointly controlling Chinese lyrics at several levels: theme and style, section layout and line count, per-line character counts, rhyme, and within-line prosodic group lengths. The representation and annotation pipeline make these requirements available together during training and generation.

LyriCo builds on prior special-token weighting and guided attention, including Octopus v2 and ChipSong. Its line-local attention bias increases with position within the current line and resets at line boundaries, directing attention toward the relevant control tokens. Our contribution lies in this task-specific formulation, its integration with the joint-control representation and annotation pipeline, and its evaluation against ordinary SFT with the backbone, training data, and structured representation held fixed. We will make these connections and distinctions explicit in the revision.

2. Ordinary SFT and variability

L1A0 denotes ordinary SFT, L3A0 weighted loss only, and L3A3 weighted loss plus attention, all with the same 1.5B backbone, training data, and structured representation. We compare existing outputs from checkpoint 6200 (the last commonly saved checkpoint) with repetition penalty 1.15:

ConfigurationLine countSection accuracyLine lengthRhymeWord format
L1A093.4095.1997.5894.0397.84
L3A095.6098.2297.8894.7898.10
L3A396.4098.8098.1694.9298.46

Values are percentages under the original per-constraint evaluator. The complete configuration improves all five metrics over ordinary SFT, and adding attention to weighted loss improves each of them at this checkpoint.

Under a strict whole-lyric criterion (complete parsing and all mechanical constraints satisfied simultaneously) on all 500 outputs, the complete configuration achieves 31.6% vs. 26.0% for ordinary SFT, a 5.6-point gain (paired, genre-stratified bootstrap 95% interval: [+0.6, +10.6]).

The matched ablations use the 1.5B backbone; the 14B and GLM experiments evaluate the complete models. A matched 14B ordinary-SFT ablation and repeated-training results are not included in the current experiments.

3. Human assessment of lyric quality

Yes. Our human preference study uses 100 prompts and five evaluators in anonymous pairwise comparisons (narrative quality, theme alignment, expressive quality; ties allowed). Results for LyriCo-14B:

ComparisonLyriCo winsTiesLyriCo lossesValid judgments
versus GPT-5.5125 (59.2%)26 (12.3%)60 (28.4%)211
versus DeepSeek-V4-Pro115 (56.1%)38 (18.5%)52 (25.4%)205

LyriCo is preferred more often than either prompted baseline. Counts are individual judgments, with multiple evaluators per prompt. We will include the protocol and pair-specific counts in the revision.

4. Interpreting the ChipSong comparison

Our evaluation on 506 released ChipSong cases measures LyriCo's control on external templates. As described in Appendix B.4, the published SongNet/ChipSong scores use different evaluation sets and scoring definitions; we will label them as literature references and present them separately from our measured results.

Thank you for the clarification and additional experiments. I now better appreciate how the joint representation and annotation pipeline enable the model to generate Mandarin lyrics conditioned jointly on the requested theme, song structure, rhyme scheme, and within-line phrasing. It's encouraging to see the improvement in strict whole-lyric compliance from 26.0% with ordinary SFT to 31.6% with the full 1.5B model. Human evaluators also favored the 14B model over both prompted baselines. However, it remains unclear how much the attention bias adds beyond loss weighting alone. On balance, I am raising my rating from 2 to 3 (weak accept).

Thank you for your thoughtful reassessment and for recognizing the value of our joint-control formulation and annotation pipeline.

These comparisons use the matched 1.5B checkpoint (6200), with repetition penalty 1.15. On the same 500 briefs, adding attention yields higher observed scores on all five individual constraint metrics.

The supplementary exploratory analysis below reports lyric-level pass rates across all 500 briefs. Each threshold requires correct overall structure and the specified proportion of lines to satisfy character-count, rhyme, and word-group constraints simultaneously:

Configuration≥90% compliant lines≥85% compliant lines≥80% compliant lines
Weighted loss only79.0%86.2%91.4%
Weighted loss + attention79.0%88.4%91.6%

At the 85% threshold, attention increases passing lyrics from 431 to 442. From a practical usability perspective, the degree of constraint satisfaction matters alongside exact compliance: the strict metric alone does not distinguish near-compliant lyrics from outputs with extensive violations.

The LLM-as-judge assessment on 100 shared briefs also gives higher mean ratings across the four reported quality dimensions:

ConfigurationFluencyTheme relevanceStyleExpression
Weighted loss only2.703.913.662.46
Weighted loss + attention2.743.963.742.53

We will focus the attention claim on these specific observed gains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.