RANDOMLLM: HOW LITTLE RANDOMNESS DOES A STRONG LOTTERY TICKET NEED?

Strong-ticket training freezes a network's weights at random values and learns only a mask over them. The standard construction for language models samples an independent random weight for every projection, so the random source holds as many independent scalars as the model has connections. We propose RandomLLM, which instead slices every random weight from a single rank-$r$ random pool and keeps a separate multivalued mask for every projection. We find that the randomness of the random weight has little influence on the strong ticket that mask learning produces: at source rank $1$, two frozen vectors supply $24,576$ independent scalars, about $10^5$ times fewer than one per connection, and the resulting model reaches $16.00$ perplexity against $15.95$ and $38.12$ against $37.21$ mean zero-shot accuracy at 3B, while a controlled three-seed experiment finds no resolvable effect from either sharing the pool or constraining its rank. Moreover, instead of accelerating this paradigm with dedicated hardware; we give a general-purpose GPU implementation that avoids rebuilding the mask at every forward pass, reducing prefill and decode latency by $6.25\times$ and $6.16\times$ and training time by $1.60\times$.