String Seed of Thought: Prompting LLMs for Distribution-Faithful and Diverse Generation

Source: https://pub.sakana.ai/ssot/ Authors: Kou Misaki, Takuya Akiba (Sakana AI) Blog date: April 2026 Paper: https://arxiv.org/abs/2510.21150 OpenReview: https://openreview.net/forum?id=luXtbX1lVK Site: Sakana AI (https://sakana.ai/) Capture note: Distill-style JS page; body recovered from HTML (interactive widgets/iframes omitted). arXiv HTML/PDF fetch was not completed in this ingest.

tl;dr

String Seed of Thought (SSoT) is a prompting method that improves LLMs’ ability to follow probabilistic instructions and generate diverse outputs. The model first generates a random string, then manipulates that string to derive the answer. No extra training or external tools; prompt-only.

SSoT instructs the LLM to (1) generate a random string as a seed, then (2) manipulate it to generate an output.

Can LLMs Flip Coins in Their Heads?

LLMs are strong on tasks with a single well-defined answer. This work asks a different question: can an LLM flip a coin in its head?

Prompt: “Flip a fair coin” → output Heads or Tails. A single trial cannot distinguish genuine sampling from always answering the plausible token “Heads”. Run ~1,000 trials with stochastic decoding (nonzero temperature, distinct decoding seeds). Count Heads vs Tails against a fair 50/50.

Result: counts stray far from 500/500. Many frontier LLMs show systematic bias under direct prompting. The same holds for >2 options and skewed probabilities.

Observation: When given a set of options and associated probabilities, frontier LLMs often fail to faithfully sample from the target distribution under direct prompting.

Application sketch: poker / Kuhn Poker. Nash equilibrium requires bluffing at specific probabilities. Failing to sample a mixed strategy leaves the model exploitable.

Goal: solve this entirely inside the LLM, by a small prompt change, without external tools. Proposed method: String Seed of Thought (SSoT).

Method

Add two instructions: (1) generate a random string in-head (no PRNG tool); (2) operate on that string in-head to simulate the stochastic decision.

Even without step-by-step recipes, models often invent a strategy such as: sum ASCII codes of the string, take modulo 2, Heads if 0 else Tails.

Experiments: SSoT substantially reduces output bias across a wide range of LLMs; reasoning models especially approach PRNG quality.

Second use: increase diversity on open-ended tasks (e.g. creative writing) with the same “generate a random string and manipulate it” instruction.

Takeaway: prompt-only SSoT can (1) faithfully sample from a given probability distribution, and (2) generate diverse answers for open-ended tasks.

Task names:

• Probabilistic Instruction Following (PIF) — e.g. fair coin
• Diversity-Aware Generation (DAG) — e.g. write a short fable

Illustrative (not measured) contrast in the blog: fair coin 78/22 biased vs 51/49 with SSoT; fable generations collapse to tortoise-and-hare without SSoT and vary with SSoT.

Simplified prompts (Appendix D.4 of the paper; full prompts in Appendix A)

PIF:

Generate a complex random string between <random_string> and </random_string>, and manipulate this string to guide any stochastic decisions within and tags.

Then, provide your final answer, enclosed within and tags.

DAG:

You must produce exactly one unique and diverse answer. To do this, first generate a complex random string between <random_string> and </random_string>, and manipulate this string to guide any stochastic decisions within and tags.

Then, provide your final answer, enclosed within and tags.

Experimental results (as reported on the blog)

Protocol for n-choice PIF (uniform and biased): 100 trials per set, repeated 10 times for error bars. Metric: Jensen–Shannon divergence (closer to 0 = more faithful).

Findings:

• SSoT substantially reduces distribution bias across a wide range of LLMs and task types.
• On DeepSeek-R1, SSoT approaches PRNG sampling quality.
• Exception: QwQ-32B on unbiased 2-choice — baseline JS × 10^-3 of 2.43 already near PRNG; SSoT slightly worse at 3.39. Paper has a failure analysis.
• Vs other prompting methods on DeepSeek-R1 (high temperature, few-shot, prompt ensembling, sequential sampling), SSoT is better across action-space sizes 2–64, near the PRNG reference.

Rock-Paper-Scissors / mixed strategies

Mixed-strategy Nash for RPS is 1/3 each. Opponents: 10 “black belt” bots from RPS Dojo Kaggle Notebook, 100 games each, score = wins − losses (−100 to +100). Bots see full history; LLM does not.

Three prompts, all aiming at varied play:

• SSoT: sample from Nash mixed strategy via random-string seed.
• Baseline: instructed to use Nash and vary play, but no randomization mechanism.
• Simple: pick winning moves and vary because opponents look for patterns; no Nash.

SSoT scores center near zero (harder to exploit). Baseline still exploitable. Simple is consistently defeated.

Diversity-Aware Generation (NoveltyBench)

Splits: curated and WildChat. k=8 responses per prompt.

Metrics:

• Distinct (1–8): classifier partitions 8 responses into functionally equivalent classes; count of unique classes.
• Utility: sum of reward scores of novel generations only, discounted by user-patience / appearance order (diversity × quality).

SSoT has the highest overall Distinct on both datasets, and matches or exceeds strongest baselines on most curated categories. Utility also rises over baseline; on the curated set a few categories (e.g. Product Recs, Opinions) are still led by Paraphrase or higher-temperature sampling on Utility.

Curated analysis categories: Creativity, Character & Entity Naming, Factual Knowledge, Product & Purchase Recommendations, Random Generation & Selection, Subjective Rankings & Opinions.

Mechanism

The prompt does not specify how to manipulate the string. Reasoning traces show models invent strategies.

PIF:

• Sum-Mod: sum ASCII values, modulo number of choices. Suited to equal probabilities (fair coin).
• Rolling Hash: hash = (hash × 31 + ASCII) mod M; threshold-split the large integer range for biased ratios (e.g. 30/70).

DAG:

• Template-based generation: decompose components (setting, traits, conflict, moral, …); Sum-Mod different string segments to pick candidates. Different strings → different combinations.

Conclusion and limitations

Prompt-only; add a few lines to the system prompt of API calls. PIF effective across the five frontier LLMs tested; DAG diversity gains reported for DeepSeek-R1 on NoveltyBench.

Limitations:

• Relies on the model inventing and executing modulo/hash strategies; weaker on smaller models with limited reasoning.
• For single-answer tasks (math, factual retrieval), SSoT is not effective and can distract.
• CoT-scaling analysis: longer reasoning traces associated with more faithful sampling — extra inference tokens can be traded for PIF fidelity in long-reasoning models.

Paper also covers theoretical analysis, model-size dependence, per-model failure modes (e.g. QwQ-32B 2-choice), and DAG output examples.