slop score
A metric that measures the presence of common AI writing habits in text, with lower scores indicating more human-like writing.
Learn
When to use it
Use a slop score when evaluating the human-likeness of text generated by AI models, beyond basic grammar and coherence checks. Slop scores quantify the presence of formulaic AI writing patterns, helping developers refine models for tasks like creative writing or customer service responses.
Quick example
In Qwen3-4B Deslop, developers aim to reduce AI's reliance on predictable writing patterns. They use the slop score to measure progress, with lower scores indicating more human-like text. Qwen3-4B Deslop includes the slop score as a core metric to guide model adjustments toward more natural writing.
prompt → model → slop score → refine model → stop
Ecosystem
Slop scores fit into the model refinement process, alongside other metrics like perplexity and BLEU scores, to assess and improve text quality.
┌─ perplexity ─┐
prompt │ slop score │→ refine model → stop
└─ BLEU score ─┘
Misconceptions
| Misconception | Rebuttal |
|---|---|
| High slop score means better text | Lower scores indicate more human-like writing |
| Slop score replaces all text metrics | It's one of several metrics used |
Trade-offs
- Human-likeness — may overlook creative AI patterns
- Specificity — requires careful calibration for different contexts