Why getting an LLM to produce readable text requires way more than clever prompts
Here’s a dirty secret about working with large language models: the hard part isn’t crafting the perfect prompt. It’s building the infrastructure around the model to make it behave consistently. I spend my nights wrestling with JSON schemas and muttering about data pipelines, not dreaming up magic incantations.
Through extensive experimentation with these systems, I’ve learned that long-form content generation is fundamentally an engineering challenge wearing a prompt engineering costume. To illustrate these principles, I’ll use creative fiction as a running example but the lessons apply equally to technical documentation, marketing content, research reports, or any substantial text you need an LLM to produce.
What We’re Actually Trying to Solve
Three things matter when generating long-form content:
- Producing text that people want to read
- Building reliable systems around unpredictable models
- Creating workflows that let you refine and improve quickly
These challenges show up everywhere. The principles transfer across domains.
The “Just Ask It” Fantasy
Everyone’s first instinct is straightforward: tell the model what you want.
“Create a mystery novel set in a remote mountain lodge where five old friends reunite, and one of them ends up dead.”
The model obliges. It produces pages of text, complete with a dramatic title. Problem solved, right?
Not even close. Here’s what you actually get:
Clichés everywhere. Every character harbors “dark secrets.” Tension comes from the same recycled thriller tropes. It reads like a hundred other mediocre books mashed together.
Characters who contradict themselves. A doctor speaks with clinical precision in one scene, then gets described as “the group clown who never took anything seriously” two pages later.
Paint-by-numbers plotting. Accusation, denial, explanation, next suspect. The same formula repeats mechanically until the story limps to a conclusion.
I’ve tested this across every major model family. Claude, GPT, Llama, Mistral, and the specific failures vary, but the fundamental problems persist. Better models won’t save us here.
Breaking It Into Pieces
The obvious next step: instead of one massive request, generate the content piece by piece. Loop through chapters. Loop through scenes within chapters.
This improves things marginally. At least you can control length. But the output still meanders without purpose. Events happen, but they don’t build toward anything. Without external structure, the model just… keeps going.
Borrowing from How Writers Actually Work
Writers don’t sit down and produce finished prose from nothing. They plan. They outline. They know where they’re headed before they start writing.
We need to give our system the same advantage.
Mapping the Journey
Before generating any actual content, establish the arc:
- Opening: What situation draws readers in? (Friends gathering at an isolated lodge)
- Escalation: What complications arise? (Murder, accusations, buried history surfacing)
- Resolution: How does it end? (Truth revealed, consequences delivered)
This framework tells the model where each piece fits in the larger whole.
Defining the Cast
Letting the model invent characters spontaneously is a recipe for inconsistency. Instead, establish upfront:
- Who are the central figures?
- What drives each of them?
- What history do they share?
- How do they relate to each other?
Real authors keep character bibles. Your AI system needs the equivalent.
Plotting the Beats
Subdivide your arc into chapters, then subdivide chapters into discrete scenes like individual moments or events that advance the story.
Chapter 1: Arrival
- Scene: Invitation arrives unexpectedly
- Scene: Awkward reunion in the lobby
- Scene: First dinner together, old tensions visible
These aren’t full prose. They’re signposts marking what needs to happen.
Background Details
Some information shapes the content without appearing directly in it. Historical context, setting details, backstory that informs behavior. Document this too.
Architecting the Pipeline
With a planning framework established, implementation becomes clearer. Each planning component becomes its own model interaction:
Step 1: Feed in the premise, get back the three-act structure
Step 2: Feed in the structure, get back character profiles
Step 3: Feed in structure and characters, get back chapter breakdown
Step 4: For each chapter, get scene-by-scene plans
Step 5: For each scene, generate actual prose
Every step produces structured data (JSON works well) that feeds into subsequent steps. This isn’t one prompt but a pipeline where each stage builds on previous outputs.
What the Prompts Look Like
Each pipeline stage needs:
- Context: Everything relevant from previous stages
- Role definition: What kind of task is this?
- Specific instructions: What exactly should this step produce?
- Output structure: How should results be formatted for downstream use?
For scene planning, your output might specify scene number, summary of what happens, target length, key emotional beats. For chapter outlines, you’d capture chapter position, title, major conflicts, and the list of scenes it contains.
Structured output isn’t optional. It’s how information flows through the system.
The Generation Engine
Once planning is complete, prose generation becomes a controlled loop:
For each chapter in the outline:
Extract chapter metadata
For each scene in this chapter:
Gather scene requirements
Include context (what's happened so far)
Generate prose within length constraints
Append to manuscriptThe model writes one manageable piece at a time, always aware of what came before and what it’s building toward.
When the Words Themselves Are the Problem
A working pipeline produces complete manuscripts. But completion doesn’t mean quality. Several text-level issues plague LLM output.
Overwrought Language
Models adore elaborate, florid descriptions:
“A smile touched her lips. Perhaps the most beautifully melancholic expression he had witnessed in all his years.”
“The music spoke through him, transforming emotion into something beyond mere sound.”
Occasionally, this works. When every paragraph drips with purple prose, it becomes exhausting.
The fix: Anchor the writing to a specific author’s voice.
Rather than vague instructions like “write elegantly,” specify: “Write in the style of [established author].” Models have absorbed enormous amounts of text from known writers. Asking them to emulate a specific voice gives them a concrete target rather than an abstract quality.
“Be concise” is interpretable a thousand ways. “Write like Hemingway” points somewhere specific.
The Same Names, Every Time
Left to their own devices, models gravitate toward certain character names obsessively. Eleanor. Marcus. Chen. Whitmore. You’ll see the same dozen names regardless of genre or setting.
More problematically, these names carry implicit characterization from training data. “Eleanor” comes pre-loaded with assumptions about who she is and how she behaves.
The fix: Specify names yourself.
This requires manual input, but providing your own character names disrupts the model’s default patterns. The associated stereotypes don’t attach to names you’ve chosen.
Monotonous Rhythm
Even with style guidance and custom characters, a subtle problem emerges. Sentence structures repeat:
“The storm intensified outside…”
“The old clock chimed midnight…”
“The accusation hung in the air…”
Article-noun-verb, over and over. Once established, the pattern self-reinforces. So, the model sees what it’s written and produces more of the same.
The fix: Inject controlled randomness.
Models can’t generate truly random output. Ask for a random word and you’ll get “serendipity” or “ephemeral” with suspicious frequency.
But we can approximate randomness through prompting. Before generating each scene opening, provide:
- An arbitrary number (seeding variation)
- An emotional state (“approach this feeling contemplative”)
- A persona angle (“write as someone who takes risks”)
- Some unrelated text (breaking pattern recognition)
Also track what sentence structures you’ve already used and explicitly instruct: “You’ve started recent paragraphs these ways. Choose differently.”
The result isn’t random, but it’s unpredictable enough to prevent monotony.
Assembling the Full System
The complete workflow:
- Premise → Story Arc: Model generates three-act structure
- Arc → Characters: Model develops cast (supplemented with your naming choices)
- Arc + Characters → Chapters: Model outlines chapter progression
- Chapters → Scenes: Model plans individual scene beats
- Scene Openings: Model generates varied sentence starters (using randomness injection)
- Full Scenes: Model writes complete prose (using all accumulated context)
Steps 4-6 repeat until the manuscript is complete.
Embracing Iteration
Real writing involves revision. This system should too.
Generate the story arc, review it, adjust what doesn’t work, regenerate. Develop characters, evaluate them, refine. The pipeline supports returning to earlier stages with feedback before proceeding.
A monolithic “write my book” prompt offers no intervention points. This architecture provides many.
Hard-Won Lessons
Structure precedes prose. Don’t let models write substantial text without establishing what that text should accomplish. Planning stages are not optional overhead. They’re essential infrastructure. This mirrors spec-driven development approaches where you define what you’re building before building it.
Data must flow cleanly. Sequential model calls require parseable outputs. When JSON generation fails (and it will), have fallback logic that asks another model call to repair the formatting.
Models have personalities. Their biases toward certain names, phrasings, and patterns aren’t bugs you can prompt away. Design around them with explicit controls.
Context windows mislead. Technical limits on input length rarely matter. Long before hitting those boundaries, models lose coherence in synthesizing information. If you’re worried about context length, the output has probably already derailed.
Evaluation remains human. You can use models to critique drafts like checking for tension, continuity, emotional resonance. These provide useful signals. But determining whether something is genuinely good? That’s still your job.
No model writes “mature” content well. Training data skews toward accessible material. Everything emerges sounding like young adult fiction. Strong style guidance partially compensates, but expect to fight this bias constantly.
Applying These Principles Beyond Fiction
While I’ve used novel writing as an example, these patterns generalize:
Technical documentation: Outline the structure first. Define key concepts and their relationships. Generate section by section with full context of what’s been covered.
Research reports: Establish the argument arc. Define the evidence that supports each point. Generate with awareness of what’s been claimed elsewhere in the document.
Marketing content: Map the customer journey. Define voice and brand guidelines explicitly. Generate pieces that reference the broader campaign context.
Code generation: Spec-driven development follows the same principle; define what you’re building before asking the model to build it.
The core insight remains constant: decompose the problem, maintain structure across generation steps, and compensate for predictable model behaviors.
The Bigger Picture
Getting coherent long-form content from language models isn’t about finding the perfect prompt. It’s about building systems that decompose the problem, maintain structure across many generation steps, and compensate for predictable model behaviors.
The models provide raw capability. The engineering determines whether that capability produces anything worth reading.
None of this makes the creative process mechanical. It just means the human creativity shifts, from writing sentences to designing systems that write sentences well. Different skill, same fundamental challenge: making something people actually want to experience.