Laya for Content Audits: Keep, Update, Merge, Remove

Every content audit ends with the same four-way decision, and every content audit runs into the same wall.

The decision is simple: for each page, do you keep it, update it, merge it into another page, or remove it?

The wall is that you have five thousand pages and no realistic way to make that decision five thousand times by hand. So you audit the top two hundred by traffic, make decisions about those, and leave the rest untouched. Which means the long tail, where the actual bloat lives, never gets reviewed.

This is a routing problem with a fixed answer set, which makes it a good fit for Laya. This article covers the complete workflow, including the two ways this particular task produces confident nonsense.

What you will finish with

A disposition for every page in your inventory, where each row carries:

A disposition: keep, update, merge, remove
A confidence value
A routing decision: auto-accept or human review
For merges, the target page
For updates, the specific reason

Who this is for: an SEO or content lead who needs a full-library audit rather than a top-pages sample.

Prerequisites:

Laya installed and running locally
A crawl of your site with page titles, headings, body text, and ideally traffic data
A written definition of what each disposition means for your site
Fifty pages you have already classified by hand, for threshold calibration

Definition of done: every page has a disposition and a confidence value, nothing below your threshold is treated as final, and you know your agreement rate on the fifty-page sample.

Define the four dispositions before you classify anything

This is the step people skip, and it is the reason their audits produce inconsistent results.

"Update" means different things to different people. To one reviewer it means rewrite the intro. To another it means add a section. To a third it means refresh the statistics. If your reviewers cannot agree on what the labels mean, the model has no chance of learning a consistent rule.

Write the definitions down, with examples.

keep    — the page serves a distinct intent, is accurate, and has no
          significant overlap with another page

update  — the page targets a valid intent but has a specific deficiency:
          outdated facts, missing sections, weak structure, or thin coverage

merge   — the page targets the same intent as another page and should be
          consolidated into the stronger of the two

remove  — the page targets no valid intent, or duplicates another page with
          no unique value, and should be deleted or redirected

Notice that each definition names the reason, not just the action. That matters because the reason is what you will act on. A "merge" without a target is not actionable.

The two failure modes specific to content audits

Before the workflow, the warnings.

Failure mode 1: the model cannot see your site

Laya has no live web access. It cannot check whether two pages are actually competing, cannot see your internal links, and cannot know that the page it is told to merge into is itself scheduled for removal.

Everything it judges has to be in the data you send it. If you send only the page's own text, it will make a reasonable decision about that page in isolation, and that decision may be wrong because it could not see the relationship to the rest of the library.

The fix: include the relevant context in the state. For merge decisions, that means the candidate target pages. For cannibalization, that means the queries the page ranks for. The model judges what you give it, and nothing more.

Failure mode 2: merge is a wide-option question

Deciding whether to merge is a simple judgment. Deciding what to merge into is a choice question with as many options as you have pages, which is exactly the shape Laya handles worst.

Remember the constraint from earlier in this series: choice options share a fixed 256-token budget in the model's output head, and accuracy falls off sharply past roughly twenty options. You cannot ask "which of these five thousand pages should this merge into."

The fix: two-stage it. First decide the disposition. Then, only for the pages marked merge, run a separate candidate-narrowing step to produce a shortlist of five to ten plausible targets, and ask the choice question against that shortlist.

This is the same candidate-narrowing pattern used elsewhere in this series: cheap filter first, judgment second.

Diagram showing pages flowing into a disposition decision, then merge-marked pages going through a separate candidate-narrowing step before a target-page choice.

One decision for the disposition, a second for the merge target. Do not combine them.

Run the disposition pass

Prepare the state

Send the page's own content plus the context the decision needs.

{
  "url": "/blog/crm-basics",
  "title": "What Is a CRM?",
  "h1": "What Is a CRM?",
  "word_count": 820,
  "primary_queries": ["what is a crm", "crm meaning"],
  "headings": ["What is a CRM?", "Why teams use one", "Common features"],
  "body_excerpt": "..."
}

Keep it focused. A full page of text is usually unnecessary; the title, headings, primary queries, and an excerpt carry most of the signal.

Write the disposition question

What should happen to this page?

keep   — serves a distinct intent, accurate, no significant overlap
update — valid intent but has a specific deficiency
merge  — same intent as another page, should be consolidated
remove — no valid intent, or duplicates another page with no unique value

Four options. Well inside the limit.

Run it in batches

A five-thousand-page audit is five thousand decisions. At local inference speeds this is an overnight job on a laptop, or a much shorter one on a GPU.

Batch by section or by template type rather than running the whole site in one undifferentiated pass. Pages that share a template tend to share a disposition, and grouping them makes anomalies easier to spot.

Store the full distribution

Keep the probability for every option, not just the winner. A page that comes back update: 0.44, keep: 0.41 is genuinely ambiguous, and you want that row flagged even though the model picked a winner.

Run the merge-target pass

Only for pages marked merge.

Narrow the candidates first

Do this with cheap similarity, not with the model. Embed the page, find the most similar pages in your inventory, and take the top five to ten.

Do not ask Laya to choose among all your pages. Ask it to choose among a shortlist that a cheap method already produced.

Ask the choice question

Which page should this page be merged into?

<option 1> — <title and one-line summary>
<option 2> — <title and one-line summary>
...
none — no existing page is a suitable merge target

Two things to notice.

Include a `none` option. Without an escape hatch, the model is forced to pick a target even when no good one exists. Forced choice produces confidently wrong merges, which are destructive.

Describe each option. A bare URL gives the model almost nothing to judge with. A title and a one-line summary give it enough.

Set your confidence threshold

Same procedure as the other workflows in this series, and it matters more here because two of the four dispositions are destructive.

Take your fifty hand-classified pages.
Run them through the disposition pass.
Compare Laya's disposition to yours. Record agreement.
Bucket by confidence.
Find the level above which agreement is good enough.

Set a higher threshold for remove and merge than for keep and update. A wrong keep costs you nothing. A wrong remove deletes a page. Use different thresholds per disposition, and route aggressively to review for the destructive ones.

A worked example, with invented numbers:

Disposition	Threshold	Auto-accepted	Agreement
keep	0.75	1,840	94%
update	0.75	1,210	91%
merge	0.90	310	96%
remove	0.90	180	97%

The point is not the specific numbers. It is that the destructive dispositions carry a stricter bar, and most of them still land in review.

Verify before you act

Three checks, in order of how much damage they prevent.

Hand-check every auto-accepted remove. All of them, not a sample. Deletion is irreversible enough that the review cost is worth paying.

Hand-check a sample of merges. Take twenty and confirm the target is genuinely the better page. A merge into the wrong target loses the content you meant to keep.

Check for systematic bias. If one disposition is over-applied, its criteria are too broad. A common version: everything with low traffic gets marked remove, which is wrong, because low traffic is not the same as no value.

Panel showing per-disposition confidence thresholds and agreement rates, with stricter thresholds on merge and remove.

Per-disposition thresholds. Illustrative numbers only.

What to do with the output

The disposition list is not the deliverable. The worklist is.

Removals. Sort by traffic and check for inbound links before deleting. A page with no traffic but twenty internal links needs its links redirected, not just removed.

Merges. Group by target page so you can do the consolidation work in one pass rather than revisiting the same target repeatedly.

Updates. Group by reason. If forty pages need the same outdated statistic fixed, that is one task, not forty.

Keeps. These are your baseline. Note what share of the library survived, because that number tells you whether your content operation is producing durable pages or churn.

Maintain it

Re-run after major content changes. A migration or a large publishing push changes the relationships between pages, which changes the dispositions.

Re-measure your threshold after any model change. A new checkpoint is a new model.

Keep the hand-labeled set. Grow it every time you review a page. After a few months you have a few hundred labels, which is the starting point for fine-tuning.

Watch the keep rate. If almost everything comes back keep, your criteria are too lenient. If almost everything comes back update, they are too vague. Both are signals about your definitions, not about the model.

What to do next

Write your four disposition definitions with examples. Then take fifty pages you have already reviewed, run them through the disposition pass, and measure your agreement.

If agreement is high on keep and update but low on merge and remove, that is the expected pattern and it is fine. Set stricter thresholds on the destructive dispositions and let the review queue do its job.

And if merge accuracy is poor even with a good threshold, check your candidate narrowing before you blame the model. Most merge failures come from a shortlist that never contained the right target.

Read the rest of the series

This article is part of a thirteen-part series on using Laya for SEO and GEO work.

Author: Clara Bennett, 10-Year Content Strategy Practitioner at Auspia. Clara writes about editorial systems, topic maps, repeatable content operations, and SEO/GEO production workflows.