About Services Tools Blog Contact us

Linking Related Documents

An overnight job that links recently changed documents in the company knowledge base to similar ones and files any that have no category.

  • Scheduled Workers & Sync
  • Professional services
  • Scheduled worker
  • Productised
  • Local/private
  • 2026

The problem

New documents landed in the knowledge base with no links, so related and duplicate material stayed hidden until someone read it. A new note that contradicted an old one never showed up as a contradiction. A suspected duplicate needs a person to decide, so it cannot be overwritten automatically.

What it does

Each night the job takes the documents changed since its last run, summarises the topic from the title and opening passages, and searches for similar documents. A very close match is suggested as a duplicate. A moderate match is labelled as related, contradicting, part of, a duplicate or a replacement. Safe link types are saved straight away, and duplicates and replacements wait as suggestions for a person.

Outcomes

  • New documents have links by the next morning.
  • Suspected duplicates are suggested and are never merged without a person.
  • Contradictions are labelled and the two documents stay separate.
  • A summary file records each run.

Simplified mockup

Interactive preview with sample data. Nothing is sent to a server.

Changed since the checkpoint

Sources walked
0
since yesterday 03:04
Safe links
0
related, contradicts, part of
Proposals
0
duplicate or supersedes
Link rows written
10
falls to 0 on a dry run
Run log
SourceBest neighbourSimilarityOutcome
Wharf lease heads of terms Wharf lease, signed 2024 0.96 Proposed duplicate
Expenses policy 2026 Expenses policy 2024 0.91 Proposed supersedes
New starter checklist Induction checklist 0.84 Linked · related
Hybrid working note Office attendance rule 0.81 Linked · contradicts
Payroll calendar, April Payroll calendar 0.86 Linked · part of
Reception refit quote Shop fit-out notes 0.31 Filed · Facilities
Data retention note Privacy notice 0.83 Linked · related
Print supplier pack Supplier checklist 0.95 Proposed duplicate
Fire drill record, March Fire drill procedure 0.80 Linked · part of
Board appendix C Board pack, first quarter 0.82 Linked · part of
Travel policy draft Expenses policy 2026 0.79 Linked · related
Unlabelled scan, 14 April No neighbour above 0.78 0.22 Filed · Inbox

At 0.95 or above the pair is a duplicate proposal. From 0.78 the pair is classified. Below that, a source with no collection is filed and the rest are skipped. Contradictions are labelled, not blended.

This pass
Checkpoint is yesterday 03:04. Safe links write immediately. Duplicates wait.

Waiting for a person

Duplicate and supersedes are never applied on their own. Declining leaves both sources. Accepting only queues a person. Nothing is overwritten.

Wharf lease heads of terms Proposed duplicate

Best neighbour is Wharf lease, signed 2024 at 0.96. The new file repeats the signed heads of terms with a fresh date on the cover.

Held. A merge would hide the signed copy.

Expenses policy 2026 Proposed supersedes

Best neighbour is Expenses policy 2024 at 0.91. Mileage and overnight rates differ. The 2024 text would stay readable until someone retires it.

Held. Superseding is a proposal, not a delete.

Print supplier pack Proposed duplicate

Best neighbour is Supplier checklist at 0.95. Same onboarding steps, with a price list stapled on the back.

Held. The price list must not disappear into the checklist.

JSON digest

02:10 dry run
0
rows written · digest only
03:04 live run
10
link rows · checkpoint moved
graph-2026-04-14.json
{ "started": "2026-04-14T03:00:04", "mode": "live", "checkpoint_from": "2026-04-13T03:04:11", "checkpoint_to": "2026-04-14T03:04:02", "sources": 12, "linked": { "related": 3, "contradicts": 1, "part_of": 3 }, "proposed": { "duplicate": 2, "supersedes": 1 }, "filed": 2, "skipped": 0, "cursor_advanced": true }
Event log
TimeEventDetail
03:01Link writtenHybrid working note contradicts Office attendance rule
03:02Duplicate proposedWharf lease heads of terms
03:02Supersedes proposedExpenses policy 2026
03:03Source filedUnlabelled scan, 14 April → Inbox
03:04Run closedDigest stored, checkpoint advanced

How it works

  1. A scheduled job starts the run each night.
  2. Find the documents changed since the last run.
  3. Summarise the topic from the title and opening passages.
  4. Search for similar documents.
  5. Save the safe link types straight away.
  6. Leave duplicates and replacements as suggestions and file anything with no category.

Pipeline

flowchart LR
  Changed["Changed sources"] --> Embed["Embed topic"]
  Embed --> Neigh["Neighbours"]
  Neigh --> Safe["Write safe links"]
  Neigh --> Proposed["Propose duplicates"]
  Neigh --> Filed["File uncategorised"]
  Safe --> Digest["Run digest"]