About Services Tools Blog Contact us

Directory Image Discovery and Recheck

Finds photographs for a venue from its own site, OpenStreetMap, Wikimedia Commons and Reddit, keeps only the links that still work, and rechecks them on a schedule.

  • Web Scraping & Crawling
  • Leisure and sport
  • Pipeline
  • Product
  • Local/private
  • 2026

The problem

A directory of venues with no photographs looks unfinished. A picture linked from another site that later disappears is worse than none. Staff cannot visit every venue site each month to see which pictures have gone, and a venue left with no images should be searched again.

What it does

The search tries the venue's own site first, then OpenStreetMap, Wikimedia Commons and Reddit. It keeps only the links that still work and picks a main image. A monthly recheck removes dead links, and a venue left with no images goes back on the list to be searched again. The venue's own site outranks the other sources.

Outcomes

  • Venue pages gain photographs without a manual hunt.
  • Dead image links are removed and no longer show as broken.
  • A venue that loses its last image is searched again.
  • A venue's own pictures rank above finds from OpenStreetMap, Wikimedia Commons and Reddit.

Simplified mockup

Interactive preview with sample data. Nothing is sent to a server.

Kiln Lane Halesend

Venue No images

Indoor hall. The page has a map pin and a website, and no photograph that still responds.

Order: own site, then a public archive, then a forum. A dead link is not kept. The hero comes from the venue site.

Sources
  1. Own site adds 4
  2. Archive adds 1
  3. Forum link does not respond
  4. Forum picture dropped
  5. Hero from the venue site
Nothing validated yet
No image
No image
No image
No image
Queue
VenueTownImagesNext
Kiln LaneHalesendNoneDiscover
Moss CourtHalesend2 storedApril recheck
River ShedHalesendHero setOK

April recheck

Kiln Lane
3 live
Moss Court
2 live
Requeue
0
Stored addresses
VenueSourceRoleStatus
Kiln LaneOwn siteHeroLive
Kiln LaneOwn siteGalleryLive
Kiln LaneArchiveGalleryLive
Moss CourtForumGalleryLive
Moss CourtArchiveGalleryLive
Kiln Lane · stays
Hero Kiln Lane hero, still responding
Own site
Kiln Lane gallery, still responding
Own site
Kiln Lane archive, still responding
Archive
Moss Court
Moss Court forum picture
Forum
Moss Court archive picture
Archive
Requeue
No venue at zero

How it works

  1. Add a venue that has no checked images to the search list.
  2. Try the venue site, then OpenStreetMap and Wikimedia Commons, then Reddit, in that order.
  3. Keep the links that work and pick a main image.
  4. On the monthly run, test each stored link and delete the dead ones.
  5. Put any venue that no longer has an image back on the list.

Pipeline

flowchart LR
  A[Venue] --> B[Own site]
  B --> C[OpenStreetMap]
  C --> D[Wikimedia Commons]
  D --> R[Reddit]
  R --> E[Live images]
  E --> F[Hero]
  E --> G[Monthly recheck]