About Services Tools Blog Contact us

Browser TLS Page Fetch

Downloads a page when ordinary tools are blocked by the site's browser checks, then returns the page, its status or readable text.

  • Web Scraping & Crawling
  • Cross-sector
  • Tool
  • Productised
  • Local/private
  • 2026

The problem

Supplier pages, size guides and shops behind a bot check refuse an automatic request. The job stops on a check page before any content is read. So product research and image downloads never get past the front door.

What it does

A person or job gives it a page address. It presents itself as a current browser, then an older browser, then a plain client if those fail. It returns the raw page, the status or readable text with scripts and menus stripped, cut to a maximum length and optionally saved to a file.

Outcomes

  • Pages that block ordinary tools return their real content.
  • A check page is never saved as if it were the content.
  • Text extraction is built in, so no second tool is needed.
  • The catalogue and image jobs reuse the same tool.

Simplified mockup

Interactive preview with sample data. Nothing is sent to a server.

Page fetch

Request

800 keeps the whole boot description. 200 cuts it and raises a truncated flag. Text view drops scripts.

Handshake
  1. Current browser handshake
  2. Stop if the body is real
  3. Strip scripts for text view
  4. Apply the length cap
Ready

Plain client, then a browser

plain client · https://trade.harbeck.example/yb-17
Challenge page

Hold on

This request did not look like a browser. No product text was returned, so the page must not be saved as the boot.

Retry ladder
  1. Plain client hits the check
  2. Refuse the challenge page
  3. Current browser handshake
  4. Keep the product body
The plain client stopped on the check

Recent fetches

trade.harbeck.example
Saved results
PathProfileStatusSaved
/yb-17Current browser200Text, within cap 800
/yb-17Plain clientChallengeDiscarded, not stored
/yb-17Older browser200Text, cap 200, truncated
/care/leatherCurrent browser200HTML file
/sizes/yb-17Older browser200Text, twelve lines

How it works

  1. Take a web address and the wanted result: page, status, or text.
  2. Try as a current browser, then an older one, then a plain request with browser-style details.
  3. Stop on the first response that is not a check page.
  4. Optionally strip scripts and menus to leave readable text.
  5. Show the result on screen or save it to a file, cut to the requested length.

Pipeline

flowchart LR
  A[URL] --> B[Browser handshake]
  B --> C[Older profile]
  C --> D[Plain request]
  B --> E[Body or text]
  E --> F[Length cap]