Protected Sites & Blocked Content

Access X.com, LinkedIn, Product Hunt, and sites that block scrapers

Many sites—X.com (Twitter), LinkedIn, Product Hunt, Amazon, paywalled blogs—block standard scrapers and headless browsers with 403 Forbidden or Cloudflare "Are you a human?" challenges. The built-in OpenClaw browser often hits these walls. On a VPS or remote server you're typically limited to headless Chrome or raw HTTP, both commonly blocked.

Two complementary solutions work with OpenClaw:

GhostFetch — Content Extraction

GhostFetch is a stealth fetch engine that returns high-density Markdown. Install it as a skill from its GitHub repo.

  • Best for: "Fetch this page's content" workflows
  • Output: Markdown you can process, summarize, or store
  • Works on: X.com, LinkedIn, Product Hunt, Amazon, many protected blogs

Use GhostFetch when you need the content of a page—text, structure—without interactive browsing.

camofox-browser — Interactive Browsing

camofox-browser is an OpenClaw plugin that runs a headless browser server built on Camoufox (Firefox fork with C++ fingerprint spoofing). Install with:

Install camofox
openclaw plugins install @askjo/camofox-browser

Exposes tools like camofox_create_tab, camofox_snapshot, camofox_click, camofox_navigate.

  • Best for: Interactive browsing on X, Product Hunt, Amazon from a server
  • Advantage: Fingerprint spoofing; bypasses many bot detection systems
  • Use case: Login flows, form filling, multi-step navigation on protected sites

When to Use Which

GhostFetch — You need the content of a page (articles, posts, product details). Fast, simple.

camofox-browser — You need to click, scroll, fill forms, or follow multi-step flows. More powerful, more setup.

For unprotected sites, the built-in browser is usually enough.

See also