Web Page to Markdown Extractor

Convert public web pages into clean Markdown, readable text, metadata, links, images, and optional HTML for AI agents, RAG, support automation, research, and no-code workflows.

Use it when you have public URLs and need LLM-ready page content without maintaining a scraper, browser script, or HTML cleanup pipeline.

At a glance

• Markdown extraction: Convert public HTTP/HTTPS pages into Markdown, plain text, metadata, links, images, and optional HTML.
• AI-ready content: Prepare pages for prompts, embeddings, RAG, summarization, classification, and agent context.
• Render control: Use fast HTTP mode by default, browser mode for JavaScript-heavy pages, or auto mode for fallback.
• Optional proxy route: Keep direct access by default or configure Apify Proxy/custom proxies for both HTTP and Chromium requests.
• Predictable runs: Limit pages, timeout per page, HTML bytes, images, links, and source HTML output.
• Inspectable outcomes: Successful pages are saved as dataset rows; failed or deferred URLs are recorded in the RUN_SUMMARY key-value record without charging a page-result event.

What can it do?

Web Page to Markdown Extractor turns public URLs into clean, LLM-ready page content.

• Convert public web pages to Markdown with readable text, page title, description, and metadata.
• Extract links and images for research, crawling, source inspection, and content workflows.
• Choose HTTP, browser, or auto rendering depending on whether the page needs JavaScript.
• Record failed URLs in RUN_SUMMARY so mixed batches remain auditable without creating paid result rows for failed work.
• Use it as a URL-to-Markdown API for AI agents, RAG, support automation, research, and no-code workflows.

What data can you extract?

For a batch-level view of failed and deferred URLs, open RUN_SUMMARY in the default key-value store. PENDING_URLS is saved before the run deadline; copy those URLs into resumeUrls to retry them in a later run.

Input example

The required input is startUrls.

{
  "startUrls": [
    { "url": "https://example.com" }
  ],
  "renderMode": "http",
  "proxyConfiguration": { "useApifyProxy": false },
  "maxPages": 10,
  "includeLinks": true,
  "includeImages": false,
  "includeHtml": false,
  "timeoutSecs": 20,
  "onlyMainContent": false,
  "maxRunSeconds": 270
}

Output example

{
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "statusCode": 200,
  "title": "Example Domain",
  "description": "This domain is for use in documentation examples without needing permission.",
  "markdown": "This domain is for use in documentation examples...",
  "text": "This domain is for use in documentation examples...",
  "links": [
    { "url": "https://iana.org/domains/example", "text": "Learn more" }
  ],
  "images": [],
  "metadata": { "viewport": "width=device-width, initial-scale=1" },
  "renderModeUsed": "http",
  "attemptedRenderModes": ["http"],
  "status": "success",
  "warnings": [],
  "error": null
}

Render modes

HTTP

Use HTTP mode for normal articles, documentation, blog posts, help pages, and static websites.

HTTP mode is fastest and usually cheapest.

Browser

Use browser mode when the page needs JavaScript before content appears.

Browser mode is useful for client-rendered sites but costs more and takes longer.

Auto

Auto mode tries HTTP extraction first, with one bounded retry for transient HTTP statuses. It then uses browser rendering when the response is a classified 401, 403, 429, block/challenge page, has no extractable Markdown, or is a small JavaScript application shell. The HTTP and browser attempts share the same per-page and whole-run deadline.

Tips for best results

• Start with one URL before running a large batch.
• Use HTTP mode unless you know the page needs JavaScript.
• Set includeHtml only when you need it.
• Set includeImages only when image URLs matter.
• Use waitForSelector for browser pages that load content slowly.
• If direct auto mode is still blocked, enable proxyConfiguration; the same route is used for both HTTP and browser attempts.
• Keep timeoutSecs realistic; long timeouts can raise costs.
• Use maxBytes to control very large pages.

Common workflows

• Turn public documentation pages into Markdown for LLM prompts.
• Extract help-center pages into a support knowledge base.
• Convert public product pages into readable summaries.
• Collect article text for monitoring or research.
• Build a URL enrichment step in a no-code automation.
• Prepare website content for embeddings and vector search.

Integrations

You can connect this actor to:

• Make scenarios for URL enrichment
• Zapier workflows for content handoff
• n8n automations for AI pipelines
• Airtable bases for research tracking
• Google Sheets exports for editorial review
• Apify webhooks for event-driven processing
• Vector databases after dataset export

API usage

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/web-page-to-markdown-extractor').call({
  startUrls: [{ url: 'https://example.com' }],
  renderMode: 'http',
  maxPages: 1
});
console.log(run.defaultDatasetId);

Python

from apify_client import ApifyClient

client = ApifyClient('APIFY_TOKEN')
run = client.actor('fetch_cat/web-page-to-markdown-extractor').call(run_input={
    'startUrls': [{'url': 'https://example.com'}],
    'renderMode': 'http',
    'maxPages': 1,
})
print(run['defaultDatasetId'])

cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~web-page-to-markdown-extractor/runs?token=APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"startUrls":[{"url":"https://example.com"}],"renderMode":"http","maxPages":1}'

MCP and AI agents

Use this actor from MCP-compatible AI tools through Apify MCP.

MCP server URL pattern:

https://mcp.apify.com?tools=fetch_cat/web-page-to-markdown-extractor

Add it to Claude Code with:

claude mcp add apify-web-page-markdown https://mcp.apify.com?tools=fetch_cat/web-page-to-markdown-extractor

Example MCP JSON configuration:

{
  "mcpServers": {
    "apify-web-page-markdown": {
      "url": "https://mcp.apify.com?tools=fetch_cat/web-page-to-markdown-extractor"
    }
  }
}

Example prompts:

• "Convert this public URL into Markdown and summarize it."
• "Extract the links from this documentation page."
• "Fetch these three article URLs and prepare text for a knowledge base."

Limits and scope

This actor processes public pages only.

It does not log in, accept private cookies, submit forms, perform social engagement, or automate arbitrary browser tasks.

Some websites block automated access. Auto mode can retry classified blocks in Chromium, and an optional proxy can be enabled without changing the output contract. Failed or challenge pages are not charged as page results and are listed in RUN_SUMMARY; a run fails when none of the requested pages can be converted.

Troubleshooting

Why is the Markdown empty?

The page may require JavaScript rendering, block automated requests, or contain mostly media. Try renderMode: auto first. If both the direct HTTP and Chromium routes are blocked, enable proxyConfiguration with a small maxPages value.

Why did browser mode cost more?

Browser rendering starts a real browser and waits for page content. Use it only for pages that need JavaScript.

How do I inspect failed URLs in a mixed batch?

Open RUN_SUMMARY in the default key-value store. Successful pages stay in the dataset, while failed or deferred URLs are listed in the summary without a page-result charge.

Legality

Use this actor only for public web pages you are allowed to access and process. Respect website terms, copyright, robots policies where applicable, privacy laws, and platform rules.

Do not use it to access private account data, bypass authentication, or collect sensitive personal information.

Support

If a URL does not extract as expected, open an issue or report a bug from the Actor page. Include the Apify run ID or run URL, input JSON, expected output, actual output, and one reproducible public URL.

Privacy and data handling

This Actor only requests the permissions needed to run the input you provide. It uses your input (such as URLs, search terms, identifiers, filters, and limits) only to fetch the requested public data from the relevant source site or API for this Actor, then writes results to your Apify dataset/key-value store.

Data may pass through Apify platform services and Apify Proxy during the run, and requests are sent only to the target site or public data provider required for this Actor's results. FetchCat does not send your inputs or outputs to advertising networks, data brokers, or model-training services, and does not retain run data outside Apify storage after the run except when you explicitly share run details for transient support debugging.

You are responsible for using this Actor lawfully, respecting the target site's terms, and avoiding unnecessary personal or sensitive data in inputs. Review the output before storing, sharing, or combining it with other data.