Semrush Domain Overview Scraper

Scrape the Semrush website for public domain overview SEO metrics, including traffic, rank, authority, backlinks, keywords, country shares, and competitor summaries.

What does Semrush Domain Overview Scraper do?

Semrush Domain Overview Scraper is an SEO data scraper for the Semrush website and public domain overview pages. Add domains, run the actor, and export structured traffic, ranking, authority, backlink, keyword, country-share, and competitor summary fields as JSON, CSV, Excel, or via API for SEO monitoring and competitive intelligence.

Use cases: who uses it

• SEO teams tracking traffic and authority snapshots for client domains.
• Growth teams benchmarking a product against competitors.
• Agencies preparing recurring domain-performance reports.
• Data teams feeding public competitive-intelligence metrics into dashboards.

Input example

{
  "domains": ["openai.com", "github.com", "stripe.com"],
  "maxItems": 3,
  "maxConcurrency": 2,
  "proxyConfiguration": { "useApifyProxy": true, "apifyProxyGroups": ["SHADER"] }
}

Output example

{
  "input": "openai.com",
  "domain": "openai.com",
  "url": "https://www.semrush.com/website/openai.com/overview/",
  "status": "success",
  "reportingPeriod": "2026-06-01",
  "visits": { "raw": "508002719", "value": 508002719, "diffPercent": -0.0945 },
  "authorityScore": { "raw": "99", "value": 99, "diffPercent": null },
  "referringDomains": { "raw": "423708", "value": 423708, "diffPercent": 0.0496 },
  "backlinks": { "raw": "64639384", "value": 64639384, "diffPercent": 0.015 },
  "topCountries": [{ "country": "United States", "share": 0.1397, "visits": 70981896 }],
  "error": null
}

Input recipes

Bulk domain snapshot

{
  "domains": ["openai.com", "github.com", "stripe.com"],
  "maxItems": 3,
  "maxConcurrency": 2
}

Competitor benchmark batch

{
  "domains": ["notion.so", "clickup.com", "asana.com", "monday.com"],
  "maxItems": 4,
  "maxConcurrency": 2
}

Input settings

Tips and limits

• Start with 3-10 domains to validate your workflow.
• Use bare domains such as example.com; URLs are accepted and normalized.
• Keep concurrency low because public Semrush pages may rate-limit aggressive bulk traffic.
• This actor extracts public overview-page summaries only. It does not log in to Semrush or export private account data.

API usage

JavaScript

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/semrush-domain-overview-scraper').call({
  domains: ['openai.com', 'github.com'],
  maxItems: 2
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient

client = ApifyClient('MY_APIFY_TOKEN')
run = client.actor('fetch_cat/semrush-domain-overview-scraper').call(run_input={
    'domains': ['openai.com', 'github.com'],
    'maxItems': 2,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~semrush-domain-overview-scraper/runs?token=MY_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"domains":["openai.com","github.com"],"maxItems":2}'

MCP and agents

Use the Apify MCP server with this actor tool enabled:

claude mcp add apify 'https://mcp.apify.com/?tools=fetch_cat/semrush-domain-overview-scraper'

JSON configuration:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com/?tools=fetch_cat/semrush-domain-overview-scraper"
    }
  }
}

Example prompts:

• "Run a Semrush overview snapshot for openai.com, github.com, and stripe.com."
• "Compare public Semrush authority and backlink metrics for these competitor domains."

Support

When opening an issue, include:

• Actor run ID or run URL.
• Full input JSON.
• Expected output.
• Actual output or status/error row.
• Reproducible public URL, for example a Semrush domain overview URL, if one specific domain fails.