Instagram Popular Search Scraper

Instagram Search Scraper exports public Instagram popular-topic search results for one or more keywords. Use it to monitor topic pages, collect public reels/posts/profile links, export Instagram search data, and feed social listening or creator research dashboards when Instagram exposes the data on logged-out public pages.

This Actor is intentionally scoped to the confirmed public /popular/<topic>/ route. It does not require Instagram login and does not claim private, authenticated, business, place, or full blended Instagram search data.

What it collects

For each keyword the Actor can save:

• Topic records with title, slug, URL, description/snippet, related topics, and reel counts when exposed
• Public reel and post URLs with shortcodes
• Public profile URLs/usernames when exposed on the topic page
• Hashtags, mentions, thumbnail/media URLs, accessibility text, and public engagement counts when present
• Transparent source/result status on saved dataset rows, with full per-keyword succeeded, empty, or failed diagnostics in the QUERY_STATUS key-value-store record

Input

• keywords — Instagram popular-topic keywords, for example travel or ceramic mugs.
• maxItems — maximum records across the whole run.
• maxItemsPerKeyword — maximum records saved for each keyword.
• maxRetries — retry count with fresh proxy sessions/backoff for transient blocks.
• locale — Accept-Language header.
• proxyConfiguration — Apify Proxy settings. The default uses your account's available automatic proxy pool; select a specific group only when your account has access to it.

Input recipes

Monitor two topics

{
  "keywords": ["travel", "ceramic mugs"],
  "maxItems": 100,
  "maxItemsPerKeyword": 50,
  "proxyConfiguration": { "useApifyProxy": true }
}

Small test run

{
  "keywords": ["fitness reels"],
  "maxItems": 20,
  "maxItemsPerKeyword": 20
}

Scheduled monitoring run

{
  "keywords": ["summer skincare", "trail running", "coffee recipes"],
  "maxItems": 90,
  "maxItemsPerKeyword": 30,
  "locale": "en-US,en;q=0.9",
  "maxRetries": 3,
  "proxyConfiguration": { "useApifyProxy": true }
}

Use this pattern for daily or weekly monitoring. Keep each keyword specific enough that the topic page has a clear intent, and set maxItemsPerKeyword to the number of new public records your dashboard can review. If one keyword is much broader than the others, split it into a separate schedule so it cannot consume the whole run limit.

API pipeline run

{
  "keywords": ["creator economy", "instagram reels tips"],
  "maxItems": 50,
  "maxItemsPerKeyword": 25,
  "locale": "en-US"
}

Use this lighter input when an external app starts runs through the Apify API and downloads the dataset after completion. The output keeps enough context to join records back to the source keyword without parsing the run input again.

Output

Dataset items include fields such as sourceKeyword, queryStatus, rank, resultType, topicTitle, topicSlug, topicUrl, url, shortcode, username, profileUrl, captionText, hashtags, mentions, viewCount, playCount, likeCount, commentCount, thumbnailUrls, mediaUrls, sourcePage, and scrapedAt.

The Actor also writes QUERY_STATUS to the key-value store so you can see which keywords succeeded, returned no items, or failed after retries. Dataset rows carry their source/result status, while full per-keyword diagnostics live in QUERY_STATUS.

Example dataset item

{
  "sourceKeyword": "travel",
  "queryStatus": "succeeded",
  "rank": 1,
  "resultType": "topic",
  "topicTitle": "Travel",
  "topicSlug": "travel",
  "topicUrl": "https://www.instagram.com/popular/travel/",
  "topicDescription": "Public Instagram topic page exposed for this keyword.",
  "topicReelCount": null,
  "relatedTopics": ["adventure", "vacation"],
  "url": "https://www.instagram.com/popular/travel/",
  "shortcode": null,
  "username": null,
  "profileUrl": null,
  "captionText": null,
  "hashtags": ["travel"],
  "mentions": [],
  "viewCount": null,
  "playCount": null,
  "likeCount": null,
  "commentCount": null,
  "thumbnailUrls": ["https://instagram.example/cdn/thumb.jpg"],
  "mediaUrls": [],
  "accessibilityText": null,
  "sourcePage": "https://www.instagram.com/popular/travel/",
  "scrapedAt": "2026-08-01T00:00:00.000Z"
}

Output fields

QUERY_STATUS key-value-store record

{
  "statuses": [
    {
      "sourceKeyword": "travel",
      "status": "succeeded",
      "itemCount": 20,
      "error": null,
      "sourcePage": "https://www.instagram.com/popular/travel/"
    },
    {
      "sourceKeyword": "ceramic mugs",
      "status": "empty",
      "itemCount": 0,
      "error": "No clean per-card records found",
      "sourcePage": "https://www.instagram.com/popular/ceramic mugs/"
    }
  ],
  "summary": {
    "saved": 20,
    "failed": 0,
    "skipped": 0,
    "remaining": 0,
    "completed": 2,
    "pending": 0,
    "pendingKeywords": []
  }
}

Output field guide

Use these groups when mapping the dataset into a warehouse, CRM, or monitoring dashboard:

• Run context — sourceKeyword, queryStatus, rank, sourcePage, and scrapedAt identify which search produced the row and when it was collected.
• Result identity — resultType, url, shortcode, topicSlug, and topicUrl help deduplicate records across repeated scheduled runs.
• Topic metadata — topicTitle, topicDescription, relatedTopics, and exposed reel counts describe the public topic page when Instagram includes those blocks.
• Creator/profile hints — username and profileUrl are saved when the public page exposes a profile link near a reel, post, or topic result.
• Text analysis fields — captionText, hashtags, mentions, and accessibility text are suitable for lightweight keyword matching, brand-safety review, and enrichment.
• Media fields — thumbnailUrls and mediaUrls are public URLs observed on the page; download them promptly if your workflow needs long-term media evidence.
• Engagement fields — viewCount, playCount, likeCount, and commentCount are optional public counters. Missing values mean Instagram did not expose the counter, not that the count is zero.

For recurring jobs, deduplicate first by url, then by shortcode when present. Keep sourceKeyword even after deduplication so you can explain why a record entered the dataset. If you merge multiple runs, prefer the most recent scrapedAt for volatile engagement counters and keep older snapshots when you need trend history.

Who is it for

Instagram Search Scraper is useful for teams that need repeatable exports from public Instagram topic discovery pages:

• Social listening teams tracking topic clusters and emerging public reels
• Creator intelligence teams collecting public profile and post links for enrichment
• Brand monitoring teams watching public Instagram topic pages around products or campaigns
• Agencies building weekly dashboards from Instagram search records
• Researchers who need timestamped, repeatable public search snapshots

API usage

Use this as an Instagram search API from the Apify API, schedules, integrations, or MCP clients. Send the same JSON input you use in Console and export the dataset as JSON, CSV, Excel, XML, or RSS.

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/instagram-search-scraper').call({
  keywords: ['travel', 'ceramic mugs'],
  maxItems: 20,
  maxItemsPerKeyword: 10,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('fetch_cat/instagram-search-scraper').call(run_input={
    'keywords': ['travel', 'ceramic mugs'],
    'maxItems': 20,
    'maxItemsPerKeyword': 10,
})

items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~instagram-search-scraper/runs?token=YOUR_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"keywords":["travel","ceramic mugs"],"maxItems":20,"maxItemsPerKeyword":10}'

After the run finishes, download results from the default dataset endpoint in JSON, CSV, Excel, XML, or RSS.

MCP and AI-agent setup

You can call this Actor from AI agents through the official focused Apify MCP endpoint:

https://mcp.apify.com?tools=fetch_cat/instagram-search-scraper

Claude CLI / Claude Code

Add the focused Apify MCP server from a terminal and pass your Apify token as an Authorization header:

claude mcp add apify-instagram-search -- npx mcp-remote "https://mcp.apify.com?tools=fetch_cat/instagram-search-scraper" --header "Authorization: Bearer <YOUR_API_TOKEN>"

Pass the Authorization: Bearer <YOUR_API_TOKEN> header to mcp-remote; setting APIFY_TOKEN alone does not send that header to the MCP endpoint.

Claude Desktop configuration

Add an MCP server entry that points to the focused Apify MCP URL and passes your Apify API token as an Authorization header:

{
  "mcpServers": {
    "apify-instagram-search": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "https://mcp.apify.com?tools=fetch_cat/instagram-search-scraper",
        "--header",
        "Authorization: Bearer <YOUR_API_TOKEN>"
      ]
    }
  }
}

Example prompts:

• "Run Instagram Search Scraper for travel and ceramic mugs, limit to 20 results total, then summarize the top result types."
• "Check the QUERY_STATUS record for my latest Instagram Search Scraper run and tell me which keywords returned no public data."
• "Export the dataset as JSON and group results by source keyword."

Workflow tips

• Use one keyword per buyer intent, campaign, topic cluster, or competitor category.
• Keep maxItemsPerKeyword small for scheduled monitoring so each run finishes quickly.
• Use QUERY_STATUS to alert on keywords that were throttled or returned no public records.
• Export CSV for spreadsheets and JSON for enrichment pipelines.
• Store run IDs in your dashboard so Support can help investigate unexpected changes.

Data quality checks

Each saved row includes sourceKeyword, queryStatus, rank, resultType, url, and scrapedAt so downstream systems can filter partial data safely. Optional fields such as captions, hashtags, mentions, thumbnails, media URLs, and engagement counts are populated only when Instagram exposes them on the public page.

Recommended validation after each scheduled run:

1. Confirm saved rows have queryStatus: "succeeded"; complete per-keyword succeeded, empty, failed, or skipped diagnostics live in the QUERY_STATUS key-value-store record.
2. Check how many dataset rows each keyword produced.
3. Deduplicate by url or shortcode before merging with historical exports.
4. Treat missing optional engagement metrics as unavailable public data, not zero.
5. Review source URLs before using the data in customer-facing reports.

Notes and limitations

Instagram can throttle or change public pages. The Actor preserves partial output when one keyword fails and records per-keyword errors in the QUERY_STATUS key-value-store record instead of hiding missing data. Fields are only populated when available on public logged-out pages.

Support

If a run fails or a keyword returns fewer rows than expected, open an Apify issue and include:

• The run ID or run URL
• The exact input JSON
• Expected output versus actual output
• One reproducible public Instagram topic URL, if available
• Any relevant QUERY_STATUS details from the key-value-store record

The QUERY_STATUS record usually shows whether Instagram throttled the request or returned no public data.