GitHub Repositories Search Scraper

Search and export public GitHub repositories by keyword, language, topic, stars, forks, and recent activity.

Turn GitHub search into a clean dataset for market research, developer relations, open-source intelligence, competitive tracking, and technical lead generation. Results can be downloaded as CSV, JSON, Excel, XML, RSS, or used through the Apify Dataset API.

At a glance

• Repository discovery: Search by product, company, framework, language, topic, or keyword and export repository URLs, names, descriptions, stars, forks, and topics.
• Market mapping: Compare ecosystems, categories, or developer tools using language, license, owner type, and activity fields.
• Competitive tracking: Monitor new or recently updated repositories around a competitor, package, or market category.
• Developer relations: Find projects that mention your SDK, API, package, or integration, then review owner and homepage context.
• API export: Send flat GitHub repository records to spreadsheets, BI tools, enrichment pipelines, or agents.

What can it do?

GitHub Repositories Search Scraper uses public GitHub repository search and returns structured rows you can sort, filter, export, and reuse from the Apify API.

• Search repositories: Enter one or more search queries such as ai agent, web scraping, or language:python stars:>1000.
• Filter results: Add a language and optional GitHub topics when you need a narrower list.
• Sort repositories: Choose best match, stars, forks, or recently updated repositories.
• Export data: Download from Apify as JSON, CSV, Excel, XML, RSS, or stream through the Dataset API.
• Enrich owners: Turn on owner details when public profile context matters for research or outreach.

Common workflows

• Developer relations: Search your-sdk-name or your-api-name to find projects that mention your product, SDK, API, or integration.
• Market research: Search topic:ai stars:>500 or another category query to find high-signal repositories sorted by popularity or freshness.
• Competitive intelligence: Search a competitor package name or framework to map public projects around that tool or market.
• Open-source intelligence: Search a vendor, package, or vulnerability-related keyword to build a monitoring list for follow-up analysis.
• Technical lead generation: Search language:rust stars:>100 or another stack-specific query to discover maintainers, companies, and active projects.
• Content research: Search mcp, agent, automation, or another trend keyword to build lists for newsletters, reports, and trend tracking.

What data can you extract?

The actor returns one dataset row per repository.

Optional owner details

Turn on includeOwnerDetails only when owner profile context is worth the extra GitHub API requests.

• Default search: Keep includeOwnerDetails set to false for faster repository search, dashboards, ranking lists, and trend monitoring.
• Owner enrichment: Set includeOwnerDetails to true for developer relations, lead research, owner segmentation, or outreach preparation.
• Extra public owner fields: Enrichment can add public owner name, company, blog, location, public repo count, followers, following, and profile timestamps.
• Larger enriched runs: Add a githubToken secret when owner details or larger searches need higher GitHub API limits.

Example input

{
  "queries": ["apify", "web scraping"],
  "language": "javascript",
  "topics": ["automation"],
  "sort": "stars",
  "order": "desc",
  "maxResults": 50,
  "includeOwnerDetails": false
}

Example output

{
  "query": "apify",
  "name": "API-mega-list",
  "fullName": "cporter202/API-mega-list",
  "htmlUrl": "https://github.com/cporter202/API-mega-list",
  "description": "A public API collection and developer resource list.",
  "ownerLogin": "cporter202",
  "ownerType": "User",
  "stars": 6733,
  "forks": 1294,
  "watchers": 6733,
  "openIssues": 15,
  "language": "JavaScript",
  "topics": ["api", "automation", "web-scraping"],
  "licenseName": null,
  "updatedAt": "2026-06-22T01:14:16Z",
  "pushedAt": "2026-01-27T18:40:57Z",
  "archived": false,
  "fork": false,
  "homepage": null,
  "cloneUrl": "https://github.com/cporter202/API-mega-list.git"
}

How to run it

1. Open the actor on Apify.
2. Add one or more repository search queries.
3. Optionally choose a language and topics.
4. Set the maximum number of repositories.
5. Start the run.
6. Download the dataset or use the API.

Search tips

• Start broad: Try a short keyword such as mcp, ai agent, web scraping, or a product name.
• Narrow by language: Use the language setting or a query qualifier such as language:python.
• Narrow by topic: Add a topic such as web-scraping, llm, automation, or react when you need a cleaner category list.
• Find popular projects: Sort by stars and start with a small maxResults value.
• Find recent activity: Sort by updated or use GitHub qualifiers such as pushed:>2026-01-01.
• Compare competitors: Run multiple queries in one input, then compare results by query, stars, forks, and pushedAt.
• Control spend: Start with maxResults: 25, inspect the table, then scale up.

Handling GitHub limits

• Small public search: Run without a token.
• Larger search: Keep maxResults bounded and split broad searches into focused queries.
• Owner enrichment: Add a GitHub token because owner details require extra API calls.
• Rate limit message: Retry with a smaller run or add a token.
• Token handling: The token is used only for GitHub API authentication during your run.

Integrations

You can connect the dataset to downstream tools.

• Send CSV exports to spreadsheets for market analysis.
• Use Make or Zapier to trigger follow-up enrichment.
• Load JSON results into a warehouse or database.
• Feed repository lists into developer-relations workflows.
• Combine repository URLs with other Apify actors for broader research.

API usage

Node.js

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fetch_cat/github-repositories-search-scraper').call({
  queries: ['apify'],
  language: 'javascript',
  maxResults: 25
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Python

from apify_client import ApifyClient
import os

client = ApifyClient(os.environ['APIFY_TOKEN'])
run = client.actor('fetch_cat/github-repositories-search-scraper').call(run_input={
    'queries': ['apify'],
    'language': 'javascript',
    'maxResults': 25,
})
items = client.dataset(run['defaultDatasetId']).list_items().items
print(items)

cURL

curl -X POST 'https://api.apify.com/v2/acts/fetch_cat~github-repositories-search-scraper/runs?token=YOUR_APIFY_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{"queries":["apify"],"language":"javascript","maxResults":25}'

MCP and AI agents

Use this Actor from MCP-compatible tools through the official Apify MCP server. The default server can search and run Actors; the focused URL exposes only this Actor.

MCP URL:

https://mcp.apify.com?tools=fetch_cat/github-repositories-search-scraper

Claude Code setup:

claude mcp add apify-github-repositories "https://mcp.apify.com?tools=fetch_cat/github-repositories-search-scraper"

Claude Desktop JSON config:

{
  "mcpServers": {
    "apify-github-repositories": {
      "url": "https://mcp.apify.com?tools=fetch_cat/github-repositories-search-scraper"
    }
  }
}

Example prompts:

• "Find 25 JavaScript repositories about Apify and summarize the top owners."
• "Export popular Python repositories about web scraping to a table."
• "Search GitHub repositories for MCP tools and rank them by stars."

Data quality notes

Repository counts and metadata come from public GitHub repository records.

Stars, forks, issues, and timestamps can change over time.

Some repositories do not have a license, homepage, topics, or recent push timestamp.

Owner email is often unavailable because many GitHub users do not publish an email on their profile.

Legality and responsible use

This actor extracts publicly available GitHub repository data.

Use the data responsibly and follow GitHub's terms, Apify's terms, and applicable laws.

Do not use exported data for spam, harassment, credential collection, or other abusive activity.

If you use owner details for outreach, comply with privacy and anti-spam rules in your jurisdiction.

Need help?

If a query behaves differently than expected, start with a small maxResults value and inspect the dataset.

Then adjust language, topic, sort, and order settings.

For larger enriched runs, use a GitHub token and keep your first test small.