Hugging Face Datasets Scraper

Export public Hugging Face dataset catalog metadata to structured datasets. Use it to find training/RAG datasets, compare licenses and tags, monitor updates, or enrich a list of known dataset IDs.

At a glance

• Primary job: Search Hugging Face datasets and save clean metadata rows.
• Input: Search keywords and/or exact dataset IDs, plus result limits and filters.
• Output: One row per dataset with IDs, URL, author, downloads, likes, tags, license/task/format fields, access flags, timestamps, and diagnostics.
• Best for: AI teams, data scientists, compliance reviewers, research analysts, RAG builders, and fine-tuning dataset discovery.

Who is it for?

This Actor is for AI engineers, data scientists, ML platform teams, compliance reviewers, RAG builders, and research analysts who need structured Hugging Face dataset metadata without manually opening dataset pages.

Common workflows

• Dataset discovery: Search topics such as finance, medical imaging, or instruction tuning and sort by downloads, likes, recency, or trending score.
• License review prep: Export raw Hugging Face tags plus parsed license, task, language, size, format, and library fields for spreadsheet review.
• Known-ID enrichment: Provide exact dataset IDs such as squad or HuggingFaceH4/ultrachat_200k and receive current public metadata.
• Monitoring: Schedule repeat runs with sort: "lastModified" and compare output datasets over time.

Input recipes

Popular finance datasets

{
  "searchQueries": ["finance"],
  "maxItems": 20,
  "sort": "downloads",
  "dedupe": true
}

Exact dataset lookup

{
  "datasetIds": ["squad", "HuggingFaceH4/ultrachat_200k"],
  "maxItems": 5,
  "dedupe": true
}

Recently modified medical imaging datasets

{
  "searchQueries": ["medical imaging"],
  "maxItems": 20,
  "sort": "lastModified",
  "dedupe": true
}

What data can you extract?

Example input

{
  "searchQueries": ["finance"],
  "datasetIds": ["squad"],
  "maxItems": 25,
  "sort": "downloads",
  "dedupe": true
}

Example output

{
  "datasetId": "squad",
  "url": "https://huggingface.co/datasets/squad",
  "author": null,
  "name": "squad",
  "description": "Stanford Question Answering Dataset...",
  "descriptionTruncated": false,
  "downloads": 12345,
  "likes": 100,
  "trendingScore": 0,
  "createdAt": "2022-03-02T00:00:00.000Z",
  "lastModified": "2024-01-01T00:00:00.000Z",
  "sha": "...",
  "gated": false,
  "private": false,
  "disabled": false,
  "accessStatus": "public",
  "tags": ["task_categories:question-answering", "language:en"],
  "license": null,
  "languages": ["en"],
  "tasks": ["question-answering"],
  "formats": ["parquet"],
  "sourceQuery": "dataset:squad",
  "fetchedAt": "2026-07-17T00:00:00.000Z",
  "schemaVersion": "1.0.0"
}

Tips for best results

• Start small: Use maxItems: 10-50 while tuning queries.
• Use focused terms: Domain phrases such as legal documents or medical imaging are usually better than one-word broad searches.
• Leave card data off by default: Enable includeCardData only when you need README text because it adds one request per saved item.
• Respect access flags: Gated datasets are marked; this Actor does not fetch gated contents.

API usage

Node.js:

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor("fetch_cat/hugging-face-datasets-scraper").call({
  searchQueries: ["finance"],
  maxItems: 20,
  sort: "downloads"
});
console.log(run.defaultDatasetId);

Python:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("fetch_cat/hugging-face-datasets-scraper").call(run_input={
    "searchQueries": ["finance"],
    "maxItems": 20,
    "sort": "downloads",
})
print(run["defaultDatasetId"])

cURL:

curl -X POST "https://api.apify.com/v2/acts/fetch_cat~hugging-face-datasets-scraper/runs?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchQueries":["finance"],"maxItems":20,"sort":"downloads"}'

MCP and AI agents

Use this Actor from MCP-compatible tools through the official Apify MCP Server.

claude mcp add --transport http apify "https://mcp.apify.com?tools=fetch_cat/hugging-face-datasets-scraper"

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=fetch_cat/hugging-face-datasets-scraper"
    }
  }
}

Example prompt: "Find 20 popular finance datasets on Hugging Face and summarize their licenses, tasks, and access flags."

Limits and caveats

• Public metadata only: The Actor uses public Hugging Face catalog endpoints and public README files only.
• No gated-content bypass: Gated/private/disabled flags are exported as metadata; protected dataset contents are not downloaded.
• Source changes: Hugging Face fields and tags can change over time.
• Diagnostics: Per-source run diagnostics are stored in the DIAGNOSTICS key-value record.

Legality and responsible use

Process only data you are allowed to access. Follow Hugging Face terms, dataset licenses, Apify's terms, and applicable laws.

Support

Open an issue from the Actor page if a run fails or output looks wrong. Include the run ID or URL, input JSON, expected output, actual output, and one reproducible public URL such as https://huggingface.co/datasets/squad.