AWS Glue crawler cost: the minimum charges that add up

Quick answer

AWS Glue crawlers bill at about $0.44 per DPU-hour in us-east-1, billed per second with a 10-minute minimum per run and a minimum of 2 DPUs, so the floor is roughly $0.15 per crawl. Data Catalog storage is free for the first million objects, then about $1 per 100,000 objects per month, and requests are free for the first million, then about $1 per million. The cost surprise comes from crawling many tables on a frequent schedule when partition projection or scheduled, targeted crawls would keep the catalog current for far less.

Glue crawlers scan your data in S3, infer schemas and partitions, and populate the Glue Data Catalog so services like Athena, Redshift Spectrum, and Glue ETL can query it. They are convenient, and their per-run cost is small, which is exactly why teams over-schedule them across many tables and end up paying more than the value the constant re-crawling delivers. Understanding the minimums is the key to keeping crawler cost trivial.

How crawlers are billed

Component	Rate (us-east-1)
Crawler run	~$0.44 per DPU-hour, 2-DPU min, 10-min min
Catalog storage	Free first 1M objects, then ~$1 per 100k/mo
Catalog requests	Free first 1M, then ~$1 per million

A crawler runs on at least 2 DPUs and bills a minimum of 10 minutes even if it finishes in one, so the floor per crawl is about 2 DPUs times 10 minutes, roughly $0.15. That is cheap for one crawler run occasionally. The cost arithmetic changes when you multiply it: 50 crawlers running every hour is 50 times 24 times $0.15, about $180 per day or $5,400 per month, much of it re-scanning data that has not changed.

The over-crawling trap

The most common Glue crawler overspend is scheduling frequent full crawls of tables that change slowly or in predictable ways. A crawler set to run every hour on a table that gets new data once a day pays for 23 unnecessary crawls. Multiply that across dozens of tables and the minimums, harmless individually, become a real monthly line. The fix is to crawl at the cadence data actually changes: hourly for genuinely hourly data, daily or on-demand for the rest.

Crawl less, or not at all

Several techniques cut crawler cost sharply. Schedule crawlers to match data arrival rather than defaulting to frequent runs. Use incremental crawls so a crawler only examines new S3 paths instead of re-scanning the whole table. Best of all, for well-structured partitioned data, skip crawlers entirely and use partition projection in Athena, which computes partitions from a pattern with no crawler at all, as covered in Athena partitioning. Many mature data lakes run crawlers rarely or never, defining schemas directly and projecting partitions.

The catalog itself

The Glue Data Catalog storage and request charges are modest for most workloads, since the first million objects and first million requests each month are free. They matter only at large scale, thousands of tables with millions of partitions, where per-partition catalog lookups from frequent queries add up. Partition projection again helps by avoiding those lookups. For the broader Glue ETL side, see Glue cost optimization and data-lake cost optimization.

Match crawl cadence to change

Glue crawlers are cheap per run and expensive per habit. The 10-minute, 2-DPU minimum means the cost lives in how often and how many, not in any single crawl. Crawl at the rate data actually changes, use incremental crawls, and replace crawlers with partition projection where the layout allows. Price your Glue and catalog configuration against the resource catalog before deploy so crawler schedules are set to real data cadence rather than an over-cautious default.

FAQ

How much does an AWS Glue crawler cost?

About $0.44 per DPU-hour in us-east-1, billed per second with a 10-minute minimum per run and a minimum of 2 DPUs, so the floor is roughly $0.15 per crawl. Cost is trivial for occasional runs but scales with frequency and table count: 50 crawlers running hourly is around $5,400 per month, much of it re-scanning unchanged data.

Why is my Glue crawler bill high?

Usually from over-scheduling: frequent full crawls of many tables that change slowly. A crawler running hourly on a table updated once a day pays for 23 unnecessary crawls, and the 10-minute, 2-DPU minimum on each adds up across dozens of tables. Crawling at the cadence data actually changes eliminates most of the cost.

How do I reduce Glue crawler cost?

Schedule crawlers to match data arrival rather than defaulting to frequent runs, use incremental crawls so only new S3 paths are examined, and for well-structured partitioned data skip crawlers entirely by using partition projection in Athena. Many mature data lakes run crawlers rarely or never, defining schemas directly and projecting partitions.

How much does the Glue Data Catalog cost?

Storage is free for the first million objects, then about $1 per 100,000 objects per month. Requests are free for the first million per month, then about $1 per million. For most workloads these are negligible; they matter only at large scale with thousands of tables and frequent per-partition lookups, which partition projection helps avoid.

How does C3X help with Glue crawler cost?

C3X prices Glue resources from Terraform including crawlers and their configuration, so the crawler footprint and its schedule-driven cost are visible in the pull request. That helps teams set crawl cadence to real data-change frequency and choose partition projection over crawlers where the layout allows, before over-scheduled crawls accumulate on the bill.

What to do next

Crawl at the rate your data actually changes. C3X prices Glue crawlers from Terraform against a live resource catalog. Start with the quickstart.