Google CloudGoogle Cloud ComposerAnalytics

`google_composer_environment` cost estimation

Managed Apache Airflow. Composer 2: ~$455/month minimum (web server $73 + scheduler $73 + workers $25/CPU + Cloud SQL $73). Composer 3 introduces serverless workers. Often the priciest 'orchestration' bill.

Cloud Composer is managed Apache Airflow. The google_composer_environment resource creates an environment running Airflow + supporting infrastructure. Pricing is complex and notoriously expensive — Composer is often the largest single bill in data engineering shops.

Composer 2 components (typical small environment): - Web server: ~$73/month (always-on n1-standard-2) - Scheduler: ~$73/month (always-on) - Workers: ~$25-$100/month each (variable based on CPU and memory) - Cloud SQL backing database: ~$73/month - GCS bucket for DAGs and logs: $0.020/GB - Cloud Logging: $0.50/GB ingested

Minimum environment: ~$455/month. A larger environment (more workers, larger backing DB) can easily reach $1,500-$5,000/month.

Composer 3 (newer, serverless workers): - Web server + scheduler: similar to Composer 2 - Workers: $0.04/vCPU-hour, scale to zero - Reduces costs for workloads with low DAG execution time

When Composer wins: - You need managed Airflow specifically (DAGs, operators, plugin ecosystem) - Operational simplicity over self-hosted Airflow on GKE - Integration with GCP services via built-in operators

When Composer is wrong: - Simple cron jobs (use Cloud Scheduler at $0.10/job/month) - Cloud Run Jobs for one-off batches - Workflows for serverless pipelines

Common cost surprise: idle Composer. Even when no DAGs run, the always-on components bill ~$220/month. For environments rarely used, this is mostly waste. Consider Composer 3's serverless workers if usage is sporadic.

c3x estimates Composer based on environment_config (node_count, machine_type, sql_machine_type for Composer 2; or worker config for Composer 3).

Terraform example

A minimal but realistic configuration that C3X can estimate.

resource "google_composer_environment" "main" {
  name   = "prod-composer"
  region = "us-central1"

  config {
    software_config {
      image_version = "composer-2-airflow-2"
    }

    workloads_config {
      scheduler {
        cpu        = 0.5
        memory_gb  = 1.875
        storage_gb = 1
        count      = 1
      }
      web_server {
        cpu        = 0.5
        memory_gb  = 1.875
        storage_gb = 1
      }
      worker {
        cpu        = 0.5
        memory_gb  = 1.875
        storage_gb = 1
        min_count  = 1
        max_count  = 3
      }
    }

    environment_size = "ENVIRONMENT_SIZE_SMALL"
  }
}

Pricing dimensions

What you actually pay for when you provision google_composer_environment.

Dimension	Unit	What's being charged
Web server	per hour	Always-on Airflow web UI host. ~$0.10/hour for small
Scheduler	per hour	Always-on Airflow scheduler. ~$0.10/hour for small
Workers	per hour or per vCPU	DAG task executors. Composer 2 has fixed workers; Composer 3 scales serverlessly. ~$0.04/vCPU-hour (Composer 3)
Cloud SQL backing DB	per hour	Airflow metadata database. Bills as a Cloud SQL instance. ~$0.10/hour for db-n1-standard-2
Cloud Logging ingestion	per GB	DAG run logs are ingested into Cloud Logging. $0.50/GB

Optimization tips

Common ways to reduce google_composer_environment cost without changing the workload.

Use Composer 3 for variable workloads

30-60% on variable workloads

Composer 3's serverless workers scale to zero. For workloads with idle periods (nights, weekends), saves substantially vs Composer 2's always-on workers.

Skip Composer for simple workflows

$450+/month

Composer's ~$455/month minimum makes it overkill for simple cron jobs. Cloud Scheduler ($0.10/job/month) + Cloud Functions or Cloud Run handles many workflows at <$10/month total. Reserve Composer for complex DAGs.

Share environments across teams

$455/month per avoided environment

One large Composer environment serving multiple teams is more cost-efficient than per-team environments. The fixed costs (web server, scheduler, DB) are amortized. Use Airflow namespaces or DAG-level access controls for separation.

Tune log retention and verbosity

Variable, often $50-$500/month

DAG logs ingest into Cloud Logging at $0.50/GB. Verbose DAGs (lots of XCom data, large log lines) can generate substantial log costs. Set log levels to INFO or WARNING in production. Configure log retention to comply only with audit requirements.

Use environment_size=SMALL when sufficient

Variable based on size reduction

Composer 2 has Small/Medium/Large environment sizes. Many production workloads fit in Small. Resize down if utilization is below 50%.

FAQ

Why is Composer so expensive?

Composer runs full Apache Airflow infrastructure: web server, scheduler, multiple workers, backing database, plus the GCP integration layer. Each component is a VM or managed service. The fixed components ($220+/month) bill whether you run DAGs or not. For workloads not justifying always-on Airflow, alternatives exist.

Should I self-host Airflow on GKE instead?

Possible savings, especially at scale. Self-hosted on GKE can be 50-70% cheaper than Composer. Trade-off: you operate the Airflow infrastructure (upgrades, scaling, troubleshooting). For data engineering teams with operational capacity, self-hosting is reasonable. For lean teams, Composer's managed value is usually worth the premium.

What about AWS MWAA or Azure Data Factory?

AWS MWAA (managed Airflow) is similarly expensive to Composer (~$300-$1000/month minimum). Azure Data Factory is cheaper for simple pipelines but different programming model (visual DAGs, less code). All three are pricier than self-hosted but easier to operate.

Can Composer scale to zero?

Composer 2: no, the web server, scheduler, and DB bill continuously. Composer 3: workers scale to zero, but web server/scheduler still bill. For true scale-to-zero workflows, Cloud Workflows or Cloud Run Jobs are alternatives.