Explanation
Why the waste happens and who it affects.
Newer AWS Glue versions - such as Glue 5.0, with 5.1 (now the default for new jobs) and 6.0 also available - include significant performance optimizations in the Spark runtime for Glue for Spark ETL jobs, whether written in PySpark or Scala. AWS reports that Glue 5.0 is 32% faster than Glue 4.0 and reduces costs by 22% on its benchmarks. These improvements often require few or no code changes, making version upgrades a simple and impactful optimization. When jobs remain on older runtimes such as Glue 3.0 or 4.0 (or Glue 2.0 and earlier, which reached end of life on April 1, 2026), they execute more slowly, consume more DPUs, and incur unnecessary cost. Jobs on Glue 4.0 or later can also use larger G.12X and G.16X workers and memory-optimized R workers, which can provide additional performance gain for some jobs.
Billing model
The pricing dimensions that drive this cost.
AWS Glue is billed per second based on the number of Data Processing Units (DPUs) used. Faster runtime versions reduce job execution time, lowering total DPU consumption and overall cost. Older versions can extend runtime duration without providing any functional benefit.
How to detect
4 checks to find it in your estate.
- Review Glue job configurations to identify jobs still running on older Glue versions such as 3.0 or 4.0, or end-of-life 2.0 and earlier
- Confirm the job is a Glue for Spark ETL job (PySpark or Scala), since the performance gains come from the Spark runtime
- Assess job execution times that remain consistently longer than expected for similar workloads
- Evaluate whether runtime upgrades have been incorporated into standard Glue job governance or deployment templates
How to fix
4 ways to remove the waste.
- Upgrade Glue for Spark jobs to the latest supported Glue runtime version to benefit from performance improvements
- Incorporate runtime version management into IaC templates and deployment pipelines to ensure consistent modernization
- Periodically review job runtimes to confirm alignment with current Glue versions and available optimizations
- Validate job output after upgrading to confirm compatibility with updated library and Spark versions
Documentation
Vendor references for pricing and configuration.