The ClusterConfig schema now includes an optional schedulingConfig property, which references the newly introduced SchedulingConfig schema. This new configuration object provides two key parameters: requestedRunDuration, which sets the maximum desired lifetime for provisioned resources (min 10 mins, max 7 days), and schedulingTimeout, which defines how long Dataproc should wait for resources to become available before failing the cluster creation request (max 13 days). These additions offer greater control over cluster lifecycle and resource allocation, enabling more efficient and cost-effective management of transient or time-bound workloads.
Medium Impact
Docs