Data Analytics

Dataform

Discovery API monitoring and real-time diff analysis for Dataform.

187
Detected Changes
4High Impact
Tracked Discovery APIs (Raw JSON):

Change History (18 · 7 breaking)

← All Services
Dataform`dataform.v1`•
Breaking

Dataform Workspaces Get Async Creation, `parent` Now Required

Dataform introduces `createLongRunning` for asynchronous Workspace creation. This new method requires the `parent` parameter, which is a breaking change if you were previously omitting it in early integrations.

Methods:
Dataform`dataform.v1beta1`•
Breaking

Dataform: Async Workspace Creation & Concurrency Control

Dataform now offers an asynchronous method for creating workspaces, `createLongRunning`, which requires a `parent` parameter. You can also set a `concurrentActionsLimit` in `InvocationConfig` for finer control over workflow execution.

Methods:
Dataform`dataform.v1`•
Medium Impact

Dataform Workflows Get Concurrency Control

Dataform now allows developers to set a maximum concurrent actions limit for workflow invocations. A new `QUEUED` state for actions provides visibility into concurrency-related delays.

Methods:
Dataform`dataform.v1beta1`•
Breaking

Dataform Adds Async Compilation, Clarifies Parent Requirement

Dataform now offers an asynchronous method to create compilation results. While marked as a breaking change, this primarily defines the required `parent` parameter for the *new* `createLongRunning` method, ensuring proper resource scoping from the start.

Methods:
Dataform`dataform.v1`•
Breaking

New Async Compilation Method in Dataform

Dataform now offers a new asynchronous method, `createLongRunning`, for creating compilation results, returning an `Operation` for tracking. This method requires a `parent` parameter.

Methods:
Dataform`dataform.v1beta1`•
Breaking

Dataform Workspaces Gain Commit History Access

Dataform now allows developers to programmatically fetch the commit history of a workspace, enhancing Git integration. A new field for personal authentication in Git remote settings is also available.

Methods:
Dataform`dataform.v1beta1`•
Released (7d lead)Breaking

Dataform adds BigQuery Unit Tests and End-User Auth

Dataform now supports native BigQuery unit testing and granular end-user authentication for workflow invocations. Note that several metadata fields for unit tests and auth are now strictly read-only.

Dataform`dataform.v1`•
Released (7d lead)Medium Impact

Dataform adds support for SQL and Notebook pipelines

Dataform is expanding beyond standard projects to support SQL and Notebook assets as first-class pipelines. New features also include GCS repository snapshotting for scheduled notebook execution.