•
Discovery API:pubsub.v1↗
High Impact
Docs

Pub/Sub Bigtable Sink: Custom Row Keys & Columns

A new columnFamilyMapping property has been added to the BigtableConfig schema. This new configuration, defined by the ColumnFamilyMapping schema, allows for flexible data mapping from Pub/Sub messages (which must be in JSON format) to Bigtable. You can now specify how row keys are constructed using either delimitedKey (concatenating message fields with a delimiter) or rowKeySchema (aligning with Bigtable's structured row key definitions). Additionally, top-level JSON fields in your messages can be mapped directly to specific Bigtable column families, providing granular control over data organization within Bigtable.