Table to Storage: configuration file
Last updated
Last updated
The GBQ to Firestore data pipeline starts with a table-to-storage (TTS) data operation. You can find the global parameters of this configuration in the Tables to storage configuration page.
This data operation executes a SQL query to extract data from BigQuery and stores it into a data file (or a set of data files if there's a lot of data to extract) in the Google Cloud Storage of your choice. You can configure it as you like, but you need to store the data into a JSON file and the SQL must follow a specific pattern.
If you want to extract data from BigQuery to load it into Firestore, then you must specify "destination_format": "NEWLINE_DELIMITED_JSON" in your configuration file.
Here is an example of TTS configuration file for a GBQ to Firestore data pipeline: