How to push results to a Google Cloud Storage bucket
After running an activation query, you can push the resulting file to a Google Cloud Storage bucket.
To do so, you will need to create a connection between the InfoSum Platform and your Google Cloud Storage bucket. The query results will then be streamed to the destination and then the connection will be removed.
Before starting, you will need the following to hand:
- A credentials JSON file
- Bucket details
You will need to do the following:
- Obtain a credentials JSON file: From the Google Management Interface, generate the key file for the service account. This is a JSON file for a service user that allows InfoSum bunker to access your Google Cloud Storage bucket. You do not need to amend this file in any way.
- Set access permissions in GCS: the service user needs to have Create and Write access to the Google Cloud Storage bucket they intend to push files to. If files are pushed to a subfolder, they may need to give explicit permission to Create and Write files to that subfolder.
You can activate results to a file using either Segment Builder or the Query Tool. You will then be taken to the Activation tab under the Activate section, where you will see some information about the campaign and its status. The total rows gives you an insight into the size of the dataset, before pushing it to the destination.
At the bottom of the page, you will see a box providing more details on the campaign, such as the query used and the name of the output column. There are three menu tabs within this box - Details, Pushes, and Push Connectors. Once you've pushed the results, the details and status will be viewable under the Pushes tab.
For now, switch to Push Connectors and use the slider to enable Google Cloud Storage. The form below now appears.
The above form contains the following fields:
- Bucket Name: Specify the bucket name (no leading Google Cloud Storage identifier, e.g. "bucket-name" not "GCS://bucket-name")
- Destination Path: Path within the bucket
- Destination Filename: User defined filename
- GCS credentials JSON: Paste the credentials JSON file text here
- Split large files: Select to split the pushed activation file into smaller files
- Max file size in MB (if splitting large files): If option 5 above is selected, use the default file size (i.e. 1024) or replace with a different maximum file size
- Suffix used when splitting large files (will be followed by the number of the file): Use the suffix shown (i.e. -part-) or replace with a different suffix
When you have completed all the required fields, click Save. When you select Save, the Activate button becomes clickable. Selecting this button begins the file stream.
Switch to the Pushes tab to see the status and check for any errors. Once the status has a green tick, the file has been successfully pushed to the Google Cloud Storage bucket.