Beacons is an app for data collaboration that is deployed in your own cloud or warehouse environment, delivering our ‘non-movement of data’ promise even before collaboration has taken place. Snowflake Beacons are deployed in your Snowflake customer account.

Pre-requisites

Snowflake setup

Beacon Deployment

1. Create a Snowflake Cloud Vault in InfoSum

2. Beacons Marketplace App Deployment in Snowflake

Next steps

(Optional) Setting up a Beacon for Destination Exports

Deleting your Beacon deployment

App Permissions

Pre-requisites

You must have an InfoSum license and account prior to starting this process. If you do not have one, please reach out to your InfoSum contact or support@infosum.com to get started.

You must have your data in a supported format, which is customer-level data. Snowflake can easily transform your data if needed to group the data by customer record. Please review these formatting guidelines and contact our team if you have any questions. 

Snowflake setup

Each Beacon deployment is one Snowflake schema and one Cloud Vault.

• Each Cloud Vault can contain multiple published datasets in InfoSum 
• Each Schema can contain multiple tables and views

This means that you need to ensure that you have a schema available that contains all your tables and views for collaboration. Please note that schemas are nested under databases; we require all the data to be available at the schema level, not the database level.

 If your tables are distributed across multiple schemas, you have two options:

1. Deploy two Beacons to two different CV: You will only be able to use the tables on these two Cloud Vaults independently, meaning you cannot publish a joint view of the data to the same Beacon. Please note there is a limit of 10 CVs per account. To install multiple instances of the same app, please follow Snowflake’s guide.
   1. This might be your preferred option if your data comes from different regions, teams, or departments/brands within your organization and cannot be co-located into the same schema.
2. Replicate or move the table/view from one schema to the other: You will then be able to access all tables and views in one Cloud Vault and combine them to create a custom recordset to publish to one single Beacon.

We do not yet support schema names that are enclosed in double quotes.

You must have the following roles/permissions in your Snowflake account to install the Beacons app

• You must use the ACCOUNTADMIN role or another role with the IMPORT SHARE and CREATE DATABASE privileges.
• You can find more information on native apps on Snowflake’s documentation.

External Network Access: InfoSum Beacons require external network access to various endpoints to function. The endpoints requirements have been listed below. Please review these, as you will be asked to approve the integration to enable egress to them once the app has been installed. More information on app permissions:

• InfoSum API endpoint - Allows outbound communication to the InfoSum platform, allowing you to perform queries and activate data with your Beacon
• AWS hosts (S3, KMS & MSK) API endpoints - Required for core Beacon functionality providing secure, cross-region support for coordination of your Beacon(s)
• Sentry - Application error reporting & performance monitoring to aid InfoSum in supporting you, should you run into any technical issues while operating your Beacon
• InfoSum address mapper (optional, only required if you wish to use the US address mapper)

(This step is optional) If you wish to activate data from your Beacon, you will need to provide details of the endpoints that are allowed. You do not need to do this if the activation takes place using your partner’s dataset or if you are only using a Snowflake Beacon Table Destination. Please see this section for more information.

Beacon Deployment

There are two distinct tasks to create a Snowflake Beacon

1. Create a Snowflake Cloud Vault
2. Deploy the Beacon app and link to your Cloud Vault

1. Create a Snowflake Cloud Vault in InfoSum

The first thing you need to do is create a Snowflake Cloud Vault in InfoSum that is connected to your Snowflake account. The Cloud Vault won’t host any data; it will simply provide an interface to prepare your data for publishing to your Snowflake hosted datasets.

Log in to InfoSum, navigate to the data management section, and click on the Cloud Vault page. 

Then click on the ‘manage cloud vaults’ button on the top right-hand side, and then the ‘create new cloud vault’ button on that page. 

Then you will be asked to select your Cloud Vault provider, where your Beacons will be hosted. In this case please select Snowflake Cloud Vault

You will then be asked to give your Cloud Vault a name - this name will identify the data management space if you have multiple.

 You’ll also need to enter:

•  Your Snowflake account: You can find this in your Snowflake account, by clicking on the account admin menu at the bottom left, and then clicking ‘view account details’ for the account you want to install the app in. This will open a pop-up that contains your account identifier. 
  
• The Cloud provider your Snowflake account is operating on. We support all cloud deployments currently leveraged by Snowflake (Amazon, Azure and Google)
• The region your cloud account is located in. The dropdown will show the name of Snowflake regions first and the InfoSum naming convention after. You can find the region your account is in in the same account admin menu described above.

This will give you a license key that you will need to authenticate in the Beacons app.

Click on finish and connect Cloud Vault to complete this process.

2. Beacons Marketplace App Deployment in Snowflake

Deploy InfoSum’s Beacons as a marketplace solution deployment into your Snowflake account. This will link your project with the virtual Cloud Vault on your InfoSum account to manage the collaboration operations.

As part of this process, you will point the InfoSum app to the tables that you wish to use for collaboration. We recommend reviewing our formatting guidelines to ensure that your data can be easily normalised and published to a Beacon.

This process is divided into two flows: installation (steps 1-4 below)  and configuration/connection to InfoSum (steps 5-8; instructions for this process can also be found in the app readme in the "about the app" section in Snowflake UI)

Please note that if your organization has already deployed the app once, you will need to create a second instace first following this Snowflake guide.

1. Find the InfoSum Beacons app in Snowflake’s marketplace and click GET ( Catalogue → Apps → InfoSum Beacons)

   https://app.snowflake.com/marketplace/listing/GZ2FRZUX5YS/infosum-infosum-beacons 
   

2. Click and Agree and continue to Snowflake native apps permissions. These are the necessary permissions for the Beacons App to connect with our platform, create datasets, and support collaboration use cases. Read more about these permissions
   
3. Give your application a name or accept the default name and click Get
   Please ensure any custom name meets the requirements below:
   - Up to 63 characters long.
   - Must start and end with an alphabetic character.
    
4. Your app install will commence. This may take a few minutes, when it’s completed, please click on ‘Configure’. 
   
   Once the app is installed, you will be able to find further configuration instructions & scripts in the 'about the app' section of the app. Below is a summary of the steps:
    
5. Configure and connect the Beacons app to InfoSum.  Review and save the external access integrations. These are necessary to process tasks within your Snowflake environment and allow certificate checks with Snowflake, Sentry telemetry, InfoSum platform communication, access to InfoSum AWS S3 buckets and InfoSum Kafka brokers. 
   You can also allow access to US address mapper if you are leveraging this service.
   Click, save and proceed to app
   
   If you need to review any of these settings, they can be found in the configurations tab of the app instance.

6. Connect the App to InfoSum. Open the installation script in a new workspace and fill in the details. You can find this script in the 'about the app' section of the app:
   
   The database/schema name must be provided in UPPERCASE. We do not yet support names that are enclosed in double quotes. If your schema name is encased in double quotes, you can rename it to contain only alphanumeric characters.
    
   1. Replace <YOUR_DATABASE> with the name of the database you want the app to use - likely the one created in the Adding Data to Snowflake section above. This can be found in the list of objects in the left-hand menu next to your worksheet.
   2. Replace <YOUR_SCHEMA> with the name of the schema within the above database you want the app to use - likely the one created in the Adding Data to Snowflake section above. This can be found in the list of objects in the left-hand menu next to your worksheet.
   3. Replace <APP_INSTALL_NAME> with the name that the app was installed as, this was set when also selecting the release channel and can be found in the list of installed apps, or in the list of objects in the left-hand menu next to your worksheet.
   4. Add the licence key of the Cloud Vault created at the start of this section to the CONFIGURE_APP() section of the workspace.
   5. You will notice a grant on tables, this allows the app access to the existing tables in the the schema. 
      1. If you have views you want to use as well as tables, then copy that grant command and change TABLES to VIEWS. 
      2. This does not allow the app access to any future created tables/views, if you add anything new after these commands have been run that you want the Beacon to use, you will have to re-run the grant commands.

7. Ensure you have selected a warehouse to run this script in, and click run (blue play button)
   
   
8. Once the queries have completed successfully, you’ll be able to see a confirmation in the results pane.
   
    

At the end of the deployment, you will see that the connection has been established with our platform if the ‘external access integration’ connection shows as ‘reviewed’.

You will also see that your Snowflake cloud vault on the platform has a  "Healthy" status once the connection is established. Please allow for some time for this to complete.

Next step

Once the above steps have been completed successfully, you will find in your Cloud Vault a reference to any tables contained inside the dataset connected to the Beacons app and you will be able to complete the data preparation steps to start collaborating. 

Setting up a Beacon for Destination Exports

This step is optional and only needs to be completed if you wish to export data to a third-party Destination using a Beacon based Dataset. 

• If you are only using a Snowflake Beacon Table Destination, you do not need to complete the steps below, as this destination can only write data from your Beacon Dataset back into your own Snowflake account. 
• If you are only using your partner's data for activation and no data will be exported from your Dataset, then you do not need to complete these steps.

To allow our software to communicate outside of the walls of your Snowflake account, you must set up the hosts that are allowed and confirm the configuration on the connections tab.

You have two options:

1. Create one exception for all Destinations (recommended): This means that activations are controlled via the InfoSum platform. To complete an export, you will need to set up each Beacon to contain export columns, grant or receive permissions that allow activation, and have an allowed and available Destination to export to.
2. Create one exception per Destination: If you wish to further restrict the approved Destinations, you can set up your rule exception to point to a specific server or third-party location. You will need to create one rule exception per destination. Activations will still be controlled via the InfoSum platform as described above. 

Open the Optional configuration installation script in a new workspace to configure the connections (You can find this in the read me section of the app):

1. Enter your host domains. 

• If you wish to restrict the allowed Destinations to a specific server, for example you may manage your own SFTP server with a specific public IP address, in which case you should enter only the IP of the server you wish the Beacon to export to. 
  • If unsure about which network range to allow here, please consult a relevant Snowflake or network administrator within your Organization
• If you wish to create one rule for all Destinations, please set a range of 0.0.0.0

Please note: the host:port must be wrapped in double quotes, and then that wrapped in single quotes.

2. Run the script - please ensure you have selected a warehouse

3. Navigate to the connections section of your app to review the External Access Integration connection. Click on review, then click on ‘connect’

Your Snowflake Beacon is now set up to allow data activation to your chosen hosts. Please remember you will still need to set up the specific Destination details and access credentials in the InfoSum platform.

Deleting your Beacon deployment

In the Snowflake UI:

• Go to Catalog → Apps → InfoSum Beacons Internal Prod
• Click the button with three little dots in the top right corner
• Click “Uninstall”  

In InfoSum platform:

• Delete the Cloud Vault associated with this Beacon

App Permissions 

Below is a list of the privileges that the InfoSum Beacons app requests on install and what they are used for:

Once the app is installed, you will also need to manually grant the app with read priviledges for the database, schema and tables (and views) that you are going to use.