In this article, we'll talk you through the main components of InfoSum Platform.
InfoSum Platform is the name of our complete solution for making privacy-safe connections between disparate data. It consists of several main components:
- Dataset Manager, which lets you manage and control access to your datasets
- Bunker, which lets you import data and convert it to InfoSum's Global Schema
- Query Tool, which helps you build, test and run Insight Query Language (IQL) queries
- Platform API, which lets you run queries using a flexible easy-to-integrate interface.
The most important thing to remember is that you import datasets and manage collaboration through a web-based UI, then have the option to perform queries programmatically.
A dataset is a single group of records - imported, for example, from a single database table. Almost any meaningful project will involve two (or more) datasets - either two datasets you own, or one which belongs to you and one which belongs to a collaborator.
The web-based Dataset Manager will be your entry-point to any InfoSum Platform project. Using Dataset Manager, you can:
- oversee the process of importing and fine-tuning datasets
- invite other organizations to collaborate with you, by allowing them to reference your datasets in anonymised, aggregate form
- and conversely, see which other organisations are allowing you to reference their datasets.
At the same time, your collaborator(s) will also use Dataset Manager to manage the process of importing their own data - and to give you permission to reference it in anonymised, aggregate form.
A Bunker is the secure storage allocated to a single dataset. When you use Dataset Manager to configure a new dataset, you'll be given a separate Bunker just for that data. And if you import several datasets, you'll have several independent Bunkers, all administered through Dataset Manager.
Each Bunker is hosted on its own private virtual server. Nobody except you - not even InfoSum - can access the raw data in your Bunker. You can give other users permission to query your dataset, but they will only be able to retrieve aggregated statistical results, never the original data.
Bunker has its own web-based interface, separate from Dataset Manager, where you can:
- import a dataset, from an existing database or CSV file
- fine-tune your data, to clean up inconsistencies or messy record-keeping
- tell InfoSum's software what each column in your dataset represents
- manage data versions, updating your dataset if new source material becomes available.
Your collaborator(s) will also import their data into their own private Bunkers. Although your collaborators give you permission to reference their data in anonymised, aggregate form, you have no access to the raw data in their Bunkers - and they have no access to yours.
The Query Tool is an interactive UI within Dataset Manager, which you can use to build, test and run queries in the Insight Query Language (IQL).
IQL is InfoSum's specialist tool for analysing data across multiple linked datasets. It helps you explore datasets in detail, generating a wealth of aggregated statistics. The original data remains securely in each party's Bunker, and is never shared or revealed. For example, using IQL, you can:
- gain insights into customers which you and your collaborator share - perhaps to profile their geographic location or ages
- statistically analyse your customer base, by using a collaborator's data to enrich your own - without revealing or processing any individual customer's data
- narrow down statistics to particular areas of interest - for example to look at people who live in a specific town or region.
The Query Tool helps you explore the datasets you can reference and explain your query - showing an interactive visualisation of the steps InfoSum Platform will use to execute it.
The explain tool helps you debug and fine-tune your query, potentially improving the quality of the result. Although you can see your query results within the Query Tool, you can also take the finished query and run it using the Platform API.
The Platform API is a programmatic interface, used to integrate InfoSum Platform into a larger solution. It is a simple RESTful interface which you can access using almost any scripting or programming language.
The Platform API's key feature is its ability to execute queries written in the IQL language. For example, you could use the API to run a query each day against potentially changing data, and report the results to an external application.
You can also use the API to find out which datasets are available for you to reference, and to get basic information about how the datasets relate to each other - for example, how large or small the intersection between them is.
You cannot create or manage datasets, or import data, using the Platform API. For these tasks, use Dataset Manager.