What is Datatap?
At its core, Datatap is a software solution that allows you to collect, manipulate, and analyze data from multiple sources, regardless of original source system and formatting. We are very proud of Datatap's flexibility, and are constantly adding new features and APIs. No matter who your partners or suppliers, or what your technology may be, Datatap probably has a unique and bespoke solution to serve you (and if we don't, it's probably planned). However, this power and flexibility can sometimes make Datatap seem intimidating, and so we created this guide to help you get started with the fundamentals before exploring the full potential of what you can do to truly understand your data in all its complexities and subtleties.
The Aims of Datatap 101
This introductory course will familiarize you with the general concepts of what Datatap can help you achieve, and guide you through the creation of your first data transfer, from your datasource all the way to an example data visualization and analytics platform.
Datastreams and Connections
The basic foundations of Datatap are Datastreams and Connections.
Connections are the virtual handshakes that start varied systems talking to each other, establishing the infrastructure to import data from your datasources.
A Datastream retrieves data from a datasource (API, File Server, etc.) and transforms it into output files (data extracts). There are numerous Datastreams available, all of which can be configured and adapted to best suit your needs, but for the sake of this tutorial we will begin with something simple.
Datatap is an incredibly flexible tool, and every Connection, Datastream, and other assets in the platform are highly configurable to suit your individual needs. In time, Datastreams can be customized to focus and filter metrics however best benefits your analysis. However, for the purpose of these tutorials, please leave all options at default values unless specifically directed otherwise. This will ensure that you get up and running with a complete data through-line as quickly as possible, and customization will be covered in later documentation. Thankfully, Datatap includes a number of out-of-the-box templates and defaults to easily get to grips with the system.
The below flow chart shows a simplified illustration of the data journey, from importing from a Datasource, all the way to export to a destination:
Schema Mapping and Transformations are both optional steps to manipulate data, from its original format to one that unifies all data across sources to allow parity, and therefore much more complete and effective analysis.
Stacks in Datatap represent distinct host servers. Stacks allow physical separation of data that you do not wish combined, such as for legal compliance (e.g. data originating in the EU being stored in an EU-located server).
Each individual Datatap Stack can be further sub-divided into Organizations. These are purely virtual distinctions rather than physical, and allow Datatap users to separate their data for analytical purposes, and equally serve as restrictions to user access, e.g. ensuring employees can access only the data to which they have a legitimate interest.
- Now that we have explained the fundamental concepts, you can move on to Your First Datastream.