These options are only functional, and therefore relevant, for Datastreams with a status of "Enabled" and which use one of the following two destinations:
- SQL Database
- Adverity Insights
For SQL Databases, Datatap automatically creates 3 additional columns in the target table:
- dt_created (creation date of current extract).
- dt_updated (date when current extract was last updated).
- dt_filename (filename containing datastream-slug [gsc-104] and hash or custom filename, e.g. gsc-104-20180708-64ecec6ab9c94a29bd72b24deeb1bfd0.csv. The Datastream slug works as the unique identifier in each instance.
- Key Columns:
Can be defined in Schema Mapping. This setting is only recommended when using 1:1 Datastream : Database Table relationships, and Key Columns always have precedence over other criteria. As soon as a column is marked as a key column within a Datastream's schema mapping, it will be used as a primary database key, upon which updates will be based.[Default=Disabled]
Will delete all rows from particular Datastream. The Datastream is recognized by its Datastream slug --> all rows in column dt_filename starting with datastream-slug == gsc-104 will be deleted. [Default=Enbabled]
Uses current filename and deletes all rows in column dt_filename holding the same value, e.g. gsc-104-20180708-64ecec6ab9c94a29bd72b24deeb1bfd0.csv. [Default=Disabled]
- Date Range:
Uses selected date column and overwrites data for current extract's earliest to latest date. [Default=Enabled]
- Regular: Datastream + Date Range delete data from same Datastream within same date range. Quick solution that behaves similar to default Adverity Insights.
- Advanced: Key Columns without any other option enabled.
Can be used whenever ALL old extracts from same Datastream should be deleted from table.
- Filename + Unique Retention Per Day:
Has the same effect as the regular case.
- Key Columns:
Can be defined in Schema Mapping and are equally treated with other options. This is very process intense, and so we advise you avoid using it unless absolutely necessary. [Default=Disabled]
Will delete all rows from particular Datastream. Adverity Insights determines this by looking at the Datatap Endpoint. [Default=Enabled]
Uses current filename and deletes all rows holding the same, e.g. gsc-104-20180708-64ecec6ab9c94a29bd72b24deeb1bfd0.csv. [Default=Disabled]
- Date Range:
Uses the column mapped to 'day' to determine date range to overwrite. [Default=Enabled]
Common Use Cases:
- Datastream + Date Range:
Delete data from same Datastream within same date range.
Can be used whenever ALL old extracts from same Datastream should be deleted.
Setting key columns to import to Adverity Insights should only be used if absolutely necessary, as they require additional import time and resources. See below for the fastest import options:
- Datastream + date range
- Datastream + date range + key columns
- Datastream + key columns