While setting up a stream, under Scheduling and Time Range you can find options to specify the time interval included in each extract and the frequency of the fetches. By default, when automatic fetches are first activated, they will occur once per day, fetching a typical range for that Datastream based on previous average preferences. "Time of Day" will be semi-randomized at a time between 00:00 and 04:00, this is in order to prevent multiple streams all triggering at the exact same time and creating bottlenecks.
Below, you can find a description about the options available, along with examples on certain setups and their expected result.
SCHEDULING
- Frequency:
How often the fetch should run automatically.
E.g.: "Every week" --> Will run at the same time once per week. - Weekday / Day of Month:
These fields will appear dynamically when the appropriate "Frequency" option is selected.
e.g. If frequency is set to "Every Month" as in the example above, "Day of Month" will appear to define upon which day of the month it will occur, in the above case every 1st of the month. - Time of day:
What time of day the stream will execute the automatic fetch. Due to the busyness and workload of the API, it is recommended to run fetches overnight, or in the early hours of the day.
TIME RANGE
- Fixed Time Range:
Will always fetch the same time range specified. - Today:
Will always provide results for the same day as the fetch is conducted, using the most recent data available up until the time of fetch.
e.g. Assuming a 1-hour delay to record results, a fetch for "Today" executed at 23:00 will produce results for 00:00 - 22:00. - Yesterday:
Will always provide results for the last complete 24-hour day. - Current "X":
Refers to the ongoing unit of time referred to in "X".
e.g. "Current Week" on a Wednesday will provide results for Monday - Wednesday. - Previous "X":
Refers to the most recent completed unit of time "X".
e.g. "Previous Week" on a Wednesday will provide results for the preceding Monday - Sunday. - Last "X":
"Last" differs from the above as it makes an absolute measurement of time, regardless of weeks, months etc.
e.g. "Last 7 days" on a Wednesday will provide results from last Wednesday - this Tuesday (inclusive). - Lifetime:
Delivers all data available from the Datasource (not compatible with all Datasources, drop-down list will auto-populate based on availability).
TIME RANGE CONFIGURATION, EXPLAINED:
Let's say that your Datastream is configured to run automatically on a daily basis (Scheduling - "run every day") and to request 1 day's data (Time Range - "Previous Day"). This means your stream will run every day at certain time, and each extract will contain data for the previous day.
If you wanted to manually fetch historical data on Google Analytics for a period from 2018-01-02 to 2018-01-08, your fetch would produce 7 extracts each containing one day's data:
| Extract | Date in the extract |
|---|---|
| #1 | 2018-01-01 |
| #2 | 2018-01-02 |
| #3 | 2018-01-03 |
| #4 | 2018-01-04 |
| #5 | 2018-01-05 |
| #6 | 2018-01-06 |
| #7 | 2018-01-07 |
Therefore, for each day in your custom-defined range, it will simulate one fetch requesting the date range defined in the settings (one day's data).
Your Datastream is configured to run automatically every week on Monday (Frequency - "Every week" and to request data for the previous 7 days (Time Range - "Last 7 days"). This means your stream will run every Monday at a certain time, and each extract will contain data from the last 7 calendar days.
If you wanted to manually fetch historical data for the first 4 weeks in January, you would select a period from 2018-01-08 to 2018-01-29. This would result in 4 extracts:
| Extract | Date range in the extract |
|---|---|
| #1 | Calendar Week 1 (2018-01-01 to 2018-01-07) |
| #2 | Calendar Week 2 (2018-01-08 to 2018-01-14) |
| #3 | Calendar Week 3 (2018-01-15 to 2018-01-21) |
| #4 | Calendar Week 4 (2018-01-22 to 2018-01-28) |
For each week in your custom defined range, it will simulate one fetch requesting the date range defined in the settings (last weeks data).
Scheduling:
- Frequency: Every Day
- Time of Day: 02:00 (2 am)
- Time range: Yesterday
Result:
The Datastream will execute the fetch automatically every night at 2am UTC, delivering the data from the previous day.
Scheduling:
- Frequency: Every Week
- Weekday: Monday
- Time of day: 02:00 (2 am)
- Time range: Previous Week
Result:
The Datastream will execute the fetch automatically every Monday at 2am UTC, delivering data of the previous week.
Scheduling:
- Frequency: Every Day
- Time of day: 02:00 (2 am)
- Time Range: Last 28 days
Result:
The Datastream will execute the fetch automatically every day delivering data of the previous 28 days. Fetching data for the same day at later points in time is useful to cover retroactive changes that can happen in the last 28 days and provide the latest accurate data.
MULTI-SCHEDULING
It's possible to set separate fetch schedules, using different criteria that will then operate independently of each other.
In this example, this stream would fetch daily reports at 00:00 UTC for the preceding calendar day, but it will also compile a report on the 1st day of each week for the entire week just completed. This example would be useful for someone wanting both immediate feedback, and a consolidated monthly report that would account for delayed attribution caused by interactions made after the initial 24 hours.
Up to 5 separate schedules can be defined.
OFFSET TO EXECUTION TIME:
Defines an offset amount, expressed as number of days from the reference date ("today" as a standard point of reference). Offset is calculated with "time range - X". Negative numbers are valid, and will set a reference point in the future. This enables you to highly customize the schedules.
Offsets Use Case 1 - Data Availability Delay
In the above example, the user wishes to create daily reports, each covering the activity of the preceding 24 hours ("yesterday"). However, what if the Datasource has an inbuilt delay due to processing time, meaning that data for "yesterday" is not actually available for an additional 24 hours? An offset solves this issue, delaying the request by 24 hours when the data is available.
Offsets Use Case 2 - Splitting Historical Data
Let's presume that a user wishes to receive reports once every two weeks, covering the previous 2 weeks data but divided into one report per week.
These two schedules will both trigger on the same date, but the offset will mean that one extract provides the most recent week, and the other will include the preceding week.
Comments
0 comments
Article is closed for comments.