Articles in this section

See more

Add Fields - "append"

Avatar
Tom Sherwood
  • Updated
Follow

Appends data from another stream or subtable to the current stream. Identical columns will be concatenated (Example 1), whilst new columns are added by default (Example 2), behaving identically to an "outer join".

Streams: The complete name(s) of the Datastream(s) you wish to be appended.
Subtables: The complete name(s) of the Subtable(s) you wish to be appended.
Meltdown: Append extracts from current datastream. Can be combined with tags.
All: If ticked, all extracts will be appended to the dataset. If unticked, only the most recent extract will be appended.
Ignore Join Date: Set to false to join only the extracts whose scheduled date is earlier than "current_extract_date". In the event of a conflict, rules set for tags will override join date.
Include Tags: Only append streams containing the tag(s) specified here. 
Exclude Tags: Exclude all streams containing the tag(s) specified here.
Errors: Specify the action to be taken if an error is encountered during the append process. Defaults to "strict".
Subtable: Perform the transformation within a subtable, named as per this field.

 

Example 1

append_example_1.png
Example Input 1
Table 1
dim_A dim_B metric_1
AAA DDD 111
BBB EEE 222
CCC FFF 333
Table 2
dim_A dim_B metric_1
GGG JJJ 444
HHH KKK 555
III LLL 666
Example Output 1
New Unified Table
dim_A dim_B metric_1
AAA DDD 111
BBB EEE 222
CCC FFF 333
GGG JJJ 444
HHH KKK 555
III LLL 666

 

Example 2

append_example_1.png
Example Input 2
Table 1
dim_A dim_B metric_1
AAA DDD 111
BBB EEE 222
CCC FFF 333
Table 2
dim_C dim_D metric_2
GGG JJJ 444
HHH KKK 555
III LLL 666
Example Output 2
New Unified Table
dim_A dim_B metric_1 dim_C dim_D metric_2
AAA DDD 111      
BBB EEE 222      
CCC FFF 333      
      GGG JJJ 444
      HHH KKK 555
      III LLL 666

 

I Ain't Afraid of No Code!

[
    [
        "append",
        {
            "all": false,
            "errors": "strict",
            "ignore_join_date": true,
            "include_tags": null,
            "subtables": null,
            "subtable": null,
            "streams": [
                "Table 2"
            ],
            "exclude_tags": null,
            "meltdown": false,
            "_comment": null
        }
    ]
]

param boolean all: If set to true, all extracts will be appended.

param str errors: ignore or raise errors.

param list include_tags: Append only streams with given tags. Without include_tags nor exclude_tags given, the most recent extract of each stream/subtable will be appended.

param list streams: Data from which datastreams should be appended.

param list subtables: Data from which subtables should be appended.

param list exclude_tags: Exclude all streams with given tags. Exclude has precedence over append.

param str comment: A comment that will appear in the Transformation UI.

Related articles

Comments

0 comments

Article is closed for comments.