Mailgun

USE CASES

If you wish to send a report or similar source of data and import it to Adverity via email. It is particularly useful to automate data import of a regularly issued report, but individual emails can also be manually imported on an ad hoc basis.

DATASTREAM CREATION

You will need to contact your Adverity Account Manager to request a Connection setup. If required, Adverity can setup the Connection using your own Mailgun account and API key, you can discuss this with your AM.

Once the Connection is functional, you will need to follow the basic steps of creating a new Datastream, with the following notable variances:

• By default, Adverity will supply the API key required to authorize the connection, rather than login using username/password. You may also wish to see Mailgun's Documentation. 

MAILGUN-SPECIFIC CONFIGURATION OPTIONS

Configuration

• Warnings: 
  If a fetch finds no available file or valid URL link, a warning will be sent to the user via the 'Issues' pane in the Overview.
• Auto Fetch:
  If ticked, Adverity will automatically run a fetch action every time an email is received by the Recipient email address.

Recipient

The recipient field informs the system of the email address to which all received email should be routed. An email address will be auto-generated upon creation of the Datastream.

File Matching

• File Pattern:
  A regex expression that will be matched to recognize attachment files.
• Zip Match:
  The file-type suffix for which Adverity should search any attached .zip files.
• Filename Date Match:
  A regex expression used to match the filename date of an attachment file, e.g. "(\d{4}-\d{2}-\d{2})"
• Filename Date Pattern:
  If attached files follow a consistent name format to define e.g. their date of creation, this field defines the date expression format that will be used, e.g. "filename-%Y-%m-%d".
• Keep filename:
  If ticked, the extract will duplicate the name of the original source file. If the source file is always of the same name, ticking this would cause each extract to also be identically named, and therefore consistently be overwritten as a new file with each new fetch.
• Has Adverity Header
  If the Datasource is also an Adverity product, then ticking this will automatically allow the Mailgun Datastream to parse header information in the standard Adverity format.

File Parsing

These options should be set to match the format of the Datasource file attached / linked to in the Mailgun email.

• Parse:
  A drop-down menu of compatible file formats compatible, select the appropriate option for your Datasource files, RAW, CSV, EXCEL, AVRO, PARQUET, JSON or NDJSON. Further options will appear based on the file format selected:
  • Source Encoding (RAW, CSV, EXCEL, AVRO, PARQUET):
    Select the encoding convention used by your file structure.
  • Delimiter (CSV):
    A one-character string used to separate fields.
  • Quote Char (CSV):
    A one-character string used to quote fields containing special characters, such as the delimiter or quotechar, or which contain new-line characters.
  • Quoting (CSV):
    Controls when quotes should be recognized when parsing files. 
  • Sheet (EXCEL):
    If your attachments are in an excel file that utilizes multiple sheets, specify the one that should be parsed (e.g. "Sheet1", "Conversion Figures").
  • Column Offset (EXCEL):
    Allows you to skip columns in the sheet that should not be imported, e.g. if Column A contains header information unnecessary to Adverity.
  • Row Offset (RAW, CSV, EXCEL, AVRO, PARQUET, JSON, NDJSON):
    Allows you to skip rows in the sheet that should not be imported, e.g. if Row 1 contains header information unnecessary to Adverity.

Email Processing

Control how the Mailgun email will be formatted e.g. files attached to the email, or URL links included in the email body.

• Store Only:
  Tick this option to skip all data processing, and store the extract in its raw format.
• Ignore Errors:
  If ticked, no error messages will be generated if Adverity receives an email containing no suitable attachment.
• Extract URLs from body:
  Tick this if you include URL links in the email body text that direct to the Datasource file location. Adverity will parse URLs in the email body and access them as Datasources to extract data in the same way it would a file attached to the email.
• Extract from HTML body:
  If ticked, parse the html body text for anchor text instead of only strict URL matches.

EVENTS

The events tab keeps a log of all historic activity on the Mailgun Datastream.

LIMITATIONS

Mailgun only retains a 3-day email history on-server.