Connector | Mailgun

USE CASES

If you wish to send a report or similar source of data and import it to Datatap via email. It is particularly useful to automate data import of a regularly issued report, but individual emails can also be manually imported on an ad hoc basis.

DATASTREAM CREATION

Request Connection from CS dept:

• Either we will create one using Adverity Mailgun Account, then user creates a Datastream using that connection
• Alternatively, they can use their own Mailgun account, but we still need to set it up for them.

You will need to follow the basic steps of creating a new Connection and Datastream, with the following notable variances:

• By default, Adverity will supply the API key required to authorize the connection, rather than login using username/password.
  
  If you wish to use your own API key, you till need to contact your Account Manager. You may also wish to see Mailgun's Documentation. 

MAILGUN-SPECIFIC CONFIGURATION OPTIONS

Configuration

• Warnings: 
  If a fetch finds no available file or valid URL link, a warning will be sent to the user via the 'Issues' pane in the Overview.
• Auto Fetch:
  If ticked, Datatap will automatically run a fetch action every time an email is received by the Recipient email address.

Recipient

The recipient field informs the system of the email address to which all received email should be routed. An email address will be auto-generated upon creation of the Datastream.

File Matching

• File Pattern:
  A regex expression that will be matched to recognize attachment files.
• Zip Match:
  The file-type suffix for which Datatap should search any attached .zip files.
• Filename Date Match:
  A regex expression used to match the filename date of an attachment file, e.g. "(\d{4}-\d{2}-\d{2})"
• Filename Date Pattern:
  If attached files follow a consistent name format to define e.g. their date of creation, this field defines the date expression format that will be used, e.g. "filename-%Y-%m-%d".
• Keep filename:
  If ticked, the extract will duplicate the name of the original source file. If the source file is always of the same name, ticking this would cause each extract to also be identically named, and therefore consistently be overwritten as a new file with each new fetch.
• Has Adverity Header
  If the Datasource is also an Adverity product, then ticking this will automatically allow the Mailgun Datastream to parse header information in the standard Adverity format.

File Parsing

• Parse:
  A drop-down menu of compatible file formats compatible, select the appropriate option for your attachment files, RAW, CSV, EXCEL, AVRO, or PARQUET.
• Source Encoding (RAW, CSV, EXCEL, AVRO, PARQUET):
  Select the encoding convention used by your file structure.
• Delimiter (CSV):
  A one-character string used to separate fields.
• Quote Char (CSV):
  A one-character string used to quote fields containing special characters, such as the delimiter or quotechar, or which contain new-line characters.
• Quoting (CSV):
  Controls when quotes should be recognized when parsing files. 
• Sheet (EXCEL):
  If your attachments are in an excel file that utilizes multiple sheets, specify the one that should be parsed (e.g. "Sheet1", "Conversion Figures").
• Column Offset (EXCEL):
  Allows you to skip columns in the sheet that should not be imported, e.g. if Column A contains header information unnecessary to Datatap.
• Row Offset (RAW, CSV, EXCEL, AVRO, PARQUET):
  Allows you to skip rows in the sheet that should not be imported, e.g. if Row 1 contains header information unnecessary to Datatap.

Email Processing

• Store Only:
  Tick this option to skip all data processing, and store the extract in its raw format.
• Ignore Errors:
  If ticked, no error messages will be generated if Datatap receives an email containing no suitable attachment.
• Extract URLs from body:
  If ticked, Datatap will parse URLs in the email body and access them as Datasources to extract data in the same way it would an attachment to the email body.
• Extract from HTML body:
  If ticked, parse the html body text for anchor text instead of only strict URL matches.
  
  

EVENTS

The events tab keeps a log of all historic activity on the Mailgun Datastream.

LIMITATIONS

Mailgun only retains a 3-day email history on-server.