How to configure a scheduled task data table export

Learn how to set up a scheduled task to automatically export data from a data table.

This task is designed to automate data export to external systems or to feed internal processes. Use it when, for example:

• you need to sync data with a CRM, a data warehouse, or BI tools
• you need to generate periodic files for partners or vendors
• you want to export only new or updated data, avoiding exporting the entire table each time

Configuration is flexible: you can filter data, transform it, and define the file format and destination to fit your needs.

Creation

Go to Settings > Scheduled tasks to view the list of scheduled tasks, which will be empty the first time you access it.

Click the New scheduled task button at the top right and define:

• Task name
• Context: Data tables
• Task type: Data table export

Then click Save, select the table you want to export, and save again. 

After creating the task, you will be directed to the first step of the configuration, “Settings,” where you will set the general configurations of the task.

On the right, you'll find the "Tasks" menu that will guide you through all configuration steps:

• Settings
• Filter on data
• Fields to export
• File parameters
• File destination

Settings

In this section you define the general settings of your task. See the article How to create a scheduled task: general settings for details.

Specifically for this scheduled task type, select the data table from which you want to automatically export data.

If you already have a file layout exported from another task (“Export configuration”), you can reuse it with Import configuration. You can upload the file via drag and drop, your account FTP or the File sharing section.

If, instead, you need to configure the data table export task from scratch, continue reading the following sections of this article.

Data filter

If you don’t want to export the entire data table, you can apply filters here by writing a query to limit the table data, working only on rows and not on columns, which you will configure in the next step. 
This way you can reduce the volume of exported data and make downstream integrations more efficient, working only on what you need.

You can set simple conditions or combine them, for example to export only orders with an amount above a certain threshold or with a purchase date after a specific point in time, and run a query test using the Calculate  the number of records button.

This configuration is typically used with $LAST_EXECUTION for incremental exports.
On the first run it is not set, so you export all records. From the following runs it is automatically populated with the date and time of the last execution. This way, you can filter only new or updated records and work by delta, avoiding re-exporting the entire dataset.
This approach is useful when you need to integrate magnews with external systems: by exporting only deltas you avoid duplicates and reduce processing time and load.

Fields to export

In this section, define the information to export: the columns correspond to the fields of the data table. 
You have full control over the file structure: you can adapt it to the format required by the system that will receive the data, without needing to intervene manually after the export.

Click Add a column for each field you want to include in the file.

For each field you can:

• Use the hand-and-note icon to manage the transformation of a field. 
  Transformation means the ability to automatically modify the values in the output file to convert them as needed. 
  For example, you can change a date format or use Value mapping: if you have a field with a list of interests (Sports, Fashion, Travel) but you need numeric codes in the exported file, you can map each value to a number, for example Sports = 1, Fashion = 2, Travel = 3
• Use the trash icon to remove the column
• Define the column order by changing its number, so they are ordered in the exported file as you prefer (e.g., column "1" will be first, column "2" will be second, and so on)

File parameters

If you have specific needs, define the characteristics of the export file; otherwise, you can move to the next step.

Here you define the final file format based on the requirements of the target system or the process that will use it.

• File name: you can use placeholders, including $OPERATIONNAME, $STARTTIMESTAMP and $OUTPUTFILEEXTENSION
• File format: you can choose among .csv, .xlsx, fixed-width file
• Remove spaces at the begining and at the end of each value
• Insert the columns headlines in the first row

If you chose to export a .csv file, you must also choose the options:

• Optimized for Excel: if enabled, adds a header to the .csv to ensure it opens correctly in Microsoft Excel
• Line separator
• Delimiters
• Text qualifier
• Text encoding
• Use compression
• Choose the type of compression: works only if you enabled the "Use compression" option

If you chose to export a fixed-width file, you must also choose the options:

• Length of the fields
• Padding character
• Line separator
• Text encoding
• Use compression

File destination

You can integrate the export directly with external systems, avoiding manual download and upload steps.

Choose where to save the generated file:

• FTP of this account, if the file must be hosted in magnews.
  In this case, select an internal FTP folder in the account where to save the file
• Remote FTP server, if the file must be deposited on a server external to magnews.
  In this case, select the remote server and the folder in it where to save the file
• Google Cloud Storage, if the file must be deposited on a Google file server.
  In this case, select the Google file server and the prefix for processed files

Task execution

Once you have set all parameters, click Save.

If you want to run it immediately, click Run now: we will ask you to confirm the immediate start of the execution.
When it ends, a detailed report informs you of the outcome. This allows you to quickly verify that the export completed successfully, especially in automated tasks.