Leverage data tables with the “New data event” and “Data periodic filter” nodes

Event-type nodes that work with data tables let you start automated flows based on records that are external to the contact profile, such as orders, policies, registrations, or any other related dataset.

You can use these nodes to create data-driven automations, react to events occurring in your external systems, and personalize communications using the information stored in data tables.

Some common examples include:

• Starting repurchase journeys using data from an eCommerce order history
• Sending reminders before an insurance policy expires using data stored in a data table
• Personalizing communications with information contained in individual records, such as purchased products, policy numbers, or other specific data

In this article, we'll cover:

• The New data event node, which reacts to the insertion of new records
• The Data periodic filter node, which runs periodic checks on data tables to identify relevant records

New data event

Use this node when you want to start an automated flow after new records are created in a data table, such as new orders, new registrations, or new policies.

You can configure the node in two ways: simple mode and advanced mode.

The node starts working only after the automated flow is activated.
Since this is an event-based trigger, you cannot manually test the node before activation: the test corresponds to a real automated flow session.

After activation, if you do not immediately see contacts entering the automated flow, wait a few minutes: processing new records may not be instantaneous.

Simple mode

Configure the node in simple mode:

• Select the association field that links the contact to the data table. For example, for a trigger based on the orders table, select the contact field that links the contact to their ordered items.
• Filter data table records so you do not consider every record indiscriminately, but only the ones you need. For example, filter orders from the last 90 days.
• Specify the field that contains the element date, meaning the record creation date in the data table.
  The node considers records as "new events" only when they have a date later than the automated flow activation date. For this reason, the field used as the event date must be of type "Date and time" and not "Date".
  If the data table does not contain a field of this type, use the Data periodic filter node.

You can also select the data table fields to make available within the automated flow session so you can use record values in other automated flow nodes or in communication content.
For each selected field, magnews automatically creates an automated flow variable composed of the table name and the field name.
For example, if the table is orders and the field is ordernumber, the variable orders_ordernumber is created.

Variables become available after the automated flow is activated.

You can use these variables:

• In a condition check node expression, using the syntax workflow.values.orders_ordernumber
• In communication content, using the placeholder [workflow:orders_ordernumber]

This lets you, for example, personalize an email with order information or create different paths in the automated flow based on values stored in the data table.

Finally, you can choose whether only subscribed contacts can enter the automated flow.

Advanced mode

Advanced mode lets you configure the node even when you do not have a contact extension.

Configure the node by specifying:

• The database the contact belongs to and that will be used to enter the automated flow
• The data table to use
• The data table field that identifies the contact
• The field that contains the record date, used by the automated flow to identify new records.
  In this case as well, the field must be of type "Date and time" and its value must be later than the automated flow activation date.
  If the data table does not contain a field of this type, use the Periodic data filter node.

You can also:

• Add filters to consider only specific records from the data table
• Select the fields to use within the automated flow session, making them available in other nodes and communication content
• Choose whether only contacts subscribed to the selected database can enter the automated flow

Filters and session fields work exactly as described for simple mode.

Data periodic filter

It works on the contents of data tables by running scheduled checks over time. Unlike the New data event node, it doesn’t trigger when a new record is created, but instead periodically filters the data table to identify records that meet specific conditions.

You can use two configuration modes: simple and advanced.

Simple mode

Configure the node in simple mode:

• Select the association field that links the contact to the data table. For example, for a trigger on the orders table, select the contact field that connects the contact to their ordered items.
• Filter the data table items, so you don’t process all records indiscriminately, but only specific ones. For example, filter today’s hoodie orders.
• Choose the data table fields you want to use in the workflow (for example, in conditions or content personalization elements). For each selected field, a workflow variable containing that information will be created.
• Configure the execution frequency of the filter, using the same parameters available in the “Periodic filter” node.

Advanced mode

Advanced mode lets you manage more complex data models. 
Make sure to specify in the SELECT query the fields you need and the field that identifies the contact; you can filter the desired records directly in the query by adding a WHERE clause.