Leverage data tables with the “New data event” and “Data periodic filter” nodes

Event-type nodes that work with data tables let you start automated flows based on records that are external to the contact profile, such as orders, policies, registrations, or any other related dataset.

You can use these nodes to create data-driven automations, react to events occurring in your external systems, and personalize communications using the information stored in data tables.

Some common examples include:

• Starting repurchase journeys using data from an eCommerce order history
• Sending reminders before an insurance policy expires using data stored in a data table
• Personalizing communications with information contained in individual records, such as purchased products, policy numbers, or other specific data

In this article, we'll cover:

• The New data event node, which reacts to the insertion of new records
• The Data periodic filter node, which runs periodic checks on data tables to identify relevant records

If you don't see the New data event or Data periodic filter nodes in the automated flow designer, make sure that the following are available:

• At least one data table
• At least one association field between a contact and a contact extension

These nodes become available only after contacts have been linked to a data table through an association field. Learn how to do it.

New data event

Use this node when you want to start an automated flow after new records are created in a data table, such as new orders, new registrations, or new policies.

You can configure the node in two ways: simple mode and advanced mode.

The node starts working only after the automated flow is activated.
Since this is an event-based trigger, you cannot manually test the node before activation: the test corresponds to a real automated flow session.

After activation, if you do not immediately see contacts entering the automated flow, wait a few minutes: processing new records may not be instantaneous.

Simple mode

Configure the node in simple mode:

• Select the association field that links the contact to the data table. For example, for a trigger based on the orders table, select the contact field that links the contact to their ordered items.
• Filter data table records so you do not consider every record indiscriminately, but only the ones you need. For example, filter orders from the last 90 days.

• Specify the field that contains the element date, meaning the date when the record was created in the data table. The field must be of type "Date and time", not "Date".
  The node considers as "new events" only records with a date later than the activation of the automated flow. For this reason, contacts associated with records dated before the activation will not enter the automated flow.
  If the data table does not contain a field of this type, use the Data periodic filter node.

You can also select the data table fields to make available within the automated flow session so you can use record values in other automated flow nodes or in communication content.
For each selected field, magnews automatically creates an automated flow variable composed of the table name and the field name.
For example, if the table is orders and the field is ordernumber, the variable orders_ordernumber is created.

Variables become available after the automated flow is activated.

You can use these variables:

• In a condition check node expression, using the syntax workflow.values.orders_ordernumber
• In communication content, using the placeholder [workflow:orders_ordernumber]

This lets you, for example, personalize an email with order information or create different paths in the automated flow based on values stored in the data table.

Finally, you can choose whether only subscribed contacts can enter the automated flow.

Advanced mode

Advanced mode gives you more control over the node configuration.
Use it when you want to explicitly choose the database from which contacts can enter the automated flow and specify which data table field identifies the contact associated with each record.

Configure the node by specifying:

• The database the contact belongs to and that will be used to enter the automated flow
• The data table to use
• The data table field that identifies the contact
• The field that contains the record date, used by the automated flow to identify new records.
  In this case as well, the field must be of type "Date and time" and its value must be later than the automated flow activation date.
  If the data table does not contain a field of this type, use the Periodic data filter node.

You can also:

• Add filters to consider only specific records from the data table
• Select the fields to use within the automated flow session, making them available in other nodes and communication content
• Choose whether only contacts subscribed to the selected database can enter the automated flow

Filters and session fields work exactly as described for simple mode.

Is the automated flow active but not receiving sessions?

If no contacts are entering the automated flow after activation, check that:

• The field selected as the event date is of type "Date and time"
• New records have a date later than the activation of the automated flow
• Any filters configured in the node are matched by the new records
• If you enabled the "Contact enters only if subscribed" option, the contact is subscribed to the database
• If you are using advanced mode, the contact associated with the record exists in the selected database

Data periodic filter

It works on the contents of data tables by running scheduled checks over time. Unlike the New data event node, it doesn’t trigger when a new record is created, but instead periodically filters the data table to identify records that meet specific conditions.

You can use two configuration modes: simple and advanced.

Simple mode

Configure the node in simple mode:

• Select the association field that links the contact to the data table. For example, for a trigger on the orders table, select the contact field that connects the contact to their ordered items.
• Filter the data table items, so you don’t process all records indiscriminately, but only specific ones. For example, filter today’s hoodie orders.
• Choose the data table fields you want to use in the workflow (for example, in conditions or content personalization elements). For each selected field, a workflow variable containing that information will be created.
• Configure the execution frequency of the filter, using the same parameters available in the “Periodic filter” node.

Advanced mode

Advanced mode lets you manage more complex data models. 
Make sure to specify in the SELECT query the fields you need and the field that identifies the contact; you can filter the desired records directly in the query by adding a WHERE clause.