Automation Studio - Status Monitoring with Notification
Already struggled with a paused automation that went unnoticed? You are not alone and there is an automation that helps you detect such issues. This article includes everything you need to know including the automation steps and code snippets.
How do automations even get paused?
It is a common pitfall that automations get paused once you open/edit one of the activities within. Users often forget to re-start automations after they are done checking them out or adjusting something. When talking about cricitcal system processes, campaigns that are critical to run on a specific day or automations that prepare your master data that all your segmentations are based on, this can be a huge issue.
How to make sure no critical automation gets paused and is unnoticed?
Consider if one of the following options works for you:
- Set a run completion notification for your most critical automations so you can directly react if you didn’t receive the notification for one of them. However it is hard too keep track if you have many of them.
- Set a run completion notification to go to an automatically monitored inbox with rules that notify you if you didn’t receive one of those emails.
- Create an automation that uses a script activity that checks your automations and notifies you if any one of those is deactivated. You just need to make sure nobody pauses this automation 😉 However, with a combination of this and the first approach, you only have one email to watch out every day instead of a multitude of run completion mails.
In this article I’ll cover option 3 in detail, so you can build it up on your own.
Setting up an “Automation-Status Monitoring” automation
In the background we need two data extensions:
- One containing a list of automations to check (manually populated by you)
- One containing the filtered results (only the automations that aren’t in a desireable state - automatically filled by the automation)
The automation setup is straigthforward and includes just three activities:
- A Script activity utilizing the Marketing Cloud API using WSProxy to check the automation status
- A SQL query activity or Filter activity to filter for automations that have an undesired status
- A verification activity that checks if the SQL returned any automations and sends you a notification in that case
This is how the automation flow looks like when fully configured:
The data extensions in the background
Both data extensions have the same structure and include only three columns where one is a helper column and not necessary from a technical perspective. The fields are as follows:
- Key (Text (255), Primary Key) → contains the external key of the automation
- Status (Text (50), Nullable) → will be filled by the script activity
- Name (Text (255), Nullable) → only for your reference to easily see which automation is impacted
Data Extension Names (also set as external keys for the respective data extension):
If you use my example code below you in an unaltered form, you need to use the exact names for the fields and Data Extensions.
Using Contact Builder, insert all the automations you’d like to monitor in the “Automation_Notification” data extension with External Key and Name of the automation.
The script activity
The script activity is where the magic happens. It fetches the automations to monitor from the “Automation_Notification” data extension, then retrieves the status via WSProxy and the Marketing Cloud API and writes it back to the data extension.
This is the code of the script activity:
The SQL activity
This activity just filters for the status values you want to get notified about. The result is written to the data extension “Automation_Notification_Results” using the “Overwrite” data action. As I monitor both scheduled and file drop automations and am only interested in the unexpectedly paused ones, I chose to filter based on the status “Inactive Trigger” and “Paused”. If you don’t have error notifications set up individually or are interested in other status values as well, you can just add them to the SQL statement.
Note: This could also be done using a filter activity instead of SQL if you desire that approach.
So you don’t have to check the data extension regularly, I included a verification activity that checks the record count in the data extension “Automation_Notification_Results” and triggers an email notification if it is above 0.
Now you are done with the configuration and just need to add all your relevant automations in the first data extension, set a desired schedule for your monitoring automation and to make sure it always runs, set a runtime notification for this automation. From now on you’ll never miss a paused automation anymore 😀