Heads up: This feature is in beta.
Please contact your account manager if you'd like to have it enabled on your account.
How to Get to Workflows
Workflows are accessed by selecting Workflows from the Settings menu:
Creating Workflows
To create a new workflow, click the New Workflow button:
Configuring workflows
Workflows are configured using an advanced configuration editor. The workflow object has three properties. These are triggers
, actions
and context
.
The triggers
property contains an array of tasks that are run to check if the workflow should be run. This might include running reports and making comparisons to test the values returned by the reports.
The actions
property contains an array of tasks that are run if all the trigger's tasks complete successfully. This might include running additional reports and generating emails that include values from reports (including those reports run as checks).
Hint: If you don't need report data for a comparison task as part of the trigger, run it as an actions task. This means that if the actions aren't triggered, the report doesn't need to run, saving Watershed from calculating the report for no reason.
The context
property contains a context object that can have any property. The values of these properties can be referenced in comparison actions as constants to compare against the result of a report. This is useful for storing settings relating to the workflow, for example setting a target average score for a workflow that sends a emails while average score for a quiz is below that value.
Thus the workflow object high level syntax looks like this:
{
"triggers": [],
"actions": [],
"context": {}
}
Tasks
Workflow tasks are the building blocks of how a workflow runs. They run in 2 distinct phases:
- Triggers
- Actions
Triggers are tasks that will be run in order to determine if a workflow should execute it's actions.
For example:
A workflow has 3 triggers: REPORT, REPORT, COMPARE and 2 actions: EMAIL, REST_REQUEST
This means that the workflow will always run 2 reports and perform a comparison on those reports. If the comparison returns true, then and only then, the workflow will execute its actions which consist of an EMAIL task followed by a REST_REQUEST task. Otherwise, if the comparison returns false or if an error is thrown, the workflow will not run the actions.
It's important to note that although any task may be used as a trigger or an action, some may not make logical sense. For example, if an EMAIL task was set as a trigger, it would send an email every single time the workflow ran its triggers (every single 30 minutes). So it can be helpful to think about a workflow as running in 2 distinct phases and separate your tasks accordingly.
For this reason, when using a Quick Actions/Triggers button the task will default to the section that makes the most sense for that particular task. We also outline the Preferred phase below in order to help clarify where a particular task may be most sensible. If the Preferred phase is "Any", that task may be equally sensible as a trigger or an action.
REPORT
Preferred phase: Any
The core component of workflows is the REPORT task. A REPORT task runs a Watershed Report to generate results. These results can be used to determine if the workflow's actions should run, and to display in emails sent by the workflow.
The following properties can be set on a REPORT task:
Property | Required | Type | Description |
---|---|---|---|
type | true | Constant - "REPORT" | Defines the type of task to be configured |
config | true | Map<String, Any> |
The config property should be a valid aggregation report configuration and must contain all required properties required by an aggregation report. It may also include any optional properties allowed by an aggregation report. You can create this configuration object using Report Builder and Advanced Configuration. |
Example Task:
"triggers": [
{ "type": "REPORT", "config": { ... } }
]
Hint: If you want to use the REPORT task to get a calculated value from a measure, in most cases it is simplest to use a CONSTANT
dimension and filter the report to the data you want. If you want to use the REPORT task to get the top or bottom items based on a measure value, you will need to use another type of dimension and the defaultSort
property. See Advanced Dimensions.
COMPARE
Preferred phase: Trigger
COMPARE tasks are used to compare two values. COMPARE task objects have four properties: type
, operator
, first
and second
.
Property | Required | Type | Description |
---|---|---|---|
type | true | Constant - "COMPARE" | Defines the type of task to be configured |
operator | true |
ENUM |
The operator property defines what comparison will be performed on the values of See below for the list of valid values. |
first | true | String | A string consisting of a context path reference to the first value which will be placed before the operator |
second | true | String | A string consisting of a context path reference to the second value which will be placed after the operator |
Valid operator
values:
> |
The first value is greater than the second. |
>= |
The first value is greater than or equal to the second. |
< |
The first value is less than the second. |
<= |
The first value is less than or equal to the second. |
== |
The first value is equal to the second. |
!= |
The first value is not equal to the second. |
Here's a complete example of a compare task that compares the value of a measure in a report task with a constant in the context:
{ "type": "COMPARE", "operator": ">", "first": "trigger[0].data[0].aggregations[2].value", "second": "minimumScore" }
Please note: It's not possible to hard code values within a compare task. All values to be compared must exist within the context.
REST_REQUEST
Preferred phase: Any
The REST_REQUEST
task is used to make an HTTP request to a public REST api. The request will stay open for a maximum of 60 seconds to wait on a reply from the url supplied in the configuration.
The REST_REQUEST task accepts the following parameters:
Property | Required | Type | Description |
---|---|---|---|
type | true | Constant - "REST_REQUEST" | Defines the type of task to be configured |
url | true | String |
The full url to call—including the proper protocol (http/https). Or a context path to a variable containing a url. Note that it must be wrapped in curly-braces. |
username | false | String |
The username to include in the basic auth header |
password | false | String | The password to include in the basic auth header |
params | false | Map<String, String|Boolean|Number> |
A map of parameter name/value pairs to pass with the request. Currently the map of parameters only accepts UTF-8 encoding. See example below. |
headers | false | Map<String, String|Boolean|Number> |
A map of header name/value pairs to pass with the request. See example below. |
body | false; Ignored unless httpMethod is set to POST |
String |
The HTTP body to send along with the request. |
httpMethod | true; Defaults to GET |
String |
The HTTP protocol method that this request should be. i.e. |
Example REST_REQUEST task:
{
"type": "REST_REQUEST",
"url": "https://example.com/api",
"params": {
"debug": "true",
"myParam": 123
},
"headers": {
"Content-Type": "application/json"
}
}
Please note: The above example is making a GET request since the default httpMethod is GET
Example of a POST REST_REQUEST task:
{
"type": "REST_REQUEST",
"url": "https://example.com/api",
"httpMethod": "POST",
"params": {
"debug": "true",
"myParam": 123
},
"headers": {
"Content-Type": "application/json"
},
"body": "{ \"somePayload\": \"withData\" }"
}
EMAIL_REPORT
Preferred phase: Action
The EMAIL_REPORT task will send an email to a list of subscribers.
Property | Required | Type | Description |
---|---|---|---|
type | true | Constant - "EMAIL_REPORT" | Defines the type of task to be configured |
subject | false | String | The subject that should appear in the "subject" heading of the email |
bodyIntro | false | String |
The message to be included. Will not use context path variables. If defined without bodyTemplate, the email will include the default Watershed email styling. See examples below. |
bodyTemplate | false | String |
The message to be included. If defined, this will override bodyIntro. Will use context path variables. See examples below. |
subscribers | false | Array<Object<String, String>> | The list of subscribers which should receive this email |
Please note: Subscribers will be sent an email asking if they want to subscribe and will only be sent further emails after they do so.
Heads Up: Subscribers will only be sent a single subscription request email. If a particular subscriber has lost their subscription email, you can resend it by removing that subscriber from the workflow, saving the workflow, and then re-adding them back and saving again. This process ensures that subscribers don't receive a subscription email every single time a workflow is saved or modified which would be extremely noisy.
Example EMAIL_REPORT configuration:
{ "type": "EMAIL_REPORT", "subject": "Average score",
"bodyIntro": "This email tells you the average score (this won't show up since we defined bodyTemplate)", "bodyTemplate": "The average score was: {{actions[0].data[0].aggregations[2].value}}", "subscribers": [ { "email": "test@example.com" } ] }
Body Template Handlebars
In addition to being used to insert values from a report, the bodyTemplate
can make use of all the CSV template handlebars functions described in CSV Templates Functions and Variables. This might be useful if you want to transform the value returned from a report in some way, or build a more complex email.
Example emails
This is an example of how an email using bodyTemplate will look:
This is an example of how an email using bodyIntro will look:
LOOP
Preferred phase: Any
The LOOP task allows a workflow to loop over a set of data and perform a single task for each individual piece of data. If the set of data is empty, the LOOP will simply return successfully without actually doing anything.
The LOOP task has the following properties:
Property | Required | Type | Description |
type | true | Constant - "LOOP" | Defines the type of task to be configured |
loopContextPath | true | String | A context path that point to a list of data with at least to iterate over |
limit | false | Number | An optional limit to place on the number of iterations to run. Should be less than 1000 or else the default limit will take precedence. |
taskToRun | true | Task | The workflow Task which should be run for each iteration over the data referenced by loopContextPath. |
Heads up: The internal limit for the LOOP task is 1,000 iterations. If the data set referenced by loopContextPath
is larger than 1,000 the LOOP task will cut off at 1,000 iterations.
Example LOOP task:
{
"type": "LOOP",
"loopContextPath": "triggers[0].data",
"limit": 10,
"taskToRun": {
"type": "EMAIL_REPORT",
"subject": ...,
"subscribers": [...],
}
}
Heads up: when using a loop you can access the data from the index of count of the loop. To access the dimension use:
{{iterator.values[0].value}}
For measures use (replacing 0 with index of the measure):
{{iterator.aggregations.[0].value}}
Context
The context
property contains an object that can have any property. The values of these properties can be referenced in comparison actions as constants to compare against the result of a report. For example, a context property might look like this:
"context": { "minimumScore": 70, "emailFromName": "John Doe" }
The context object property names can be anything except "triggers", "actions", or "context".
Context Path
A context path is a representation of where inside the context object a certain piece of data exists.
For constants that are defined in the configuration, you can simply use the name of the property like so:
"first": "minimumScore"
To reference a dynamic value (such as a report measure, or the results of a previous task) you must first reference the task you wish to reference by using the number of its order within the phase that it is defined.
For example:
A workflow runs a REPORT task as the first trigger and we would like to reference the value of the first measure of that REPORT task within a COMPARE task. The first property might look like this:
"first": "triggers[0].data[Y].aggregations[0].value"
Notice the use of 0
instead of 1
because lists and array begin with 0, so the first trigger would be triggers[0]
In this example, Y
would be the row to examine—again 0-based. So to view the first result Y
would be 0
.
General Context Path Formulas
Measures:
triggers[x].data[Y].aggregations[Z].value
where
- X is the index of the task (replace "triggers" with "actions" if referencing a task within the "actions" phase)
- Y is the index of the dimension item (always 0 if a CONSTANT dimension is used)
- Z is the index of the measure.
Dimensions:
triggers[X].data[Y].values[0].label
where
- X is the index of the task (replace "triggers" with "actions" if referencing a task within the "actions" phase)
- Y is the index of the dimension item (always 0 if a CONSTANT dimension is used)
Hint: Replace label
with value
to reference a dimension item id.
Jobs
The Jobs tab includes a list of every time a workflow has run, including the time and whether or not the job was successful.
Heads up: Jobs will only appear on this list if the trigger successfully runs and passes; only successful jobs and jobs that error in the actions phase will be listed. Where jobs do get listed as failing, no error message is provided. Use the debugging feature to fully test the template before activating it.
Debugging
The workflow debugging tool can be used when writting a workflow to test each stage to confirm that it is working, and to see any errors. The debugging tool can be accessed in two different ways:
- Select Debug Workflow from the gear menu on the workflows list page.
- When editing an existing workflow, select Save and Debug or Debug at the bottom of the editor.
The debug tool will then run through each task of the workflow until is completes or errors. Messages indicating progress through the workflow are shown in blue, while errors are shown in red.
Heads up: Debug error message functionality is in alpha, it may not always be clear from the error what the issue is. However, the error will at least show which task is failing and the error messages may be useful to a Watershed developer investigating the issue.
Schedule
By default, workflows will run every 30 minutes. This means that, if you're using an email task, you'll get an email once every 30 minutes so long as the triggers pass. In most cases you'll want to reduce this so you get emails less regularly. This is done using the intervalOverride
property.
The intervalOverride
property takes the value in the format of a cron expression. For example, the expression to run a workflow every day at midnight is "0 0 0 ? * * *". Here's an explanation of how that works:
- The first zero means the cron will run at second 0
- The second zero means the cron will run at minute 0
- The third zero means the cron will run at hour 0
- The fourth character, the questionmark, represents the day of the week. A question-mark means to ignore this value and instead use the day of the month.
- The fifth character, star, means to run every day of the month.
- The sixth character, star, means to run every month of the year.
- The last character means to run every year.
The table below provides some additional examples:
Cron Expression | When the workflow will run |
---|---|
* * * ? * * * |
Every 30 minutes (because the minimum period is 30 minutes). |
0 */30 * ? * * * |
Every 30 minutes. |
0 0 * ? * * * |
Every hour. |
0 13 * ? * * * |
Every hour sometime between 13 and 43 minutes past. |
0 0 0 ? * * * |
Every day. |
0 0 0 ? * 2 * |
Every week on a Monday. |
0 0 0 24 * ? * |
Every month on the 24th day of the month. |
0 0 0 ? * 2 2020 |
Every Monday in 2020. |
All dates and times are in UTC 0 timezone.
Hint: You can use this formatter to create a cron expression.
Complete example workflows
This first example workflow that will send a daily email with the number of people active in the account in the last day, when there have been at least 1 person active in the account in the last day.
{
"intervalOverride": "0 0 0 ? * * *", "actions": [ { "type": "EMAIL_REPORT", "subscribers": [ { "email": "example@example.com" } ], "bodyTemplate": "There are {{triggers[0].data[0].aggregations[0].value}} people were active yesterday! You will get this email every 30 minutes.", "subject": "How many people were active yesterday?" } ], "triggers": [ { "type": "REPORT", "config": { "filter": { "dateFilter": { "dateType": "trailing", "trailingAmount": 1, "trailingType": "days" } }, "type": "leaderboard", "dimensions": [ { "type": "CONSTANT" } ], "measures": [ { "name": "Person Count", "aggregation": { "type": "DISTINCT_COUNT" }, "valueProducer": { "type": "STATEMENT_PROPERTY", "statementProperty": "actors.person.id" } } ] } }, { "type": "COMPARE", "operator": "!=", "first": "triggers[0].data[0].aggregations[0].value", "second": "zero" } ], "context": { "zero": 0 }, "active": true }
The next example workflow can be used to send a daily email when the first item in a report filtered to show data since the end of yesterday is different to the same report filtered to show data until the end of the previous day. The workflow is run 15 minutes after midnight UTC0 (and reports use a UTC0 timezone) so as to be as up to date as possible (since midnight is the point in time when "today" becomes "yesterday"), but also allow some margin of error to avoid the workflow running just before midnight instead.
This workflow can be used for an alert triggered when there is a new item at the top of a leaderboard. In this case the leaderboard is organized by person, but this approach will work with any dimension.
{ "intervalOverride": "0 15 0 ? * * *",
"actions": [ { "type": "EMAIL_REPORT", "subscribers": [ { "email": "example@example.com" } ], "bodyTemplate": "New leader! Previous leader: {{triggers[0].data[0].values[0].label}} New leader: {{triggers[1].data[0].values[0].label}}", "bodyIntro": "", "subject": "New Leader!" } ], "triggers": [ { "type": "REPORT", "config": { "type": "leaderboard", "filter": { "not": { "dateFilter": { "dateType": "trailing", "trailingAmount": 1, "trailingType": "days", "fieldName": "timestamp" } } }, "dimensions": [ { "type": "STATEMENT_PROPERTY", "statementProperty": "actors.person.id" } ], "measures": [ { "name": "Interaction Count", "aggregation": { "type": "COUNT" }, "valueProducer": { "type": "STATEMENT_PROPERTY", "statementProperty": "id" } } ], "defaultSort": "-measure[0].value" } }, { "type": "REPORT", "config": { "type": "leaderboard", "filter": { "not": { "dateFilter": { "dateType": "trailing", "trailingAmount": 0, "trailingType": "days", "fieldName": "timestamp" } } }, "dimensions": [ { "type": "STATEMENT_PROPERTY", "statementProperty": "actors.person.id" } ], "measures": [ { "name": "Interaction Count", "aggregation": { "type": "COUNT" }, "valueProducer": { "type": "STATEMENT_PROPERTY", "statementProperty": "id" } } ], "defaultSort": "-measure[0].value" } }, { "type": "COMPARE", "operator": ">", "first": "triggers[0].data[0].values[0].value", "second": "triggers[1].data[0].values[0].value" } ], "context": {}, "active": true }