An important question to ask when evaluating the effectiveness of training is how the performance of people who completed the training compares with people who did not complete it. This walkthrough shows you how to set up a line report comparing people who have and have not completed a piece of training on a particular performance KPI.

Before starting this walkthrough, you will need to have data in Watershed about training completion and about job performance.

1. Create a Line Report

To get started, create a simple Line report showing the KPI you want to look at over time. Filter the report by the group of people you want to look at, update the report text and add the appropriate KPI measure. In this example we are looking at the Average CSAT Rating of the Customer Service group.

In our example scenario, let’s pretend that we want to see whether or not attending the “Module 1” training event at the end of October helps to reduce an expected seasonal drop in CSAT rating in November.

2. Filter by people who attended training

This shows us the average CSAT rating for all people in the Customer Service group. Now let’s filter the measure to show only people that attended the Module 1 event. Open up advanced configuration and add the following to the measure filter. In your report, you should change the activity id and verb id to match the training or learning experience you want to evaluate the impact of.

You can also rename the measure to “Attended”.

"peopleFilter":{
  "activityIds": {
    "ids": [
      "http://program.module-1-event.example.com"
    ],
    "regExp": false
  },
  "verbIds": {
    "ids": [
      "http://adlnet.gov/expapi/verbs/attended"
    ]
  }
}

3. Filter by people who did not attend training

Now copy the measure to create a 2nd measure called “Did not attend”. Wrap the people filter in a not filter like so:

"not": {
  "peopleFilter": {
    "activityIds": {
      "ids": [
        "http://program.module-1-event.example.com"
      ]
    },
    "verbIds": {
      "ids": [
        "http://adlnet.gov/expapi/verbs/attended"
      ]
    }
  }
}

The complete configuration for our example report looks like this:

{
  "filter": {
    "groupCustomIds": [
      "Customer Service"
    ]
  },
  "dimensions": [
    {
      "type": "TIME",
      "timePeriod": "MONTH"
    }
  ],
  "measures": [
    {
      "name": "Attended",
      "aggregation": {
        "type": "AVERAGE"
      },
      "valueProducer": {
        "type": "STATEMENT_PROPERTY",
        "statementProperty": "result.score.raw"
      },
      "filter": {
        "activityIds": {
          "ids": [
            "http://csat.example.com/csat/rating"
          ]
        },
        "peopleFilter": {
          "activityIds": {
            "ids": [
              "http://program.module-1-event.example.com"
            ]
          },
          "verbIds": {
            "ids": [
              "http://adlnet.gov/expapi/verbs/attended"
            ]
          }
        }
      }
    },
    {
      "name": "Did Not Attend",
      "aggregation": {
        "type": "AVERAGE"
      },
      "valueProducer": {
        "type": "STATEMENT_PROPERTY",
        "statementProperty": "result.score.raw"
      },
      "filter": {
        "activityIds": {
          "ids": [
            "http://csat.example.com/csat/rating"
          ]
        },
        "not": {
          "peopleFilter": {
            "activityIds": {
              "ids": [
                "http://program.module-1-event.example.com"
              ]
            },
            "verbIds": {
              "ids": [
                "http://adlnet.gov/expapi/verbs/attended"
              ]
            }
          }
        }
      }
    }
  ],
  "includePeopleWithoutStatements": false,
  "hiddenMeasures": [],
  "line": true,
  "singleGraph": true,
  "allowGaps": false,
  "fillGapsWithZeroes": false,
  "type": "line-chart"
}

Save the report.