With Watershed's CSV Import Feature, you can bring both xAPI and groups data into Watershed with a flat CSV file. This is done by first creating an xAPI template or creating a template for organizational hierarchy data. Users can then upload files with new data on a regular basis.

To avoid having to manually upload the CSV, you can use Watershed’s CSV Data Import API to send the CSV data to Watershed automatically.

This guide outlines the allowed resources and methods of the CSV Data Import API. In order to interact with the CSV Data Import API, you will need authentication credentials and an organization id (org id).

Upload Data as a CSV

Request URL: /api/organizations/[org-id]/csv-import

Method: POST

Expected response code: 202 Accepted

Request headers

If you are using a code library for sending files, it will likely take care of setting the proper Content-Type header.  Consult your code library documentation.

If you are building a request directly in your code, you will need the following header:

Header	Details
Content-Type	The content type for this request is multipart/form-data. The content type header should also include the boundary used in the request content. E.g. multipart/form-data; boundary=BOUNDARY

Request parameters

Parameter	Details
template	Name (or id) of the template to use for the import import.
fileSize	Size of the file in bytes.  This number must be exact.  Optional if file size < 10Mb.
notify	Email address to notify with details once the import completes (or fails). Can be a single email address or a comma-separated lists of email addresses.

 

Request content

The content of the request uses the multipart/form-data format. Most libraries for sending files will properly handle setting up the request body without having to worry about details such as the boundary separator string. Consult the documentation for your library of choice.

If you find that you have to build a request body in your own code, a single part contains the contents of the csv file. For example:

--BOUNDARY
Content-Disposition: form-data; name="file"; filename="name.csv"
Content-Type: text/csv

header A, header B
cell 1A, cell 1B
cell 2A, cell 2B
--BOUNDARY--

Response

A successful response contains the id of the import job that was started.

Preview a CSV Upload

The preview request is similar to the upload request except that it returns generated statements and any errors, instead of storing them in Watershed.

Request URL: /api/organizations/[org-id]/statement-import/preview

Method: POST

Expected response code: 200 OK

Request headers

Header	Details
Content-Type	The content type for this request is multipart/form-data. The content type header should also include the boundary used in the request content. E.g. multipart/form-data; boundary=BOUNDARY

 

Request parameters

Parameter	Details
template	Name (or id) of the template to use for the import import.

 

Request content

The content of the request is multipart. The first part is the csv file and the second part is a form parameter containing the size of the csv file. For example:

--BOUNDARY
Content-Disposition: form-data; name="file"; filename="name.csv"
Content-Type: text/csv

header A, header B
cell 1A, cell 1B
cell 2A, cell 2B

--BOUNDARY
Content-Disposition: form-data; name="filesize"
52
--BOUNDARY--

Response

A successful response contains an object with two properties, failedRows and statements. Both of these properties contain an object, the properties of which are strings named with the row numbers of failed or successful rows.

Within the failedRows object, the value of each property is an error message outlining the error encountered with that row. For example (line breaks added for clarity):

"failedRows": {
    "1388": "Statement must include an ID, and the ID must
    be a UUID. The current value of the field is: 
    some-invalid-id."
 }

Within the statements object, the value of each property is an array of xAPI statements generated by that row.

Fetch CSV Imports

Gets details of csvs that have been imported and not voided. This might be useful if you need to void an import via API (see below).

Request URL: /api/organizations/[org-id]/csv-import

Method: GET

Expected response code: 200 OK

Request parameters

Parameter	Details
id	Id of the import to get. If not included all imports will be returned.
template	Name of template to filter by. Returns only those imports using this template.

 

Response

A successful response returns a collection object with count and results properties. The count property contains a count of the results. The results property contains an array of import objects, the properties of which are outlined in the table below.

Property	Details
id	Id of the import.
created	ISO 8602 timestamp of when the CSV was imported
version	Not necessary
size	Size of file?
originalFilename	The name of the file imported
file	The name of the file imported
mediaType	Always text/csv
hashFileName
userName	The full name of the user who uploaded the file.
statemenIds	A comma separated string listing the ids of all the statements generated by the import.
alllStatementIds	An array of ids of all the statements generated by the import.
failedRows	An object whose properties correspond to row numbers of rows that errored. The value of each property is the error generated for that row.
template	The name of the template used for the import

Void an Import

If a CSV has been uploaded in error, it is possible to delete the import and void all statements created. If the file had groups information, voiding the file will remove any people, groups, and permissions created by the import, even if the information had been edited by a later import. It won't revert any changes made to people and groups that previously existed.

Request URL: /api/organizations/[org-id]/csv-import/[import-id]

Method: DELETE

Expected response code: 200 OK

Response

A successful responses returns no content.

Code samples

This section provides reference code samples for the above API calls using Python 3.

Code: Upload Data as a CSV

import os
import requests

# Get the CSV Data
## Using open to get the csv data
with open('my_csv.csv', 'r') as infile:
csv_data = infile.read()

## Alternatively using Pandas to get the csv data:
import pandas as pd
df = pd.read_csv('my_csv.csv')
csv_data = df.to_csv(index=False)

# Create the request
## Get the size of the file
filesize = len(csv_data.encode('utf-8'))
### Alternatively use the os module to get the file size:
filesize = os.path.getsize('my_csv.csv')

files = {'file': (filename, csv_data)}
notification_email = "your_email@example.com"
## Alternatively, a list of emails
notification_email = "your_email1@example.com,your_email2@example.com"

template_name "My Template"
ws_key = "your_watershed_key"
ws_secret = "your_watershed_secret"
url = f"https://sandbox.watershedlrs.com/api/organizations/5664/csv-import?template={template_name}&fileSize={filesize}&{notify}"
r = requests.post(url, auth=(ws_key, ws_secret), files=files, timeout=1200)

Code: Preview a CSV Upload

# Using the same code from above, we change the url to:
url = f"https://sandbox.watershedlrs.com/api/organizations/5664/statement-import/preview?template={template_name}&fileSize={filesize}&{notify}"
r = requests.post(url, auth=(ws_key, ws_secret), files=files, timeout=1200)

Code: Fetch CSV Imports

# Getting all imports
url = "https://sandbox.watershedlrs.com/api/organizations/5664/csv-import"
res = requests.get(url, headers=headers, auth=(ws_key, ws_secret), timeout=60)

# Getting all imports for a specific template
url = f"https://sandbox.watershedlrs.com/api/organizations/5664/csv-import?template={template_name}"
res = requests.get(url, headers=headers, auth=(ws_key, ws_secret), timeout=60)

Code: Void a CSV Import

# Get the import ID from either the csv_import request above or from the CSV Data Page on the UI by hovering over the "Source" link
url = f"https://sandbox.watershedlrs.com/api/organizations/5664/statement-import/{import_id}?deleteLrsStatements=true"
res = requests.delete(url, headers=headers, auth=(ws_key, ws_secret), timeout=60)