Data Imports

Follow

Applies to: Pro, Plus, & Enterprise Plans

Introduction

This article details the process and best practices for importing various types of data into Absorb. Please note that these imports are an additional service that incurs a fee and technical resources on both the Absorb, and client sides. If you simply need to import Users in bulk, we have a User Import tool built into the Admin User Interface (UI) for all clients.

In the following sections, we will heavily reference an Absorb Data Mapping document. As such, for all mentions of fields and/or files, readers should be referencing the Absorb Data Mapping document unless otherwise indicated. We recommend downloading the spreadsheet to ensure that you can review it, and become familiar with the Instructions sheet/tab.

Contents:

Historical Data Import

Absorb has been designed to allow data to be imported from other Learning Management Systems (LMS). This allows LMS clients new to Absorb to continue making use of their historical information for both reporting, and User experience purposes as soon as they start using our product. These types of data imports are done only once, usually during the onboarding process for new Absorb clients, to allow organizations a seamless transition from one system to another. 

CSV Sync Integration

CSV sync integrations are essentially recurring historical data imports - however each subsequent import after the first isn't truly, historical. The difference here being that following the initial import, which is usually a full extract from the previous LMS, Absorb schedules automatic processing of those files uploaded onto our FTPS site at a pre-determined frequency. For example, we can automatically process a file containing new or updated User records each night at 3:00 AM. The client's HRIS or other system would automatically export the file(s) which would drop onto Absorb's FTPS each night at 2:30 AM.

Tip: We will provide the FTPS server and account details as part of the implementation. If you require the files be picked up from your own secure FTP server, please discuss and confirm with the contact working with you at Absorb. You can find more information in our FTP Support for Integrations article.

External ID

One of the many important concepts that will be discussed in this article is the use of ExternalId. This field exists as a means to uniquely identify and link related LMS records. The value of this special field can either be ‘real’, as taken from some external system, or they can be purpose-generated for the sole use of your Absorb data import.

An example would be importing Users and their Course enrollments into Absorb, where the Course Enrollment table's User_ExternalId would refer to the User table's ExternalId, thereby relating the two elements during the import.

 

Import Process

During the initial scoping call, we will discuss requirements and timelines, as well as walk you through the steps of an import process. The first major deliverable that we typically work towards is obtaining a sample file set, and a completed Data Import Requirements document. Once received, our development team reviews your sample file set, and proceeds to import the data into a sandbox environment. For that reason, it is very important that the sample file set is provided in the exact final format that your final production data will be provided in. Any change to the data format after import work begins can result in delays and/or added cost.

Tip: It's usually best to provide real data in your sample files instead of mock data. This allows us to identify any data quality issues early on in the process.

A slightly difference process is followed for "Standard" vs. "Custom" imports. Most imports are considered "Standard" unless specifically scoped, quoted and discussed otherwise with us.

 

Standard Import

A Standard Import requires the client to provide their import files in a standard format, defined in the Absorb Data Mapping Spreadsheet. Any modifications needed beyond these stock formats will require a Custom Import (as outlined in the next section). 

Steps of a Standard Import: 

  • Scoping call and confirmation of requirements (complete the Data Import Requirements document)
  • Client sample files provided (must match Absorb Data Mapping document exactly
  • Import to sandbox environment
  • Data revision and re-import (if required > maximum of 2 re-imports* allowed
  • Client review and approval
  • Final data files provided (generally just prior to, or after launch) 
  • Final import to sandbox
  • Final approval from client
  • Final import to production
  • Client review and confirmation

Custom Import

A Custom Import allows for greater flexibility in the file formats that an Absorb client can provide for the import process of their historical data. The companion Absorb Data Mapping document is still used to identify available data fields. With this type of import, the order of a client’s columns, their field delimiters, and custom logic can all be modified. Of course, this is all subject to scoping and approval.
 
Steps of a Custom Import: 
  • Scoping call and confirmation of requirements (complete the Data Import Requirements document)
  • Client sample files provided
  • Custom data mapping completed (fields mapped as per the Absorb Data Mapping document) 
  • Import to sandbox environment
  • Data revision and re-import (if required > maximum of 4 re-imports* allowed
  • Client review and approval
  • Final data files provided (generally just prior to or after launch) 
  • Final import to sandbox
  • Final approval from client
  • Final import to production
  • Client review and confirmation
*A re-import involves the client providing updated data files, generally in order to resolve issues with data accuracy and/or consistency identified after a previous importation event. Files should be provided in the same format as the previously imported versions. Slight modifications such as adding or dropping a single column may be allowed provided they are discussed with Absorb first. However, be advised that Absorb reserves the right to consider any modification made to a file format after the initial import development has begun as out of scope, additional work.
 

File Format

Files should be provided in CSV (Comma Separated Values) format. The Instructions tab of the Data Mapping Spreadsheet provides further directions for naming and creating your CSV files. If you're unfamiliar with the CSV format, we recommend reading our, "What is a CSV file and how do I save my spreadsheet as one?"
 

Requirements

The Data Import Requirements document should be reviewed and completed with our assistance. Please review this document carefully as it contains all of the options and configuration that we will use for this data import. 

Common Elements

This section will detail some of the more commonly imported elements, their dependencies, and special considerations. Keep these in mind when formulating your file set.

Users:

All Users

In order to import Users - which includes Learners, Administrators, and Instructors - Absorb requires the client’s Users file at a minimum. Generally a Departments file is also required, though the client may wish to set up their departments manually through the Absorb Admin UI first.

Please Note:

  • The ExternalId field must be filled in during setup.
  • All Admin Users (IsAdmin=1) will be assigned either a single role, or no role at all. This will be decided by the client prior to carrying out the data import if no other files are provided.

Administrators

Advanced user imports can also configure the assignment of different Admin access roles per each User (with the inclusion of a Role Assignments file), restrict Admin access to Users in certain departments (by incorporating a Department Admins file), and/or limit Admin access to Users in certain groups (by adding a Group Admins file).

Special Considerations

Any User referenced in the Supervisor_ExternalId field must also be set as an Admin User (i.e. IsAdmin=1). Clients should note that all Admin Users will be assigned whatever default admin role is chosen for the import unless a Role Assignments file is also provided.

Custom Field columns should be named exactly as they are named in the Admin UI, without spaces. E.g., "My Custom Field" in the Admin site becomes the "MyCustomField" column in your file.

Courses, Curricula and Course Bundles:

Metadata vs Content
It is important to note that Absorb only imports Course metadata such as name, description, and some Course settings, etc. We do not import the actual Course content itself like SCORM files, videos, etc. This means that the uploading of Course content still needs to be completed on a Course-by-Course basis through the Admin UI once the initial importation is completed. Some Absorb clients prefer to complete their Course setup entirely through the Admin UI for this reason. If this situation applies to your organization, then please ensure that you fill out each Course’s External ID field if Absorb is required to import any other entities that must be linked to Courses. E.g., Course Enrollments, etc.
 
Minimum Requirements
In order to import Online Courses and Instructor Led Courses (ILC), Absorb requires a Courses file at a minimum. Individual Lessons within each Online Course can also be populated using the Lessons file, however clients must still upload and link content for each of those lessons to the Course through the Admin UI. Scheduled instances of your ILC can be populated using the Sessions and Classes files, but note that the Classes file must also be accompanied by a Venues file.
 
Multiple Courses
Curricula (learning paths) and Course Bundles contain courses within them and can also be imported using their respective files. 
 
Special Considerations
The AdminVisibilityDepartmentId field restricts Admin visibility to specific Courses in the Admin UI only. This setting is generally not required unless you need to segregate your Course visibility to different sets of Admins. Should your import require this, it is recommended that you set up and test examples of this through the Admin portal before applying the settings in a bulk import.
 
Certificate_IsEnabled must be enabled before setting any of the other Certificate fields (e.g. Certificate_ExpireType). The same sequence also applies to EvaluationEnabled, ToggleEmails, SendNudgeEmails, AllowReEnroll and AllowFailure in that you must enable these fields first in order to set other fields related to them. Be advised that not all of these fields are applicable to Curricula and Course Bundles however.
 

Course Progress and Completion Data:

General
In order to import Learner Course progress and completion data such as scores, progress, start dates, completion dates, etc., Absorb requires the Course Enrollments file at a minimum. This file will set Learner activity at the Course level only, meaning that each Learner will have a score, status, etc., for the Course itself but not for each individual Lesson under each Course. This level of detail is usually sufficient. Please note that the Course Enrollment file refers to both Users and Courses, so those files must also be provided or Users and/or Courses must be set up through the Admin UI with the External ID fields populated before the import commences.
 
Additional Detail
If an Absorb client deems that more granular learner activity is needed, then they may provide Absorb with both Course Enrollments (Course level) and Enrollment Activities (lesson level) files as part of their import inputs. It’s very important to note that Absorb will also require Courses and Lessons files as well, if asked to incorporate this level of detail.
 
Learning Paths
If a client has Courses that are organized into learning paths with learner activity and enrollment that they would like to preserve, then a Curriculum Enrollments file should also be provided. Curricula themselves must also be provided either in a file or created manually through the Admin UI, as described above for the Courses Enrollments file.
 
Certificates / Badges
Absorb clients may also want to preserve or issue their existing learner certificates or competencies (badges) in the new LMS. In this case they would need to provide Absorb with the Certificates and/or Competencies files. Please note that Certificates must be set up, either within the Courses and/or Curricula files, or manually through the Admin UI, before importing the Certificates file. Competencies must be set up through the Admin UIbefore importing the Competencies file.
 
Special Considerations - Course and Curriculum Enrollments:
Absorb supports the concept of multiple enrollments in a single Course or curriculum. The IsActive field is used to control this. Generally, if a client is planning to import only a single enrollment per User and course pair then this value should be set to 1 (Active). 
 
However, if a client’s User had taken the Course twice, then the client would likely set the oldest enrollment record to 0 (Historical) and the most recent enrollment record to 1 (Active). It is important to point out that historical enrollments are available for reporting only and are not displayed in the Learner UI. For this reason it is very important that enrollments that should be deemed as ‘active’ are correctly set to 1 when, and where appropriate.
 

Absorb Assessments:

Question and Answers 
Questions and answers for use in either Absorb Exams, Quizzes and/or Surveys can also be imported. To accomplish this, both the Questions and Answers files are required. Absorb clients also have the option to provide the Question Banks file to group questions for use in multiple assessment scenarios.
 

Data Validation

Before providing your files to Absorb please take a moment to review and ensure they pass these checks:
  • All External IDs are unique. 

  • All referenced entities do exist when one file refers to an entity in another file. E.g., The User file referring to a Department_ExternalId from the Department file can be found within said file. References should either occur in another file or have been previously been set up through the Admin UI.

  • All special characters (e.g. © ● ™ etc.) are represented correctly in the files. Some software and/or export processes can automatically replace these with incorrect characters.

  • All dependencies have been checked and verified. E.g., if enabling certificate expiry for a Course (Certificate_ExpireType), also ensure the issuance of a certificate for that Course (Certificate_IsEnabled) has also been enabled.

  • All active/inactive flags such as fields like ActiveStatus, IsActive, etc., are set correctly. Double check documentation to ensure fields are set to 1 or 0 as is appropriate.

  • All column headers are correct. Confirm that they have no spaces and match the data mapping document. E.g., “FirstName” instead of "First Name".

  • All field values are fully accurate. Double check data in a text editor like Notepad or other plain text editor to confirm. Programs like Excel may remove leading 0’s, truncate very long numbers, and may change your original data in unexpected ways.
 

File Set Examples

The following examples are meant to illustrate which combinations of files you might provide us with. Please note that clients may elect to choose a different combination of files which is acceptable, as long as they observe the dependencies noted in the previous section.
 
(See the next section for samples of actual file content)
 
Example 1 – Basic: 
  • Departments
  • Users
  • Courses / Curricula / Course Bundles
  • Course Enrollments / Curriculum Enrollments
Example 2 – Advanced User: 
  • Departments
  • Users
  • Role Assignments
  • Department Admins / Group Admins
  • Courses / Curricula / Course Bundles
  • Course Enrollments / Curriculum Enrollments
Example 3 – ILCs: 
  • Departments
  • Users
  • Courses / Curricula / Course Bundles
  • Sessions
  • Classes
  • Venues
  • Course Enrollments / Curriculum Enrollments
Example 4 – Advanced Learner Activity: 
  • Departments
  • Users
  • Courses / Curricula / Course Bundles
  • Lessons
  • Course Enrollments / Curriculum Enrollments 
  • Enrollment Activities
  • Certificates / Competencies


Appendix: Data Examples

Examples of the pre-importation ‘working’ data for some of the more popular import files are shown below.

Departments:

Spreadsheet

 
Text
Name,ExternalId,Parent_ExternalId
Demo Organization,00001,
Sales,00002,00001
Support,00003,00001
Human Resources,00004,00001
North America Sales,00005,00002
European Sales,00006,00002
  

Users:

Spreadsheet

 
Text
Username,Password,FirstName,LastName,ActiveStatus,Department_ExternalId,ExternalId,IsAdmin,IsLearner,EmailAddress,CustomField,Supervisor_ExternalId
john.doe@demo.com,Password5467,John,Doe,0,00002,100012,1,1,john.doe@demo.com,Custom Value,
jane.doe@demo.com,Password5812,Jane,Doe,0,00002,100013,0,1,jane.doe@demo.com,Custom Value 2,100012

 

Courses:

Spreadsheet

 
Text 
Name,ExternalId,TypeName,ActiveStatus,Description,Certificate_IsEnabled,ToggleEmails,SendNewEnrollmentEmail,SendEnrollmentCompletionEmail,AllowReEnroll,AllowFailure
Marketing 101,901,OnlineCourse,0,Course description 1,0,1,1,1,1,1
Introduction to Calculus,902,InstructorLedCourse,0,Course description 2,1,1,1,0,,1

 

Course Enrollments:

Spreadsheet

 
Text
User_ExternalId,Course_ExternalId,IsActive,Status,DateStarted,DateCompleted,Score,Credits,AcceptedTermsAndConditions,DateExpires,Session_ExternalId
100012,901,1,1,2015/06/05,,,,1,2016-06-05,
100012,901,0,3,2005/11/15,2006/01/06,70,3,1,,
100013,902,1,3,2010/04/10,2010/07/30,87,,1,,647

 

Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

0 Comments

Article is closed for comments.