Assessment with BadgeKit
As an issuer, if you use an assessment process to award badges, the BadgeKit Web app and API provide a range of tools you can benefit from.
The Web app stores pending applications, with reviewers able to log into BadgeKit and view application details, make awarding decisions and include earner feedback. Together with the Web app, the API provides a series of endpoints and webhooks you can plug your site into in order to complete a full assessment system for your earners.
BadgeKit has been developed to facilitate a typical assessment process, while still keeping you in control over the data for your community of earners.
This guide explains assessment for issuers who are using BadgeKit API and the BadgeKit Web App for badge creation, publishing and assessment.
First, an explanation of what assessment means for Open Badges. Badges are earned in different ways:
- Issuer issues a badge directly to an earner email address.
- Issuer gives the earner a claim code on completing a learning or other achievement; the earner enters the claim code into the issuer site and is awarded a badge.
- Badge is issued automatically following completion of other badges (this is a milestone badge).
- Earner applies for a badge, including supporting evidence; assessor reviews the application and decides whether to award the badge; issuer contacts earner, forwarding feedback from the assessor.*
*This is the assessment process.
The assessment of an Open Badge of course varies depending on the badge and issuer context. BadgeKit facilitates the following stages:
- Including criteria, rubrics and notes during badge creation
- Issuer site can then list criteria information so that earners can decide whether to apply/ choose evidence to include
- Submitting applications for badges, including evidence (can be text, image or link)
- Viewing applications alongside criteria, rubrics and notes
- Accessing earner-submitted evidence
- Submitting awarding decisions, including detailed feedback relevant to badge criteria and application as a whole
- Notification when an application review is submitted
- Issuer site can then contact the earner, offering the badge in successful cases
- Issuing badges
- Issuer site can create badge instances to award to specific earners (optionally when the earner accepts an offered badge following a successful application)
- Notification when a badge is issued
- Issuer site can then follow up, e.g. offering to push the badge to a backpack
When you create a new badge in the BadgeKit Web app, you can include multiple items of information that are relevant to assessment. In particular, the Criteria section lets you define what exactly is required of the earner to be awarded the badge.
You can set the number of Criteria items, then specify details for each one. If you mark a Criteria item as required, BadgeKit will use this to automatically calculate whether an earner has met the requirements for the badge during assessment.
You should include a description for each Criteria item - this may be displayed to potential applicants as well as to the badge reviewer during assessment. You can also include a note for the reviewer on each Criteria item - you can use this to guide the assessment decision, including information that will be presented to the reviewer but not the earner.
In the Criteria section in BadgeKit, you will also notice that you can include a link to external material relevant to the badge. This could be a rubric or, for example, an educational standard relevant to the badge.
As you can imagine, not all of these items will be required for every badge you create, but they are there to facilitate various types of badge.
Once you publish a badge in BadgeKit, it becomes available for listing on your site via the API.
See the following API guides relevant to these parts of the process:
- Retrieving Badges - retrieving available badges and displaying the details to earners on your site (you can present badge information including Criteria)
- Submitting Applications - allowing earners to submit applications for listed badges, including evidence
- Using BadgeKit API - general intro to the API
Once an earner has applied for a badge, their application will become visible in the BadgeKit Web app - in the Applications > Pending section.
What a particular BadgeKit user sees in the pending applications section depends on their account permissions and on their system context.
When you give access to a user with a particular login, you can configure the systems, issuers and programs that user can access badges for. You can also configure what the user can do while logged in, including drafting, publishing and reviewing.
Logged in users can also toggle between the different systems, issuers and programs they have permissions for. This means that the pending applications you see at any time are for the system, issuer or program whose context you are in and have reviewing permissions for.
When an earner applies for a badge on the issuer site, the issuer can use the BadgeKit API endpoints to forward the application data. BadgeKit API then stores this information and the Web app in turn retrieves it from the API for display to reviewers.
The BadgeKit pending applications section lists applications by badge and applicant email address. Reviewers can select individual applications to review, at which point they see the badge data together with the earner application info.
The application data presented during review includes the badge Description:
The reviewer can also see the Evidence submitted by the earner:
Alongside the evidence is the Criteria for the badge:
This includes the information set when the badge was created. The reviewer can check each criteria item and set whether or not the earner has met it using a radio button. The reviewer can also optionally include a comment on each criteria item.
BadgeKit will update the display with an indicator of whether or not the required criteria have been met. In the Finish section, the reviewer can include several items of general feedback to the earner before submitting the review:
When a reviewer submits an assessment for a badge application, the BadgeKit API sends a notification to the issuer webhook. Issuer sites can configure webhook URLs to which BadgeKit will send notifications of badging events - this is done in the BadgeKit API database. The issuer site can include code at the webhook location to process these notifications.
When an application review is submitted, the API sends the webhook the following information:
- application data
- with badge info (includes criteria)
- review info
- includes feedback from reviewer (general and on specific criteria)
- approved status (true or false)
The webhook code can therefore check whether or not the application has been approved, then communicate with the earner as appropriate, including a selection of the data items sent from the API.
Even if an application is marked as approved, this does not mean that the badge is automatically issued - the issuer site must issue the badge using the API endpoints. Depending on your earners, you may wish to offer the badge to successful applicants, waiting for their acceptance before issuing the badge. Alternatively the badge can be issued whenever an approved review is received by the webhook.
Applications will continue to appear in the pending list in BadgeKit following review, until they are marked as processed via the API, which the issuer site must also do explicitly. You can either do this in the webhook for application reviews being submitted or can do it when the earner accepts their badge.
See Application Review Webhooks for an overview of receiving data in a webhook for application reviews, responding to the earner to offer them a badge if approved.
Whether you decide to issue a badge as soon as an approved review is received by the webhook or following earner interaction to accept an offered badge, the process is the same. The issuer site is required to carry out two actions when issuing a badge following assessment:
- creating a badge instance
- marking the application as processed
Note that BadgeKit will not carry out any communication with the earner when a badge is issued. Instead, the API sends notification to your webhook URL when a badge is issued - you can then contact the earner directly.
BadgeKit API includes an endpoint for creating a badge instance (this is how you actually issue a badge following assessment using the API). Issuer sites can pass the earner email address to the endpoint for a particular badge to create an instance for it. The API returns information about the new badge instance as confirmation that the badge has been issued - by which we mean that a badge instance has been created for the earner email. The API also sends notification to the webhook URL.
The API also includes an endpoint for updating existing applications - issuer sites can use this to mark an application as processed by including a processed date.
See Awarding Badges for an overview of issuing badges using the API.
When a badge is issued, BadgeKit API sends another notification to the issuer webhook including the assertion data for the new badge. The issuer can then contact the earner to carry out any follow-up tasks such as offering to push the new badge to a backpack.
The badge issued notification occurs when a badge is issued using any route (i.e. it doesn't just fire for badge issuing following assessment, but for any badges issued via BadgeKit, for example using claim codes or the "Issue by Email" option).
To recap, here's an overview of a typical badge issuing process using assessment with BadgeKit:
- Issuer creates a new badge and publishes it in the BadgeKit Web app.
- Issuer site retrieves a list of published badges, presents them to earners.
- Issuer site includes facility for earners to submit applications for listed badges, including evidence.
- Issuer site forwards new applications for badges to BadgeKit API.
- Applications received by BadgeKit API are listed within the BadgeKit Web app pending applications section.
- Reviewers log into BadgeKit and review applications (viewing submitted evidence), submitting feedback and indicator of whether each criteria item has been met.
- API sends application review info to issuer webhook.
- Issuer webhook code communicates with earner regarding their application, offering the badge if approved.
- Earner interacts with issuer site to accept the badge.
- Issuer site creates a new badge instance for the earner email.
- Issuer site marks the earner application processed (application stops appearing in BadgeKit).
- API sends badge instance created info to issuer webhook.
- Issuer follows up with earner regarding their new badge.
BadgeKit has been designed so that issuers can choose which of the component parts in the API and app to use, so you do not need to implement the process exactly as it is described here. The processes are also designed to keep the issuer in control over earner data, which BadgeKit itself does not store.