Certica Products

Response Scores

Updated Jul 30, 2021

By default, most interactions within an item authored in ItemLogic are boolean scored - that is, the possible scores achievable by the candidate for each interaction are 0 or 1. When this logic is applied to composite items (items with more than one interaction), each of the boolean outcomes are summed to form the item-level score.

Traditionally, the boolean scoring model works; it is what comes standard with many assessment platforms. It does not, however, adequately reward the candidate for more advanced interactions, where a range of potential scores would be a better fit in assessing their knowledge of a particular subject.

Notes:

  • Inline Text, Order, Match, and Matrix responses are the exceptions. While they may look like one interaction visually in the item editor, each choice or text entry and each matrix row represent a singular interaction. For example, an inline text interaction with two gaps would potentially be worth two points. Take this into consideration when defining your custom interactions.
  • It is vital that you check with your assessment platform partner(s) to ensure they support the type of scoring you define in ItemLogic.

Custom Scoring

To set up a custom score for an item in the item editor, define the possible choices. Then select all appropriate keys for the selected response.

Then click the Custom Scoring checkbox.

Click the Edit option.

Enter the values in the score fields.

Click Save (or press Enter if your cursor is still inside of one of the editable fields).

Note: When defining scores where (0 > x < 1), use values that are divisible by 0.25 or 0.10. Do not assign partial scoring in thirds. JavaScript-based scoring engines fail when adding 0.33 values and values of 0.33 only sum to 0.99.

Min, Max, and Def Fields

The +0.50 values next to each correct choice are likely to make immediate sense. Each selected option awards +0.50 points to the candidate. Selecting both options results in a score of 1. The Min, Max, and Def fields control the following:

  • Def: Default value. All options not that do not have a defined score (in this example: helium, carbon, nitrogen, and chlorine) are assigned this value. The recommended value of this field is below zero (ex: -1, -0.5).
  • Min: The minimum possible score that can be earned in this interaction.
  • Max: The maximum possible score that can be earned in this interaction.

Default Values Explained

To better explain the role of the default value, the table below shows some example outcomes.

Default Value Scenario Outcome
-1.00 Candidate selects two incorrect options (helium, carbon).
Raw candidate score for interaction would be: (-1 + -1 = -2). Since the Min field is set to 0, the candidate would receive 0 points.
-1.00 Candidate selects both correct options (hydrogen, oxygen), but then selects an incorrect option (helium).
Candidate would receive (0.5 + 0.5 + -1) = 0 points. This scoring model awards the candidate partial credit for selecting each option, but full penalty when any incorrect option is selected.
-0.50 Candidate selects both correct options (hydrogen, oxygen), but then selects an incorrect option (helium).
Candidate would receive (0.5 + 0.5 + -0.5) = 0.5 points. If the candidate were to select any more incorrect options, the score would be 0 points.
0 Candidate selects all options, correct and incorrect.
Candidate would receive (0.5 + 0 + 0 + 0.5 + 0 + 0) = 1 point. "What?" That is why a negative default value is imperative! There most always needs to be a penalty for selecting an option that is not correct, even partially.

Specifying Keys

Specifying Keys

Activated keys represent the desired candidate response.

You can award partial credit for incorrect selections. Do not activate keys for incorrect answers even when they offer partial credit.

Note: If you are creating an matching interaction, activate each of the matches that represent a correct pairing.

Item Min/Max

Item Min/Max

A higher-level of scoring for an item is also available for controlling the minimum and maximum values allowed for an item as a whole.

In the item editor, click the Customize button [1] and set the Min Score and Max Score values for the overall item [2]. This overwrites any Min or Max values you may have provided for the responses.

Note: If you raise the Max value for an item, remember to use custom scoring for the responses so that a candidate can actually earn more than 1 point.

Scoring Algorithms

Scoring Algorithms

The scoring algorithm for an item has the final say in how an item is to be scored. Although "Boolean" is often the default for an item bank, you can change this default if you have administrative access to an item bank. If you are an author or editor of an item, you can change the scoring algorithm for an interaction part or for an entire item that may have multiple parts.

To change the default scoring algorithm for all items in a bank:

  1. From within an item bank, click on the Settings tab [1].
  2. Select an option from the Scoring Algorithm drop-down menu [2].
  3. Click Save Settings [3].

This algorithm applies to the entire item, which is important to know especially if you have multiple interactions within an item such as in a multi-part item.

Change Scoring for Stem Interaction

To change the change the scoring algorithm for any stem interaction:

  1. In the item editor, click Customize.
  2. Choose a scoring algorithm:
    • Default (for the item bank; see above)
    • Partial (credit given if any correct responses selected)
    • All-or-Nothing (full credit given for only all correct responses)
    • Boolean (0 if any response is incorrect; otherwise 1)
    • PARCC theme/idea (customized for PARCC items)

Change Scoring for Multi-Part Item

You can also set the scoring algorithm for a multi-part item:

  1. From the item editor, go to the Settings tab.
  2. Make a selection in the "Scoring Algorithm" drop-down menu. 
    • Default (for the item bank; see above)
    • PARCC Profile (customized for PARCC items)
    • Partial - Dependent (credit given for the second interaction only when first interaction is 100% correct; not for use on items with more than two interactions) 
    • All-or-Nothing (full credit given only when all items are correct)
    • Boolean (0 if any item part is incorrect; otherwise 1)

Verifying Custom Scores

Understanding each of the controls is vital to authoring a properly scored interaction, particularly for more advanced interactions. The best way to verify your authoring is by testing the item yourself.

When you preview or render an item, try answering it in various ways to verify you have the scoring rules set as desired. The earned score and the maximum score achievable in the item will be displayed at the bottom of the item when you click Check Answer.

Previous Article Applying Tags to Items or Resources
Next Article Scoring Rubrics