5.5 Checklist and Complete Workflow

As a summary the following list describes the typical workflow that is required for implementing automatic scoring in the CBA ItemBuilder:

  • Prepare the implementation of scoring by defining explicit User Defined IDs (see 3.7.4) for all components that should be used for scoring. It is not possible to define scoring using the automatically generated User Defined IDs that start with a $-sign.

  • Define a task as an entry point for the CBA ItemBuilder project. Since the scoring definition is done per task, a task must always be defined first (see section 3.6 for details).

  • When tasks are defined, define Classes. For each variable that should be included in the result data for a particular Task define one variable. For all newly created items activate the option Use first active hit per class (applies to all classes).

  • Define Hit-Conditions, that evaluate to true if the required conditions are fulfilled (see section 5.3.2. Order the Hit-conditions and add a default condition with the hit-syntax true as last condition. This will ensure that each class has one active hit (see section 5.3.3). For a usual workflow Miss-conditions are not needed, neither are Weights.

  • Extract string information using the result_text()-operator, if necessary. While hits serve as values of categorical variables, the Result-Text can be used to capture numerical or text responses.

  • Assign hits to classes. Classes fulfill the function of variables in the scoring of CBA ItemBuilder projects. Besides the unique name of the class (variable name), a description of the class can be entered in the Class Comment (variable description).

  • Decide how to handle missing response and implement, if necessary, additional Hit-Conditions for omitted responses and not reached questions (see section 5.3.11).

  • Test the scoring implementation using the Scoring Debug Window. If the option Use first active hit per class (applies to all classes) was not activated, make sure that exactly one hit (or miss) is active for each class at any point in time (see 8.4.2).