Preface
This book describes the CBA ItemBuilder, a tool implemented over the last years for developing and delivering complex items and tasks. The CBA ItemBuilder is a development environment for assessments enabling researchers, domain experts, and teachers to develop complex items without requiring any specific programming skills. Possible item types include but are not limited to: web environment simulations (including browser, search engines, etc.), desktop application(s) simulation and simulations using finite-state machines. The CBA ItemBuilder has been used in several national and international projects and is constantly being improved. It is available free of charge for non-commercial projects. This book describes how to use the CBA ItemBuilder for different user groups involved in the use of technology-based assessments as part of Open Educational Resources.
The latest version can be found online as pdf, epub or as html at github.
Suggested Citation:
DIPF, Frankfurt am Main, Germany. https://doi.org/10.5281/zenodo.10359757
Why read this book
This book is for item authors and researchers preparing and organizing assessments or analyzing the result data or log data from computer-based assessments.
For the following use-cases we have created section that allow to reach a particular goal without the need to read the entire book:
Quick start section 1.4 describes how CBA ItemBuilder project files that contain one or multiple tasks with single items or units can be previewd. If you are new to computer-based task authoring tools, you can download existing ItemBuilder projects, open them in the CBA ItemBuilder and use the preview function to view the items ‘live’. This allows, for instance, item reviewers to use the CBA ItemBuilder to view (and review) items.
Quick start section 1.5 provides the necessary information to explore the item scores, that are created by a particular task defined in an CBA ItemBuilder project file. This allows, for instance, domain experts to verify the scoring of items, that is to relate test-taker’s responses to result variables provided by the items.
Quick start section 1.6 shows how identifiers that can be found in (log) data can be linked to parts of the instrument, as it is required for scientists after obtaining data collected with the CBA ItemBuilder. This allows, for intsance, using mock items to explore interactive computer-based tasks.
Quick start section 1.7 explains how to revise existing items (e.g., adapt, adjust, modify or improve content). The ability to adapt the content of computer-based items without having to consult with the programmers or designers of the item material is one of the important prerequisites for open computer-based assessments.
Quick start section 1.8 summarizes how programmers can integrate content generated with the CBA ItemBuilder into their Web application. Moreover, it is explained how HTML content can be used as part of CBA ItemBuilder projects.1
Quick start section 3.3 gives hands-on instruction how to create simple single-page items using the CBA ItemBuilder. This section is intended for users who want to familiarize themselves with the CBA ItemBuilder while using the software.
Quick start section 7.1 provides an hands-on example how to use the assessment components created in section 3.3 to realize a typical offline deployment.
All quick start sections are cross-referenced to the relevant sections in the main text, allowing readers to dive deeper into the subject matter if needed.
Structure of the book
For researchers, teachers, lecturers, content experts, students and Ph.D. students, student assistants, and interested readers who want to go deeper into the use of the CBA ItemBuilder, the book starts with a short primer on computer-based assessments in chapter 2.
Building on the objectives and some basic vocabulary, chapter 3 then describe the implementation of assessment components using static components provided by the CBA ItemBuilder. The static content mainly covers the visible elements of items, instruction pages, etc., and includes the development of items using multiple pages and dialogs.
If the displayed task content should change either time-controlled or in reaction to observed test-taker behavior, the items become dynamic. Chapter 4 deals with the enrichment of assessment components, such as items, with dynamic content.
Chapter 5 of this book is then devoted in detail to the evaluation of answers, the so-called Scoring of computer-based items.
For users who would like to plan and create new assessments, chapter 6 provides further tips and suggested solutions for practical challenges in creating computer-based assessments using the CBA ItemBuilder.
Chapter 7 summarizes possibilities to combine single CBA ItemBuilder project files to complete assessments and introduces software tools and platforms to deliver assessments in practice.
Chapter 8 finally deals with workflows, i.e., typical processes for preparing, testing, delivering, and archiving computer-based assessments.
The book concludes with a discusses of the CBA ItemBuilder as a tool in the context of open science and reproducible research.
Contributors
Starting with version 9.0 of the CBA ItemBuilder, item-projects created with this authoring tool can be flexibly integrated into various HTML-based execution environments without the need for special server components. I have taken this milestone as an opportunity to combine the existing documentation and the CBA ItemBuilder Wiki in a new user book-length documentation.
This book would not be possible without the support of
- all previous and current CBA ItemBuilder users,2
- Frank Goldhammer (Center for Technology-based Assessment), Eckhard Klieme (retired Director of the department of Educational Quality and Evaluation and Research Fellow at DIPF) and Marc Rittberger (Director of the Information Center for Education),
- Robert Baumann, Ingow Barkow, Paul Libbrecht and Daniel Schiffner (IT engineers at the Center for Technology-Based Assessment),
- Margit Mikula, Marisa Herrmann, Gabriele Gissler and Britta Upsing (CBA Item author support and early users of the CBA ItemBuilder at the Center for Technology-Based Assessment),
- Rachel Ghebrehawariat and Angelika Sichma (project assistants at the Center for Technology-Based Assessment),
- all former and current doctoral students at the Center for Technology-Based Assessment,
- student assistants at the Center for Technology-Based Assessment,
- and many others.
The idea of creating an authoring tool for the implementation of complex interactive items goes back to Jean Paul Reef and was supervised conceptually and organizationally by Heiko Rölke in the first years at the Center for Technology-based Assessment at the DIPF (Rölke 2012).
Special thanks go to the long-standing cooperation partners Softcon GmbH / naragro AG (Michel Dorochevsky, Constantin von Kirschten and the entire team).
