2.4 Item Presentation and Navigation

The term Navigation (or interface navigation) within assessments refers to the provided possibilities for test-takers to switch back and forth between items. Software tools and assessment platforms provide different interfaces for navigating between Tasks, and as the ICT/ATP Guidelines for Technology-Based Assessment suggest,²² test-takers should be informed about navigating between tasks and have opportunities to practice navigation before the assessments.

Screen Layout: In order to be able to describe the options for navigation within assessments, one must first be aware of how the instructions and items in a computer-based assessment are presented on screen. Either only one item is displayed in full screen (Full Page Items, see section 2.4.1), or one or more additional visual components, for instance, for navigation, progress display, etc., are presented together with the item (Integrated Item Presentation, see section 2.4.6) on screen.

Combining multiple text entries, for instance, in C-tests (see section 2.4.3) or more general as so-called Cloze tests or in an Embedded Answers response format results in screen layout with multiple interactions per page. Text entry response formats (and other response formats as well) typically define one focused element, resulting in a Within-Item Navigation. Combining multiple items, for example, items with a common stimulus can also result in dependencies between assessment items. For this reason, it may be helpful to further distinguish between navigation within related items (referred to as Units, see Within-Unit Navigation in section 2.4.4) and navigation between items (see Between-Unit / Test-Level Navigation in section 2.4.5). Finally, for operational reasons, some assessment components have a unique role. Examples of this are so-called stop items, which are used in group tests to synchronize test processing in terms of time (see section 2.4.2).

Presentation Size: The planned screen layout determines how much space the assessment content can occupy (i.e., the required Size of items on screen). Screen size is usually specified in pixels. In addition, the Resolution (also measured in Pixels) must be distinguished from the actual size of the display, which depends on the physical size of the screens (usually specified in Inches of the diagonal). Finally, for the translation from Pixels (resolution) to Inches (display size), the pixel density as the amount of pixels per inch (often specified as dpi, i.e., dots per inch) and potential zoom factors must be taken into account. Zoom factors can be device-specific (i.e., a magnification implemented, for instance, by the operating system) or in browser-based deliveries also implemented by the web browser.

How to do this with the CBA ItemBuilder? A property CBA Presentation Size is defined for each assessment component created with the CBA ItemBuilder. As discussed in section 3.2.2, the CBA Presentation Size defines the aspect ratio of item material as presented on screen, while the test deployment software (see chapter 7 can proportionally scale the content to fit on screen).

Aspect Ratio and Screen Orientation: The ratio of the width to the height of a screen is called the aspect ratio, resulting in terms like 4:3 (e.g., screens with a resolution of 1024x768 pixels), 16:9 (e.g., screens with a resolution of 1920x1080 pixels), etc., which are also used to characterize screens. Finally, the aspect ratio also determines the orientation of screens. If the width is larger than the height, one speaks of Landscape Format, if the height is larger than the width of Portrait Mode.

Windows Size and Browser Size: For web-based delivered assessments or other forms of delivery that are not standardized using a kiosk solution, the window size or the size of the browser must also be considered. In the worst case, this can cause the user to change the display size during test processing. On some operating systems, even for web-based deliveries, the browser full-screen mode can at least prevent the window size from changing.

Proportional Scaling: When assessments are not run on uniform hardware, determining the exact Item Size is often tricky in practice. It is even often impossible, especially if the item contents are only displayed embedded (see section 2.4.6) or if the available display area may even change due to the window size. This is countered by the demand to design as precisely as possible what the test-takers see in the context of a standardized assessment at a given time and how they can interact with the item material. One possible solution to this challenge is to implement Proportional Scaling of item content. This uses the available space on the screen while maintaining the aspect ratio until either height or width is exhausted.

Scrolling: An obvious way to display more content on a screen than the screen size allows is to use scrollbars. In fact, in HTML-based implementations of computer-based assessments, these often appear automatically. Vertical scrolling, in particular, should be used with caution for diagnostic reasons (see section 2.2.1). For detailed analyses, it may also be necessary to infer later from the data what a test-taker saw at a point in time (see section 2.8). Finally, it must be considered whether the entire screen or only parts of the display should scroll, for example, if a navigation area is to be permanently displayed on the screen.

Responsive Design: A more typical way for websites to deal with different resolutions and screen sizes is the use of so-called Responsive Designs. In Responsive Designs, the arrangement and display of the content are adapted to the actual available area (View Port). In the context of computer-based assessments, the use of responsive designs must be weighed up against the extent to which this can be reconciled with the goal of standardizing assessments.

2.4.1 Full Page Items

Full Page Items represent an item design in which minimal non-solution interactions with the assessment platform are required. Full Page Items can be implemented without scrolling and with scrolling, as shown in Figure 2.5. The central feature of Full Page Items is that they are presented exclusively on the screen, without being surrounded by a navigation area. Necessary navigation elements are displayed within the item when they should be available.

FIGURE 2.5: Example for full page navigation. (html|ib).

The design and presentation of items should always clarify whether they are instructional text or a question. In addition, it should be explained (either in the instruction at the beginning of the assessment or through short instructional texts) how the item can be processed and answered. If multiple pages are used, the navigation buttons must be repeated on each page.

How to do this with the CBA ItemBuilder? The creation of Full Page Items is the standard procedure for the CBA ItemBuilder when only the necessary navigation elements for switching between pages and finishing the task (see Next-Taskcommand in section 3.12.1) are placed on each page.

Forced Choice: A unique response format in which items are typically displayed without navigation is Forced Choice. In this format, which is also used for questionnaires, the test takers are not offered the option of not answering. As shown in the examples in Figure 2.6, the system automatically navigates to the next question as soon as an answer is given.

FIGURE 2.6: Example for forced choice navigation. (html|ib).

Multiple forced choice questions can also be presented on one page and still be administered in a fixed order, as the example in Figure 2.6 shows.

FIGURE 2.7: Example for multiple forced choice questions per page. (html|ib).

Blocked Item Response In cognitive and non-cognitive assessments, rapid responding (see Rapid Guessing and Rapid Responding in section 2.5.3) is a phenomenon that compromises the validity of responses. A naive way to limit response elicitation is to disable the buttons for a limited time. Example 3 in Figure 2.6 shows this concept in combination with the forced-choice response format, Figure 2.8 shows this concept with other response formats (see also Persic-Beck, Goldhammer, and Kroehne 2022).

FIGURE 2.8: Examples for Blocked Item Response with different response formats (html|ib).

2.4.2 Breaks / Stop Items

For different operational reasons, it may be necessary to interrupt and pause the flow of computerized instruments or to insert positions where test-takers should take a break. According to the logic described in this section, these pause pages are Full Page Items, since no further navigation should be possible.

Magic Word: In school assessments, test administration is often organized as on-site group tests with a test administrator present in the room. If the assessment is administrated using secure offline environments (e.g. in a kiosk mode, see 7.2.3), breaks for the group can be easily implemented with a Magic Word (i.e., a piece of information that functions as a password), which is only announced by the test administrator at the time when every test-takers should start with a following section.

Dashboards: Computer-based assessments are also often administered in network environments (e.g., web-based), enabling centralized control across test-takers. So-called Dashboards for test management can show, for example, which task a test-taker is currently working on or whether a test taker needs support. Dashboards can also be used for the control of group testing or for monitoring online test administrations, for instance, by remote interviewers.

2.4.3 Within-Item Navigation

As soon as several input fields or components with an input focus are placed on a page, the possibility or the necessity for a Within-Item Navigation arises. Only particular response formats, such as Point and Click response formats (see, for instance, section 3.9.10 for so-called ImageMaps) and Drag and Drop response formats (see section 4.2.6) do not work with input focus.

C-Test (Text Completion Test): In the C-Test (Text Completion Test) in Figure 2.9,²³, the text entries can be made in any order. Nevertheless, switching from one gap to the next with the tab key is also possible to allow fast editing.

FIGURE 2.9: Example for a C-Test / Text Completion Test (html|ib).

How to do this with the CBA ItemBuilder? If many input fields are combined on one page, as shown in the example in Figure 2.9, then controlling the tab order and setting your input focus to the first text field is recommended. Item authors can define the tab order (see section 3.7.7), and input focus can also be set with a particular operator (see section 4.4.6).

If more than one answer can be given on a page, it may also make sense not to restrict the processing order due to construct-related considerations. An example of a test originally developed for paper-based assessment is shown in Figure 2.10. In this so-called Digit Symbol Substitution Test (representation based on Jaeger 2018), it is implicitly assumed that the test is processed in sequential order.

FIGURE 2.10: Example for a Digits Symbol Substitution Test (html|ib).

In the computerization with the CBA ItemBuilder shown in Figure 2.10, the processing can happen in any order and it must be additionally explained in the instruction whether only the consecutive items should be evaluated.

2.4.4 Within-Unit Navigation

When multiple tasks share a common stimulus, the tasks and the stimulus material are often referred to as a Unit. Within Units, dependencies can arise, i.e., answering individual subtasks is influenced by preceding or subsequent subtasks. Accordingly, computerization often allows subtasks to be answered and re-answered in any order, which leads to so-called Within-Unit-navigation. Figure 2.11 illustrates an example for within-unit navigation based on the separation between Question and Stimulus, as used, for instance, in the Programme for International Assessment of Adult Competencies (PIAAC, see Figure 2.2. in OECD 2019 that shows the PIAAC user interface). ²⁴

FIGURE 2.11: Example for question and stimulus navigation (html|ib).

The left side in Figure 2.11 shows the individual questions, which can be navigated between using the > and < buttons. The split stimulus can be seen permanently on the right side in Figure 2.11. The stimulus itself can provide further opportunities for interaction for complex interactive items. The ? button indicates how additional editing help can be integrated into the unit.

How to do this with the CBA ItemBuilder? The separation of items into different areas, e.g., a stimulus area and individual task-pages as shown in Figure 2.11, is possible in the CBA ItemBuilder by using either special page types (e.g., X-Pages, see section 3.4.2) or by using partial pages (e.g., PageAreas, see section 3.5.4).

2.4.5 Between-Unit / Test-Level Navigation

For technical reasons, there is no need to restrict navigation within computerized tests. Just as test-takers could flip back and forth in paper-based test booklets, they could do the same in computer-based assessments.

Figure 2.12 shows schematically how free navigation could look, illustrated for a unit with only one task, a unit with multiple tasks and scrolling, and a unit with a text stimulus and multiple tasks.

FIGURE 2.12: Examples for free Between-Unit navigation (html|ib).

How to do this with the CBA ItemBuilder? For implementing assessments with the CBA ItemBuilder, it must be acknowledged that completely free navigation with the means of the CBA ItemBuilder is only possible within Tasks. Each CBA ItemBuilder Task can request navigation to a next and a previous Task (see Runtime Commands in section 3.12), but the delivery software is responsible for responding to these requests. However, delivery software can also provide an additional user interface that can be used to drive any CBA ItemBuilder Tasks (see Integrated Item Presentation in the next section).

From a psychometric/diagnostic perspective, it should be considered what reason free navigation is necessary and whether the advantages of this item presentation outweigh the disadvantages already present with paper-based assessment. Suppose test-takers can freely navigate between a more significant number of items. In that case, it can be assumed that the influence of test-taking strategies such as skipping items when needed and returning later, is increased.

In practice, the options for navigation are often restricted, either due to psychometric considerations (for instance, to support adaptive testing), or because of operational considerations (for instance, to simplify the implementation of time restriction). For instance, backward navigation across Units (i.e., the possibility to go back to a previous unit) can be disabled in CBA (see, for instance, Cochran et al. 2020). As shown in Figure 2.13, test-takers can move freely only within a unit. When test-taking requests to navigate forward from the last task or page of a unit, a pop-up message is displayed informing the test-taker that it will be impossible to return to the previous tasks.

FIGURE 2.13: Examples for restricted Between-Unit navigation (html|ib).

How to do this with the CBA ItemBuilder? Feedback to test-takers about potential navigation restrictions is just one of the many options for providing targeted information during test-taking (see 2.9.1). They can be realized with the CBA ItemBuilder, for instance, using dialog pages (see section 3.15), so-called Page Areas (see section 3.5.4), or by so-called Conditional Links (see section 4.3).

2.4.6 Integrated Item Presentation

In various contexts, items are displayed and integrated within other web application or web-based deliveries. This includes, for instance, the integration of CBA ItemBuilder items within QTI-based deployment software (such as TAO) as Portable Custom Interaction (PCI) (see section 7.4), or more general embedding CBA ItemBuilder items using the so-called TaskPlayer-API (see section 7.7). Another possible scenario is the integration of assessments within learning management systems (e.g. Moodle) based on the LTI interface (see section 7.5.8).

In scenarios where assessment components are integrated into other deliveries, the following points need special consideration:

• Presentation Size: The size of the item displayed within other software environments is often smaller than the available space on the screen. Therefore, the items may need to be made smaller so that they can be answered reasonably in the embedded presentation. This can also result in the display of scroll bars on different screen sizes if the web application adjusts the size of the integrated items (responsive design).
• Re-Visits / Restore: The integration of items within external applications may result in items being terminated and re-entered by external navigation. To ensure a coherent picture for the test-takers and that the content shows the last working state on each new visit, the items must save their state before exiting.
• Full Screen: Full-screen presentation control must take into account when items are integrated into other web applications.

How to do this with the CBA ItemBuilder? The CBA Presentation Size must be defined for each CBA ItemBuilder project (see section 3.2.2). The TaskPlayer-API allows to store the item state at each point in time (see section 7.7). Commands to control full screen presentation (see section 3.12.3) might not work in Integrated Item Presentation.

Suppose assessment components, which themselves allow inner navigation or should not be terminated at any time, are integrated into other environments (e.g., via PCI in TAO, see section 7.4). In that case, the navigation in the external application must be blocked.

References

Baghaei, Purya, and Mona Tabatabaee. 2015. “The C-Test: An Integrative Measure of Crystallized Intelligence.” Journal of Intelligence 3 (2): 46–58. https://doi.org/10.3390/jintelligence3020046.

Cochran, Gary L., Jennifer A. Foster, Donald G. Klepser, Paul P. Dobesh, and Allison M. Dering-Anderson. 2020. “The Impact of Eliminating Backward Navigation on Computerized Examination Scores and Completion Time.” American Journal of Pharmaceutical Education 84 (12): ajpe8034. https://doi.org/10.5688/ajpe8034.

International Test Commission and Association of Test Publishers. 2022. Guidelines for Technology-Based Assessment.

Jaeger, Judith. 2018. “Digit Symbol Substitution Test: The Case for Sensitivity Over Specificity in Neuropsychological Testing.” Journal of Clinical Psychopharmacology 38 (5): 513–19. https://doi.org/10.1097/JCP.0000000000000941.

———. 2019. Beyond Proficiency: Using Log Files to Understand Respondent Behaviour in the Survey of Adult Skills. OECD Skills Studies. OECD. https://doi.org/10.1787/0b1414ed-en.

Persic-Beck, Lothar, Frank Goldhammer, and Ulf Kroehne. 2022. “Disengaged Response Behavior When the Response Button Is Blocked: Evaluation of a Micro-Intervention.” Frontiers in Psychology 13 (November): 954532. https://doi.org/10.3389/fpsyg.2022.954532.

---

22. “Planning for TBAs should include the design of tutorials for navigation of test elements.”(International Test Commission and Association of Test Publishers 2022)↩︎

23. The content for this example is taken from Baghaei and Tabatabaee (2015)↩︎

24. The example item NavigationExampleStimulusQuestion.zip is created using simple pages (see section 3.4.1), components of type PageArea (see section 3.5.4) and a navigation implemented completely in the finite-state machine (see section 4.4).↩︎