This document is a technical reference for the CBA ItemBuilder and the CBA runtime.
Its chapters fall into three groups:
Chapters most relevant for assessment content authoring in the CBA ItemBuilder:
Chapters explaining mechanisms available to run assessments with the CBA runtime:
Chapters relevant for assessment results analysis:
A summary table of all display components is given here.
A table of all display component attributes is given here.
Some components (e.g. Button and MenuItem) can trigger a command specified in the Command
attribute.
The available commands are:
If you assign a command and a state machine event to a button React will first execute the assigned command and afterwards it will trigger the state machine event.
The menu items to switch to bookmarked pages in the menu opened by the BOOKMARK command (and in the global ‘browser context menu’ and in the menu opened by the Bookmark button) display a page icon for the bookmarked page besides the page URL. This page icon is the resource specified in the ImageReference
attribute of the page’s Frame.
For the display of overlapping components the following rules apply:
The following table lists the layers from bottom to top:
Layer | Component Types Included in Layer |
---|---|
Frames Layer | TabfolderFrame, TaskbarFrame, WebBrowserFrame, Frame, Panel, PageArea, ChildArea, WebChildArea, TabfolderGroup, WebBrowserToolbar, TreeChildArea, TreeView |
Images Layer | All types of buttons if in Image-Button mode: TabButton, TaskbarButton, TaskbarStartButton, HomeButton, ForwardButton, BackButton, Bookmark, Button; ImageField, ImageMap, ImageValueDisplay, IconValueDisplay |
Lines Layer | LineHorizontal, LineVertical, Rectangle |
Animations Layer | Audio, Video, AudioVideoDisplay, VideoValueDisplay, Timer |
Text and Interactions Layer | All types of buttons if not in Image-Button mode: TabButton, TaskbarButton, TaskbarStartButton, HomeButton, ForwardButton, BackButton, Bookmark, Button; ComboBox, CheckBox, Link, SimpleTextField SingleLineInputField, InputField HTMLTextField, TextField, ImageTextField, RadioButtonGroup, MenuBar, Menu, ExternalPageFrame, Table, Tree, TableCell, TableCellEditor, VariableValuDisplay, TextValueDisplay, VariableValueInput, SpinnerVariableInput, ScaleVariableInput |
Several components embed complete pages as their display content:
Embedding the same page in different components will result in independent page instances at assessment run time: Modifications (like typing in text in an InputField or selecting a CheckBox) in one page instance will not affect other instances of the same page; scoring calculations will differentiate between the page instances involved. See the section about Component References in the chapter about DSL Operators down below on how User Defined ID paths provide a mechanism to refer to component on specific page instances.
It is possible to embed pages that contain page embedding components. The setEmbeddedPage
operator can modify the nesting structure at assessment run time. The system restricts the depth of page nestings to 15 levels at assessment run time to avoid resource problems freezing the browser (e.g. due to a page inadvertently nesting itself): When the limit is reached an error message text is embedded in place of yet another nested page.
The position of the embedded page in the embedding page area depends on the top level frame of the embedded page:
Frame
with the Dialog attribute set to DIALOG
or MODAL_DIALOG
the frame appears at an offset position inside the embedding page area given by the frame’s X,Y
attributes. (Embedding a dialog page is uncommon. The item author might use this to get the X,Y attributes of the frame into play.)Besides embedded pages the system supports the display of up to six pages at the same time:
Dialog
attribute set to ‘UNDEFINED’.Dialog
attribute set to ‘DIALOG’. You can open these dialogs by task initialization rules, page links or the openDialogPage
operator, all targeting a page with the Dialog
attribute set to ‘DIALOG’.Dialog
attribute set to ‘MODAL_DIALOG’. You can open these dialogs by task initialization rules, page links or the openDialogPage
operator, all targeting a page with the Dialog
attribute set to ‘MODAL_DIALOG’.Where the pop up dialog appears depends on the technique used to open it:
X,Y
attributes of the Frame
in the target dialog page.X,Y
to specify the position of the dialog explicitly. If these parameters are not given the dialog will appear in the center of the available screen area.Besides the dialogs described above the SEARCH button command may open a page (as configured in the project global properties) as a modal dialog.
A timer event starts its waiting period as soon as the statemachine reaches a state that can accept the event. Once the waiting period is over the event fires. If the statemachine enters another state that can also accept the timer event during the running wait period, the timer will restart its wait period. The same applies if the statemachine reenters the original state via a ‘=>’ rule (e.g. TimingState => TimingState {...}
). The timer will continue with the old wait period if an internal transition (e.g. Timing State internal {...}
) inside the original state takes place.
If a state accepts more than one timer event all of these timer events will start their waiting periods.
The length of the waiting period is given as seconds in decimal form. The system schedules waiting periods in units of milliseconds (i.e. up to three digits after the decimal point) but the actual event firing might not occur with this accuracy.
A listing of all operators available in the languages to specify
Many operators use parameters that refer to a display component. Such a reference consists of a list of user defined IDs (see the component attribute ‘User Defined ID’) separated with a dot. For brevitiy we call such a list a ‘User Defined ID Path’. Some examples:
ContinueButton
: A single User Defined ID refers to a display component on a page that is not embedded in another page.AreaOne.ButtonToGoBack
: This referes to a display component (with ID ‘ButtonToGoBack’) on a page that is embedded in a page area with ID ‘AreaOne’. The page area might be a WebPageArea component in a web-browser page or the ChildArea in a taskbar page or the PageArea in a simple page.For pages bearing the page tag ‘xPage’ and not the page tag ‘standardPage’ the reference path starts in the X-Page. For all other pages it starts in the standard page.
Similar to a ‘User Defined ID Path’ uniquely identifying a display component each node in a tree is identified by a ‘Node Path ID’ with this format:
<parent Node Path ID>_<node ID>-<node number>
where
<parent Node Path ID>
is the Node Path ID of the parent node. A top-level node uses the User Defined ID of the tree.<node ID>
is
<node number>
is the number of the node in the branch (from 0 to n-1).The scheme applies for item design time and assessment run time. At design time the Node Path IDs are recalculated if the User Defined ID of a node is modified or if nodes are deleted. At assessment time adding or deleting a node has no impact on the Node Path ID labeling of other nodes, i.e. holes in the numbering scheme of siblilngs ar not fille up when nodes are deleted or new nodes are created.
The API for JavaScript code embedded in an ExternalPageFrame supports four features:
The JavaScript code embedded in an ExternalPageFrame may use the ‘post message’ mechanism provided by HTML5 to send events at assessment run time to the CBA runtime like this:
window.parent.postMessage(data, targetOrigin)
The parameters in this call are:
data
A string representing a JSON object with these attributes:
attribute | type | explanation |
---|---|---|
microfinEvent | <string> |
The id of an FSM event to be triggered in the CBA runtime. If not provided no FSM event is triggered. |
microfinVariable | {variableName: <string>, newValue: <number>} |
Set a variable in the CBA runtime to a new value. If not provided no variable value is changed. (This does the same as setVariable . We keep it for backwards compatibility only. For new code you should use setVariable instead.) |
setVariable | {variableName: <string>, newValue: <string or number or bool>}, newType: <string, optional> |
Set a variable in the CBA runtime to a new value. Implicitly creates the variable it does not exist yet and newType is provided. (If newType is provided and the variable exists already the types must match. You cannot change the type of an already existing variable.) |
getVariable | {variableName: <string>, callId: <string>} |
Get the value of a variable in the CBA runtime. The CBA runtime will send back a postMessage event with a this payload: {result: { name: <string, name of variable>, type: <string, type of variable>, value: <string or number or bool, current value of the variable>}, callId: <string>} . If getVariable is not provided, the CBA runtime will not send back a message. |
traceMessage | {…} | A message object to be written to the trace log (in an entry of type JavaScriptInjected). The message object must be serializable as JSON string. If not provided no trace log entry is written. |
traceType | <string> |
A string to be written as ‘type’ in the details structure of the trace log entry. If not provided the details structure in the trace log entry has no type attribute. (The trace log entry’s main type will be JavaScriptInjected in any case. The type value given here appears inside the details structure of the trace log entry.) |
userDefIdPath | <string> |
The user defined ID path of the ExternalPageFrame component as it is defined in the designer. This field becomes part of the JavaScripInjected trace log entry. If not provided the attribute in the log entry remains empty. |
indexPath | <string> |
The index path of the ExternalPageFrame component. This field becomes part of the JavaScriptInjected trace log entry. If not provided the attribute in the log entry remains empty. |
traceCount | <number> |
The increment add to the userInteraction counter in the CBA runtime due to the event transmission. Must be an integer greater or equal 0. If not provided the counter is not incremented. |
Example:
var data = {
'microfinEvent': 'resetEvent',
'microfinVariable': { variableName: 'VarA', newValue: 0 },
'userDefIdPath': 'myExternalCodeFrame',
'traceMessage': { errorLevel: 3, errorMessage: 'Triggered reset of test via FSM event.' }
'traceCount': 1
}
Note:
If the content in an ExternalPageFrame is embedded in the item itself (i.e. an ‘internal URL’ is specified for the ExternalPageFrame), the URL given to the IFrame contains two additional query parameters userDefIdPath
and indexPath
. These parameters contain the user defined ID path and the index path of the ExternalPageFrame component instance. They can be used to fill the corresponding attributes in the data structure to transmit for trace log entries.
targetOrigin
A string that specifies what the origin of the target window must be for the event to be dispatched. This is given either as simple asterisk *
(which indicates no preference) or as a URI. For more details refer to documentation for the windows.postMessage mechanism, e.g. [https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage].
The DSL operator callExternalPageFrame
sends an event via the ‘post message’ mechanism provided by HTML5 to the embedded JavaScript code.
The values given as parameters in the operator call appear in the transmitted event as a JSON array with one entry per given value.
Example:
The operator called like this in the statemachine rules
callExternalPageFrame(PageFrameUserDefId, "doItProcedure", 13, "simpleMode")
will deliver this JSON structure as event payload:
[ "doItProcedure", 13, "simpleMode"]
The CBA runtime supports an API to backup/restore state used by the embedded JavaScript code (i.e. a ‘snapshot’ for the state used by the JavaScript code):
getState()
on the content window of the IFrame and saves the object returned internally.setState(savedState)
on the content window of the IFrame with the state saved before destroying the IFrame.For the mechanism to work the embedded JavaScript code must define the two methods getState() and setState(savedState) on the global (i.e. window) level:
getState()
expects no parameters and must return the state to be saved across the destroy/recreate phase.setState(savedState)
expects one parameter which is the state saved before the desctruction of the IFrame. It must restore the state of the code embedded in the IFrame.The CBA runtime skips the state save/restore steps if the browser denies access to the code running in the IFrame due to cross origin restrictions.
The CBA runtime skips the save step if the code running in the IFrame does not define the getState()
method.
The CBA runtime skips the restore step if the code running in the IFrame does not define the setState(savedSate)
method.
The CBA runtime polls for the setState(savedState)
method to appear with one try every 20 milliseconds up to a maximum of 100 tries.
The React runtime calls the getState()
method synchronously, i.e. it will destroy the IFrame immediately once the getState()
method returns.
The CBA runtime also calls the getState()
method synchronously before writing a snapshot to the trace log to obtain up-to-date data from the embedded JavaScript code.
The JavaScript code embedded in an ExternalPageFrame may declare variables of an item at item design time.
The mechanism is triggered by an ‘Import variables’ command that the item author must run while editing a page. The command sends an event via the ‘post message’ mechanism provided by HTML5 to the embedded JavaScript code. The event contains a payload object like this:
{
callId: <String containing the phrase importVariables>
}
The embedded JavaScript code must respond to this with a ‘post message’ event containing a payload like this:
{
callId: <String, the callId sent by the command triggering the response>
intialVariables: [
{
name: <String>
defaultValue: <String>
type: <String, one of 'Integer', 'Number', 'String', 'Boolean'>
namedValues: [
{
name: <String>
value: <String>
}
]
},
...
]
}
We officially support the CBA Item Builder on Windows only. To install the CBA Item Builder follow these steps:
allFeatures.cbalicense
) into the directory. Do this before you start the ItemBuilder for the first time. See the chapter on license files for more details.cba-itembuilder.ini
file if required. (The delivered default configuration should work in most cases without modifications.) See the chapter below for details on the configuration options.File/Directory | Description |
---|---|
workspace | directory for temporary data |
workspace/Templates | directory for template files |
workspace/.metadata/.log | main log file |
cba-itembuilder.ini | main configuration file, see chapter below |
<temporaryFilesFolder>/cba-item-builder<port> | Folder used for temporary files. The appended port number distinguishes CBAItemBuilder instances running in parallel, see the -Djetty.port configuration option. The <temporaryFilesFolder> is the folder for temporary files provided by the operating system (e.g. C:/User/<username>/AppData/Local/Temp). |
The table below describes the configuration options available for use in the cba-itembuilder.ini file.
Option | Values | Description |
---|---|---|
-DAllowFSMDebugging=false | true (default) , false | Enables Ctrl+M to open the State Machine debug window. |
-DAllowScoreDebugging=false | true (default) , false | Enables Ctrl+S to open the Scoring debug window. |
-DAllowTraceDebugging=false | true (default) , false | Enables Ctrl+Y to open the Trace debug window. |
-DdefaultItemHeight=NNN | integer (default is 715) | Default value for the preferred item height. |
-DdefaultItemWidth=NNN | integer (default is 722) | Default value for the preferred item width. |
-Djetty.port=NNNN | integer (default is 7070) | Port used to publish the preview to the browser. You must change the default value if you want to run more than one ItemBuilder in parallel since two ItemBuilder instances cannot share the same port. |
-DsimpleTextEditorWrap=no | yes (default), no | Enable automatic word wrap in the text editor for SimpleTextField and InputField (applies at item authoring time). |
-DbrowserChoice=chrome | chrome, internal | Sets the default for the preview browser preference. No setting will use the standard browser of the operating system. |
-DpreviewAutoSave=always | always, confirm, never (default) | Sets a default for the auto-save operation at each preview: With ‘always’ each preview will save the item implicitly, with ‘confirm’ the user is asked whether to save or not, with ‘never’ the preview will not implicitly save the item. |
-DisTreeViewShowsAll=yes | yes, no (default) | The Component Subtree View (only available with an ‘experimental’ license) will show additional elements for debugging purposes. |
-DonlineHelpUrl=String | URL string | The URL to the online help used by the Help->HelpOnline command. Default is https://cba.itembuilder.de |
-Dapp=true | true, false (default) | For developers only. (Gets React app from application.manager/app/locations/react folder.) |
License files control the available feature set in an ItemBuilder installation. Place serveral license files in the installation directory to combine the required feature set:
File name | Feature Set | Description |
---|---|---|
textBlocksEditing.cbalicense | Text Block Editor | The command ‘Edit all text blocks’ iterates through all text fields where text blocks are defined and allows adjustments of text blocks. |
textEditing.cbalicense | Text Editor | The command ‘Edit all text fields’ opens a table of all translation relevant texts which can be edited. |
xliff.cbalicense | Import XLIFF | Commands to import an XLIFF file including previewing and finalizing the translation. No modifications of the item layout and structure are allowed. |
xliffEditor.cbalicense | Translation XLIFF Editor | Command translate an time including starting the ‘XLIFF (OLT) Editor’, import an XLIFF file, preview and finalize the translation. No modifications of the item layout and structure are allowed. |
xliffVerify.cbalicense | Translation Verification | Commands to import an XLIFF file, preview, verify and finalize the translation. ‘Verify text translation’ iterates through all text fields where text blocks are defined and displays the translated and original text side by side for final review and text block adjustment. The XLIFF editor is not accessible. No modifications of the item layout and structure are allowed. |
standard.cbalicense | Standard | All features except MicroFIN and MicroDYN features, also excluding ValueMaps, MicroDYN pages, state machine editors, variables, XLIFF translation related commands, ‘Text Block Editor’ and ‘Text Editor’. |
microFIN.cbalicense | State Machine | All ‘Standard’ features and additionally state machine editors, ValueMaps and state machine variables. |
microDYN.cbalicense | MicroDYN | All ‘Standard’ features and additionally MicroDYN functionality including MicroDYN pages and MicroDYN variables. |
extContent.cbalicense | External Content | All ‘Standard’ features and additionally the external content explorer. |
allFeatures.cbalicense | All Features | All features above. |
experimental.cbalicense | Exeperimental | Experimental features not yet for widespread public use. |
The temporary data in the folder workspace/.metadata
will reflect some restrictions implied by the license file in use. These will not be overwritten when a new license file is added after the ItemBuilder’s first run. If you don’t see expected extensions to the feature set due to an additional license file, drop the workspace/.metadata directory and restart the ItemBuilder. Be aware that this will also drop the settings that you configured in the Preferences dialog.
The CBA runtime internally separates two layers:
The ItemBuilder preview uses the embedded controller layer for historical reasons. In future releases we might drop the embedded controller layer in the CBA runtime and make the ItemBuilder preview use the Task Player API instead.
New or customized execution environments should implement their own controller that uses the Task Player API directly, i.e. they should use the CBA runtime as a plain task player and ignore the embedded controller layer. The Electron Execution Environment example shows how to implement a controller that uses the Task Player API only.
The Task Player API uses the the ‘post message’ mechanism provided by HTML5 to send events between the CBA runtime and the external controller.
At startup the task player sends a ‘ready’ message via the Task Player API and waits for the external controller to give commands via the Task Player API. You may provide the window and the domain URI that the task player should use to send the initial ‘ready’ message as URL parameters or in a configuration structure named ‘cba_runtime_config’ declared in the global JavaScript scope (i.e. as window.cba_runtime_config). The cba_runtime_config mechanism has precedence: The task player will ingore an URL parameter if the corresponding value is given in the cba_runtime_config structure.
Here is a table of the parameters used to control the initial behaviour of the task player:
Query parameter or structure attribute | Values | Description |
---|---|---|
eventDomainUri | <string> |
The task player will use this string as target origin when sending the initial taskPlayerReady message. If the parameter is not given it will use the domain of the calling URL. |
eventTargetWindow | self or parent or opener |
The task player will send the initial taskPlayerReady message to the window specified by this parameter. Use parent if the task player runs in a child window (like an IFrame) and should send the message to the parent window. Use self if the task player should send the message to its own window. Use opener if the task player should send the message to the window that opened the task player’s window. The task player uses self as the default if no value is given. |
initialWaitingMesssagePrimary | <string> |
The task player will show this message at the top of the page which it displays while waiting for configuration data to arrive. |
initialWaitingMesssageSecondary | <string> |
The task player will show this message in the middle of the page which it displays while waiting for configuration data to arrive. |
startTaskPlayerEvent | <string> |
The task player will wait for an event dispatched with this name on the task player’s window. If not specified the task player will not wait but start immediately. You might use this to get your postMessage receiver ready before the task player sends the ‘task player ready’ message. |
Examples:
https://myServer/testTakerPage?eventDomainUri=*&eventTargetWindow=parent
window.cba_runtime_config = {eventDomainUri: '*', eventTargetWindow='parent'}
From the API’s point of view the task player follows this state transition table:
State | ->Event-> | <-Event<- | State | Remarks |
---|---|---|---|---|
Initial | task player ready | (never) | NotLoggedIn | The task player switches to the NotLoggedIn state as soon as it is ready to receive commands via the Task Player API. |
NotLoggedIn | ‘setUserId’ | ‘logout’ | NoTaskRunning | The events are triggered by the Task Player API messages setUserId and logout . |
NoTaskRunning | ‘startTask’ | ‘stopTask’ | TaskRunning | The events are triggered by the Task Player API messages startTask and stopTask . |
A basic, minimal sequence of TaskPlayer API interactions will usually contain these steps (with the involved Task Player methods given in square brackets):
taskPlayerReady
]setTraceContextId
]setTraceLogTransmissionChannel
]showLogin
, loginDialogClosed
]setUserId
]addItem
]setTaskSequencer
]startTask
]stopTask
, startTask
]taskSwitchRequest
] and will continue running the current task until the task sequencer sends commands to stop or switch to another task [Task Control: stopTask
, startTask
].Other, common interactions are:
setWaitMessages
]insertMessageInTrace
]logout
]getOldScoringResult
, getOldScoringResultReturn
] or for the currently running task. [Scoring Control: getScoringResult
, getScoringResultReturn
]. To retrieve the scoring result at the end of a task, stopping the task first and then calling getOldScoringResult(...)
runs faster than getScoringResult()
first and stopping the task afterwards.setTraceContentFilter
]The Task Player API provides more methods for specific cases (e.g. pause task execution, preload tasks state, activate debugging windows, force trace log transmissions, trigger additional snapshots written into the trace log, configure audio/video recording transmissions, send state machine event to running task). See the Task Player API Reference for more details.
A table of all Task Player API methods is given here.
The task player uses two different streams to send assessment session traces and results to the execution environment:
The Task Player API provides similar configuration methods for both streams (see sections about trace control and recordings control respectively in the API reference).
The task player supports these transmission channels for both streams:
The tracing data stream collects accruing trace data for a configurable time span before transmitting it in chunks to the execution environment.
The recording data stream waits for each recording until it is finished before it sends the recorded data to the execution environment.
The CBA-ItemBuilder saves items on disk as ZIP files. To run such an item using the Task Player API your controller will need the following parts of the item included in the ZIP file:
config.json
contains the JSON-serialized item configuration description that you should use as value of the itemConfig parameter in the addItem call. The file might not be there if the item author saved the item in an inconsistent state.resources
and external-resources
contain the resources that your server should provide as static content on HTTP GET requests issued by the running TaskPlayer. You specify the URLs targeting your server as parameters resourcePath
and externalResourcePath
in the addItem call.The file stimulus.json
contains a JSON-serialized, short description of the item, that the controller may use to obtain meta information about the item. The file might not be there if the item author saved the item in an inconsistent state. The attributes in the file are:
Attribute | Value Type | Description |
---|---|---|
runtimeCompatibilityVersion | String | The version of the runtime library to be used for displaying this item. |
itemName | String | The name of the item. |
itemWidth | Integer | The width of the preferred item size. |
itemHeight | Integer | The height of the preferred item size. |
tasks | List of Strings | The names of the tasks defined by the item. |
resources | List of resource entries | The list of image/audio/video resources potentially downloaded by the item at run time. See the next table for a description of the entry structure. |
externalResources | List of resource entries | The list of html/javaScript resources potentially used by the item at run time as IFrame content anchor. See the next table for a description of the entry structure. |
Each resource entry provides the following attributes:
Attribute | Value Type | Description |
---|---|---|
pathInZip | String | The path of the resource file inside the item ZIP file. |
size | Integer | The approximate size of the resource file in bytes. |
type | String | The type of the resource, one of image, audio, video, urlTarget. |
To rename an item without the ItemBuilder you have to change the following files inside the item ZIP file:
File | Description |
---|---|
internal.json | Change the value of the projectName field to the new item name. |
config.json | Change the value of the name field of the top level object to the new item name. (The field is a sibling of the runtimeCompatibilityVersion field.) |
stimulus.json | Change the value of the itemName field to the new item name. |
All the other files inside the ZIP file serve internal purposes of the CBA-ItemBuilder or the EE4CBA. You don’t need them for the TaskPlayer based approach.
The Task-Player API opens a path to implement deployment scenarios with limited server connections. Here are options and observations that form the building blocks for these implementations:
file:/
protocol to fetch the resources.With these options one can implement controller layers that support deployment scenarios with limited server connections:
The task player library remains agnostic to the deployment scenario: You can use the same version of the task player library for all cases.
The Example Execution Environment is a minimalistic example of an execution environment that uses the basic features of the TaskPlayer API. It presents an assessment based on a linear sequence of tasks configured by a JSON configuration file. It transmits trace log and recording results by sending HTTP POST requests to an external server as described in Result Upload Streams. It supports different CBA runtime versions (versions 9.2 and above should work) in a single installation.
Prerequisites:
Installation Units:
To install the example execution environment perform the following steps:
npm install
to install all required third party libraries.The execution environment is configured by modifying files in these subfolders of the folder <InstDir>/public:
Folder | Explanation |
---|---|
controller |
The file config.json contains a JSON structure with basic settings. See the chapter about controller configuration below for details. |
assessments |
The file config.json contains a JSON structure listing the tasks that will be executed during each user session. See the chapter about assessments configuration below for details. |
items |
The folder contains the items that define the tasks used during the user sessions. See the chapter about the items folder below for details. |
react-runtime |
The folder contains the CBA runtime libraries (in the subfolders css and js ) and HTML files required to integrate the CBA runtimes into the execution environment. You need to touch this only if you want to add more CBA runtimes to the execution environment. See the chapter about the react-runtime folder below for details. |
The example execution environment comes preconfigured with an example controller configuration file, an example task sequence, some CBA runtime versions and a set of example items.
The file public/controller/config.json
contains a JSON object with these attributes:
Attribute | Description |
---|---|
traceLogTransmission |
This configures the channel to transmit the trace log written during the user sessions. The attribute is optional. If not specified the trace log will appear as messages in the browser console. If it is specified the trace log is sent via HTTP POST requests to a server or via postMessage events to a browser window. The value for the attribute must be an object as described in the paragraph following this table. |
mathJaxCdnUrl |
This configures a CDN URL where items with MathJax content will try to download the MathJax libraries. The attribute is ignroed for items that do not use the MathJax embedding feature of the HTMLText component. |
itemSize |
This configures the size used by the execution environment to display an item. The atrribute value is an object with two integer attributes height and width . |
players |
The value for this attribute is an array of objects. Each object configures a CBA runtime with three string attributes: playerId is an arbitrary but unique id for the CBA runtime, runtimeVersion is the runtime version of the CBA runtime and frameContentFile is the name of the HTML file that integrates the CBA runtime into the execution environment. To reduce overhead in the browsers you may drop all CBA runtimes that you do not actually need from the list. See the chapter about the react-runtime folder below. |
showPlayerInfo |
This boolean attribute switches a header line on or off. This header line shows, which CBA runtime is currently active. |
userId |
This attribute is optional. If it appears, the value for this attribute must be an object with one string attribute urlParameter which specifies the name of an optional URL parameter used to specify a user ID for the session. If this parameter appears in the calling URL the execution environment will use the given value as user ID for the session. Otherwise it will show a login dialog to obtain a user ID. |
As value of the traceLogTransmission
attribute you may specify one of two types of objects:
The attributes of the HTTP channel configuration object are:
Attribute | Description |
---|---|
channel |
The string "http" . |
transmitUrl |
Specifies the target URL for the HTTP POST requests. |
interval |
Specifies the interval between HTTP POST requests (in milliseconds). |
httpTimeout |
Specifies the timeout to wait for the return status of each HTTP POST request. |
The attributes of the PostMessage channel configuration object are:
Attribute | Description |
---|---|
channel |
The string "postMessage" . |
targetWindowType |
Specifies the target window for the events, one of the values parent , self or opener . |
targetOrigin |
Specifies the URL origin of the target window (i.e. the target origin for the postMessage calls sending the events). Use * if you don’t want restrict the target origin. |
interval |
Specifies the minimum interval between postMessage events (in milliseconds). |
You may use a EE4CBA server as target for the trace log HTTP POST requests: Use an URL like http://localhost:8101/ee4cba-api/trace
as transmitUrl
attribute value. The transmitted trace logs will appear in the EE4CBA server as if the user session was hosted by the EE4CBA. See the description of the EE4CBA for details on how to finally retrieve the trace log information from there. You may have to tweak the configuration of the Wildfly server to avoid Cross-Origin-Resource-Sharing (CORS) issues. See https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS for a general explanation of the problems involved here and http://www.mastertheboss.com/web/jboss-web-server/how-to-configure-cors-on-wildfly/ for details on how to do this for the Wildfly server.
See the description of the trace log transmission channel in the chapter about the TaskPlayer API and the chapter about the TraceLog for details if you want to use another server as target for the HTTP POST requests. You might have to configure the server to allow CORS as mentioned above for EE4CBA as transmission target.
See the description of the trace log transmission channel in the chapter about the TaskPlayer API for details if you want to use the Post Message channel configuration. For this option to be useful you probably have to configure a trace log listener in the MessageReceiver
or add an event listener on the receiving window. See the section about development for details on the MessageReceiver
.
The file public/assessments/config.json
contains a JSON object with a single attribute tasks
that has an array of objects as value. Each object describes a task in the task sequence of the assessment. The attributes of these objects are:
Attribute | Description |
---|---|
item |
The name of the item defining the task. |
task |
The name of the task to run. |
scope |
A key to establish different instances of the same task in a user session. You may use any string as key value. Objects in the array with the same values for all three attributes refer to the same task instance: At each occurrence in the array the task instance will continue where it was stopped at its previous occurrence, i.e. the user ‘returns’ to the task at its previous state. |
The folder public/items
contains the items that define the tasks used during the assessment.
For each item unpack the item ZIP file into its own sub folder. Use the item’s name as name for its sub folder. (Note: The item’s name might differ from the name of the item’s ZIP file. To find out the actual name of the item look into the stimulus.json file inside the ZIP file or open the item with the CBA Item Builder.)
For convenience you may also put the ZIP files themselves into the public/items
folder. But be aware that the execution environment will simply ignore these. It will only use the sub folders public/items/<item-name>
with the unzipped item content.
The folder react-runtime
contains the CBA runtimes that the execution environment supports.
For each CBA runtime this folder must contain:
css
subfolder.js
subfolder.<div>
element with id root
and class loader
which will be filled with the content provided by the CBA runtime. You could use one of the already provided HTML files as a template.To make a CBA runtime actually available you have to add an entry in the players
list in the controller configuration file. See the chapter about the controller configuration for details. Be aware that each CBA runtime that you add to the players list, will be loaded and run on the browser - even if none of your items actually requires it.
During assessment the execution environment automatically picks a compatible CBA runtime for each item that it loads.
To start the example execution environment run the command npm start
in the <InstDir>. This will fire up a basic HTTP server. To run an assessment session use a browser and access the URL http://localhost:3000.
To make the execution environment available on the network interfaces of your computer perform the following steps:
npm install -g http-server
.npm run build
in the <InstDir>. This will create a subfolder build
.http-server
.You may deploy the content of the <InstDir>/build folder as static content to any HTTP server or your choice. In combination with some other web application configured as trace log transmission target this might be sufficient for small studies with limited requirements:
If you need to use another URL root path (like cba-assessment
in http://192.168.94.13:3000/cba-assessment
) add an attribute homepage
(like "homepage": "cba-assessment"
) to the JSON object in the file package.json
before running the npm run build
command.
You may use the example execution environment as a starting point to develop your own execution environment taylored to your hosting environment.
The folder <InstDir>/src contains all sources. All the code is written in TypeScript to improve readability. You might combine this with plain JavaScript if you are not comfortable with TypeScript.
We use React to implement the GUI part but this is limited to the top most layers of the application (index.tsx, App.tsx, PlayerFrame.tsx) and does not diffuse into those parts that you will most likely want to extend (TaskSequencer.tsx and Controller.tsx).
Here is a diagram showing the overall structure of the code:
The following table gives a short summary of the main components. See the comments in the code for more explanations.
File | Description | Modify? |
---|---|---|
index.html |
The template for the entry point file of the application. The template establishes the top level <div> element that the index.tsx file will use to attach the App.tsx component. The template will be filled by the build command with further <link> tags that load all our libraries. |
rarely |
index.tsx |
The first part of the initialization of our application: Read the controller configuration, initialize the MessageReceiver.tsx (for postMessage events) and put the App.tsx component into the top level <div> . |
rarely |
App.tsx |
Establish PlayerCatalog.tsx , ItemCatalog.tsx and TaskSequencer.tsx . Configure the controller by calling the main method in Controller.tsx . Wrap each CBA runtime in a PlayerFrame.tsx . |
rarely |
Controller.tsx |
The controller coordinating all actions required to run the tasks list in the CBA runtimes. Currently the controller is just a method that attaches listeners to the MessageReceiver.tsx . These listeners send messages to the task players via the MessageSender.tsx The controller delegates the decision which task to run next to the TaskSequencer.tsx . See the comments in the file for detailed explanations. |
usually |
TaskSequencer.tsx |
The component deciding which task to run next. Our implementation supports a fixed, linear sequence of tasks only. You may implement your own algorithm here. | usually |
PlayerFrame.tsx |
An Iframe that contains the frameContent<XXX>.html which configures and loads the CBA runtime. Registers the CBA runtime as task player in the PlayerCatalog . |
rarely |
frameContent<XXX>.html |
Each of these files configures a CBA runtime in the window.cba_runtime_config structure and loads it via the link and script tags. You have to adapt the names of the main.xxx.css file and the main.xxx.js file for each CBA runtime version. |
frequently |
MessageSender.tsx |
A method used to send messages to the TaskPlayer running in the CBA runtime. This implements the full TaskPlayer API already (even though we use only part of it in our example code). Therefore, you usually should not need to extend it. | rarely |
MessageReceiver.tsx |
A Service that receives messages from the task players running in the CBA runtimes. Code handling those messages attaches itself as listener on this service. The service implements the full TaskPlayer API already (even though we use only part of it in our example code). Therefore, you usually should not need to extend it. | rarely |
PlayerCatalog.tsx |
A catalog of all CBA runtimes (i.e. task players). | rarely |
ItemCatalog.tsx |
A catalog of the items that are already loaded on all compatible task players. | rarely |
FileDownload.tsx |
A bunch of helper methods to download configuration files and item description files. You might adapt this for more attributes in the configuration files or even introduce new configuration files. | sometimes |
To upgrade to a new version of the CBA runtime add a new entry in the players
list in the controller configuration and add the CBA runtime files and the integrating HTML file as described in the chapter on the react-runtime
folder.
If you don’t need to support older CBA runtime versions you may drop the files of the old CBA runtime, its integrating HTML file and its entry in the players
list.
(In case of compatibility breaking changes in the TaskPlayer API you may also have to adapt your code accordingly. We will limit such breaking changes to really important improvements that we cannot easily achieve otherwise.)
Prerequisites:
Installation Units:
To install the EE4CBA follow these steps:
Create a separate directory for the CBA Execution Environment <InstDir> e.g. “C:/EE4CBA”.
Unpack the ZIP archive “ee4cba-wildfly-10.1.0.Final.zip” in <InstDir>.
Unpack the ZIP archive “ee4cba-standalone-<R.V.S>.zip” in “<InstDir>/ee4cba-wildfly-10.1.0.Final/standalone”.
Clean up and delete temporary files in WildFly (when updating an installation):
“<InstDir>/ee4cba-wildfly-10.1.0.Final/standalone/tmp/*”.
Remove the content repository (when updating an installation), it is not always compatible to previous releases:
“<InstDir>/ee4cba-wildfly-10.1.0.Final/bin/jackrabbit” and
all directories in “<InstDir>/ee4cba-wildfly-10.1.0.Final/data/ee4cba_assets/”
Note: Due to the path length limit in Windows of 255 Bytes we recommend installing the CBA Execution Environment in a directory rather near to the root of the file system.
The following parameters in startup file “ee4cba-wildfly-10.1.0.Final/bin/standalone.conf.bat” (in Windows) are relevant (in line with JAVA_OPTS definition):
Parameter | Comment |
---|---|
-DAllowFSMDebugging=true | Allows displaying the Finite State Machine (FSM) Viewer (by Ctrl+M) for debugging the FSM rule logic (value must be “true”). Only intended for item development. |
-DAllowScoreDebugging=true | Allows displaying the Scoring Viewer (by Ctrl+S) when previewing a task (value must be “true”). Only intended for item development. |
-DAllowTraceDebugging=true | Allows displaying the Trace Events Viewer (by Ctrl+Y) when previewing a task (value must be “true”). Only intended for item development. |
-DuseServices=true | Use services for test person authentication and management of tests.(default is false) |
The Administration console is protected by a password (default “admin”). It can be configured in “<InstDir>/ee4cba-wildfly-10.1.0.Final/standalone/configuration/ee4cba_config.xml”:
<category name="adminauthentication">
<property name="adminauthentication.password" value="admin"/>
</category>
To start and stop the execution environment use the scripts:
To see the React based test person view use an URL like
http://localhost:8101/cba_react_app/
To activate special extensions you can add further parameters to the URL like this:
http://localhost:8101/cba_react_app/?headerFooterMenu=true&headerButtons=true
Here is a table of all available URL parameters:
Parameter | Comment |
---|---|
headerFooterMenu=true | Activates the menu bars to navigate between tests and tasks. |
headerButtons=true | Activates the buttons to specify the current Interest and Competence level at the top of the screen. |
To see the administrator view use an URL like
http://localhost:8101/ee4cba/admin
To retrieve trace logs for the React based architecture you have two options:
Use the ‘Export Results’ button in the EE4CBA Administrator’s GUI. This will download the trace logs for the React based test runs as JSON file react_traces.json (together with all the RAP based snapshot and trace files). See Trace Log Entries for a description of data contained in the react_traces.json file.
Use a HTTP-based API provided by EE4CBA:
Get an overview about the users and the sessions they attended (very useful for the following available endpoints): http://localhost:8101/ee4cba-api/sessions
Get the sessions attended by one user (‘someUserId’ in the example): http://localhost:8101/ee4cba-api/sessions/someUserId
Drop all user data kept in the EE4CBA sessions: http://localhost:8101/ee4cba-api/drop-session-data
To drop all user data for a specific user append the user ID (‘someUserId’ in the example): http://localhost:8101/ee4cba-api/drop-session-data/someUserId
Get all trace logs kept in the EE4CBA currently (not recommended if the system collected lots of traces, rather use more specific calls ): http://localhost:8101/ee4cba-api/traces. See Trace Log Entries for a description of the returned data.
To get all trace logs for a specific user append the user ID (‘someUserID’ in the example) to the URL: http://localhost:8101/ee4cba-api/traces/someUserID. See Trace Log Entries for a description of the returned data.
To get all trace logs for a specific session append the session ID (‘aSessionID’ in the example): http://localhost:8101/ee4cba-api/traces/someUserID/aSessionID. See Trace Log Entries for a description of the returned data.
Get all the recordings kept in the EE4CBA currently (not recommended if the system collected many recordings, rather use more specific calls): http://localhost:8101/ee4cba-api/recordings
To get all the recordings for a specific user append the user ID (‘someUserId’ in the example): http://localhost:8101/ee4cba-api/recordings/someUserId
To get all the recordings for a specific session append the session ID (‘aSessionId’ in the example): http://localhost:8101/ee4cba-api/recordings/someUserId/aSessionId
To get all the recordings for a specific source component in a specific session append the source component ID (‘aComponentId’ in the example): http://localhost:8101/ee4cba-api/recordings/someUserId/aSessionId/aComponentId
To get a specific recording for a specific source component in a specific session append the number of the recording (‘aNumber’ in the example): http://localhost:8101/ee4cba-api/recordings/someUserId/aSessionId/aComponentId/aNumber
This chapter describes the trace log created by the runtime library. The main part of the trace log is a list of trace log entries describing the events that happened during the assessment run.
Each trace log entry contains these attributes common to all log entry types:
Further attributes of the log entry depend on the type of the entry. They form the ‘details’ structure of the trace log entry.
For entries triggered by a mouse click the ‘details’ structure contains the mouse click coordinates: clientX, clientY, pageX, pageY, screenX and screenY.
Here is an example of a trace log entry:
{
entryId: ‘32356’,
timestamp: ‘2018-05-14T17:06:27.510+0200’,
type: ‘Link’
details: {
indexPath: ‘...’,
userDefIdPath: ‘...’
}
}
We describe the types of trace log entries created by the runtime library in a listing with one paragraph per entry type. These paragraphs contain:
For each additional attribute the table specifies:
MANDATORY
or OPTIONAL
.<boolean>
or <string>
.{...}
. Another table describing the complex value type follows below the attributes table.[...]
. Another table describing the list item type follows below the attributes table.You find a listing of the entry type paragraphs here.
The chapter about the Task Player API explains how to retrieve the trace logs directly from the runtime library via the trace log transmission channel. The chapter about EE4CBA explains how to retrieve trace logs from EE4CBA.
The trace log transmission channel of the TaskPlayer API and the API of the EE4CBA execution environment use different top-level structures to add meta data about the trace log and to wrap the list of trace log entries:
The transmission channel of the TaskPlayer API sends a JSON object with attributes metaData
and logEntriesList
in each transmission. The logEntriesList
is an array of the trace log entries as described above. The metaData
attribute contains an object with details about the running session (sessionId
as specified by TaskPlayer API call setTranceContextId
, userId
as specified by TaskPlayer API call setUserId
, timestamps and version information). The CBA release distribution provides a JSON schema for this in the file traceLogCallback.schema.json.
The API of EE4CBA returns an array of objects with attributes userId
and sessions
. The sessions
attribute contains an array of objects that describe the sessions for the user. Each of the session objects contains an attribute sessionId
and an attribute dataEntries
. The dataEntries
attribute is an array of the trace log entries as described above. The CBA release distribution provides a JSON schema for this in the file traceLogEE4CBA.schema.json.
This chapter describes the scoring result structure created by the runtime library. This structure appears
getScoringResult
or getOldScoringResult
call in the TaskPlayer APItaskResult
in the taskSwitch
entries of the trace log.The scoring result consists of name-value pairs with values of type integer, boolean or string.
It is represented as a single JSON object with one attribute per name-value pair: The attribute name is the name of the name-value pair, the value of the attribute is the value of the name-value pair.
The name-value pairs to appear in the scoring result are configured by the item containing the running task. In the remaining part of this chapter we describe the name-value pairs for items that are created with the CBA ItemBuilder.
Items created with the CBA ItemBuilder calculate the scoring result based on hits, misses, hit-classes and miss-classes. The names of these hits, misses and classes appear in the names of the value pairs.
Example:
For a hit named allMatchesFound
pairs like these will appear:
"hit.allMatchesFound": true
"hitText.allMatchesFound": "we got it"
"hitWeighted.allMatchesFound": 3
In this documentation we describe them with patterns with name placeholders and type designators like this:
hit.<hitName>: <bool>
hitText.<hitName>: <string>
hitWeighted.<hitName>: <number>
Name placeholders are:
<hitName>
: The name of the hit. The pair will appear once for each hit.<missName>
: The name of the miss. The pair will appear once for each miss.<className>
: The name of the class. The pair will appear once for each class.Type designators are:
<bool>
: A boolean value.<string>
: A string value.<number>
: A number value.The following sub-chapters describe the name-value pairs that items built with CBA ItemBuilder will create.
Name | Type | Description |
---|---|---|
hit.<hitName> |
<bool> |
Is the hit activated? |
hitText.<hitName> |
<string> |
The result text for the hit (result_text operator). |
hitWeighted.<hitName> |
<number> |
The activated weight of the hit (0 if hit is not activated). |
hitClass.<hitName> |
<string> |
The class of the hit (no pair if no class is assigned). |
hitsCount |
<number> |
The number of activated hits. |
hitsAccumulated |
<number> |
The accumulated weight of all activated hits. |
Name | Type | Description |
---|---|---|
miss.<missName> |
<bool> |
Is the miss activated? |
missText.<missName> |
<string> |
The result text for the miss (result_text operator). |
missWeighted.<missName> |
<number> |
The activated weight of the miss (0 if miss is not activated). |
missClass.<missName> |
<string> |
The class of the miss (no pair if no class is assigned). |
missesCount |
<number> |
The number of activated misses. |
missesAccumulated |
<number> |
The accumulated weight of all activated misses. |
Name | Type | Description |
---|---|---|
classHitWeighted.<className> |
<number> |
The accumulated weight of the activated hits in the class (no pair if there are no hits in the class). |
classMissWeighted.<className> |
<number> |
The accumulated weight of the activated misses in the class (no pair if there are no misses in the class). |
classFirstActiveHit.<className> |
<string> |
Pair appears for calculation mode “first active hit/miss” only: The first active hit in the class (no pair if there are no hits in the class, an empty string if none is activated). |
classFirstActiveMiss.<className> |
<string> |
Pair appears for calculation mode “first active hit/miss” only: The first active miss in the class (no pair if there are no misses in the class, an empty string if none is activated). |
classResult.<className> |
<boolean> |
true if all hits in the class are activated and no misses in the class are activated, false otherwise. |
classMaxName |
<string> |
The class with the maximum activated weight. (If there is more than one class with the maximum activated weight we pick just one of them.) |
classMaxWeighted |
<string> |
The activated weight of the class with the maximum activated weight. |
Name | Type | Description |
---|---|---|
nbUserInteractions |
<number> |
The number of user interactions since the (re-)start of the task. |
nbUserInteractionsTotal |
<number> |
The number of user interactions in all previous runs of the same task (in the same scope). |
firstReactionTime |
<number> |
The time (in milliseconds) until the first user interaction happened after the (re-)start of the task. |
firstReactionTimeTotal |
<number> |
The accumulated first-reaction-times of all previous runs of the same task (in the same scope). |
taskExecutionTime |
<number> |
The time (in milliseconds) since the (re-)start of the task. |
taskExecutionTimeTotal |
<number> |
The accumulated execution times of all previous runs of the same task (in the same scope). |
Name | Type | Description |
---|---|---|
totalResult |
<number> |
1 if all hits and no misses are active, 0 otherwise. |