+ ASSESSMENT PLANNING

When arranging for an assessment, any data held in the system can be reported upon to support planning and preparation. Lists of eligible applicants, sites and other examination details can also be generated as reports based on demographics, schedules and registration details. The Advanced Search feature allows administrators to design and refine reports and save them down for ready use in the future.

An interface between the dashboard and internal finance systems, as well as payment registration confirmation details, creates a streamlined administration environment. The system can merge registered candidate details with exam and payment details and confirmed and approved results.

+ WEB PORTAL CANDIDATE REGISTRATION

The EXAMINA+ test booking capability allows candidates to create a profile that can be used over time and across multiple exams. Eligibility for registration and enrolment by candidates is measured against preconfigured criteria that can include registered address, age cohort, pre-qualification approval, number of attempts per candidate or exam type and any other criteria. Administration team members also have an interactive dashboard to supervise, monitor, amend and withdraw registration on behalf of applicants.

The EXAMINA+ Registration Module supports identity management, including the bulk provisioning of users and allows SSO access from external systems. It supports the registration of candidates by providing a capability to upload and store candidate data, as well as import data via CSV files. This module also manages the allocation of user IDs and passwords and is currently being extended to incorporate facial recognition capabilities provided by NEC.

+ AUTOMATIC AND MANAGED COMMUNICATION

The EXAMINA+ system can automatically communicate with applicants and candidates through web portals. Alerts can be sent through a user’s profile page via SMS and/or email to notify recipients to apply to sit their examination, register their details, confirm payment, registration or application, examination details or any unforeseen changes.

+ PREFERRED VENUE SELECTION

EXAMINA+ will be integrated with the RMS Online Booking System to create test schedules.

Once their profile is created, candidates can select their preferred venue and examination time and the system will upload the resulting schedule for administration by internal staff and venue management.

Candidates who elect to withdraw from exams can do so through their private portal. All information included in the request, such as reasons and/or supporting information is linked to the candidate and payment confirmation details.

+ ON-DEMAND SCHEDULE MODIFICATIONS

The EXAMINA+ system is pre-integrated with the Opturion Event Planning and Timetabling system (EPT). This makes it easy to modify schedules due to different requirements and constraints (for example, unexpected attendees or extra activities), quickly finding the right schedule that will minimise disruption.

The EPT solution enables permitted users to change features of the schedule and run the scheduler as many times as necessary, ensuring the ability to satisfy the revised requirements.

+ EXAM DAY TEAM SCHEDULE MANAGEMENT/ ROSTERING AND UNEXPECTED ABSENCE MANAGEMENT

The integrated Opturion optimiser deploys multiple algorithms across many rapidly changing variables and allows customers to quickly run and rerun solutions to efficiently manage exam day schedules, staff rosters and unexpected absences.

Where resource constraints exist, the platform uses the advanced optimisation platform to develop the most appropriate schedules given the constraints.

Sessions are managed using the administration module in the platform but invigilators are given control of sessions in the case of high stakes examinations.

+ PAPER DELIVERY AND THIRD PARTY OPTIMISATION

CRM features can be configured as part of the solution. Alternatively, interfaces to existing CRM systems can be built to support communications with third party providers.

+ HUMAN MARKING AND PRINTING COORDINATION 

Settings can be changed within the marking schedule so the optimal form of marking is utilised. In the case of an OMR Paper (which is generally to be marked by machine), if the system confidence level is set to low it will move to human marking.

The integrated Item Bank item types include: multiple-choice questions (including single choice and both restricted or unrestricted choices), case study scenarios, choose any three, ranking items in order, short answer, OSCE, essay questions and drag and drop. Each item can be linked to themes, competencies and curriculum frameworks and items can be stored against tags for future reference.

+ ONLINE ITEM CREATION AND EDITING

Item Bank supports the creation of items in Rich Text Format, large graphics and multimedia files (including audio, video and images), flexible tagging frameworks and the automatic generation of exam papers based on user-defined criteria

+ TRANSFER OF ITEMS FROM EXTERNAL BANKS

The EXAMINA+ platform supports the importing of various types of statistical information, test result data and tags. Items can be imported and then stored for editing and preparation of new test papers via automated QTI-compliant transfers. Alternatively, manual import can also be achieved through the selection of Excel or CSV file packages. The interfaces stipulate data format, validation, timing, transfer quality and security.

+ PAPER COMPILATION AND CONTROL

Authorised users can create tests using a manual process or by utilising the blueprint framework. The framework allows users to select items at random based on selected criteria that includes tags, question categories, question types and degrees of difficulty.

Once initiated, items are assigned as drafts and will progress through the review and approval process. The item status will be recorded at each stage and authors can assign reviewers and review periods. When a task arises for a reviewer, they will receive a system notification to advise them of the pending job. Comments and feedback can be added to items whilst they are being reviewed.

A full history of item authoring and reviewing is stored, with all items being version controlled. All actions taken (including view, creation, update and deletion) are fully tracked and traced.

+ SECURE DATA MANAGEMENT

Data is stored at the primary platform and replicated in real time to the secondary platform. The primary platform has redundancy for both the application and database virtual servers and this is also reproduced in the secondary platform.

Periodic data backup procedures will be undertaken to a dedicated backup storage array. This data mirror will be remotely backed up to a secure storage facility located within the Genix Melbourne office.

All appropriate security measures are applied to protect the data according to any agreed data security classifications.

+ FINALISED ASSESSMENTS

Once refined and approved, the assessment is locked and no further changes are possible. Finalised assessments are then moved to the test bank and from here can be delivered to candidates.

Assessments can also be integrated with external test banks and other sources of examination content using web services.

The marking and scoring features of EXAMINA+ support a combination of machine marking and human marking based on user-friendly marking keys. EXAMINA+ provides a facility to automatically mark multiple-choice and short answer questions, with essay and case study questions routed to markers who can access and mark candidate responses online.

The marks generated by the marking exercise are collated for each candidate and then combined with any results from automated marking if applicable, before being made available in the Administration Portal for review. All scores and results are stored within the system and can be reported against related data, including candidate details, exam category, score, final result, date and venue.

The results notice can be printed directly from the Administrator dashboard or linked to individual preferences and results can be sent immediately if required. Once published, candidates will also be able to review their results from their private dashboard.

+ ONLINE MARKING ENVIRONMENT ESTABLISHMENT

Administrators set up the marking templates and assign markers and reviewers for each examination. Complete papers for a certain number of candidates or the same question across a larger number of candidates can be allocated. Markers will select appropriate marking keys, review candidate responses and record the assigned marks for each answer.

The system can automatically score and check responses to ensure that no question remains unmarked and this is complemented by adaptable marking workflows that can be tailored as required.

+ PAPER AND ONLINE ANSWERS VIEWABLE ONSCREEN

Once exams are complete, the online marking platform will commence automatic marking processes and trigger manual marking workflows. Optical mark recognition (OMR) technology captures information by digitally scanning documents filled out by hand. OMR provides several advantages over manual data collection such as increased accuracy, efficiency and overall cost.

If hand written responses need to be uploaded to the system, they will also be available for manual markers to review online.

+ DOUBLE MARKING / CONTROL MARKING

Double blind marking for human-marked responses is available and rules can be configured to determine how discrepancies in marker outcomes should be managed.

The Administrator can set marking thresholds for the allowable differential between two markers. If the differential exceeds the allowed value, then the system will refer to the set rule to determine if the two markers are given the responsibility of agreeing to a mark or if the marks are sent to a third party for review. These rules together with the number of rounds of double marking are fully configurable.

+ REVIEW OF MARKING OUTCOMES

The background audit function will record all actions undertaken and stamp each with detailed tags, allowing any issues or incidents to be reviewed in detail. A full audit trail is provided, with onscreen replays of user actions if needed.

Once finalised, the marks generated by the marking exercise are collated for each candidate and then combined with the results from any automated marking associated with the paper. Marks are then made available through the Administration Portal for review, where Reviewers and Administrators with the appropriate permissions can then access the marks and underlying images and conduct reviews as required.

The EXAMINA+ system supports the delivery of secure high stakes, low stakes and on demand linear assessments. The module contains functionality to support assessment administration including the creation of assessment events and test groups, as well as the allocation of resources such as rooms and invigilators. Assessments are prepared and published prior to an examination sitting and candidate and invigilator information is added to each paper, session and location.

When undertaking the test, candidates are presented with a captive browser that fully restricts their access to the exam system only. The system conforms to international and Australian government standards for both content usability and accessibility and provides a clear and consistent experience for all users. Within the browser, candidates can progress through the questions in order of appearance or move to any question at any time. The summary pane on the left of the exam window provides the ability to quickly see the question status and scroll through the question numbers. Questions can also be flagged for review, the colour of the question in the summary pane turning red for easy identification when scrolling.

+ ON DEMAND PRACTICE EXAMS

On demand exams (including practice exams) do not require a captive browser or an invigilator. This feature allows candidates to gain experience in the exam interface prior to sitting formal examinations.

Practice exams are made available to authorised candidates through a secure URL. Access can be gained through special or everyday credentials, for a defined period or unlimited time frame. Candidates can also be selected for access or granted universal privileges for the practice environment.

+ MULTI-FORMAT QUESTION ANSWERING

The examination module supports extensive question banks with multiple choice, written answers, image analysis (e-Film Reading) and case scenarios. Questions can be developed collaboratively and leverage highly graphical, dynamic images.

+ AUTOMATIC RESPONSE SAVING

The system automatically saves responses when a candidate clicks to move onto a different question and a periodic saving mechanism also occurs at defined points throughout the exam. As candidates progress through the exam, the responses are stored and can be returned to at any time. In the event of a major disruption within the room or at network level, the last saved responses across the sitting cohort are stored in two data centres, providing far greater confidence that the latest candidate answers will be able to be recalled.

+ TIME REMAINING MONITORING

The time remaining is clearly visible for candidates as they progress through the questions. Candidates can elect to complete an exam at any time during the examination window. If the exam time expires, the candidate will unable to enter any further responses.

+ INTEGRATED SURVEYS AND CANDIDATE FEEDBACK

Completion of exams can trigger an on screen survey for candidates to provide direct and immediate feedback.

Organisation and communication with examiners, invigilators, markers, reviewers and administrators

Invigilators manage exam sessions through their own secure interface. In the lead up to an exam, candidate lists are generated and made available to the assigned invigilator at a pre-determined time.

System background processes judge the workloads of all possible team members and make active decisions on assignments, which can subsequently be accepted or reassigned if the chosen member is absent or needs support.

+ ONSITE CANDIDATE VERIFICATION

Invigilators are presented with a registration screen that records the status of each stage of candidate registration. Candidate profiles are displayed against their unique id-number and clicking the ‘Verify’ button confirms that they have successfully completed all the manual identity checks required for the exam. Once satisfied, the invigilator assigns a machine and establishes logon rights. They can also record any notes at the time of verification or at any time during the exam, such as in the event of time changes or password resets.

Administrators have immediate, on-demand access to the entire invigilator record including candidate attendance lists. A QR code on the front of each candidate’s exam paper also provides complete details on both the candidate’s personal information and specifics of their exams and bookings.

+ INCIDENT CAPTURE, REPORTING AND INVESTIGATION

The invigilator module is configured to support the activities undertaken in the administration of tests. Invigilators can verify candidate identities and capture all incidents, events and points of note for each exam and site directly into the system.

The advanced audit capabilities of EXAMINA+ support the incident management process. Every action undertaken in the system by candidates, invigilators, markers and administrators is captured and stamped with the action type, date, time, user ID, IP address, screen viewed, time viewed, field altered, previous field content, and new field content.

+ SPECIAL NEEDS SUPPORT

When registering for an exam, candidates can be prompted to submit special requests, such as medical conditions, with supporting information that is uploaded with the registration. All active requests are assigned unique identifiers and managed as individual cases within the solution. The request will trigger the appropriate workflow depending upon the request type and the system will assign the first activity to an available team member with the appropriate role.

EXAMINA+ provides invigilators and candidates with various mechanisms to manage special needs requirements, including invigilator controlled time extension, exam suspension and pausing and candidate controlled font size switcher and finish time.