Workshop description Thursday, 4th of April 2019


Workshop 1: Computer-/Web-/Tablet based Exams

Dipl.-Inform. Med. Jörn Heid

UCAN offers the appropriate examination software for various application scenarios. For example, if you want to use a computer pool, CAMPUS can help you to perform fail-safe and legally compliant assessments. If you want to improve your OSCE, you can save yourself with tOSCE paper and the error-prone copying of the results. If, on the other hand, you would like to carry out higher-quality and safe examinations regardless of location, tEXAM is a possible candidate.

If you want to offer your students a progress test UCAN offers a platform to perform tests at home or even on smartphones while traveling including advanced concepts like key feature, videos or confidence ratings.

Target group: For newcomers, this workshop will introduce the three systems, allowing you to plan the first use of this software.

Take home message: Participants can take the first steps in their faculty to implement Computer-/ Web- /Tablet- based Exams.

Workshop 2: Scan your IMS-Exams with Klaus: Well-established Processes Revisited

Pauline Müller

As a member of UCAN you have access to Klaus/Edgar in order to implement scan-exams. If you already use this tool, you certainly already have established a valid process for handling your own exams. Haven´t you? Take the chance to assess your current approach and check the new features that have been introduced recently: K-Prim-questions on exam sheets with tabular layout, double-sided fixed answer sheets which can be easily distributed by simply copying the sheets.

The workshop offers the opportunity to rethink your processes in order to work even more efficiently in the future. We present a practical example of an exam with known participants, printed seat numbers on the sheets and flexible choice of the permutation for each participant. Together, we go through the scanning process and review the results. Questions are welcome at any time and we are happy to discuss your ideas and suggestions. There will also be room to debate whether and how exams on paper should look like in the future. And of course, we would like to know which features are important for you to improve your work with Klaus/Edgar.

Target group: All users of Klaus/Edgar are invited to learn about new features, to exchange experiences and to develop requirements for future usage in this workshop.

Take home message: Users of Klaus/Edgar are aware of their options with the new features in Klaus/Edgar are able to implement them into their exam processes. New requirements and ideas for new or improved features of Klaus/Edgar are taken into account in the development planning.

Workshop 3: Assessing and providing feedback via video annotation

Dipl. Päd. Anna Mutschler, Dipl.- Inform. Med. Winfried Kurtz

To assess skills, attitudes as well as knowledge and give feedback to the students’ performance, videos can be used for various purposes in formative or summative exams. Analogous to written, oral or practical exams (e.g. OSCE´s), it is necessary to determine, what exactly should be assessed and to which points the students should be given feedback (e.g. through checklists, horizons of expectations).

The quality of the physician-patient communication has a significant impact on the treatment, the satisfaction and health-outcome of patients. Against this background we will use medical interviewing as an example to give an insight into the usage of the annotation of videos. A simulated doctor-patient interview will be recorded on video, annotated according to certain criteria of medical interviewing and communication skills. After providing the feedback to the workshop participants, we will discuss together further developments of this module, which is part of the aPortfolio.

Target group: Teachers and assessors

Take home message: The participants get to know the unlimited possibilities of video-assessment and gain their first experiences.

Workshop 4: Exams according to German administrative law I

Dr. Philipp Verenkotte

This workshop discusses exams from a legal perspective. There are a lot of requirements set by German administrative law that need to be fulfiled. Especially computer-based exams may cause a lot of uncertainty of what is legally possible and what has to be considered. Part I of this Workshop series provides the basics; in part II which takes place on Friday, we discuss your specific problems.

Note: The workshop is held in German.

Target group: Everyone who is involved in designing, holding or correcting exams at a German university or college.

Take home message: Participants will

-              understand the requirements of German administrative law concerning exams

-              know the stumbling blocks concerning computer- based exams

-              identify the main approaches to solve these problems and

-              learn how to design and conduct exams legally compliant

Workshop 5: Quantitative analyses of medical examinations I–Introduction to examination statistics

Dr. Andreas Fleig

Written examinations constitute an important part of any medical curriculum. Relying on established statistical indicators (e.g. difficulty, selectivity and reliability), quality assurance efforts are able to identify conspicuous questions and items. The group will also discuss communication strategies towards lecturers and administration.

Target group: In the course of the workshop, participants are invited to ask questions and exchange experiences on quality assurance efforts / quantitative exam evaluations at their own university. Statistical knowledge is not a prerequisite for participation.

Take home message: During this workshop, participants will receive an introduction to the significance and practicability of these indicators for different question formats of written examinations (single select, multi select, Kprime, etc.).

Workshop description Friday, 5th of April 2019

Workshop 1: tEXAM advanced

Dipl.-Inform. Med. Jörn Heid

In this workshop we will dive deep into the set-up and management of the tEXAM app and server components. Device management, MDM integration, backup strategies, fraud prevention, security settings including passwords and SSL certificates will be discussed here. We will also have time to exchange experiences with the software and to talk about how the workflow and the software can be improved in the future.

Target group: Partners who are already using tEXAM or will do in the near future

Take home message: Discovered ways to improve the management of tEXAM exams

Workshop 2: Testing communicative competencies: Reflection Report, Situational Judgement Test and MiniCe

Dr. Barbara Hinding, Dr. Lena Selgert

Teaching and testing of communicative competences are firmly anchored in the German Licensing Regulations for Physicians, and the Master Plan for Medical Studies 2020 also supports the promotion of communicative competences. Results from surveys of faculties show that the examination of communicative competences is currently very heterogeneous. If communication is tested at all, then it is more knowledge-based than practical competence-oriented. Only a minority uses practical examination formats. In this case, the assessment criteria often differ from faculty to faculty with the result that they are hardly comparable.

In this workshop we will work out concrete examples for the three test formats Reflection Report, Situational Judgement Test and MiniCex (Mini-Clinical Evaluation Exercise).

Based on the learning objectives for the doctor-patient interaction formulated in the National Competence-Based Learning Objectives Catalogue for Medicine (NKLM), we will use case vignettes and examination tasks to elaborate test examples and select, discuss and, if necessary, modify evaluation criteria in small working groups.

Target group: Everyone interested in the development of tests of communicative competence.

Take home message: The participants receive suggestions and ideas for solutions for testing communicative competences for the formats worked on, as well as the opportunity for content and methodological exchange.

Workshop 3: Digital Scenarios in Use - 6 Steps into Digitization subject-specific proposals about the realization of digital scenarios

Prof. Jürgen Handke

This workshop uses interdisciplinary as well as subject-specific concepts in order to create digital elements for teaching. We will focus on using Open Educational Resources (OER) and will show how these can be compiled to small learning packages that can be integrated into the process of teaching and learning. This includes the discussion of license aspects.

In addition, digitized scenarios for universities are shown and will be critically examined: flexible on campus formats (FLOCKs), online courses, MOOCs and pMOOCs.

Literature (German): Handke, Jürgen. 2014. Patient Hochschullehre. Marburg: Tectum Verlag.
Video Link: Six Steps - A Roadmap Towards Digitization in Linguistics

Take home message: The participants will

-         identify the problems of traditional teaching

-         formulate solutions in a digital world and transfer them to their discipline

-         compile OER into small learning packages

-         understand and apply the new possibilities of in-class tuition

Workshop 4: Exams according to German administrative law II

Dr. Philipp Verenkotte

The workshop takes a look at exams from a legal perspective. In Part I (on Thursday) of this Workshop series we talked about the basics. Now we will discuss about your specific problems.

Note: The workshop is held in German.

Target group: Everyone who is involved in designing, holding or correcting exams at a German university or college.

Take home message: Participants will

-              understand the requirements of German administrative law concerning exams

-              know the stumbling blocks concerning computer-based exams

-              identify the main approaches to solving these problems and

-              learn how to design and conduct exams legally compliant

Workshop 5: Quantitative analyses of medical examinations II – Generalizability analysis of OSCEs

Dr. Andreas Möltner, Mirijam Lehmann, Sevgi Timbil

In addition to written exams, objective structured clinical examinations (OSCEs) are well established in medical curricula. Though, assessing the reliability of OSCEs poses a particular challenge as, during the course of the OSCE, i) different examiners might oversee a single station and ii) a single examiner might be responsible for more than just one station. As an approach to account for such aspects, the workshops discusses the framework of generalizability theory. Using R statistical software (packages lme4 and lmerTest), the workshop offers a practical example for carrying out the analyses using data from a peer-assessed formative OSCE in general medicine at the Medical Faculty of the University of Heidelberg. The OSCE is conducted at the end of the pre-clinical section (4th semester) and carried out by 32 students that serve as ‘examiners’. For each of the 300 participating students, the OSCE comprises 4 out of overall 13 stations.

Target group: Basic knowledge of examination statistics is advantageous for participating in the workshop.

Take home message: Topics of the workshop:

-                    Model of generalizability theory

-                    Gathering the data from the tOSCE output files

-                    Analysis:

  • variance components of facets “student”, “station”,” rater” and “station x rater”
  • generalizability and dependability