Learning Outcomes of Interactive Online Cases in Veterinary Epidemiology and Bacteriology Education: A Survey Study

Introduction

Veterinary epidemiology and bacteriology are mandatory basic subjects in the veterinary student curriculum. Bacteriology, a sub-discipline of microbiology, is traditionally taught to university students through a combination of lectures and laboratory exercises.^1,2 In recent years, largely accelerated by the onset of the COVID-19 pandemic, various online digital tools have been developed for microbiology education, such as virtual laboratory exercises and simulations and other teaching methods such as Wikipedia writing and posting on social media.^3–7 Veterinary epidemiology is currently taught through a combination of lectures and different exercises such as creating communication handouts.^8,9 In addition, the use of various online tools such as blogs, discussion boards and quizzes has increased in the teaching of epidemiology in recent years.^10,11 Online learning generally has several advantages: it is more flexible and more accessible, and it enables the student to study at their own pace, repeat learning exercises as many times as needed, and receive instant feedback.¹¹ Online teaching does not require in-person classrooms and a wide selection of case scenarios can be presented to the students, also cases that would not be possible to work on in class, for example, bacteriological diagnostics cases involving biosafety level 3 pathogenic bacteria.^11,12

Case-based learning (CBL) is an educational approach in which students solve real-life cases. Background information on the patient or situation, and support material such as laboratory results is provided for the students.¹³ CBL offers the students an opportunity to utilize their prior knowledge to solve the cases, and can improve clinical reasoning, problem-solving and communication skills and enhance learning motivation.^13–15 CBL activities may be available for students as in-class activities such as discussions or in an e-learning format.^16,17 Over recent years, case-based e-learning has become more popular in veterinary medical education. It has been used for teaching veterinary students pathology and parasitology, biochemistry and clinical sciences.^12,17–21 Online cases have also been utilized for teaching veterinary epidemiology or microbiology. However, studies of case-based online teaching of veterinary microbiology have thus far almost always combined some other subject, such as pathology and parasitology or epidemiology and food safety aspects with microbiology.^18,22 In addition to describing the case-based teaching method, Duckwitz et al also studied the learning outcomes of these interdisciplinary online cases and found that the majority of students evaluated their self-assessed learning outcomes positively.²² Only a few reports exist on the use of online cases that focus solely on veterinary microbiology or epidemiology, and these have not studied the effect of CBL on learning outcomes.^23,24 Thus, more information is needed to determine how well veterinary epidemiology- and bacteriology-related learning outcomes can be achieved using case-based online teaching.

The primary aim of this study was to evaluate the learning outcomes of interactive online learning tools, developed at the Faculty of Veterinary Medicine at the University of Helsinki, in veterinary epidemiology and bacteriology. The learning outcomes of the veterinary students were investigated over multiple academic years through learning surveys, which the veterinary students completed both before and after their learning exercises. Specifically, cognitive skills were tested with a pre- and post-exercise logical organization test and a narrative question. Generic academic skills, such as critical thinking and problem-solving, were evaluated with a post-exercise self-evaluation.

Materials and Methods

Study Design and Participants

This study investigated the effects of two internet-based learning tools that offer interactive epidemiology and bacteriology cases on the learning outcomes of veterinary students studying these subjects.

This study contains two parallel but independent sections, one for epidemiology and one for bacteriology. Both sections had similar design: a prospective pre-post survey with paired within-subject comparisons. The study was conducted over several academic years which are specified below.

The cohorts (Figure 1) for the sections were detached. For the epidemiology section of this study, the cohort consisted of V3 (third-year) students from year 2015 to year 2018 who responded to a learning survey when solving a specific epidemiology case. Students for whom this case was their first were included in the study. At the time of the learning exercise, the V3 students were taking mandatory “Infection epidemiology” and “Evidence-based veterinary medicine” courses that included lectures and exercises. Solving two or three epidemiology cases during the course was mandatory. The students solved the epidemiology cases on their own, either at the university during their curriculum or outside it.

Figure 1 Flow diagram of study participants. The diagram shows the progression of veterinary students´ participation during the study as they use either the epidemiology or the bacteriology learning tool. a For the epidemiology learning tool pre- and post-exercise test data was excluded if student did not answer both pre- and post-exercise test (n=22) or if the case solved was not student’s first one in this learning tool (n=68). b For the bacteriology learning tool pre- and post-exercise test data was excluded if the assessment/self-evaluation form was empty (n=3) or duplicate (n=1).

For the bacteriology section of this study, the cohort consisted of V2 (second-year) students from years between 2016 and 2020 who solved any bacteriology case of their choice and responded to a learning survey. The students were instructed to respond to the learning survey once, when solving their first case. At the time of this learning exercise, the students were taking a mandatory “Infection microbiology” course that included lectures and mandatory laboratory exercises.² Each student solved the bacteriology case on their own outside the university curriculum. Solving the cases was voluntary for students.

Learning Tools (Intervention)

The intervention in both sections of this study was the use of a specified learning tool. These two learning tools were developed in the Faculty of Veterinary Medicine (FVM) of the University of Helsinki, and both were in the Finnish language.

The epidemiology learning tool aims to enhance students’ professional competence in the prevention and control of animal diseases. Its purpose is to enable students to solve a virtual outbreak case that mimics a real-life situation that veterinarians may encounter in their practice. The students have to proceed in a logical order to identify the agent responsible for the outbreak in each case and they must implement all necessary measures. After reading the medical history of the case, the students have to formulate a working hypothesis, also known as a list of differential diagnoses. They can then request additional information from the animal owner, examine the animals, determine the required preventive and control measures, and contact the competent authority. Ideally, after all the necessary steps of the case have been completed, only one diagnosis should remain, and it is expected to be the correct one. Finally, additional information about the case, including details on, for example, the agent that caused the outbreak and the required measures, is displayed on screen.

The bacteriology learning tool aims to improve students’ professional competence by increasing their laboratory reasoning skills and their ability to interpret laboratory test results. In each learning tool case, the student’s goal is to determine which bacterium is the pathogen responsible for the symptoms of the animal/s. After reading the medical history of the animal/s, the student must first choose the appropriate sample for laboratory analysis from the list given. Next, the student has to analyze the sample by conducting conventional bacteriological laboratory tests in the virtual laboratory. The tests must be performed in a logical order and only tests relevant to the case can be conducted. The results of the tests are given to the student as photographs of actual test results. Control photographs are also available during the exercise, for example, photographs of uncultured growth media. After completing all the necessary tests, the student can make the diagnosis by writing down the name of the bacterium. If the student has written the correct answer, an information box opens on the screen, displaying extra information about the solved case.

Learning Surveys

The effects of the epidemiology and bacteriology learning tools on the learning outcomes of veterinary students were studied using an internet-based feedback survey. The questionnaires were in Finnish and consisted of two parts: one that assessed cognitive learning outcomes and another that evaluated generic skills. The cognitive test was divided into pre- and post-exercise sections, each comprising a logical organization test and a narrative question. Details of the questionnaires of both learning tools are presented below.

In the cognitive part of the epidemiology learning survey, the logical reasoning of the students was tested before the exercise (the pre-exercise logical organization test) using a list of 12 tasks related to an outbreak investigation (Box 1). The students had to organize all these tasks into the order they considered most appropriate. After the exercise, the students were asked to complete the post-exercise survey, which contained the same logical organization test as in the pre-exercise survey, now called post-exercise logical organization test.

Box 1 Logical Organization Test in Epidemiology-Related Survey

In the cognitive part of the bacteriology learning survey, the logical organization test evaluated the students’ logical reasoning before the exercise (the pre-exercise logical organization test) with a list of tasks for determining a bacterium from a clinical sample in a laboratory. The students arranged eight tasks into the order they considered the most appropriate (Box 2). Immediately after solving the case, the students were instructed to complete the post-exercise survey, which contained the same question as the pre-exercise survey, now called the post-exercise logical organization test.

Box 2 Logical Organization Test in Bacteriology Related Survey

In addition to the logical organization test, the cognitive learning outcomes were also tested with a narrative question in both learning surveys. In the epidemiology learning survey, students were instructed to explain to the animal owner, through a narrative, what they were doing in this outbreak (Explain to the owner, through a narrative, what you are doing in this outbreak). In the bacteriology learning survey, students were asked to write a narrative response to the question on the bacterial identification procedure (Explain in your own words to someone unfamiliar with the topic how the bacterial identification process is carried out). The same questions were included in both pre- and post-exercise parts of the surveys. To obtain a better understanding of the quality of learning, these narrative responses were evaluated using the Structure of the Observed Learning Outcome (SOLO) method.²⁵ The pre- and post-exercise narratives were classified by combining them with the taxonomy of the SOLO learning outcomes method.²⁵ The SOLO taxonomy was chosen because of its success in previous evaluation studies in veterinary education.²⁶ The evaluation was carried out in accordance with the reasoning of Biggs and Tang (Table 1) and by using verbs that paralleled the SOLO taxonomy.²⁷ The SOLO evaluation of all the data was done by a single person. As the students learn, the complexity of their thinking increases at each SOLO level. The two ends of the response spectrum were: when a student missed the point and offered no response (blank field) at the prestructural level, and when a student created an abstract general principle or hypothesis from the given information at an extended abstract level because their answer went beyond what was given in the original information. When comparing pieces of information, a student can score between the multistructural and relational levels.²⁷ The difference is that the understanding of the information is integrated, and not merely produced as a list.

Table 1 Selected Verbs from the Structure of the Observed Learning Outcome (SOLO) Taxonomy.27

The generic skills part of the survey was similar for both learning tools. The questionnaire contained ten generic skills, shown in Box 3. The students were asked to self-evaluate whether they had learned each skill during the exercise, using the following scale: 4 = a lot, 3 = quite a lot, 2 = little, 1 = not at all, and I cannot say.

Box 3 List of Generic Skills in Self-Evaluation Questionnaire

Statistical Analysis

The learning survey responses were compared in paired groups using the Wilcoxon signed rank test and IBM SPSS Statistics (28.0.0.0) software. The alpha level used was p<0.05. All 95% confidence intervals (CI) for the percentages were calculated using the EpiTools online calculator and the Wilson score interval.^28,29

In the logical organization test no missing data was detected. In the narrative responses of the blank fields were categorized by SOLO taxonomy to the prestructural level and thus there were no missing data. In the generic skills’ self-evaluation “I can’t say” answers were categorized as missing data and excluded from the analysis.

Ethical Considerations

Participating in both the epidemiology and bacteriology learning survey was voluntary, and students were asked to give a written permission to use and publish any anonymized data from the survey. The participants’ informed consent included publication of anonymized responses/direct quotes. Only data on the students who had given their consent to participate were used in the study. According to the guidelines of the Finnish National Board on Research Integrity (The ethical principles of research with human participants and ethical review in the human sciences in Finland; available at: https://tenk.fi/sites/default/files/2021-01/Ethical_review_in_human_sciences_2020.pdf), this study does not fall under the remit of any research design that requires an ethics review. Our study complies with the Declaration of Helsinki.

Results

Changes in Learning Outcomes

Between 2015 and 2018, 104 of the V3 students who solved an epidemiology learning tool case and completed the learning survey were included in the evaluation of learning outcomes (Figure 1). In the logical organization test, all 104 V3 students made errors in the pre-exercise part. However, most of the students (n=75, 72% [95% CI 63–80%]) improved their results in the post-exercise test, and some (n=11, 15% [95% CI 8–24%]) even obtained a perfect score. Overall, the decrease in the median of the error count (from 3 to 2) was statistically significant (p<0.01).

In the SOLO-based evaluation of the narrative responses, most of the 104 V3 students’ (n=55, 53% [95% CI 43–62%]) scores remained at their pre-exercise level after the learning exercise, mostly at either the relational (21% [95% CI 14–30%]) or multistructural (24% [95% CI 17–33%]) level. Only 3% (95% CI 1–8%) of the post-exercise answers were of the “same as above” type, and no copy-paste responses were detected. Changes in learning outcomes occurred among 47% (95% CI 38–57%) of the V3 students. These changes were mostly shifts between multistructural and interpretatively close relational levels (43% [95% CI 30–57%] of all differences) or from no response (prestructural level in classification scale) to some other level (27% [95% CI 16–40%] of all differences). In total, the distribution of responses moved significantly (p<0.01) towards the higher end of the SOLO classification.

We will find a disease and confirm a diagnosis. We will find out where the disease has come from (V3 student, unistructural answer).

Animals are examined – samples are taken – diagnostics – limitations are given – an epidemiological investigation is carried out – follow-up measures depending on the disease (another V3 student, multistructural list as answer).

Between 2016 and 2020, 191 V2 students who solved a bacteriology case and completed the learning survey were included in the study (Figure 1). In the logical organization test, most (n=143, 75% [95% CI 68–81]) of the students put all the tasks in the correct order both before and after the learning exercise. Of the 40 students who made errors in the pre-exercise test, 73% (95% CI 57–84%) improved their results in the post-exercise test, decreasing the median of error count in this group from 3 to 0 (p<0.001). A similar but smaller improvement was also seen among all respondents (n=191, p=0.007).

In the SOLO-based evaluation of the V2 narrative responses, most of the answers remained at the pre-exercise level after the learning exercise, at either the relational (58%, 95% CI 51–64%) or multistructural (14%, 95% CI 9.5–19%) level. In both the pre- and post-exercise tests, the students usually based their answers on the list of actions given in the logical organization test instead of writing the response in their own words. In addition, of the post-exercise narrative responses, 18% (95% CI 13–24%) were either “same as above”, or created by copy-pasting the pre-playing answer without further explanations. Changes were seen in the learning outcomes of 21% (95% CI 16–28%) of the V2 students. The differences were mostly shifts from the multistructural to the relational level (34% [95% CI 22–49%] of all differences) or from the unistructural to the multistructural level (39% [95% CI 26–54%] of all differences). Changes from the multistructural to the unistructural or from the relational to the multistructural levels were also observed (5% [95% CI 1.2–16%] and 17% [95% CI 8.5–31%], respectively). Overall, these changes generated a slight but significant (p=0.002) shift of distribution towards higher SOLO levels.

First, we need a sample with bacteria, milk or blood. The sampling process depends on the sample type, location, and what is suspected. The goal is to get a sample with the right bacterium, not a contaminated sample. Then we try to identify the bacteria based on its appearance, growth and biochemical characteristics. We can identify it using microscopy and the increasing information about the bacteria (V2 student, relational answer with several related issues).

Self-Evaluation of Generic Skills

The students’ responses in the self-assessments of their generic skills were very similar for both the epidemiology and bacteriology learning tools (Figure 2, Supplementary tables S1 and S2). In both learning exercises, the majority of the students reported that they had improved their reasoning, problem-solving and information-processing skills, and their mean scores ranged from 2.7 to 2.8 and 2.9 to 3, respectively. Most of the students also noted an improvement in their critical thinking, creativity and continuous learning skills, and their mean scores ranged from 2.2 to 2.6 and 2.2 to 2.4, respectively. In contrast, opinions on teamwork, communication and computer skills were more divided. These skills had more “I can’t say” answers (up to 20%, data not shown) than the others, and the remaining answers were generally evenly distributed between “Not at all” and “Little” or “Quite a lot”. The ethical practice skill responses differed from all the other skill responses, as it was the only one with learning outcomes that varied in the two learning exercises. In epidemiology, slightly over half of the students felt they had learned this skill, whereas in bacteriology, slightly more than half felt they had not learned it at all.

Figure 2 Self-assessed learning scores of epidemiology and bacteriology learning exercise.

Discussion

The goal of this study was to investigate the effects of two internet-based learning tools that offered interactive epidemiology and bacteriology cases on the learning outcomes of veterinary students.

The cognitive skills learning outcomes showed a slight but statistically significant improvement according to both logical reasoning and the SOLO-based evaluation for both learning tools. This improvement was, however, smaller than we expected, for both learning tools. The reasons for this should be studied further, but the students’ prior knowledge of the topic is one possible explanation. This study´s epidemiology learning exercise was the students’ first “hands-on” outbreak investigation, and most of the students made several mistakes in both the pre- and post-exercise logical organization tests. As cognitive learning usually requires repetition, solving one case may not have been enough to achieve more visible results, especially as the sequence of an epidemiology outbreak investigation is quite complicated (12 steps in our logical organization test).^30,31 If the learning survey had been performed after the students had solved several epidemiology cases rather than after their first case, improvements in cognitive skills might have been more notable. The situation with the bacteriology learning tool was quite the opposite regarding prior student experience as all the students had had practical laboratory exercises prior to participating in the study and were already familiar with the sequence of the bacteriological diagnostics. Therefore, only a minority of the students made mistakes in the pre-exercise logical organization test and were thus able to improve their performance in the post-exercise test. In addition, the sequence of the bacteriology logical reasoning test was somewhat simpler (only eight steps in our test) than the corresponding epidemiology sequence.

According to the students’ self-evaluations, solving the epidemiology or the bacteriology cases improved many of their generic skills, for example, their reasoning and continuous learning skills, which are important “day one competences” for veterinarians.³² One of the main objectives of developing these learning tools for the FVM students was indeed to improve their reasoning skills, which are needed in both clinical work and when reading scientific material.³² Most of the generic skills that improved after the learning exercises were the same for both learning tools. The greatest difference between these learning tools regarding improvement of generic skills was in ethical practice, a skill that was learned during the epidemiology exercise but not so much during the bacteriology exercise. This may be due to the different nature of these learning exercises, as during the epidemiology exercise the students also had to be in contact with the virtual animal owner and the virtual competent authority, whereas the bacteriology exercise was done entirely in a virtual laboratory. In addition, the V3 students doing the epidemiology exercise were one year further in their studies than the V2 students doing the bacteriology exercise, and this may have affected their accumulation of ethical practice skills more positively. One limitation of this study is that the generic skills’ learning outcomes were self-assessed by the students and not objectively measured. However, objective measurements of these skills would have been difficult.

Our study also has some limitations related to its design. Initially, we aimed for an equal number of students to use the learning tools within the study period for both learning surveys. However, since the use of the bacteriology learning tool was voluntary, the accumulation of the participants was slower compared to the epidemiology section of the study. Consequently, we planned a longer study period for the bacteriology section but, unfortunately, due to the Covid-19 pandemic and the resulting changes in teaching, the collection of participants had to be halted in 2020. This, combined with the relatively small difference between the pre- and post-exercise results when comparing all participants in the bacteriology section, resulted in a smaller sample size than desired. Nonetheless, as estimated using EpiTools online calculator (data not shown), the sample size in the bacteriology section is sufficient for comparing the students who made errors in the pre-exercise test. As the difference between pre- and post-results in the epidemiology section of the study was more pronounced, the sample size in it is valid for comparisons across the entire student group. The difference in the number of included respondents in the epidemiology and bacteriology surveys must also be kept in mind if the two activities are to be compared. Although nearly the same number of students participated to both learning surveys (Figure 1), the number of students included in the epidemiology section of the study (n=104) was much smaller than in the bacteriology section (n=191). In the epidemiology section, a larger portion of data had to be excluded compared to the bacteriology section. There are two main reasons for this. First, the survey form in the epidemiology section was designed in such a way that respondents could easily forget to complete the latter (post-test) part of the survey, resulting in the exclusion of 22 students. Second, students in the epidemiology section were able to use the learning tool before taking the survey, which led to the exclusion of 68 students. Another study-related factor that might have biased the results of this study is that, due to scheduling reasons, only one evaluator was used in the SOLO evaluation for this study. Due to the small differences in the evaluation criteria between the SOLO taxonomy levels, and thus, difficulty of the evaluator to maintain consistency in evaluation, technical evaluation errors may have appeared. If it had been feasible, using multiple evaluators and establishing inter-rater reliability would have been preferable.

The difference in participation conditions, whether or not the learning exercise was compulsory (epidemiology exercise) or voluntary (bacteriology exercise) for the students, may have had an effect on the students’ learning experiences. It is possible that only the most subject-motivated V2 students were interested in doing the bacteriology exercise resulting to overall more positive learning outcomes. Even though the participation conditions of the two learning activities differed, the survey was voluntary for both the V2 and V3 students.

The students’ subjective perceptions of the interest and usefulness of the topics of our learning tools may have influenced their performance in the exercises and the number of study participants. For example, epidemiology has been previously described as too theoretical to appeal to veterinary students.⁸ Only a minority of American veterinary students had a primary career interest in epidemiology and up to 30% of Australian veterinary students did not consider epidemiology useful for their future career.^9,33,34 Our students may have shared these perceptions. Medical microbiology, on the other hand, has shown to be an engaging subject for medical students.³⁵ We did not find any previous research on this topic of veterinary students. As bacteriology studies in veterinary microbiology courses usually also include laboratory exercises, as is the case at FVM, this subject is more likely seen as less theoretical than epidemiology.² Consequently, this may have made it more motivating for FVM students. One of the targets of developing these realistic online cases for FVM students was to increase their motivation to study epidemiology and bacteriology and to benefit from deeper learning. In the future it would be interesting to study whether solving epidemiology or bacteriology online cases influences the motivation of students to study these subjects.

The students’ prior background knowledge of epidemiology and bacteriology could also potentially have influenced the results of our study. However, most of the participants had similar levels of theoretical knowledge of epidemiology and bacteriology before the learning experience, as FVM students have a strict curriculum, which is the same for everyone. Some students might, however, have had some prior knowledge of epidemiology or bacteriology from before their studies at the FVM. As we measured the changes in the learning outcomes by comparing the difference between each student’s paired pre- and post-exercise answers, we believe that possible prior knowledge of epidemiology or bacteriology had no considerable effect on the results of this study.

As both learning tools were updated and developed further after this study, the epidemiology and bacteriology online cases that the students solved during this study are no longer available in their original form. After this study, the epidemiology learning tool was further developed in an Erasmus+ project and is now freely accessible in English.³⁶ The updated bacteriology learning tool is still used in teaching and follows the same principles and logic as the earlier version. The updated bacteriology learning tool is freely available in Finnish.³⁷

Conclusion

Internet-based interactive cases developed for learning veterinary epidemiology and bacteriology enhanced students’ cognitive and generic skills. The results provide evidence that internet-based interactive cases have beneficial effects on learning outcomes in veterinary education and can help further develop these learning tools. Wider application of internet-based interactive cases would provide veterinary students with more opportunities to solve real-life cases while simultaneously gaining important academic cognitive and generic skills.