Kategorie: ICWL

The usefulness of search results and task outcome

Keynote by Prof. Dr. Pertti Vakkari

In evaluating search systems there is a growing trend to complement the established effectiveness indicator topical relevance by the usefulness of search results. Usefulness refers to the contribution of search results to a larger task generating information search. Task outcome signifies the end-product of a task performance process. It is evident that the whole search process contributes to task outcome. The aim of my presentation is to give a systematic account of the characteristics and results of studies focusing on the usefulness of search results or task outcomes in larger tasks containing searching. I summarize how the usefulness of search results, and search outcomes are defined and operationalized in empirical studies. I categorize various types of usefulness and outcomes and various predictor types used in studies. Finally, I summarize which factors significantly predict the usefulness of search results as well as search outcomes.

Exercises Recommendation Method Based on Machine Learning

Zhizhuang Li, Zhengzhou Zhu

Different exercises containing the same knowledge point have different effects on improving students‘ mastery of the knowledge point. Recommending exercises with higher effect for students to improve their mastery of knowledge points can help students improve their learning efficiency and reduce their learning burden. However, the existing exercise recommendation method does not consider the effect of improving students‘ mastery of knowledge points after doing the recommended exercises. This paper presents a method of exercises recommendation based on machine learning. This method can recommend more suitable exercises to students according to the category they belong to. Firstly, we use linear regression and EM algorithm to accurately model the students‘ mastery of each knowledge point. For each knowledge point, students are divided into several categories according to their mastery of the knowledge point and their average mastery of all knowledge points. For each knowledge point, according to the student history answer record, find out the exercise that can make each kind of student get bigger promotion respectively. For the students who need to recommend the exercises that contain the specified knowledge points, we first use the k-nearest neighbor algorithm to classify the students, and then recommend the exercises suitable for the students. It has been proved by experiments that this method can help students to achieve greater improvement in the same number of exercises.

Promoting Inclusion in Vocational Education and Training programs using Open Educational Resources

Silvia Margarita Baldiris Navarro, Laura Mancera, Levinton Licona, Jorge Bacca, Cecilia Avila, Yurgos Politis, Lizbeth Goodman, Jutta Treviranus

Open Educational Resources (OER) have been declared by UNESCO as a promising tool to address inclusion in educational settings. However, recent studies support the need to pay more attention to the accessibility and inclusion capacity of OER. In this context, the study presented in this paper provide evidence about the benefits of adopting Universal Design for Learning and the Web Content Accessibility Guidelines to support the OER creation considering the variability of students in Vocational Education and Training (VET) pro-grams. For doing so, it was created and validated some OER on VET setting in Colombia. Results evidence the OER support the achievement of high level of motivation and academic performance in the participants.

Towards the design and deployment of an Item Bank: An analysis of the requirements elicited

Claudia-Melania Chituc, Marisa Hermann, Daniel Schiffner, Marc Rittberger

Assessments represent an important phase in the learning process. Advancements in information and communication technologies determined the development of e-learning software tools which support e-learning activities, including e-assessment. The increasing usage of summative and formative e-assessments led to the increasing challenge of managing available items, such as test questions. The concept of an item bank is meant to support teachers and students alike, to provide an overview when taking assessments or creating exams. This article presents an on-going R&D project towards the design and deployment of an item bank, and discusses its role within a service-oriented system architecture which enables the execution of activities related to e-assessment, ranging from item design and test creation, to the analysis of event logs generated by test-takers. The research methodology followed for the requirements elicitation and main findings are presented, and directions for future work are discussed.

Designing a User-Friendly Educational Game for Older Adults

Louise Sauve, David Kaufman, Patrick Plante

Commercially available games present challenges in terms of their ease of use for a large number of older adults because their needs are not well known. Given the importance of a well-constructed educational gaming interface and the costs in-volved in its development, it is important to identify the ergonomic requirements to be considered during the design process to ensure that the game be adapted to the characteristics of seniors. In a study of seniors aged 55 and older, we created and tested an educational game, „In Anticipation of Death“, in order to measure usability in the sense of determining the intuitive capacity of the game (user-friendliness).
This paper presents the variables of the study, the way we adapted the game Solitaire for seniors and the results of an experiment done with 42 older players. The latter showed a high degree of satisfaction with game navigation, the display mode and gameplay equipment. Recommendations are presented to guide the development of online educational games for seniors.

Cheating Detection Method Based on Improved Cognitive Diagnosis Model

Zhizhuang Li, Zhengzhou Zhu, Teng Yang

Cheating in examinations destroys the principles of fairness and justice in evaluation. Cheating detection is of great practical significance. Traditional cheating detection methods have many disadvantages, such as difficult to detect covert equipment cheating, multi-source cheating, difficult to distinguish plagiarists from plagiarists, difficult to distinguish plagiarists from victims, or plagiarism from coincidences. In this paper, the concept of knowledge point mastery Index is introduced to measure students‘ mastery of a certain knowledge point, and a test method of cheating based on improved cognitive diagnostic model is proposed. This method calculates the weight of each knowledge point in every examination question through linear regression and EM algorithm according to students‘ historical learning behavior, and then calculates students‘ mastering degree of knowledge point based on historical answers. Then calculate the mastering degree of knowledge point based on the examination results. Finally, we compare the mastering degree of knowledge point based on the examination results and the historical answers to detect students‘ cheating situation. The experiments show that the average precision ratio of this method is 31.3% higher than that of the method based on the false-same rate, 23.9% higher than that of the method based on the false-same rate and the right-same ratio, 96.2% higher than that of the method based on the Person-Fit index. The recall ratio is 74.8% higher than that based on the false-same rate, 114.0% higher than that based on the false-same rate and the right-same ratio, and 27.8% higher than that based on the Person-Fit index.

Measuring Similarity for Observation of Learners’ Syntactic Awareness in Web-Based Writing Environments

Min Kang, Koichi Kawamura, Shuai Shao, Harumi Kashiwagi, Kazuhiro Ohtsuki

Writing in foreign language is a struggle task for learners and revising learners’ writings is time consuming for teachers as well. For this reason, writing support systems have been proposed a lot and one of main functions is to automatically detecting and revising errors in learners’ writings. However, the detection technologies are in progress and the effectiveness of error revision feedback is in argument. Meanwhile, numerous efforts in teaching writing have been made to enhance learners’ writing proficiency and reduce errors. Reading is emphasized to be one of important strategies. However, for the problems what linguistic knowledge learners pay attention to and how they use the knowledge into their writings in web-based learning, there is few studies reported. In this paper, we performed a reading-to-write experiment in a web-based writing environment and analyzed reading materials and learners’ writings to explore how to observe learners’ awareness on syntactic structures in the materials. Sentence patterns that proposed in our previous works are introduced to categorize sentences and the syntactic similarities be-tween reading materials and writings are calculated. The experimental results revealed that students showed higher comprehension on content but poor attention on syntactic structures in reading activities if the structures are not significant salient. It is assumed that the similarity measure is effective to ob-serve students’ awareness on syntactic structures in materials and further works are needed to automatically observe the awareness.

Measuring Students‘ Stress with Mood Sensors: First Findings

Henri Kajasilta, Mikko-Ville Apiola, Erno Lokkila, Ashok Kumar Veerasamy, Mikko-Juss Laakso

Stress and stress management play a major part in the modern lifestyle when considering wellbeing. Short time stress can work as a motivator and is not necessarily a bad thing, but high stress levels in the long run are linked to burn-out. This may lead to expending person’s psychological resources and can cause a long time absence from proactive actions. Our sample included 17 students who were instructed to keep diaries for one week and describe their feelings during studying activities. Combining the diaries and Moodmetric data allowed separating the different studying activities from the data and also differentiate non-study time from studying. We found a strong correlation between non-study and study Moodmetric value averages. Even when comparing daily Moodmetric values, the correlation was strong and significant in the 95% confidence level. We got 53 non-study and study average pairs from the students. This lead to the assessment that stress levels are a comprehensive case, where it is difficult to determine all the causing factors. The sample size was not large enough to make reliable deductions from single studying activities. With larger sample sizes it would became more reasonable to research individual activities. It would also be beneficial to extend both the time students wear the rings and the time they keep diaries, so we get longer and more reliable periods of students‘ life and observed arousal levels. This could also allow interventions for highly stressed students.

Cross-cultural Reflections of Tertiary Students on ICT-Supported Innovations

Dagmar El-Hmoudova, Martina Manenova

The paper focuses on the perception of innovations from the perspective of students. By innovations, in this case, we understand information and com-munication technologies. The aim of the research project is to compare the perception of innovations from the view of Czech and Asian students. We proceeded from Rogers‘ theory of diffusion of innovations (Rogers uses the words technolgy and innovation as synonyms), which defines five categories of adopters as classification of individual members of the social system, based on innovativeness. Diffusion is seen as a process, while innovations are passed on to other members of a particular social system during a certain time unit and through certain information channels. As a research tool, we used the Kankaarinta questionnaire. The research group consisted of Czech and Asian university students. Given the expansion and use of technology in Asian countries, we assumed that Asian students would be more inclined to innovate than the Czech ones. We worked with comparable groups (186 Czech students and 159 Asian students). Due to the fact that Czech students were from the Faculty of Education, female students prevailed in that group. Asian group of students was gender-balanced. In both groups, early majority prevailed, with a statistically significant difference between the two groups. Asian students seemed more innovative.

Audience Response Systems Reimagined

Sebastian Mader, François Bry

Audience response systems (ARS) allow lecturers to run quizzes in large classes by handing to technology the time-consuming tasks of collecting and aggregating students‘ answers. ARSs provide immediate feedback to lecturers and students alike. The first commercial ARSs emerged in the 1990s in form of clickers, i.e., transmitters equipped with a number of buttons, which impose restrictions on possible questions – most often, only multiple choice and numerical answers are possible.
Starting at the early 2010s, the ubiquity of smartphones, laptops, and tablet computers paved the way for web-based ARSs which, while running on technology that provides more means for input and a graphical display, still have much in common with their precursors: Even though more types of questions besides multiple choice are supported, the full capability of web-based technology is still not fully exploited. Furthermore, they also do not adapt to a student’s needs and knowledge, and often restrict quizzes to two phases: Answering a question and viewing the results.
This article first examines the current state of web-based ARSs: Question types found in current ARSs are identified and their support in a variety of ARSs is examined. Afterwards, three axes on which ARSs should advance in the future are introduced: Means of input, adaption to students, and support for multiple phases. Each axis is illustrated with concrete examples of quizzes.