Difficulty Ranking Mechanisms for Automatic Question Generation of Multimedia Contents

Abstract

Automatic Question Generation (AQG) system has shown its potential of reducing teacher�s workloads in producing vast questions for students� evaluation. The common techniques to generate such automatic questions for textual contents can be worked on syntactic or semantic approach, while that for multimedia contents by knowledge graph-based approach. However, the current AQG frameworks have not generated sufficient questions to reflect the actual knowledge for student assessment in an effective manner, primarily due to the lack of knowledge sources for multimedia contents. As consequence, these frameworks are then constrained to generate questions that are only suitable for assessing specific levels of student proficiency. Although the similarity-based approach can offer difficulty ranking mechanism for knowledge aspect, it cannot produce questions that are applicable to assess cognitive skills. In this research, an integrated AQG framework is proposed for generating variance in questions and option choices with difficulty ranking mechanism for multimedia contents, which could reflect the actual knowledge and cognitive skills for student assessment effectively. A new knowledge representation model using a graph database is constructed by integrating Natural Language Processing (NLP) techniques and external knowledge sources, such as WordNet and Open Linked Data source, and time attribute data for describing the multimedia contents. The unified difficulty ranking mechanism is proposed in accordance with this framework for yielding better capability than traditional similarity-based difficulty ranking. In addition, a new cognitive-based ranking mechanism according to Bloom�s taxonomy is especially proposed to handle time-based attribute data of multimedia contents for cognitive skills assessment. The evaluation of proposed unified difficulty ranking mechanism performs with 94% on our experimental dataset and 75% accuracy on public dataset. In this regard, the proposed hybrid cognitive and unified similarity-based difficulty ranking mechanisms can be well support for ranking various knowledge proficiencies and cognitive level questions that are generated from multimedia contents. Hence, the proposed AQG framework could generate sufficient questions to reflect the actual knowledge and cognitive skills for student assessment in an effective manner.

Automatic Question Generation (AQG) system has shown its potential of reducing teacher�s workloads in producing vast questions for students� evaluation. The common techniques to generate such automatic questions for textual contents can be worked on syntactic or semantic approach, while that for multimedia contents by knowledge graph-based approach. However, the current AQG frameworks have not generated sufficient questions to reflect the actual knowledge for student assessment in an effective manner, primarily due to the lack of knowledge sources for multimedia contents. As consequence, these frameworks are then constrained to generate questions that are only suitable for assessing specific levels of student proficiency. Although the similarity-based approach can offer difficulty ranking mechanism for knowledge aspect, it cannot produce questions that are applicable to assess cognitive skills. In this research, an integrated AQG framework is proposed for generating variance in questions and option choices with difficulty ranking mechanism for multimedia contents, which could reflect the actual knowledge and cognitive skills for student assessment effectively. A new knowledge representation model using a graph database is constructed by integrating Natural Language Processing (NLP) techniques and external knowledge sources, such as WordNet and Open Linked Data source, and time attribute data for describing the multimedia contents. The unified difficulty ranking mechanism is proposed in accordance with this framework for yielding better capability than traditional similarity-based difficulty ranking. In addition, a new cognitive-based ranking mechanism according to Bloom�s taxonomy is especially proposed to handle time-based attribute data of multimedia contents for cognitive skills assessment. The evaluation of proposed unified difficulty ranking mechanism performs with 94% on our experimental dataset and 75% accuracy on public dataset. In this regard, the proposed hybrid cognitive and unified similarity-based difficulty ranking mechanisms can be well support for ranking various knowledge proficiencies and cognitive level questions that are generated from multimedia contents. Hence, the proposed AQG framework could generate sufficient questions to reflect the actual knowledge and cognitive skills for student assessment in an effective manner.