Employing Transfer Learning for the Automation of Short Answer Grading
Keywords:
ASAG, ML, NLP, Language, Processing, Deep Learning, Data SetsAbstract
To accommodate the ever-increasing number of pupils, automated short answer grading (ASAG) has recently attracted interest in the field of education. We take a look at the latest developments in ASAG research as it relates to the impact of recent advances in ML and NLP on the discipline. In this study, we add to the existing literature by giving a thorough evaluation of newly published methods that use deep learning techniques. We focus in on the shift from features that are hand-engineered to representation learning methods, which automatically learn task-specific features from massive data sets. Word embeddings, sequential models, and attention-based strategies frame our examination of deep learning methods. We found that learned representations alone do not help to produce the greatest outcomes, but they rather function in a complimentary fashion with hand-engineered features, which is how deep learning affects ASAG differently from other domains of natural language processing. Combining the strength of the semantic descriptions offered by modern models with the meticulously constructed characteristics, such as in transformer designs, is undoubtedly the key to top performance. We highlight problems and suggest a future research agenda for tackling them.
Downloads
References
Michael Mohler, Razvan C. Bunescu, and RadaMihalcea, ‘Learning to grade short answer questions using semantic similarity measures and dependency graph alignments.’, in ACL, pp. 752–762, (2021).
K. Papineni, S. Roukos, T. Ward, and W. J. Zhu, ‘Bleu: a method for automatic evaluation of machine translation’, Technical report, IBM Research Report, (2021).
DaiX. et al.( 2020) An analysis of simple data augmentation for named entity recognition.
BayerM. et al. (2022) A survey on data augmentation for text classificationACMComput. Surv.
BurrowsS. et al. (2022) The eras and trends of automatic short answer grading Int. J. Artif. Intell. Educ.
MichaelMohler,RazvanBunescu,andRadaMihalcea.2012.LearningtoGradeShortAnswerQuestionsusingSemanticSimilarityMeasuresandDependencyGraphAlignments.InProceedingsofthe49thAnnualMeetingoftheAssociationforComputationalLinguistics:HumanLanguageTechnologies.AssociationforComputationalLinguistics,Portland,Oregon, USA, 752–762.
DetmarMeurers,RamonZiai,NielsOtt,andStaceyMBailey.2012.Integratingparallelanalysismodulestoevaluatethe meaning of answers to reading comprehension questions.International journal of continuing engineering educationandlife-longlearning21(2012),355–369.
Myroslava O. Dzikovska, Rodney D. Nielsen, and Chris Brew. 2012.Towards Effective Tutorial Feedback for ExplanationQuestions:ADatasetandBaselines.InProceedingsofthe2012ConferenceoftheNorthAmericanChapteroftheAssociationforComputationalLinguistics:HumanLanguageTechnologies.AssociationforComputationalLinguistics,Montréal, Canada,200–210.
MilenKouylekov, Luca Dini, AlessioBosca, and Marco Trevisan. 2013.Celi: EDITS and Generic Text Pair Classification.In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the SeventhInternationalWorkshoponSemanticEvaluation(SemEval2013).AssociationforComputationalLinguistics,Atlanta,Georgia,USA,592–597.
OmerLevy,TorstenZesch,IdoDagan,andIrynaGurevych.2013.InSecondJointConferenceonLexicalandComputa-tional Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval2013). Association for Computational Linguistics, Atlanta, Georgia, USA, 285–289.
MichaelHeilmanandNitinMadnani.2013.ETS:DomainAdaptationandStackingforShortAnswerScoring.InSecondJoint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh InternationalWorkshoponSemanticEvaluation(SemEval2013).AssociationforComputationalLinguistics,Atlanta,Georgia,USA,275–279.
NielsOtt, Ramon Ziai, Michael Hahn, and DetmarMeurers. 2013.CoMeT: Integrating different levels of linguisticmodelingformeaningassessment.InSecondJointConferenceonLexicalandComputationalSemantics(*SEM),Volume2:ProceedingsoftheSeventhInternationalWorkshoponSemanticEvaluation(SemEval2013).AssociationforComputationalLinguistics, Atlanta, Georgia, USA, 608–616.
SergioJimenez,ClaudiaBecerra,andAlexanderGelbukh.2013.SOFTCARDINALITY:HierarchicalTextOverlapforStudentResponseAnalysis.InSecondJointConferenceonLexicalandComputationalSemantics(*SEM),Volume2:ProceedingsoftheSeventhInternationalWorkshoponSemanticEvaluation(SemEval2013).AssociationforComputationalLinguistics, Atlanta, Georgia, USA, 280–284.
LakshmiRamachandran,JianCheng,andPeterFoltz.2015.IdentifyingPatternsForShortAnswerScoringUsing Graph-based Lexico-Semantic Text Matching. In Proceedings of the Tenth Workshop on Innovative Use of NLP forBuildingEducationalApplications.AssociationforComputationalLinguistics,Denver,Colorado,97–106.
Ahmed EzzatMagooda, Mohamed A. Zahran, Mohsen A. Rashwan, Hazem M. Raafat, and Magda B. Fayek. 2016.Vector Based Techniques for Short Answer Grading. In Proceedings of the Twenty-Ninth International Florida ArtificialIntelligence Research Society Conference, FLAIRS 2016, Key Largo, Florida, USA, May 16-18, 2016, Zdravko Markov andIngrid Russell (Eds.). AAAI Press, 238–243
Md Arafat Sultan, Cristobal Salazar, and Tamara Sumner. 2017.Fast and Easy Short Answer Grading with HighAccuracy.InProceedingsofthe2016ConferenceoftheNorthAmericanChapteroftheAssociationforComputationalLinguistics: Human Language Technologies. Association for Computational Linguistics, San Diego, California, 1070–1075.
ShouryaRoy,HimanshuS.Bhatt,andY.Narahari.2018.AnIterativeTransferLearningBasedEnsembleTechniqueforAutomatic Short Answer Grading.ArXivabs/1609.04909 (2018).
Lucas B. Galhardi, Helen Senefonte, Rodrigo de Souza, and Jacques Brancher. 2019.Exploring Distinct Features forAutomaticShortAnswerGrading.InAnaisdoXVEncontroNacionaldeInteligênciaArtificialeComputacional.SBC,Porto Alegre, RS, Brasil, 1–12.
YamanKumar,SwatiAggarwal,DebanjanMahata,RajivRatnShah,PonnurangamKumaraguru,andRogerZimmer-mann. 2020.Get IT Scored Using AutoSAS - An Automated System for Scoring Short Answers. In AAAI.
BrianRiordan,AndreaHorbach,AoifeCahill,TorstenZesch,andChongMinLee.2017.Investigatingneuralarchitectures for short answer scoring. In BEA@EMNLP.
Sachin Kumar, SoumenChakrabarti, and Shourya Roy. 2017.Earth Mover’s Distance Pooling over Siamese LSTMsforAutomaticShortAnswerGrading.InProceedingsoftheTwenty-SixthInternationalJointConferenceonArtificialIntelligence,IJCAI-17.2046–2052.
SwarnadeepSaha,TejasI.Dhamecha,SmitMarvaniya,RenukaSindhgatta,andBikramSengupta.2018.SentenceLevelorTokenLevelFeaturesforAutomaticShortAnswerGrading?:UseBoth.InArtificialIntelligenceinEducation,CarolynPensteinRosé,RobertoMartínez-Maldonado,H.UlrichHoppe,RoseLuckin,ManolisMavrikis,KaskaPorayska-Pomsta,Bruce McLaren, and Benedict du Boulay (Eds.). Springer International Publishing, Cham, 503–517
Tianqi Wang, Naoya Inoue,Hiroki Ouchi, TomoyaMizumoto,and KentaroInui. 2019.InjectRubrics into ShortAnswerGradingSystem.InProceedingsofthe2ndWorkshoponDeepLearningApproachesforLow-ResourceNLP(DeepLo2019).AssociationforComputationalLinguistics,HongKong,China,175–182.
WaelHassanGomaaandAlyAlyFahmy.2019.Ans2vec:AScoringSystemforShortAnswers.InTheInternationalConferenceonAdvancedMachineLearningTechnologiesandApplications(AMLTA2019),AboulEllaHassanien,Ah-mad TaherAzar, TarekGaber, RoheetBhatnagar, and Mohamed F. Tolba (Eds.). Springer International Publishing,Cham,586–595.
ChulSung,TejasIndulalDhamecha,andNirmalMukhi.2020.ImprovingShortAnswerGradingUsingTransformer-BasedPre-training.InArtificialIntelligenceinEducation,SeijiIsotani,EvaMillán,AmyOgan,PeterHastings,BruceMcLaren, and Rose Luckin (Eds.). Springer International Publishing, Cham, 469–481.
HuiQi,YueWang,JinyuDai,JinqingLi,andXiaoqiangDi.2020.Attention-BasedHybridModelforAutomaticShortAnswer Scoring. In Simulation Tools and Techniques, Houbing Song and Dingde Jiang (Eds.). Springer InternationalPublishing, Cham, 385–394.
Tianqiao Liu, Wenbiao Ding, Zhiwei Wang, Jiliang Tang, Gale Yan Huang, and Zitao Liu. 2019.Automatic ShortAnswer Grading via Multiway Attention Networks.ArXivabs/1909.10166 (2020).
ChulSung,Tejas I. Dhamecha, SwarnadeepSaha, Tengfei Ma, V. Pulla Reddy, and Rishi Arora. 2021.Pre-TrainingBERT on Domain Resources for Short Answer Grading. In EMNLP/IJCNLP
LishanZhang,YuweiHuang,XiYang,ShengquanYu,andFuzhenZhuang.2022.Anautomaticshort-answergradingmodelforsemi-open-endedquestions.InteractiveLearningEnvironments 0,0(2019),1–14.
YuanZhang,ChenLin,andMinChi.2023.Goingdeeper:Automaticshort-answergradingbycombiningstudentandquestionmodels.UserModelingandUser-AdaptedInteraction30,1(01Mar2020),51–80.https://doi.org/10.1007/s11257-019-09251-6.
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
All papers should be submitted electronically. All submitted manuscripts must be original work that is not under submission at another journal or under consideration for publication in another form, such as a monograph or chapter of a book. Authors of submitted papers are obligated not to submit their paper for publication elsewhere until an editorial decision is rendered on their submission. Further, authors of accepted papers are prohibited from publishing the results in other publications that appear before the paper is published in the Journal unless they receive approval for doing so from the Editor-In-Chief.
IJISAE open access articles are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. This license lets the audience to give appropriate credit, provide a link to the license, and indicate if changes were made and if they remix, transform, or build upon the material, they must distribute contributions under the same license as the original.
