Search

Research list

Math Word Problem (MWP) solving involves understanding mathematical questions expressed in natural language and deriving the appropriate mathematical equations. Traditional approaches heavily rely on simple lexical pattern matching, limiting their flexibility in diverse real-world scenarios. Our co-authored paper introduces ATHENA (Attention-based THought Expansion Network Architecture), designed to mimic human cognitive processes for more generalized and robust mathematical reasoning.
Let's explore how ATHENA achieves robust performance and why this matters for mathematical reasoning in AI.

Research Background

Math word problem (MWP) solving involves translating complex linguistic descriptions into mathematical expressions. Traditional models tend to memorize lexical patterns rather than understand mathematical principles and procedures, limiting their ability to generalize to unseen or slightly varied problems.
Consider the following cases: calculating the area of a rectangle and determining how many items can be evenly distributed across containers. While both require multiplication, they involve different types of conceptual understanding.
Previous methods have struggled with two key aspects in reaching human-level understanding:
•
Conceptual knowledge: Understanding how mathematical principles apply in various contexts.
•
Procedural knowledge: The ability to deduce answers step by step through logical reasoning.
ATHENA is specifically designed to bridge this gap, enabling the model to expand its reasoning capabilities by mimicking human cognitive processes.

Methodology: ATHENA

ATHENA: Mathematical Reasoning with Thought Expansion
NLP
Math Word Problem
ML
Language models (LMs) have become remarkably proficient in natural language reasoning tasks by leveraging the rationales they generate to enhance their reasoning capabilities. These rationales serve a dual purpose: they not only help improve reasoning performance but also justify model decisions. The problem? Perfect rationales are often impossible to obtain.
Our research, How Ambiguous Are the Rationales for Natural Language Reasoning? A Simple Approach to Handling Rationale Uncertainty, explores this issue in depth. The study introduces AURA (Ambiguous Rationale Utilization for Robust Answering), a novel two-stage reasoning framework that enhances model robustness in the face of ambiguous rationales. Let’s dive into the details of this study and its implications for natural language reasoning.

Research Background and Problem Statement

Language models (LMs) have achieved significant progress in solving complex reasoning tasks that require commonsense knowledge, as well as in handling challenging multiple-choice questions. In particular, recent advancements have enabled models to leverage generated rationales to further enhance reasoning performance. Nevertheless, obtaining perfect rationales from models (or even from humans) is practically impossible. While human annotation can improve the quality of rationales, it is extremely costly and does not guarantee perfect conditions.
More importantly, learning the patterns of rationales, which are generated based on an enormous number of different statements, is nearly impossible. This is because rationales inherently contain various normative concepts, which justify different ways of thinking or acting—in other words, the same question may have different explanations. As a result, this diversity causes models to experience uncertainty, making it challenging to learn rationales effectively.
Therefore, our study analyzes the impact of ambiguous rationales on the performance of natural language reasoning models and proposes a method to effectively handle ambiguity.

Methodology

Natural Language Reasoning with Rationales

To begin, we introduce the concept of Natural Language Reasoning with Rationales. In this study, we focus on Multiple Choice Question Answering (MCQA) tasks, where a model is given a question qq and a set of answer choices A={ai}A=\{a_i\}.
First, the model computes a plausibility score ρ(q,ai)\rho(q,a_i) for each answer choice and predicts the optimal answer a^\hat{a} by selecting the choice with the highest score:
Handling Ambiguous Rationales in Natural Language Reasoning
Natural Language Reasoning
NLP
ML
Abstract. With an increasing interest in personalized learning, active research is being conducted on knowledge tracing to predict the learner’s knowledge state. Recently, studies have attempted to improve the performance of the knowledge tracing model by incorporating various types of side information. We propose a knowledge tracing method that utilizes the learner’s language proficiency as side information. Language proficiency is a key component of comprehending a question’s text and is known to be closely related to students’ academic performance. In this study, language proficiency was defined with Elo rating score and time window features, and was used in the knowledge tracing task. The dataset used in this study contains 54,470 students and 7,619,040 interactions, which were collected from a real-world online-learning platform. We conducted a correlation analysis to determine whether the language proficiency information of students was related to their ability to solve math word problems. In addition, we examined the effect of incorporating the language proficiency information on the knowledge tracing models using various baseline models. The analysis revealed a high correlation between the length of word problems and students’ language proficiency. Furthermore, in experiments with various baseline models, utilizing the language proficiency information improved the knowledge tracing model’s performance. Finally, when language proficiency information was incorporated, the cold start problem of the knowledge tracing model was mitigated. The findings of this study can be used as a supplement for educational instruction.

Introduction

As interest in personalized online learning grows, the research in the field of knowledge tracing (KT) to model a learner’s knowledge state also increases. Many studies have recently been conducted to improve the performance of the KT model by using various side information, such as response time [31,33], the number of attempts [10,43], question text [21,36], and relationship between concepts [9,23].
On the other hand, previous studies have found that the academic achievement in one subject is closely related to achievement in other subjects. For example, math grades are significantly correlated with science grades [16,39], while English grades have a significant impact on math and science grades [3,4]. As a result, if learning data from multiple subjects in which students participated can be collected, problem-solving information from one subject can be used to predict achievement in other subjects.
In particular, if we collect problem-solving performance data for students’ first language subjects (e.g., Korean, English, French, etc.), we can estimate their language proficiency (LP) information. LP is an important factor when learners acquire domain knowledge, and it has been shown to have a significant impact on academic performance in a variety of subjects [6,15,30,35]. Moreover, LP is an important factor in the learner’s problem solving. This is owing to the fact that learners must be able to read and comprehend the problem to complete it correctly [6]. For example, in mathematics, students with low LP are known to struggle with long word problems [37]. Therefore, LP information extracted from students’ problem-solving data of language subjects is likely to be useful for the KT task to predict their future performance in other subjects.
However, despite the fact that LP is an important factor in predicting students’ academic performance, to our knowledge, no study in the KT field has used this information. In addition, studies that used academic achievement data in one subject to predict the student’s knowledge state in another subject have not been sufficiently conducted.
In this study, we propose a KT method using LP information extracted from Korean problem solving data from students. First, we analyzed whether LP information was related to students’ math word problem performance. In addition, through experiments with real-world datasets, we predicted the knowledge state using students’ LP information and investigated whether using this information was effective in improving the performance of the KT model. Furthermore, we investigated whether using LP information can help mitigate the cold start problem.
As a result, the following research questions (RQs) were posed in this study, in an attempt to find answers.
Language Proficiency Enhanced Knowledge Tracing
Knowledge Tracing
Student Modeling
Learning Analytics
Language Proficiency
Data matters. Accordingly, data augmentation has been an important ingredient for boosting performances of learned models. Prior data augmentation methods for few-shot text classification have led to great performance boosts. However, they have not been designed to capture the intricate compositional structure of natural language. As a result, they fail to generate samples with plausible and diverse sentence structures. Motivated by this, we present the data augmentation using lexicalized probabilistic context-free grammars that generates augmented samples with diverse syntactic structures with plausible grammar.
The Lexicalized PCFG parse trees consider both the constituents and dependencies to produce a syntactic frame that maximizes a variety of word choices in a syntactically preservable manner without specific domain experts. Experiments on few-shot text classification tasks demonstrate that ALP enhances many state-of-the-art classification methods.
We delve into the train-val splitting methodologies when a data augmentation method comes into play. We argue empirically that the traditional splitting of training and validation sets is sub-optimal compared to our novel augmentation-based splitting strategies that further expand the training split with the same number of labeled data. Taken together, our contributions on the data augmentation strategies yield a strong training recipe for few-shot text classification tasks.

Using Self-Training for Learning with limited data?

Learning with Limited Data using Compositionality in Language
NLP
ML
오늘은 교육 전문가들의 의견을 반영하여, 실용적으로 실제 교육 현장에서 활용하기 용이한 학습 성취 예측 방법을 제안한 연구를 소개하도록 하겠습니다.

학습 성취 예측에 대한 선행 연구들

학생의 학습 성취를 미리 예측할 수 있다면, 적절한 교육적인 조치를 취함으로써 학습의 효과를 높이는 데 기여할 수 있습니다. 이러한 기대 하에 학생의 학습 성취를 예측하기 위한 연구들이 오랜 시간 꾸준히 진행되어 왔습니다. 학습자 해당 분야에 대한 선행 연구들을 분석하면 다음과 같은 특징을 발견할 수 있습니다.
•
데이터가 수집된 강좌의 수가 연구마다 다양합니다. 하나의 강좌로부터 데이터를 수집한 경우도 있고, 많게는 700개에 가까운 강좌로부터 데이터를 수집한 경우도 있습니다.
•
다양한 머신러닝 알고리즘이 사용됩니다. 공통적으로 많이 사용되는 알고리즘으로는 decision tree (DT), Naïve Bayes (NB), multi-layer perceptron (MLP), support vector machine (SVM), logistic regression (LR), random forest (RF), and K-nearest neighbor (KNN) 등이 있으며, 앙상블 기법도 많이 사용됩니다.
•
‘위험군’을 예측하는 task의 경우, 위험군을 정의하는 방식이 다양하게 존재합니다. 예를 들어, 특정 학점을 기준으로 위험군을 정의하기도 하고(예: C학점 이하), 전체 수강 학생들의 평균 점수를 기준으로 위험군을 정의하기도 합니다.
•
모델 학습에 사용되는 속성에 대한 교육적인 배경 이론이 다양합니다. 주로 사용되는 배경 이론으로는 Moore의 상호작용 이론, 구성주의 이론, 자기 주도 학습 이론 등이 있습니다.
•
모델 학습에 사용되는 속성들을 선정하는 과정에 큰 관심을 두지 않는 경우가 많습니다. 해당 과정에 대해 설명한 연구들의 경우 상관관계 분석을 이용하거나, 선행 연구들을 참조한 경우가 많습니다.
오늘 소개드릴 연구에서는, 이렇게 꾸준히 진행되어 온 학습 성취 예측 기술에 대해 한 가지 의문을 제기합니다. “과연 이 기술들을 실제 교육 현장에서 사용할 수 있을까?”
교육 현장의 목소리를 반영한 LMS 데이터 기반의 실용적인 학습 성취 조기 예측 방법 제안
LMS
Education
Personality detection identifies a person's characteristic patterns in the online text he or she creates. Active participants on social media yield a considerable amount of online posts implying their psychological status. Having emerged personalized systems using online postings, natural language processing tasks requires psycholinguistic knowledge to detect individuals' personality traits. This emerging task is currently in demand for extensive application scenarios such as personalized recommendation systems, dialogue systems, and more.
Yet prior personality detection has been under-explored and restricted. Existing methods require resource-intensive and time-consuming professional tags to infer personality traits using ground-truth information. Those using deep neural networks have still focused on extra resources of lexical features in addition to the labeled training data. The performance heavily relying on such manual resources as Linguistic Inquiry and Word Count (LIWC) and Medical Research Council (MRC) restricts the pre-trained model's psycholinguistic proficiency and constrains the method in words only specified in the dictionaries. Not only utilizing the dictionaries but pre-trained language models are not in proper use.
We have three assumptions regarding pre-trained language models (PLMs) to minimize these issues. First, personality detection methods utilizing pre-trained language models (PLMs) rely on shortcutted lexical cues than understanding the knowledge in texts. Second, those lexical cues harm PLMs to perform well on out-of-distribution or unseen examples. Third, PLMs are capable of acknowledging the causal relations across out-of-distribution datapoints. We aim to generalize models well to unseen data by inducing models to emerge relevant knowledge through causal relations

What is Personality Detection?

Most work on personality detection studies the model performance with an evaluation metric such as Big Five models. The Big Five models utilize four different psychological attributes of analyzing how one gains energy (extraversion), how one processes information (openness to experience), how one makes decisions (agreeableness), and how one presents themselves to the outside world (conscientiousness), and how one he feels anxiety and depression (neuroticism).

Performance Depending on Training Set Ratios?

Emerging Knowledge from Pre-Trained Language Models for Psycholinguistic Analysis
Psycholinguistic Analysis
NLP
ML