Question: Evaluating RateMyProfessors reviews across institutions, disciplines, and cultures: Identifying the hallmarks of an excellent professor.

Rating or Not? Semantic Direction Utilized for Unsupervised Categorization of Opinions

This work introduces a straightforward unsupervised learning technique for categorizing opinions as positive endorsements ("thumbs up") or negative critiques ("thumbs down"). The categorization of an opinion is determined by averaging the semantic direction of the phrases within that contain adjectives or adverbs. A phrase is considered to have a positive semantic direction when associated with favorable attributes (e.g., "delicate subtleties") and a negative semantic direction when linked to unfavorable traits (e.g., "extremely cavalier"). Herein, the semantic direction of a phrase is quantified as the mutual information between the phrase and the term "outstanding" subtracted from the mutual information between the phrase and the term "substandard". An opinion is classified as a positive endorsement if the average semantic direction of its constituent phrases is positive. When tested on 410 opinions from the Epinions platform, spanning four distinct categories (automobiles, financial institutions, films, and travel locales), the method yields an average precision of 74%. The precision ranges from 84% for automobile evaluations to 66% for film reviews, demonstrating the versatility of this unsupervised categorization approach.

Content Memory (Learning Ability) As Comprehension 82 Vocabulary Cs .30 ( ) .23 .31 ( ) .31 .31 .35 ( ) .29 .48 .35 .38 ( ) .30 .40 .47 .58 .48 ( ) As judged against these latter values, comprehension (.48) and vocabulary (.47), but not memory (.31), show some specific validity. This transmutability of the validation matrix argues for the comparisons within the heteromethod block as the most generally relevant validation data, and illustrates the potential interchangeability of trait and method components. Some of the correlations in Chi's (1937) prodigious study of halo effect in ratings are appropriate to a multitrait-multimethod matrix in which each rater might be regarded as representing a different method. While the published report does not make these available in detail because it employs averaged values, it is apparent from a comparison of his Tables IV and VIII that the ratings generally failed to meet the requirement that ratings of the same trait by different raters should correlate higher than ratings of different traits by the same rater. Validity is shown to the extent that of the correlations in the heteromethod block, those in the validity diagonal are higher than the average heteromethod-heterotrait values. A conspicuously unsuccessful multitrait-multimethod matrix is provided by Campbell (1953, 1956) for rating of the leadership behavior of officers by themselves and by their subordinates. Only one of 11 variables (Recognition Behavior) met the requirement of providing a validity diagonal value higher than any of the heterotrait-heteromethod values, that validity being .29. For none of the variables were the validities higher than heterotrait-monomethod values. A study of attitudes toward authority and nonauthority figures by Burwen and Campbell (1957) contains a complex multitrait-multimethod matrix, one symmetrical excerpt from which is shown in Table 6. Method variance was strong for most of the procedures in this study. Where validity was found, it was primarily at the level of validity diagonal values higher than heterotrait-heteromethod values. As illustrated in Table 6, attitude toward father showed this kind of validity, as did attitude toward peers to a lesser degree. Attitude toward boss showed no validity. There was no evidence of a generalized attitude toward authority which would include father and boss, although such values as the VALIDATION BY THE MULTITRAIT-MULTIMETHOD MATRIX

Self-serving biases, found routinely in Western samples, have not been observed in Asian samples. Yet given the orientation toward individualism and collectivism in these 2 cultures, respectively, it is imperative to examine whether parallel differences emerge when the target of evaluation is the group. It may be that Asians show a group-serving bias parallel to the Western self-serving bias. In 2 studies, group-serving biases were compared across European Canadian, Asian Canadian, and Japanese students. Study 1 revealed that Japanese students evaluated a family member less positively than did both groups of Canadian students. Study 2 replicated this pattern with students' evaluations of their universities. The data suggest that cultural differences in enhancement biases are robust, generalizing to individuals' evaluations of their groups.


Question: Evaluating RateMyProfessors reviews across universities, subjects, and societies: Uncovering the hallmarks of an excellent educator.

This paper introduces an innovative approach to sentiment analysis, focusing on the identification and categorization of subjective viewpoints within text documents, such as classifying movie reviews as positive or negative. Our method employs a sophisticated machine-learning technique that specifically targets the subjective sections of the text for analysis. To achieve this, we utilize advanced graph theory algorithms to efficiently locate and isolate the minimum cuts in the document's structure, which helps in accurately identifying the context and sentiment across sentences. This technique significantly enhances the incorporation of cross-sentence contextual information, improving the overall accuracy of sentiment polarity detection. This method has been successfully applied and published in the Proceedings of the ACL in 2004, demonstrating its effectiveness in sentiment analysis tasks.

**Question**: The Role of Emotions in Social Media: A Comparative Study of Facebook and Twitter

**Documents**: Emotion and Twitter
We present a study of emotion on Twitter. Our first goal is to identify which emotions are expressed on Twitter, and how frequently. Our second goal is to determine how these emotions are expressed: through lexical choice, emoticons, or both. We use a crowdsourcing technique to identify emotion-bearing tweets, and then use a machine-learning algorithm to classify tweets according to the emotions they express. We find that happiness is the most frequently expressed emotion on Twitter, and that it is expressed primarily through lexical choice. Publication info: Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing.

**Similar Documents**: In this research, we conduct an in-depth analysis of emotional expression on Twitter, focusing on the types of emotions conveyed and their frequency. Our investigation aims to understand the mechanisms through which these emotions are communicated, including the use of specific vocabulary, emoticons, or a combination of both. To identify tweets laden with emotion, we employ a crowdsourcing method, followed by the application of a machine-learning model to categorize these tweets based on the emotions they convey. Our findings reveal that happiness is the predominant emotion expressed on Twitter, primarily communicated through word choice. This study, published in the Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing, provides valuable insights into the emotional landscape of Twitter.

This study of 805 Finnish teachers working in elementary, secondary, and vocational schools tested 2 interaction hypotheses. On the basis of the job demands–resources model, the authors predicted that job resources act as buffers and diminish the negative relationship between pupil misbehavior and work engagement. In addition, using conservation of resources theory, the authors hypothesized that job resources particularly influence work engagement when teachers are confronted with high levels of pupil misconduct. In line with these hypotheses, moderated structural equation modeling analyses resulted in 14 out of 18 possible 2-way interaction effects. In particular, supervisor support, innovativeness, appreciation, and organizational climate were important job resources that helped teachers cope with demanding interactions with students.

Flipped classroom approach has been increasingly adopted in higher institutions. Although this approach has many advantages, there are also many challenges that should be considered. In this paper, we discuss the suitability of this approach to teach computer programming, and we report on our pilot experience of using this approach at Qatar University to teach one subject of computer programming course. It is found that students has positive attitude to this approach, it improves their learning. However, the main challenge was how to involve some of the students in online learning activities.

Seven experts on personality measurement here discuss the viability of public-domain personality measures, focusing on the International Personality Item Pool (IPIP) as a prototype. Since its inception in 1996, the use of items and scales from the IPIP has increased dramatically. Items from the IPIP have been translated from English into more than 25 other languages. Currently over 80 publications using IPIP scales are listed at the IPIP Web site (http://ipip.ori.org), and the rate of IPIPrelated publications has been increasing rapidly. The growing popularity of the IPIP can be attributed to Wve factors: (1) It is cost free; (2) its items can be obtained instantaneously via the Internet; (3) it includes over 2000 items, all easily available for inspection; (4) scoring keys for IPIP scales are This article represents a synthesis of contributions to the presidential symposium, The International Personality Item Pool and the Future of Public-Domain Personality Measures (L.R. Goldberg, Chair) at the sixth annual meeting of the Association for Research in Personality, New Orleans, January 20, 2005. Authorship order is based on the order of participation in the symposium. The IPIP project has been continually supported by Grant MH049227 from the National Institute of Mental Health, U.S. Public Health Service. J.A. Johnson’s research was supported by the DuBois Educational Foundation. The authors thank Paul T. Costa Jr., Samuel D. Gosling, Leonard G. Rorer, Richard Reed, and Krista Trobst for their helpful suggestions. * Corresponding author. Fax: +1 814 375 4784. E-mail address: j5j@psu.edu (J.A. Johnson). 0092-6566/$ see front matter  2005 Elsevier Inc. All rights reserved. doi:10.1016/j.jrp.2005.08.007 L.R. Goldberg et al. / Journal of Research in Personality 40 (2006) 84–96 85 provided; and (5) its items can be presented in any order, interspersed with other items, reworded, translated into other languages, and administered on the World Wide Web without asking permission of anyone. The unrestricted availability of the IPIP raises concerns about possible misuse by unqualiWed persons, and the freedom of researchers to use the IPIP in idiosyncratic ways raises the possibility of fragmentation rather than scientiWc uniWcation in personality research.  2005 Elsevier Inc. All rights reserved.

This chapter reviews basic issues of theory and method in approaches to research on teaching that are alternatively called ethnographic, qualitative, participant observational, case study, symbolic interactionist, phenomenological, constructivist, or interpretive. These approaches are all slightly different, but each bears strong family resemblance to the others. The set of related approaches is relatively new in the field of research on teaching. The approaches have emerged as significant in the decade of the 1960s in England and in the 1970s in the United States, Australia, New Zealand, and Germany. Because interest in these approaches is so recent, the previous editions of the Handbook of Research on Teaching do not contain a chapter devoted to participant observational research. Accordingly, this chapter attempts to describe r~s and their theoretical _nresup~s in considerable detail and does not attempt an exhaustive review of the rapidly growing literature in the field. Such a review will be appropriate for the next edition of this handbook. From this point on I will use the term interpretive to refer to the whole family of approaches to participant observational research. I adopt this term for three reasons: (a) It is more inclusive than many of the others (e.g., ethnography, case study); (b) W Blake

There is a trend in Irish universities to utilise the benefits of the e-learning as a mechanism to improve learning performance of campus-based students. Whilst traditional methods, such as face-to-face lectures, tutorials, and mentoring, remain dominant in the educational sector, universities are investing heavily in learning technologies, to facilitate improvements with respect to the quality of learning. The technology to support reuse and sharing of educational resources, or learning objects, is becoming more stable, with interoperability standards maturing. However, debate has raged about what constitutes effective use of learning technology. This research expands upon a study carried out in 2003 examining students’ perceptions of e-learning in a large undergraduate accounting class environment. As a result, improvements were made to the instructional design of the course, to enable students to engage interactively with content. The subsequent study, reported in this paper, adopted a broad range of techniques to understand students’ learning experience in depth. The findings of this research provide an insight into how these students really work and learn using technologies, if at all. It is hoped that our findings will improve the experience for both students and lecturers who engage in teaching and learning through this medium. 502 British Journal of Educational Technology Vol 36 No 3 2005 © British Educational Communications and Technology Agency, 2005. Introduction A recent trend in higher education is to create and provide online access to course materials. Over the past two decades academics and institutes of higher education have been diversifying their delivery of instruction through new Internet media such as learning management systems, asynchronous distance learning, and online classrooms amongst a myriad of other burgeoning educational technologies. This combination of traditional face-to-face lectures or tutorials, and web-based course content is better known as “blended learning,” purporting to blend the best aspects of real and virtual environments. Many Irish universities have invested in architectures and platforms to support their teaching staff in delivering material to students in a blended manner. Other institutions have adopted a lecturer-driven approach, whereby teaching staff are left to their own devices to supplement their lectures and tutorials with online material, hosted via their own web servers and typically open source, freeware software, or basic web pages. The reasons behind the drive to incorporate technology into the educational process are threefold. First, pressure to utilise information and communications technology (ICT) at university level comes from changes in the student demography. The rise of “full time part time students” is a phenomenon of recent years, where school leavers take part-time jobs whilst attending university, leaving less time for evening tutorials or weekend study. In addition, there is a drive for what is known as lifelong learning, whereby adults are increasingly returning to institutions of higher education to take supplementary courses whilst in full-time employment, or during short career breaks. In this regard, the Irish government has made a commitment to have 15% of adults in continuing higher education by 2006. The figure currently stands at less than 5% (Department of Education and Science, 2000). This movement to adult continuing education was also reflected in a statement made by Information Commission Society (Ireland): “The issue of integration IT to the teaching process is an important part of future improvements, which it will be crucial to pursue” (Boylan, 2000, p. 31). Second, changes in the market for delivery of education is also shifting. Private for-profit higher education institutions are offering a wide range of certificate and degree courses. The Irish Open University Initiative “Oscail” and its UK equivalent offer distance learning diploma and degree courses. Many of these courses are delivered through ICT, along with weekend face-to-face tutorials. This distance learning market has also seen the arrival of new entrants, such as the Atlantic University Alliance, a collaboration between National University of Ireland, Galway, University College Cork, and the University of Limerick. Similar undertakings are underway in other institutes of higher education, whereby expertise in delivering blended learning within the institution is being extended towards distance learning courses by using a significant amount of ICT to overcome geographic and time barriers. Third, another pressure on traditional institutions of higher education comes from innovations in new technologies. John Seely Brown and Paul Duiguid claim that these technologies “offer new ways to think of producing, distributing and consuming acaWhat students think about e-learning 503 © British Educational Communications and Technology Agency, 2005. demic material. As with so many other institutions, new technologies have caused universities to rethink not simply isolated features but their entire mission and how they go about it” (Seely Brown & Duiguid, 2000). This sentiment is also being echoed by government policy towards higher education where the Irish State is attempting to set certain parameters of performance covering such areas as equity of access, commitment to e-learning and to ICT, commitment to life-long learning, and outreach to other communities. In the subject domain of accountancy, the integration of technology into the classroom is also becoming a central issue for accounting educators for an additional reason. Accountancy professionals in industry require competency in accountancy-related information technology (IT) skills. Higher education institutions, in response, have tried to simulate authentic learning scenarios in the accountancy class and incorporate IT topics into their programmes. However, the efficacy of introducing technology in accounting education remains unclear. Positive learning effects associated with IT in accounting education have been noted (Abdolmohammadi et al , 1998; Fetters, McKenzie & Callaghan, 1986; Friedman, 1981). Equally, evidence has shown that technology has negatively impacted on learning (Dickens & Harper, 1986; Togo & McNamee, 1997). Indeed, some critics harshly criticise its introduction to educational contexts. Selwyn (2002, p. 172) claims in generic terms that “...in the final analysis educational technology remains a diverting pastime and ‘add-on’ to the curriculum for some teaching and learners, whilst for most in education its impact has been slight.” This paper hopes to look at some of the issues raised by individual students with regard to the use of technology in the teaching and learning process. Rather than focusing on macro issues, such as the political decisions behind the integration of technology, this research aims to examine individuals’ acceptance or rejection of using ICT. Exploring the perspective of the individual student when discussing the use of technology is paramount to beginning to understand the nature of ICT in higher educational settings. This is of importance both to policy makers and education technologists in the debate on the role of ICT in education at an organisational level. The module and the medium This research focused on one module of The Principles of Accounting being taught to approximately 600 first-year undergraduate students at the University of Limerick. The module under investigation was part of a wider course, leading to a degree in Business Studies or Law and Accounting. The majority of students were between 17 and 19 years of age, coming from distributed second-level schools in Ireland. A minority were Erasmus students from various European countries, or mature students over the age of 23. For all, except overseas students, this was their first experience in using an e-learning platform, although most had previously informal used the web to gather information, or prepare coursework in second-level education, prior to entering university. This module was delivered by using a blended learning approach, supplementing weekly lectures, tutorials, and laboratory sessions with online course content, interac504 British Journal of Educational Technology Vol 36 No 3 2005 © British Educational Communications and Technology Agency, 2005. tive quizzes, and Excel tasks. A range of different e-learning assessments complemented the traditional weekly meetings. Students were asked to submit a compulsory paperbased accounting case study, and additionally were offered two optional ICT assignments, a stream of online multiple-choice quizzes, and an Excel project. A module web site provided course details, additional readings, and supplementary links. Employing multiple teaching methods simultaneously is a form of blended learning (Saunders & Werner, 2003). Alternative instructional resources can stimulate positive effects on accounting students’ learning experiences, according to (Rebele et al , 1998). Furthermore, instructional innovations are desirable to develop accounting students’ IT competencies (Albrecht & Sack, 2000). Figure 1 illustrates the staged delivery of both the traditional and online elements of the module. The university does not provide institutional support for e-learning on a campus-wide scale, but two other modules in their first year also used a blended learning delivery approach, providing lecture notes online, or sample answers, in the subject area of economics and mathematics. Both of these other modules differ in marking scheme and in the extent to which online supports are provided such as discussion boards, or positive marking schemes for continuous online assessments. As such, blended learning is not yet a norm across modules for first-year students at this current time. Methods The research developed on experience gained in a previous study on a prior

This paper presents a simple unsupervised learning algorithm for classifying reviews as recommended (thumbs up) or not recommended (thumbs down). The classification of a review is predicted by the average semantic orientation of the phrases in the review that contain adjectives or adverbs. A phrase has a positive semantic orientation when it has good associations (e.g., “subtle nuances”) and a negative semantic orientation when it has bad associations (e.g., “very cavalier”). In this paper, the semantic orientation of a phrase is calculated as the mutual information between the given phrase and the word “excellent” minus the mutual information between the given phrase and the word “poor”. A review is classified as recommended if the average semantic orientation of its phrases is positive. The algorithm achieves an average accuracy of 74% when evaluated on 410 reviews from Epinions, sampled from four different domains (reviews of automobiles, banks, movies, and travel destinations). The accuracy ranges from 84% for automobile reviews to 66% for movie reviews.

This study aims to examine the possibility of implementing the Augmented Reality (AR) application in higher education by answering three questions: Which extended faculty at the Saudi Electronic University are familiar with such applications, what the perceptions do they hold toward using it in education, and what barriers do they believe may hinder implementing this technology. An online survey was designed and distributed to faculty members at two colleges selected randomly to collect data from participants. Even though the faculty were at an accepted level of familiarity with this technology, they did not use it in their classes. Results showed that faculty have a positive attitude toward the use of AR and have confidence in its potential to enrich the learning environment. This result is connected to the AR advantages the faculty were in agreement with. Results also showed that faculty have concerns about some barriers that might impact implementation of AR in the education environment, such as the lack of technical support.

The overall goal of our Software Engineering Teamwork Assessment and Prediction (SETAP) project is to develop effective machine-learning-based methods for assessment and early prediction of student learning effectiveness in software engineering teamwork. Specifically, we use the Random Forest (RF) machine learning (ML) method to predict the effectiveness of software engineering teamwork learning based on data collected during student team project development. These data include over 100 objective and quantitative Team Activity Measures (TAM) obtained from monitoring and measuring activities of student teams during the creation of their final class project in our joint software engineering classes which ran concurrently at San Francisco State University (SFSU), Fulda University (Fulda) and Florida Atlantic University (FAU). In this paper we provide the first RF analysis results done at SFSU on our full data set covering four years of our joint SE classes. These data include 74 student teams with over 380 students, totaling over 30000 discrete data points. These data are grouped into 11 time intervals, each measuring important phases of project development during the class (e.g. early requirement gathering and design, development, testing and delivery). We briefly elaborate on the methods of data collection and describe the data itself. We then show prediction results of the RF analysis applied to this full data set. Results show that we are able to detect student teams who are bound to fail or need attention in early class time with good (about 70%) accuracy. Moreover, the variable importance analysis shows that the features (TAM measures) with high predictive power make intuitive sense, such as late delivery/late responses, time used to help each other, and surprisingly statistics on commit messages to the code repository, etc. In summary, we believe we demonstrate the viability of using ML on objective and quantitative team activity measures to predict student learning of software engineering teamwork, and point to easy-to-measure factors that can be used to guide educators and software engineering managers to implement early intervention for teams bound to fail. Details about the project and the complete ML training database are downloadable from the project web site.

Despite its continuing popularity as the social network site par excellence, the educational value of Facebook has not been fully determined, and results from the mainstream educational paradigms are contradictory, with some scholars emphasizing its pedagogical affordances (e.g., widening context of learning, mixing information and learning resources, hybridization of expertise) and others cautioning against its use for educational purposes. Moreover, systematic reviews about documented educational usage of Facebook as a learning environment are lacking. This article attempts to provide a critical overview of current studies focusing on the use of Facebook as a technology-enhanced learning environment, with the aim of exploring the extent to which its pedagogical potential is actually translated into practice. Only empirical studies published in peer-reviewed academic journals with a specific focus on Facebook as a learning environment have been considered for the review. The authors conducted a comprehensive literature search that identified 23 relevant articles that were subsequently analysed according to a simplified list of guidelines. These articles were further analysed and recoded through a set of emerging categories. The results show that pedagogical affordances of Facebook have only been partially implemented and that there are still many obstacles that may prevent a full adoption of Facebook as a learning environment such as implicit institutional, teacher and student pedagogies, and cultural issues. Finally, a broad observation on the implications of the study is developed with some suggestions for future research.

In this paper we introduce a novel computational technique of extraction of personality traits (HEXACO) of employees from Enterprise Social Media posts. We deal with challenges such as not being able to use existing survey instruments for scoring and not being able to directly use existing psychological studies on written text due to lack of overlapping words between the existing dictionary and words used in Enterprise Social Media. Using our approach we are able to infer personality traits (HEXACO) from posts and find better coverage and usage of the extended dictionary.


Question: Evaluating RateMyProfessors reviews across institutions, disciplines, and cultures: Identifying the hallmarks of an excellent professor.

This document introduces an innovative method for conducting phrase-level sentiment analysis that effectively distinguishes between neutral and polar expressions, subsequently clarifying the polarity of the identified polar expressions. The proposed system boasts the capability to autonomously ascertain the contextual polarity for a substantial portion of sentiment-related expressions, yielding outcomes that markedly surpass baseline measures. Specifically, in the realm of evaluating professorial performance across various institutions, disciplines, and cultures, this technique can be employed to decipher the nuanced sentiments expressed in student evaluations on platforms like RateMyProfessors. By accurately discerning the polarity of comments within their specific contexts, educators and academic administrators can gain valuable insights into the attributes that students perceive as indicative of a good professor. This method's precision in sentiment analysis enhances the reliability of such evaluations, facilitating more informed decisions regarding teaching effectiveness and student satisfaction across diverse academic environments.

Scholars in various disciplines have considered the causes, nature, and effects of trust. Prior approaches to studying trust are considered, Including characteristics of the tnistor, the trustee, and the role of risk. A definition of trust and a model of its antecedents and outcomes are presented, which integrate research from multiple disciplines and differentiate trust from similar constructs. Several research propositions based on the model are presented.

Learning predictors for student retention is very difficult. After reviewing the literature, it is evident that there is considerable room for improvement in the current state of the art. As shown in this paper, improvements are possible if we (a) explore a wide range of learning methods; (b) take care when selecting attributes; (c) assess the efficacy of the learned theory not just by its median performance, but also by the variance in that performance; (d) study the delta of student factors between those who stay and those who are retained. Using these techniques, for the goal of predicting if students will remain for the first three years of an undergraduate degree, the following factors were found to be informative: family background and family’s social-economic status, high school GPA

The convergent and discriminant validities of well-being concepts were examined using multitrait-multimethod matrix analyses (D. T. Campbell & D. W. Fiske, 1959) on 3 sets of data. In Study 1, participants completed measures of life satisfaction, positive affect, negative affect, self-esteem, and optimism on 2 occasions 4 weeks apart and also obtained 3 informant ratings. In Study 2, participants completed each of the 5 measures on 2 occasions 2 years apart and collected informant reports at Time 2. In Study 3, participants completed 2 different scales for each of the 5 constructs. Analyses showed that (a) life satisfaction is discriminable from positive and negative affect, (b) positive affect is discriminable from negative affect, (c) life satisfaction is discriminable from optimism and self-esteem, and (d) optimism is separable from trait measures of negative affect.


Question: Evaluating RateMyProfessors reviews across universities, subjects, and societies: Uncovering the indicators of an excellent professor.

This paper introduces a novel method for performing context-sensitive phrase-level polarity analysis in sentences, focusing on determining the subjective orientation of phrases. Our methodology integrates lexical scoring derived from the Dictionary of Affect in Language (DAL), which is expanded through WordNet, enabling automatic scoring for a wide range of words in the input text without the necessity for manual annotation. To further refine the analysis, we incorporate n-gram analysis to take into account the influence of context on the phrase's polarity. We combine the DAL scores with syntactic structures from the sentences, and then extract n-grams from these structures. Additionally, the polarity of each syntactic element within the sentence is utilized as a feature in our model. Our experimental outcomes reveal substantial enhancements over a baseline that relies on the majority class, as well as over a more complex baseline that uses lexical n-grams alone. This demonstrates the effectiveness of our approach in capturing nuanced contextual information in phrase-level polarity analysis.

Expertise retrieval has been largely unexplored on data other than the W3C collection. At the same time, many intranets of universities and other knowledge-intensive organisations offer examples of relatively small but clean multilingual expertise data, covering broad ranges of expertise areas. We first present two main expertise retrieval tasks, along with a set of baseline approaches based on generative language modeling, aimed at finding expertise relations between topics and people. For our experimental evaluation, we introduce (and release) a new test set based on a crawl of a university site. Using this test set, we conduct two series of experiments. The first is aimed at determining the effectiveness of baseline expertise retrieval methods applied to the new test set. The second is aimed at assessing refined models that exploit characteristic features of the new test set, such as the organizational structure of the university, and the hierarchical structure of the topics in the test set. Expertise retrieval models are shown to be robust with respect to environments smaller than the W3C collection, and current techniques appear to be generalizable to other settings.

This paper articulates a framework and criteria that can be used to evaluate the quality of theories. While the framework and criteria have general applicability, my focus is the evaluation of theories within the information systems discipline. To illustrate the usefulness of the framework and criteria, I show how they can be employed to pinpoint the strengths and weaknesses of a theory which, based upon citation evidence, has had a significant impact on other researchers within the information systems discipline. Because the evaluation of existing theories often provides the basis for refining existing theories or building new theories, I also show how the framework and criteria can be used to inform the development of high-quality theory.

BACKGROUND
Research has shown how academic emotions are related to achievement and to cognitive/motivational variables that promote achievement. Mediated models have been proposed to account for the relationships among academic emotions, cognitive/motivational variables, and achievement, and research has supported such mediated models, particularly with negative emotions.

AIMS
The study tested the hypotheses: (1) self-regulation and the positive academic emotions of enjoyment and pride are positive predictors of achievement; and (2) enjoyment and pride both moderate the relationship between self-regulation and achievement.

SAMPLE
Participants were 1,345 students enrolled in various trigonometry classes in one university.

METHODS
Participants answered the Academic Emotions Questionnaire-Math (Pekrun, Goetz, & Frenzel, 2005) and a self-regulation scale (Pintrich, Smith, Garcia, & McKeachie, 1991) halfway through their trigonometry class. The students' final grades in the course were regressed to self-regulation, positive emotions, and the interaction terms to test the moderation effects.

RESULTS AND CONCLUSIONS
Enjoyment and pride were both positive predictors of grades; more importantly, both moderated the relationship between self-regulation and grades. For students who report higher levels of both positive emotions, self-regulation was positively associated with grades. However, for those who report lower levels of pride, self-regulation was not related to grades; and, for those who reported lower levels of enjoyment, self-regulation was negatively related to grades. The results are discussed in terms of how positive emotions indicate positive appraisals of task/outcome value, and thus enhance the positive links between cognitive/motivational variables and learning.

School of Business Administration, Southwestern University of Finance and Economics, Sichuan, P. R. of China; Business School, Shantou University, Guangdong, P. R. of China; Information Technology Management, School of Business, University at Albany, Albany, NY, U.S.A; Department of Management, Hong Kong University of Science and Technology, Hong Kong, P.R. China; Department of Finance and Decision Sciences, School of Business, Hong Kong Baptist University, Hong Kong, P.R. China

Predicting student academic performance has long been an important research topic in many academic disciplines. The present study is the first study that develops and compares four types of mathematical models to predict student academic performance in engineering dynamics – a high-enrollment, highimpact, and core course that many engineering undergraduates are required to take. The four types of mathematical models include the multiple linear regression model, the multilayer perception network model, the radial basis function network model, and the support vector machine model. The inputs (i.e., predictor variables) of the models include student’s cumulative GPA, grades earned in four pre-requisite courses (statics, calculus I, calculus II, and physics), and scores on three dynamics mid-term exams (i.e., the exams given to students during the semester and before the final exam). The output of the models is students’ scores on the dynamics final comprehensive exam. A total of 2907 data points were collected from 323 undergraduates in four semesters. Based on the four types of mathematical models and six different combinations of predictor variables, a total of 24 predictive mathematical models were developed from the present study. The analysis reveals that the type of mathematical model has only a slight effect on the average prediction accuracy (APA, which indicates on average how well a model predicts the final exam scores of all students in the dynamics course) and on the percentage of accurate predictions (PAP, which is calculated as the number of accurate predictions divided by the total number of predictions). The combination of predictor variables has only a slight effect on the APA, but a profound effect on the PAP. In general, the support vector machine models have the highest PAP as compared to the other three types of mathematical models. The research findings from the present study imply that if the goal of the instructor is to predict the average academic performance of his/her dynamics class as a whole, the instructor should choose the simplest mathematical model, which is the multiple linear regression model, with student’s cumulative GPA as the only predictor variable. Adding more predictor variables does not help improve the average prediction accuracy of any mathematical model. However, if the goal of the instructor is to predict the academic performance of individual students, the instructor should use the support vector machine model with the first six predictor variables as the inputs of the model, because this particular predictor combination increases the percentage of accurate predictions, and most importantly, allows sufficient time for the instructor to implement subsequent educational interventions to improve student learning. 2012 Elsevier Ltd. All rights reserved.

We apply a new bibliometric measure, the h-index (Hirsch, 2005), to the literature of information science. Faculty rankings based on raw citation counts are compared with those based on h-counts. There is a strong positive correlation between the two sets of rankings. We show how the h-index can be used to express the broad impact of a scholar’s research output over time in more nuanced fashion than straight citation counts.

Recent studies appreciate that "MOOCs bring an impetus of reform, research and innovation to the Academy" [9]. Even though MOOCs are usually developed and delivered as independent online courses, experiments to wrap formal university courses around existing MOOCs are reported by teachers and researchers in different articles [3], [4], [8], [17]. This paper describes a new approach, in which the participation of students in different MOOCs was integrated in a blended course run on a social mobile LMS. The topics of MOOCs delivered on specific platforms and having particular characteristics were connected with the Fall 2013 undergraduate course of Web Programming, at University Politehnica Timisoara, Romania, facilitated by the first author, the co-authors providing a shadow/peer facilitation, for tailoring the course scenario. The main parts of this study deal with: a) The reasons to integrate MOOCs in the university course. b) How the course was designed, how the students’ activities on different MOOC platforms were assessed and integrated in the course scenario. c) The results of a survey that evaluates students’ experiences related to MOOCs: a number of MOOC features were assessed [14]; answers to a few problems are also analysed: Did the participation in MOOCs support students to clarify and expand the course issues? What are students' suggestions for a more active participation in MOOCs? By comparing the learning scenarios of MOOCs with the Web Programming blended course, how can the course and its virtual space be improved? Do students consider that the MOOCs phenomenon is important for professional and personal development? The conclusions of the paper can be used by other teachers/instructors for integrating MOOCs in the courses they deliver/facilitate.


Question: Evaluating RateMyProfessors reviews across universities, subjects, and societies: Uncovering the hallmarks of an excellent professor.

This paper introduces a sophisticated methodology for analyzing sentiments within opinions, addressing a complex issue in the realm of affective computing. We propose a system capable of automatically identifying individuals who express opinions on a specific topic, alongside the sentiment underlying those opinions. This system is equipped with dual functionalities: a module dedicated to assessing the sentiment of individual words and another for synthesizing sentiments within a sentence. To refine our approach, we evaluate different models for classifying and amalgamating sentiment at both the word and sentence levels, achieving encouraging outcomes. This research not only contributes to the field of sentiment analysis but also holds potential for applications in understanding public opinion, customer feedback, and online reviews across various domains, including education, where it can be pivotal in evaluating the effectiveness of teaching methods and professorial performance.

We review research and theory on the HEXACO personality dimensions of Honesty-Humility (H), Agreeableness (A), and Emotionality (E), with particular attention to the following topics: (1) the origins of the HEXACO model in lexical studies of personality structure, and the content of the H, A, and E factors in those studies; (2) the operationalization of the H, A, and E factors in the HEXACO Personality Inventory-Revised; (3) the construct validity of self-reports on scales measuring the H factor; (4) the theoretical distinction between H and A; (5) similarity and assumed similarity between social partners in personality, with a focus on H and A; (6) the extent to which H (and A and E) variance is represented in instruments assessing the "Five-Factor Model" of personality; and (7) the relative validity of scales assessing the HEXACO and Five-Factor Model dimensions in predicting criteria conceptually relevant to H, A, and E.

An enduring issue in higher education is student retention to successful graduation. National statistics indicate that most higher education institutions have four-year degree completion rates around 50%, or just half of their student populations. While there are prediction models which illuminate what factors assist with college student success, interventions that support course selections on a semester-to-semester basis have yet to be deeply understood. To further this goal, we develop a system to predict students’ grades in the courses they will enroll in during the next enrollment term by learning patterns from historical transcript data coupled with additional information about students, courses and the instructors teaching them. We explore a variety of classic and state-of-the-art techniques which have proven effective for recommendation tasks in the e-commerce domain. In our experiments, Factorization Machines (FM), Random Forests (RF), and the Personalized Linear Multiple Regression model achieve the lowest prediction error. Application of a novel feature selection technique is key to the predictive success and interpretability of the FM. By comparing feature importance across populations and across models, we uncover strong connections between instructor characteristics and student performance. We also discover key differences between transfer and non-transfer students. Ultimately we find that a hybrid FM-RF method can be used to accurately predict grades for both new and returning students taking both new and existing courses. Application of these techniques holds promise for student degree planning, instructor interventions, and personalized advising, all of which could improve retention and academic performance.

Yunglung Chen, Yuping Wang, Kinshuk, Nian-Shing Chen Department of Information Management, National Sun Yat-sen University, No. 70, Lienhai Rd., Kaohsiung 80424, Taiwan School of Languages and Linguistics, Griffith University, Nathan, 4111, Australia School of Computing and Information System, Athabasca University, Athabasca, Canada henryylchen@gmail.com, y.wang@griffith.edu.au, kinshuk@athabascau.ca, nschen@mis.nsysu.edu.tw

In what case do you like reading so much? What about the type of the qualitative research evaluation methods integrating theory and practice book? The needs to read? Well, everybody has their own reason why should read some books. Mostly, it will relate to their necessity to get knowledge from the book and want to read just to get entertainment. Novels, story book, and other entertaining books become so popular this day. Besides, the scientific books will also be the best reason to choose, especially for the students, teachers, doctors, businessman, and other professions who are fond of reading.

It has long been recognized that personality test scores are influenced by non-test-relevant response determinants. Wiggins and Rumrill (1959) distinguish three approaches to this problem. Briefly, interest in the problem of response distortion has been concerned with attempts at statistical correction for "faking good" or "faking bad" (Meehl & Hathaway, 1946), the analysis of response sets (Cronbach, 1946,1950), and ratings of the social desirability of personality test items (Edwards, 19 5 7). A further distinction can be made, however, which results in a somewhat different division of approaches to the question of response distortion. Common to both the Meehl and Hathaway corrections for faking good and faking bad and Cronbach's notion of response sets is an interest in the test behavior of the subject(S). By social desirability, on the other hand, Edwards primarily means the "scale value for any personality statement such that the scale value indicates the position of the statement on the social desirability continuum . . ." (1957, p. 3). Social desirability, thus, has been used to refer to a characteristic of test items, i.e., their scale position on a social desirability scale. Whether the test behavior of 5s or the social desirability properties of items are the focus of interest, however, it now seems clear that underlying both these approaches is the concept of statistical deviance. In the construction of the MMPI K scale, for example, items were selected which differentiated between clinically normal persons producing abnormal te¥Tpfpfiles~snd^cTinically abnormal individuals with abnormal test profiles, and between clinically abnormal persons with normal test profiles and abnormal 5s whose test records were abnormal. Keyed responses to the K scale items tend to be statistically deviant in the parent populations. Similarly, the development of the Edwards Social Desirability Scale (SDS) illustrates this procedure. Items were drawn from various MMPI scales (F, L, K, and the Manifest Anxiety Scale [Taylor, 1953]) and submitted to judges who categorized them as either socially desirable or socially undesirable. Only items on which there was unanimous agreement among the 10 judges were included in the SDS. It seems clear that the items in Edwards SDS would, of necessity, have extreme social desirability scale positions or, in other words, be statistically deviant. Some unfortunate consequences follow from the strict use of the statistical deviance model in the development of-sOcialTtesirSbTBty scales. With items drawn from the MMPI, it is apparent that in addition to their scalability for social desirability the items may also be characterized by their content which,^n a general sense, has pathological implications. When a social desrrabtltty^scale constructed according to this procedure is then applied to a college student population, the meaning of high social desirability scores is not at all clear. When 5s given the Edwards SDS deny, for example, that their sleep is fitful and disturbed (Item 6) or that they worry quite a bit over possible misfortunes (Item 35), it cannot be determined whether these responses are attributable to social desirability or to a genuine absence of such symptoms. The probability of occurrence of the symptoms represented in MMPI items (and incorportated in the SDS)

Expertise retrieval has been largely unexplored on data other than the W3C collection. At the same time, many intranets of universities and other knowledge-intensive organisations offer examples of relatively small but clean multilingual expertise data, covering broad ranges of expertise areas. We first present two main expertise retrieval tasks, along with a set of baseline approaches based on generative language modeling, aimed at finding expertise relations between topics and people. For our experimental evaluation, we introduce (and release) a new test set based on a crawl of a university site. Using this test set, we conduct two series of experiments. The first is aimed at determining the effectiveness of baseline expertise retrieval methods applied to the new test set. The second is aimed at assessing refined models that exploit characteristic features of the new test set, such as the organizational structure of the university, and the hierarchical structure of the topics in the test set. Expertise retrieval models are shown to be robust with respect to environments smaller than the W3C collection, and current techniques appear to be generalizable to other settings.


Question: A method for extracting triples from unstructured text using a parser.

This paper highlights DBpedia, a pioneering initiative aimed at transforming Wikipedia's rich content into a structured, accessible format on the Web. DBpedia's mission is to meticulously extract meaningful data from Wikipedia and subsequently publish this information, enabling complex queries and fostering connectivity between Wikipedia and other web-based datasets. By converting Wikipedia's vast repository into consumable datasets, DBpedia facilitates both human and machine interaction, offering a sophisticated gateway to knowledge. The paper delves into the intricate process of data extraction and the methodologies employed to ensure the information's availability on the Web. Additionally, it explores the burgeoning applications developed by the DBpedia community, demonstrating the practical integration of DBpedia content within various websites. Moreover, the paper underscores the significance of interlinking DBpedia with other open data sources, positioning DBpedia as a pivotal hub in the evolving landscape of the Web of Open Data. This interconnection not only enriches the data ecosystem but also exemplifies the potential for DBpedia to act as a central node in a network of interconnected datasets, fostering a more integrated and accessible web of knowledge.

In this report, we explore various methods that are applied to solve NER. In section 1, we introduce the named entity problem. In section 2, various named entity recognition methods are discussed in three three broad categories of machine learning paradigm and explore few learning techniques in them. In the first part, we discuss various supervised techniques. Subsequently we move to semi-supervised and unsupervised techniques. In the end we discuss about the method from deep learning to solve NER.

This paper examines the Stanford typed dependencies representation, which was designed to provide a straightforward description of grammatical relations for any user who could benefit from automatic text understanding. For such purposes, we argue that dependency schemes must follow a simple design and provide semantically contentful information, as well as offer an automatic procedure to extract the relations. We consider the underlying design principles of the Stanford scheme from this perspective, and compare it to the GR and PARC representations. Finally, we address the question of the suitability of the Stanford scheme for parser evaluation.

Natural Language Interface to Database (NLIDB) systems convert a Natural Language (NL) query into a Structured Query Language (SQL) and then use the SQL query to retrieve information from a database. The main advantage with this function of an NLIDB system is that it makes information retrieval much easier and more importantly, it also allows non-technical users to query a database. In this work, we present an effective usage of tokens in an NL query to address various problems in an NLIDB system. The conversion of an NL query to SQL query is framed as a token classification problem, using which we unveil a novel method of mapping an NL query to an SQL query. Our approach reasonably addresses domain dependency which is one of the major drawbacks of NLIDB systems. Concepts Identification is a major component of NLIDB systems (Gupta et al., 2012; Srirampur et al., 2014). We improve Concepts Identification (Srirampur et al., 2014) by making use of Stanford Parser Dependencies. Our approach is more robust than previously proposed methods. In addition, we also discuss how Concepts Identification can be applied to address Ellipsis in a dialogue process. In addition to providing results to a user, it is essential to provide a relevant and a compact set of results. We propose a new problem of generating a compact set of results to a user query. At a higher level, user-system interactions are modeled based on patterns frequently observed between a user’s current query and his previous queries, while interacting with a system. Using these models, we propose a novel method of system prompting to help a user obtain a smaller and a relevant set of results. In addition to providing a compact and a relevant set of results, it is imperative that answers of an NLIDB system are comprehensible even by the people who are less familiar with a common language like English. NLIDB systems use Natural Language Generation (NLG) modules to provide answers in the form of sentences. It is important to make sure that an answer generated by an NLG module is very simple to understand. This brings in the problem of text simplification, wherein, one of the most crucial and initial sub-problems is Complex Word Identification (CWI). We address the problem of CWI by distinguishing words as simple and complex. A plethora of classifiers were explored to identify the complex words. This information helps us in improving the simplification of an NLIDB system’s final output to a user. To summarize, in addition to addressing problems within an NLIDB system, this work touches postNLIDB problems like results processing. All the proposed issues are tackled using tokens in an NL query as the basic and a driving unit of force.

Identifying incorrect content (i.e., semantic error) in text is a difficult task because of the ambiguous nature of written natural language and themany factors that canmake a statement semantically erroneous. Current methods identify semantic errors in a sentence by determining whether it contradicts the domain to which the sentence belongs. However, because thesemethods are constructedon expected logic contradictions, they cannot handle new or unexpected semantic errors. In this paper, we propose a new method for detecting semantic errors that is based on logic reasoning. Our proposed method converts text into logic clauses, which are later analyzed against a domain ontology by an automatic reasoner to determine its consistency. This approach can provide a complete analysis of the text, since it can analyze a single sentence or sets of multiple sentences. When there are multiple sentences to analyze, in order to avoid the high complexity of reasoning over a large set of logic clauses, we propose rules that reduce the set of sentences to analyze, based on the logic relationships between sentences. In our evaluation, we have found that our proposed method can identify a significant percentage of semantic errors and, in the case of multiple sentences, it does so without significant computational cost. We have also found that both the quality of the information extraction output and modeling elements B Dejing Dou dou@cs.uoregon.edu Fernando Gutierrez fernando@cs.uoregon.edu Nisansa de Silva nisansa@cs.uoregon.edu Stephen Fickas fickas@cs.uoregon.edu 1 Department of Computer and Information Science, 1202 University of Oregon, Eugene, OR 97403, USA of the ontology (i.e., property domain and range) affect the capability of detecting errors.

In this thesis, we present a novel approach for modeling textual entailment using lexicalsemantic information on the level of predicate-argument structure. To this end, we adopt information provided by the Berkeley FrameNet repository and embed it into an implemented end-to-end system. The two main goals of this thesis are the following: (i) to provide an analysis of the potential contribution of frame semantic information to the recognition textual entailment and (ii) to present a robust system architecture that can serve as basis for future experiments, research, and improvement. Our work was carried out in the context of the textual entailment initiative, which since 2005 has set the stage for the broad investigation of inference in natural-language processing tasks, including empirical evaluation of its coverage and reliability. In short, textual entailment describes inferential relations between (entailing) texts and (entailed) hypotheses as interpreted by typical language users. This pre-theoretic notion captures a natural range of inferences as compared to logical entailment, which has traditionally been used within theoretical approaches to natural language semantics. Various methods for modeling textual entailment have been proposed in the literature, ranging from shallow techniques like lexical overlap to shallow syntactic parsing and the exploitation of WordNet relations. Recently, there has been a move towards more structured meaning representations. In particular, the level of predicate-argument structure has gained much attention, which seems to be a natural and straightforward choice. Predicate-argument structure allows annotating sentences or texts with nuclear meaning representations (“who did what to whom”), which are of obvious relevance for this task. For example, it can account for paraphrases like “Ghosts scare John” vs. “John is scared by ghosts”. In this thesis, we present an approach to textual entailment that is centered around the analysis of predicate-argument structure. It combines LFG grammatical analysis, predicate-argument structure in the FrameNet paradigm, and taxonomic information from WordNet into tripartite graph structures. By way of a declarative graph matching algorithm, the “structural and semantic” similarity of hypotheses and texts is computed and the result is represented as feature vectors. A supervised machine learning architecture trained on entailment corpora is used to check textual entailment for new text/hypothesis pairs. The approach is implemented in the SALSA RTE system, which successfully participated in the second and third RTE challenge. While system performance is on a par with that of comparable systems, the intuitively expected strong positive effect of using FrameNet information has not yet been confirmed. In order to evaluate different system components and to assess the potential contribution of FrameNet information for checking textual entailment, we conducted a number of experiments. For example, with the help of a gold-standard corpus, we

For many domains, structured knowledge is in short supply, while unstructured text is plentiful. Knowledge Base Population (KBP) is the task of building or extending a knowledge base from text, and systems for KBP have grown in capability and scope. However, existing datasets for KBP are all limited by multiple issues: small in size, not open or accessible, only capable of benchmarking a fraction of the KBP process, or only suitable for extracting knowledge from title-oriented documents (documents that describe a particular entity, such as Wikipedia pages). We introduce and release CC-DBP, a web-scale dataset for training and benchmarking KBP systems. The dataset is based on Common Crawl as the corpus and DBpedia as the target knowledge base. Critically, by releasing the tools to build the dataset, we enable the dataset to remain current as new crawls and DBpedia dumps are released. Also, the modularity of the released tool set resolves a crucial tension between the ease that a dataset can be used for a particular subtask in KBP and the number of different subtasks it can be used to train or benchmark.

To take the first step beyond keyword-based search toward entity-based search, suitable token spans ("spots") on documents must be identified as references to real-world entities from an entity catalog. Several systems have been proposed to link spots on Web pages to entities in Wikipedia. They are largely based on local compatibility between the text around the spot and textual metadata associated with the entity. Two recent systems exploit inter-label dependencies, but in limited ways. We propose a general collective disambiguation approach. Our premise is that coherent documents refer to entities from one or a few related topics or domains. We give formulations for the trade-off between local spot-to-entity compatibility and measures of global coherence between entities. Optimizing the overall entity assignment is NP-hard. We investigate practical solutions based on local hill-climbing, rounding integer linear programs, and pre-clustering entities followed by local optimization within clusters. In experiments involving over a hundred manually-annotated Web pages and tens of thousands of spots, our approaches significantly outperform recently-proposed algorithms.

Actually in the Hospital Information Systems, there is a wide range of clinical information representation from the Electronic Health Records (EHR), and most of the information contained in clinical reports is written in natural language free text. In this context, we are researching the problem of automatic clinical named entities recognition from free text clinical reports. We are using Snomed-CT (Systematized Nomenclature of Medicine – Clinical Terms) as dictionary to identify all kind of clinical concepts, and thus the problem we are considering is to map each clinical entity named in a free text report with its Snomed-CT unique ID. More in general, we are developed a new approach for the named entity recognition (NER) problem in specific domains, and we have applied it to recognize clinical concepts in free text clinical reports. In our approach we apply two types of NER approaches, dictionary-based and machine learning-based. We use a specific domain dictionary-based gazetteer (using Snomed-CT to get the standard clinical code for the clinical concept), and the main approach that we introduce is using a unsupervised shallow learning neural network, word2vec from Mikolov et al., to represent words as vectors, and then making the recognition based on the distance between candidates and dictionary terms. We have applied our approach on a Dataset with 318.585 clinical reports in Spanish from the emergency service of the Hospital “Rafael Méndez” from Lorca (Murcia) Spain, and preliminary results are encouraging. Key-Words: Snomed-CT, word2vec, doc2vec, clinical information extraction, skipgram, medical terminologies, search semantic, named entity recognition, ner, medical entity recognition

In this paper we give an overview of the Knowledge Base Population (KBP) track at the 2010 Text Analysis Conference. The main goal of KBP is to promote research in discovering facts about entities and augmenting a knowledge base (KB) with these facts. This is done through two tasks, Entity Linking – linking names in context to entities in the KB – and Slot Filling – adding information about an entity to the KB. A large source collection of newswire and web documents is provided from which systems are to discover information. Attributes (“slots”) derived from Wikipedia infoboxes are used to create the reference KB. In this paper we provide an overview of the techniques which can serve as a basis for a good KBP system, lay out the remaining challenges by comparison with traditional Information Extraction (IE) and Question Answering (QA) tasks, and provide some suggestions to address these challenges.

We present an overview of TARSQI, a modular system for automatic temporal annotation that adds time expressions, events and temporal relations to news texts.


Question: A method for extracting triples from unstructured text using a parser.

Introducing ClausIE: A Clause-Based Open Information Extraction Framework

This paper presents ClausIE, a pioneering clause-based method for open information extraction, aimed at identifying relationships and their arguments within natural language text. Unlike previous approaches, ClausIE innovatively distinguishes between the identification of "useful" information within a sentence and its representation as extractions. Specifically, ClausIE leverages an understanding of English grammar to first isolate clauses from the input sentence and then categorize each clause based on the grammatical role of its components. This nuanced approach enables ClausIE to generate extractions with high precision, with the added flexibility of tailoring the extraction format to suit the specific application. Relying on dependency parsing and a limited set of domain-independent lexicons, ClausIE processes sentences individually without the need for post-processing or training data, be it labeled or unlabeled. Comparative analyses across various real-world datasets have revealed that ClausIE outperforms existing methods in both recall and precision, demonstrating its effectiveness on both high-quality and noisy web-based text.

We present a question answering system over DBpedia, filling the gap between user information needs expressed in natural language and a structured query interface expressed in SPARQL over the underlying knowledge base (KB). Given the KB, our goal is to comprehend a natural language query and provide corresponding accurate answers. Focusing on solving the non-aggregation questions, in this paper, we construct a subgraph of the knowledge base from the detected entities and propose a graph traversal method to solve both the semantic item mapping problem and the disambiguation problem in a joint way. Compared with existing work, we simplify the process of query intention understanding and pay more attention to the answer path ranking. We evaluate our method on a non-aggregation question dataset and further on a complete dataset. Experimental results show that our method achieves best performance compared with several state-of-the-art systems.

We present a parser for Abstract Meaning Representation (AMR). We treat Englishto-AMR conversion within the framework of string-to-tree, syntax-based machine translation (SBMT). To make this work, we transform the AMR structure into a form suitable for the mechanics of SBMT and useful for modeling. We introduce an AMR-specific language model and add data and features drawn from semantic resources. Our resulting AMR parser improves upon state-of-the-art results by 7 Smatch points.

As part of the information fusion task we wish to automatically fuse information derived from the text extraction process with data from a structured knowledge base. This process will involve resolving, aggregating, integrating and abstracting information via the methodologies of Knowledge Representation and Reasoning into a single comprehensive description of an individual or event. This report surveys the key principles underlying research in the field of Knowledge Representation and Reasoning. It represents an initial step in deciding upon a Knowledge Representation and Reasoning system for our information fusion task. APPROVED FOR PUBLIC RELEASE

Populating large-scale structured databases from unstructured sources is a critical and challenging task in data analytics. As automated feature engineering methods grow increasingly prevalent, constructing sufficiently large labeled training sets has become the primary hurdle in building machine learning information extraction systems. In light of this, we have taken a new approach called data programming [7]. Rather than hand-labeling data, in the data programming paradigm, users generate large amounts of noisy training labels by programmatically encoding domain heuristics as simple rules. Using this approach over more traditional distant supervision methods and fully supervised approaches using labeled data, we have been able to construct knowledge base systems more rapidly and with higher quality. Since the ability to quickly prototype, evaluate, and debug these rules is a key component of this paradigm, we introduce DDLite, an interactive development framework for data programming. This paper reports feedback collected from DDLite users across a diverse set of entity extraction tasks. We share observations from several DDLite hackathons in which 10 biomedical researchers prototyped information extraction pipelines for chemicals, diseases, and anatomical named entities. Initial results were promising, with the disease tagging team obtaining an F1 score within 10 points of the state-of-the-art in only a single day-long hackathon's work. Our key insights concern the challenges of writing diverse rule sets for generating labels, and exploring training data. These findings motivate several areas of active data programming research.

A wealth of knowledge is encoded in the form of tables on the World Wide Web. We propose a classification algorithm and a rich feature set for automatically recognizing layout tables and attribute/value tables. We report the frequencies of these table types over a large analysis of the Web and propose open challenges for extracting from attribute/value tables semantic triples (knowledge). We then describe a solution to a key problem in extracting semantic triples: protagonist detection, i.e., finding the subject of the table that often is not present in the table itself. In 79% of our Web tables, our method finds the correct protagonist in its top three returned candidates.

Knowledge graphs (KG) and open information extraction systems (OIE) both represent information by a set of (subject, relation, object)-triples. In KGs, arguments and relations are taken from a closed vocabulary, i.e., entities are disambiguated and relations are assumed to be semantically distinct. In OIE, arguments and relations are extracted from natural language text. There is no predefined or closed vocabulary, many different mentions may refer to the same entity, and relations can be semantically close but not equal, e.g. “visited by” and “frequently visited by”. The set of OIE triples can be viewed as an open knowledge graph (OKG). In this paper, we study the problem of open link prediction: Given a mention of an entity and an open relation (e.g., “A. Einstein” and “is a native of”), predict correct entity mentions (e.g., “Imperial Germany”) using solely the OKG. Models that aim to solve the open link prediction task sucessfully need to be able to reason about different entity mentions, as well as the relations between them, in a fully automatic, unsupervised way. As a benchmark for this task we design an OKG dataset, an evaluation protocol and present and evaluate a number of baseline models. In particular, we compare a token-based co-occurrence model, a non token-based link prediction model, and several token-based link prediction models. Our results indicate that the latter combination is beneficial.

While Named Entity extraction is useful in many natural language applications, the coarse categories that most NE extractors work with prove insufficient for complex applications such as Question Answering and Ontology generation. We examine one coarse category of named entities, persons, and describe a method for automatically classifying person instances into eight finergrained subcategories. We present a supervised learning method that considers the local context surrounding the entity as well as more global semantic information derived from topic signatures and WordNet. We reinforce this method with an algorithm that takes advantage of the presence of entities in multiple contexts.

We describe a neural network model that jointly learns distributed representations of texts and knowledge base (KB) entities. Given a text in the KB, we train our proposed model to predict entities that are relevant to the text. Our model is designed to be generic with the ability to address various NLP tasks with ease. We train the model using a large corpus of texts and their entity annotations extracted from Wikipedia. We evaluated the model on three important NLP tasks (i.e., sentence textual similarity, entity linking, and factoid question answering) involving both unsupervised and supervised settings. As a result, we achieved state-of-the-art results on all three of these tasks. Our code and trained models are publicly available for further academic research.

We introduce a neural semantic parser which is interpretable and scalable. Our model converts natural language utterances to intermediate, domain-general natural language representations in the form of predicate-argument structures, which are induced with a transition system and subsequently mapped to target domains. The semantic parser is trained end-to-end using annotated logical forms or their denotations. We achieve the state of the art on SPADES and GRAPHQUESTIONS and obtain competitive results on GEOQUERY and WEBQUESTIONS. The induced predicate-argument structures shed light on the types of representations useful for semantic parsing and how these are different from linguistically motivated ones.1

The inability of reliable text extraction from arbitrary documents is often an obstacle for large scale NLP based on resources crawled from the Web. One of the largest problems in the conversion of PDF documents is the detection of the boundaries of common textual units such as paragraphs, sentences and words. PDF is a file format optimized for printing and encapsulates a complete description of the layout of a document including text, fonts, graphics and so on. This paper describes a tool for extracting texts from arbitrary PDF files for the support of largescale data-driven natural language processing. Our approach combines the benefits of several existing solutions for the conversion of PDF documents to plain text and adds a language-independent post-processing procedure that cleans the output for further linguistic processing. In particular, we use the PDF-rendering libraries pdfXtk, Apache Tika and Poppler in various configurations. From the output of these tools we recover proper boundaries using on-the-fly language models and languageindependent extraction heuristics. In our research, we looked especially at publications from the European Union, which constitute a valuable multilingual resource, for example, for training statistical machine translation models. We use our tool for the conversion of a large multilingual database crawled from the EU bookshop with the aim of building parallel corpora. Our experiments show that our conversion software is capable of fixing various common issues leading to cleaner data sets in the end.


Question: A method for parsing and extracting triples from unstructured text documents.

In the context of automata theory, a fundamental concept is the Pumping Lemma, which aids in demonstrating that certain languages are not regular. Consider the language L defined over the alphabet {0, 1}, and let's assume for the sake of contradiction that L is regular. Let n be the constant provided by the Pumping Lemma, distinct from any other variable n used in the definition of L. Select w = 0^n10^n, a string that falls within L. When we decompose w into three segments, x, y, and z (w = xyz), we observe that the length of xy is bounded by n, implying that y contains solely 0's. Consequently, the string xz, which must be a member of L if L is regular, exhibits a discrepancy in its structure. It comprises less than n 0's followed by a single 1 and exactly n 0's, a configuration not found in L. This contradiction disproves the initial assumption, thereby confirming that L is not a regular language. This technique showcases the utility of the Pumping Lemma in the analysis of language regularity within automata theory.

Research on information extraction (IE) seeks to distill relational tuples from natural language text, such as the contents of the WWW. Most IE work has focussed on identifying static facts, encoding them as binary relations. This is unfortunate, because the vast majority of facts are fluents, only holding true during an interval of time. It is less helpful to extract PresidentOf(Bill-Clinton, USA) without the temporal scope 1/20/93 1/20/01. This paper presents TIE, a novel, information-extraction system, which distills facts from text while inducing as much temporal information as possible. In addition to recognizing temporal relations between times and events, TIE performs global inference, enforcing transitivity to bound the start and ending times for each event. We introduce the notion of temporal entropy as a way to evaluate the performance of temporal IE systems and present experiments showing that TIE outperforms three alternative approaches.

We aim to automatically generate natural language narratives about an input structured knowledge base (KB). We build our generation framework based on a pointer network which can copy facts from the input KB, and add two attention mechanisms: (i) slot-aware attention to capture the association between a slot type and its corresponding slot value; and (ii) a new table position self-attention to capture the inter-dependencies among related slots. For evaluation, besides standard metrics including BLEU, METEOR, and ROUGE, we also propose a KB reconstruction based metric by extracting a KB from the generation output and comparing it with the input KB. We also create a new data set which includes 106,216 pairs of structured KBs and their corresponding natural language descriptions for two distinct entity types. Experiments show that our approach significantly outperforms state-of-the-art methods. The reconstructed KB achieves 68.8% 72.6% F-score.1

Natural Language Interface to Database (NLIDB) systems convert a Natural Language (NL) query into a Structured Query Language (SQL) and then use the SQL query to retrieve information from a database. The main advantage with this function of an NLIDB system is that it makes information retrieval much easier and more importantly, it also allows non-technical users to query a database. In this work, we present an effective usage of tokens in an NL query to address various problems in an NLIDB system. The conversion of an NL query to SQL query is framed as a token classification problem, using which we unveil a novel method of mapping an NL query to an SQL query. Our approach reasonably addresses domain dependency which is one of the major drawbacks of NLIDB systems. Concepts Identification is a major component of NLIDB systems (Gupta et al., 2012; Srirampur et al., 2014). We improve Concepts Identification (Srirampur et al., 2014) by making use of Stanford Parser Dependencies. Our approach is more robust than previously proposed methods. In addition, we also discuss how Concepts Identification can be applied to address Ellipsis in a dialogue process. In addition to providing results to a user, it is essential to provide a relevant and a compact set of results. We propose a new problem of generating a compact set of results to a user query. At a higher level, user-system interactions are modeled based on patterns frequently observed between a user’s current query and his previous queries, while interacting with a system. Using these models, we propose a novel method of system prompting to help a user obtain a smaller and a relevant set of results. In addition to providing a compact and a relevant set of results, it is imperative that answers of an NLIDB system are comprehensible even by the people who are less familiar with a common language like English. NLIDB systems use Natural Language Generation (NLG) modules to provide answers in the form of sentences. It is important to make sure that an answer generated by an NLG module is very simple to understand. This brings in the problem of text simplification, wherein, one of the most crucial and initial sub-problems is Complex Word Identification (CWI). We address the problem of CWI by distinguishing words as simple and complex. A plethora of classifiers were explored to identify the complex words. This information helps us in improving the simplification of an NLIDB system’s final output to a user. To summarize, in addition to addressing problems within an NLIDB system, this work touches postNLIDB problems like results processing. All the proposed issues are tackled using tokens in an NL query as the basic and a driving unit of force.

We present a question answering system over DBpedia, filling the gap between user information needs expressed in natural language and a structured query interface expressed in SPARQL over the underlying knowledge base (KB). Given the KB, our goal is to comprehend a natural language query and provide corresponding accurate answers. Focusing on solving the non-aggregation questions, in this paper, we construct a subgraph of the knowledge base from the detected entities and propose a graph traversal method to solve both the semantic item mapping problem and the disambiguation problem in a joint way. Compared with existing work, we simplify the process of query intention understanding and pay more attention to the answer path ranking. We evaluate our method on a non-aggregation question dataset and further on a complete dataset. Experimental results show that our method achieves best performance compared with several state-of-the-art systems.

Existing knowledge-based question answering systems often rely on small annotated training data. While shallow methods like relation extraction are robust to data scarcity, they are less expressive than the deep meaning representation methods like semantic parsing, thereby failing at answering questions involving multiple constraints. Here we alleviate this problem by empowering a relation extraction method with additional evidence from Wikipedia. We first present a neural network based relation extractor to retrieve the candidate answers from Freebase, and then infer over Wikipedia to validate these answers. Experiments on the WebQuestions question answering dataset show that our method achieves an F1 of 53.3%, a substantial improvement over the state-of-the-art.

The success of Semantic Web will heavily rely on the availability of formal ontologies to structure machine understanding data. However, there is still a lack of general methodologies for ontology automatic learning and population, i.e. the generation of domain ontologies from various kinds of resources by applying natural language processing and machine learning techniques In this paper, the authors present an ontology learning and population system that combines both statistical and semantic methodologies. Several experiments have been carried out, demonstrating the effectiveness of the proposed system. Keywords-Ontologies, Ontology Learning, Ontology Population, Latent Dirichlet Allocation

The inability of reliable text extraction from arbitrary documents is often an obstacle for large scale NLP based on resources crawled from the Web. One of the largest problems in the conversion of PDF documents is the detection of the boundaries of common textual units such as paragraphs, sentences and words. PDF is a file format optimized for printing and encapsulates a complete description of the layout of a document including text, fonts, graphics and so on. This paper describes a tool for extracting texts from arbitrary PDF files for the support of largescale data-driven natural language processing. Our approach combines the benefits of several existing solutions for the conversion of PDF documents to plain text and adds a language-independent post-processing procedure that cleans the output for further linguistic processing. In particular, we use the PDF-rendering libraries pdfXtk, Apache Tika and Poppler in various configurations. From the output of these tools we recover proper boundaries using on-the-fly language models and languageindependent extraction heuristics. In our research, we looked especially at publications from the European Union, which constitute a valuable multilingual resource, for example, for training statistical machine translation models. We use our tool for the conversion of a large multilingual database crawled from the EU bookshop with the aim of building parallel corpora. Our experiments show that our conversion software is capable of fixing various common issues leading to cleaner data sets in the end.

Text document clustering plays an important role in providing intuitive navigation and browsing mechanisms by organizing large sets of documents into a small number of meaningful clusters. The bag of words representation used for these clustering methods is often unsatisfactory as it ignores relationships between important terms that do not cooccur literally. In order to deal with the problem, we integrate core ontologies as background knowledge into the process of clustering text documents. Our experimental evaluations compare clustering techniques based on precategorizations of texts from Reuters newsfeeds and on a smaller domain of an eLearning course about Java. In the experiments, improvements of results by background knowledge compared to a baseline without background knowledge can be shown in many interesting combinations.

In this paper, we present methods for automatically finding abstract, legally relevant concepts in case texts and demonstrate how they can be used to make predictions of case outcomes, given the texts as inputs.In a set of experiments to test these methods, we focus on the open question of how best to represent legal text for finding abstract concepts. We compare different ways of representing legal case texts in order to test whether adding domain knowledge and some linguistic information can improve performance.We found that replacing individual names by roles in the case texts led to better indexing, and that adding certain syntactic and semantic information, in the form of Propositional Patterns that capture a sense of "who did what", led to better prediction. Our experiments also showed that of three learning algorithms, Nearest Neighbor worked best in learning how to identify indexing concepts in texts.In these experiments, we introduced a prototype system that can reason with text cases; it analyzes a case, predicts its outcome considering other cases in the database, and explains the prediction, all starting with a textual description of the case's facts as input.

In this paper we give an overview of the Knowledge Base Population (KBP) track at TAC 2010. The main goal of KBP is to promote research in discovering facts about entities and expanding a structured knowledge base with this information. A large source collection of newswire and web documents is provided for systems to discover information. Attributes (a.k.a. “slots”) derived from Wikipedia infoboxes are used to create the reference knowledge base (KB). KBP2010 includes the following four tasks: (1) Regular Entity Linking, where names must be aligned to entities in the KB; (2) Optional Entity linking, without using Wikipedia texts; (3) Regular Slot Filling, which requires a system to automatically discover the attributes of specified entities from the source document collection and use them to expand the KB; (4) Surprise Slot Filling, which requires a system to return answers regarding new slot types within a short time period. KBP2010 has attracted many participants (over 45 teams registered for KBP 2010 (not including the RTEKBP Validation Pilot task), among which 23 teams submitted results). In this paper we provide an overview of the task definition and annotation challenges associated with KBP2010. Then we summarize the evaluation results and discuss the lessons that we have learned based on detailed analysis.