Question: Sentiment analysis of financial microblogs for bearish and bullish market trends.

This paper outlines an innovative method for the automatic creation, assessment, and deployment of BabelNet, a comprehensive multilingual semantic network. The cornerstone of our approach is the amalgamation of lexicographic and encyclopedic data from WordNet and Wikipedia. Furthermore, we utilize Machine Translation techniques to augment the lexical content for a broad spectrum of languages. To validate the accuracy and breadth of BabelNet, we conduct in vitro trials using both new and established gold-standard datasets. Our findings reveal that BabelNet exhibits high-quality standards and extensive coverage. We also demonstrate that our lexical resource is capable of effectively performing monolingual and cross-lingual Word Sense Disambiguation. Thanks to its extensive lexical range and unique semantic connections, BabelNet has achieved cutting-edge outcomes in three distinct SemEval evaluation tasks, positioning it at the forefront of semantic network research and application.

**Question**: The Role of Social Media in Shaping Public Opinion

**Documents**: Social media and the decision to participate in political protests: Evidence from Tunisia
The aim of this paper is to assess the role of social media in the decision to participate in a protest in Tunisia. We exploit a unique dataset that combines survey data on individual protest participation with data on the number of Internet users and the number of Facebook users at the municipal level. We use a difference-in-differences (DiD) strategy to compare the change in the probability of participating in a protest between municipalities with high and low levels of Internet and Facebook use before the start of the Arab Spring. We find that the use of social media has a positive and significant effect on the probability of participating in a protest. Specifically, we estimate that a one standard deviation increase in the number of Facebook users increases the probability of participating in a protest by 3.5 percentage points. We also find that the effect of social media is stronger for women and for individuals with higher levels of education. Our results suggest that social media can play an important role in mobilizing people to participate in political protests.

**Similar Documents**: This research endeavors to evaluate the impact of social media on the inclination to engage in political demonstrations, using Tunisia as a case study. Our investigation utilizes a unique dataset that merges individual-level protest participation surveys with municipal-level statistics on Internet and Facebook usage. Employing a difference-in-differences (DiD) methodology, we contrast the shifts in protest participation likelihood between areas with high and low Internet and Facebook penetration rates prior to the Arab Spring's inception. Our findings reveal a positive and substantial association between social media

The popularity of social media in recent years has increased a lot and created new opportunities to study the interactions of different groups of people. Because of this it comes to possible to study the user‟s opinion. Two popular topics in the study of social networks are community detection and finding trends. Trend Analysis is an ongoing field of research in Data mining field, May referred as an “Opinion Mining” by some researchers. Trend Analysis is the computational treatment of opinions, sentiments and subjectivity of text. The related field to Trend Analysis contains finding emotions in a statement which attracted researchers recently are discussed. The main target of this survey is to give nearly full image of Trend Analysis along with the community detection and their related fields with brief details. The main contributions of this paper include the overall analysis to bridge the concept between trend analyses with community detection. To find out the anomaly detection in a community with using the concept of trend analysis is a difficult task.

In this paper, we address the tasks of recognition and interpretation of affect communicated through text messaging in online communication environments. Specifically, we focus on Instant Messaging (IM) or blogs, where people use an informal or garbled style of writing. We introduced a novel rule-based linguistic approach for affect recognition from text. Our Affect Analysis Model (AAM) was designed to deal with not only grammatically and syntactically correct textual input, but also informal messages written in an abbreviated or expressive manner. The proposed rule-based approach processes each sentence in stages, including symbolic cue processing, detection and transformation of abbreviations, sentence parsing and word/phrase/sentence-level analyses. Our method is capable of processing sentences of different complexity, including simple, compound, complex (with complement and relative clauses) and complex–compound sentences. Affect in text is classified into nine emotion categories (or neutral). The strength of the resulting emotional state depends on vectors of emotional words, relations among them, tense of the analysed sentence and availability of first person pronouns. The evaluation of the Affect Analysis Model algorithm showed promising results regarding its capability to accurately recognize fine-grained emotions reflected in sentences from diary-like blog posts (averaged accuracy is up to 77 per cent), fairy tales (averaged accuracy is up to 70.2 per cent) and news headlines (our algorithm outperformed eight other systems on several measures).

Cantonese is an important dialect in some regions of Southern China. Local online users often represent their opinions and experiences on the web with written Cantonese. Although the information in those reviews is valuable to potential consumers and sellers, the huge amount of web reviews make it difficult to give an unbiased evaluation to a product and the Cantonese reviews are unintelligible for Mandarin Chinese speakers. In this paper, standard machine learning techniques naive Bayes and SVM are incorporated into the domain of online Cantonese-written restaurant reviews to automatically classify user reviews as positive or negative. The effects of feature presentations and feature sizes on classification performance are discussed. We find that accuracy is influenced by interaction between the classification models and the feature options. The naive Bayes classifier achieves as well as or better accuracy than SVM. Character-based bigrams are proved better features than unigrams and trigrams in capturing Cantonese sentiment orientation. 2010 Elsevier Ltd. All rights reserved.

We present a computational framework for understanding the social aspects of emotions in Twitter conversations. Using unannotated data and semisupervised machine learning, we look at emotional transitions, emotional influences among the conversation partners, and patterns in the overall emotional exchanges. We find that conversational partners usually express the same emotion, which we name Emotion accommodation, but when they do not, one of the conversational partners tends to respond with a positive emotion. We also show that tweets containing sympathy, apology, and complaint are significant emotion influencers. We verify the emotion classification part of our framework by a human-annotated corpus.

Twitter and other microblogging services have become indispensable sources of information in today's web. Understanding the main factors that make certain pieces of information spread quickly in these platforms can be decisive for the analysis of opinion formation and many other opinion mining tasks.
 This paper addresses important questions concerning the spread of information on Twitter. What makes Twitter users retweet a tweet? Is it possible to predict whether a tweet will become "viral", i.e., will be frequently retweeted? To answer these questions we provide an extensive analysis of a wide range of tweet and user features regarding their influence on the spread of tweets. The most impactful features are chosen to build a learning model that predicts viral tweets with high accuracy. All experiments are performed on a real-world dataset, extracted through a public Twitter API based on user IDs from the TREC 2011 microblog corpus.

BACKGROUND
Mental illness is quickly becoming one of the most prevalent public health problems worldwide. Social network platforms, where users can express their emotions, feelings, and thoughts, are a valuable source of data for researching mental health, and techniques based on machine learning are increasingly used for this purpose.

OBJECTIVE
The objective of this review was to explore the scope and limits of cutting-edge techniques that researchers are using for predictive analytics in mental health and to review associated issues, such as ethical concerns, in this area of research.

METHODS
We performed a systematic literature review in March 2017, using keywords to search articles on data mining of social network data in the context of common mental health disorders, published between 2010 and March 8, 2017 in medical and computer science journals.

RESULTS
The initial search returned a total of 5386 articles. Following a careful analysis of the titles, abstracts, and main texts, we selected 48 articles for review. We coded the articles according to key characteristics, techniques used for data collection, data preprocessing, feature extraction, feature selection, model construction, and model verification. The most common analytical method was text analysis, with several studies using different flavors of image analysis and social interaction graph analysis.

CONCLUSIONS
Despite an increasing number of studies investigating mental health issues using social network data, some common problems persist. Assembling large, high-quality datasets of social media users with mental disorder is problematic, not only due to biases associated with the collection methods, but also with regard to managing consent and selecting appropriate analytics techniques.

Sentiment analysis has been widely researched in the domain of online review sites with the aim of generating summarized opinions of product users about different aspects of the products. However, there has been little work focusing on identifying the polarity of sentiments expressed by users during disaster events. Identifying sentiments expressed by users in an online social networking site can help understand the dynamics of the network, e.g., the main users’ concerns, panics, and the emotional impacts of interactions among members. Data produced through social networking sites is seen as ubiquitous, rapid and accessible, and it is believed to empower average citizens to become more situationally aware during disasters and coordinate to help themselves. In this work, we perform sentiment classification of user posts in Twitter during the Hurricane Sandy and visualize these sentiments on a geographical map centered around the hurricane. We show how users' sentiments change according not only to users’ locations, but also based on the distance from the disaster.

Opinion mining is an interested area of research, which epitomize the customer reviews of a product or service and express whether the opinions are positive or negative. Various methods have been proposed as classifiers for opinion mining such as Naïve Bayesian, and Support vector machine, these methods classify opinion without giving us the reasons about why the instance opinion is classified to certain class. Therefore, in our work, we investigate opinion mining of Arabic text at the document level, by applying decision trees classification classifier to have clear, understandable rule, also we apply parallel decision trees classifiers to have efficient results. We applied parallel decision trees on two Arabic corpus of text documents by using parallel implementation of RapidMiner tools. In case of applying parallel decision tree family on OCA we get the best results of accuracy (93.83%), f-measure (93.22) and consumed time 42 Sec at thread 4, one of the resulted rule is Urdu language lines. In case of applying parallel decision tree family on BHA we get the best results of accuracy (90.63%), f-measure (82.29) and consumed time 219 Sec at thread 4, one of the resulted rule is Urdu language lines.

Social media now plays a pivotal role in electoral campaigns. Rapid dissemination of information through platforms such as Twitter has enabled politicians to broadcast their message to a wide audience. In this paper, we investigated the sentiment of tweets by the two main presidential candidates, Hillary Clinton and Donald Trump, along with almost 2.9 million tweets by Twitter users during the 2016 US Presidential Elections. We analyzed these short texts to evaluate how accurately Twitter represented the public opinion and real world events of significance related with the elections. We also analyzed the behavior of over a million distinct Twitter users to identify whether the platform was used to share original opinions and to interact with other users or whether few opinions were repeated over and over again with little inter-user dialogue. Finally, we wanted to assess the sentiment of tweets by both candidates and their impact on the election related discourse on Twitter. Some of our findings included the discovery that little original content was created by users and Twitter was primarily used for rebroadcasting already present opinions in the form of retweets with little communication between users. Also of significance was the finding that sentiment and topics expressed on Twitter can be a good proxy of public opinion and important election related events. Moreover, we found that Donald Trump offered a more optimistic and positive campaign message than Hillary Clinton and enjoyed better sentiment when mentioned in messages by Twitter users.

BACKGROUND
Early identification and intervention are imperative for suicide prevention. However, at-risk people often neither seek help nor take professional assessment. A tool to automatically assess their risk levels in natural settings can increase the opportunity for early intervention.

OBJECTIVE
The aim of this study was to explore whether computerized language analysis methods can be utilized to assess one's suicide risk and emotional distress in Chinese social media.

METHODS
A Web-based survey of Chinese social media (ie, Weibo) users was conducted to measure their suicide risk factors including suicide probability, Weibo suicide communication (WSC), depression, anxiety, and stress levels. Participants' Weibo posts published in the public domain were also downloaded with their consent. The Weibo posts were parsed and fitted into Simplified Chinese-Linguistic Inquiry and Word Count (SC-LIWC) categories. The associations between SC-LIWC features and the 5 suicide risk factors were examined by logistic regression. Furthermore, the support vector machine (SVM) model was applied based on the language features to automatically classify whether a Weibo user exhibited any of the 5 risk factors.

RESULTS
A total of 974 Weibo users participated in the survey. Those with high suicide probability were marked by a higher usage of pronoun (odds ratio, OR=1.18, P=.001), prepend words (OR=1.49, P=.02), multifunction words (OR=1.12, P=.04), a lower usage of verb (OR=0.78, P<.001), and a greater total word count (OR=1.007, P=.008). Second-person plural was positively associated with severe depression (OR=8.36, P=.01) and stress (OR=11, P=.005), whereas work-related words were negatively associated with WSC (OR=0.71, P=.008), severe depression (OR=0.56, P=.005), and anxiety (OR=0.77, P=.02). Inconsistently, third-person plural was found to be negatively associated with WSC (OR=0.02, P=.047) but positively with severe stress (OR=41.3, P=.04). Achievement-related words were positively associated with depression (OR=1.68, P=.003), whereas health- (OR=2.36, P=.004) and death-related (OR=2.60, P=.01) words positively associated with stress. The machine classifiers did not achieve satisfying performance in the full sample set but could classify high suicide probability (area under the curve, AUC=0.61, P=.04) and severe anxiety (AUC=0.75, P<.001) among those who have exhibited WSC.

CONCLUSIONS
SC-LIWC is useful to examine language markers of suicide risk and emotional distress in Chinese social media and can identify characteristics different from previous findings in the English literature. Some findings are leading to new hypotheses for future verification. Machine classifiers based on SC-LIWC features are promising but still require further optimization for application in real life.


Question: Sentiment analysis of financial microblogs for bearish and bullish trends.

This research introduces an innovative unsupervised learning technique aimed at categorizing reviews into positive (thumbs up) or negative (thumbs down) sentiments. The classification process hinges on calculating the average semantic orientation of phrases within each review that include adjectives or adverbs. A phrase exhibits a positive semantic orientation when associated with favorable terms (for instance, "subtle nuances"), whereas a negative orientation is indicated by adverse associations (such as "very cavalier"). The semantic orientation of a specific phrase is quantified by the mutual information linking the phrase to the term "excellent," lessened by the mutual information relating the phrase to "poor." A review is designated as positive if the mean semantic orientation of its constituent phrases is above zero. When tested on a diverse dataset of 410 reviews from Epinions, encompassing four distinct categories (automobiles, banks, movies, and travel destinations), the algorithm yields an average accuracy of 74%. Performance varies across categories, with the highest accuracy of 84% observed for automobile reviews and the lowest at 66% for movie reviews. This method offers a straightforward yet effective approach to gauging sentiment in unsupervised review classification.

In the past years we have witnessed Sentiment Analysis and Opinion Mining becoming increasingly popular topics in Information Retrieval and Web data analysis. With the rapid growth of the user-generated content represented in blogs, wikis and Web forums, such an analysis became a useful tool for mining the Web, since it allowed us to capture sentiments and opinions at a large scale. Opinion retrieval has established itself as an important part of search engines. Ratings, opinion trends and representative opinions enrich the search experience of users when combined with traditional document retrieval, by revealing more insights about a subject. Opinion aggregation over product reviews can be very useful for product marketing and positioning, exposing the customers’ attitude towards a product and its features along different dimensions, such as time, geographical location, and experience. Tracking how opinions or discussions evolve over time can help us identify interesting trends and patterns and better understand the ways that information is propagated in the Internet. In this study, we review the development of Sentiment Analysis and Opinion Mining during the last years, and also discuss the evolution of a relatively new research direction, namely, Contradiction Analysis. We give an overview of the proposed methods and recent advances in these areas, and we try to layout the future research directions in the field.

User generated content on the microblogging social network Twitter continues to grow with significant amount of information. The semantic analysis offers the opportunity to discover and model latent interests’ in the users’ publications. This article focuses on the problem of uncertainty in the users’ publications that has not been previously treated. It proposes a new approach for users’ interest discovery from uncertain information that augments traditional methods using possibilistic logic. The possibility theory provides a solid theoretical base for the treatment of incomplete and imprecise information and inferring the reliable expressions from a knowledge base. More precisely, this approach used the product-based possibilistic network to model knowledge base and discovering possibilistic interests. DBpedia ontology is integrated into the interests’ discovery process for selecting the significant topics. The empirical analysis and the comparison with the most known methods proves the significance of this approach.

While much work has recently focused on the analysis of social media in order to get a feel for what people think about current topics of interest, there are, however, still many challenges to be faced. Text mining systems originally designed for more regular kinds of texts such as news articles may need to be adapted to deal with facebook posts, tweets etc. In this paper, we discuss a variety of issues related to opinion mining from social media, and the challenges they impose on a Natural Language Processing (NLP) system, along with two example applications we have developed in very different domains. In contrast with the majority of opinion mining work which uses machine learning techniques, we have developed a modular rule-based approach which performs shallow linguistic analysis and builds on a number of linguistic subcomponents to generate the final opinion polarity and score.

Automatic sentiment analysis provides an effective way to gauge public opinion on any topic of interest. However, most sentiment analysis tools require a general sentiment lexicon to automatically classify sentiments or opinion in a text. One of the challenges presented by using a general sentiment lexicon is that it is insensitive to the domain since the scores assigned to the words are fixed. As a result, while one general sentiment lexicon might perform well in one domain, the same lexicon might perform poorly in another domain. Most sentiment lexica will need to be adjusted to suit the specific domain to which it is applied. In this paper, we present results of sentiment analysis expressed on Twitter by UK energy consumers. We optimised the accuracy of the sentiment analysis results by combining functions from two sentiment lexica. We used the first lexicon to extract the sentiment-bearing terms and negative sentiments since it performed well in detecting these. We then used a second lexicon to classify the rest of the data. Experimental results show that this method improved the accuracy of the results compared to the common practice of using only one lexicon.

Political views frequently conflict in the coverage of contentious political issues, potentially causing serious social problems. We present a novel social annotation analysis approach for identification of news articles' political orientation. The approach focuses on the behavior of individual commenters. It uncovers commenters' sentiment patterns towards political news articles, and predicts the political orientation from the sentiments expressed in the comments. It takes advantage of commenters' participation as well as their knowledge and intelligence condensed in the sentiment of comments, thereby greatly reduces the high complexity of political view identification. We conduct extensive study on commenters' behaviors, and discover predictive commenters showing a high degree of regularity in their sentiment patterns. We develop and evaluate sentiment pattern-based methods for political view identification.

Probabilistic topic models have been widely used for sentiment analysis. However, most of existing topic methods only model the sentiment text, but do not consider the user, who expresses the sentiment, and the item, which the sentiment is expressed on. Since different users may use different sentiment expressions for different items, we argue that it is better to incorporate the user and item information into the topic model for sentiment analysis. In this paper, we propose a new Supervised User-Item based Topic model, called SUIT model, for sentiment analysis. It can simultaneously utilize the textual topic and latent user-item factors. Our proposed method uses the tensor outer product of text topic proportion vector, user latent factor and item latent factor to model the sentiment label generalization. Extensive experiments are conducted on two datasets: review dataset and microblog dataset. The results demonstrate the advantages of our model. It shows significant improvement compared with supervised topic models and collaborative filtering methods.

Online social and news media generate rich and timely information about real-world events of all kinds. However, the huge amount of data available, along with the breadth of the user base, requires a substantial effort of information filtering to successfully drill down to relevant topics and events. Trending topic detection is therefore a fundamental building block to monitor and summarize information originating from social sources. There are a wide variety of methods and variables and they greatly affect the quality of results. We compare six topic detection methods on three Twitter datasets related to major events, which differ in their time scale and topic churn rate. We observe how the nature of the event considered, the volume of activity over time, the sampling procedure and the pre-processing of the data all greatly affect the quality of detected topics, which also depends on the type of detection method used. We find that standard natural language processing techniques can perform well for social streams on very focused topics, but novel techniques designed to mine the temporal distribution of concepts are needed to handle more heterogeneous streams containing multiple stories evolving in parallel. One of the novel topic detection methods we propose, based on -grams cooccurrence and topic ranking, consistently achieves the best performance across all these conditions, thus being more reliable than other state-of-the-art techniques.

Sentiment Analysis of tweets is a complex task, because these short messages employ unconventional language to increase the expressiveness. This task becomes even more difficult when people use figurative language (e.g. irony, sarcasm and metaphors) because it causes a mismatch between the literal meaning and the actual expressed sentiment. In this paper, we describe a sentiment analysis system designed for handling ironic and sarcastic tweets. Features grounded on several linguistic levels are proposed and used to classify the tweets in a 11-scale range, using a decision tree. The system is evaluated on the dataset released by the organizers of the SemEval 2015, task 11. The results show that our method largely outperforms the systems proposed by the participants of the task on ironic and sarcastic tweets.

Emoticons (e.g., :) and :( ) have been widely used in sentiment analysis and other NLP tasks as features to machine learning algorithms or as entries of sentiment lexicons. In this paper, we argue that while emoticons are strong and common signals of sentiment expression on social media the relationship between emoticons and sentiment polarity are not always clear. Thus, any algorithm that deals with sentiment polarity should take emoticons into account but extreme caution should be exercised in which emoticons to depend on. First, to demonstrate the prevalence of emoticons on social media, we analyzed the frequency of emoticons in a large recent Twitter data set. Then we carried out four analyses to examine the relationship between emoticons and sentiment polarity as well as the contexts in which emoticons are used. The first analysis surveyed a group of participants for their perceived sentiment polarity of the most frequent emoticons. The second analysis examined clustering of words and emoticons to better understand the meaning conveyed by the emoticons. The third analysis compared the sentiment polarity of microblog posts before and after emoticons were removed from the text. The last analysis tested the hypothesis that removing emoticons from text hurts sentiment classification by training two models with and without emoticons in the text, respectively. The results confirms the arguments that: 1) a few emoticons are strong and reliable signals of sentiment polarity and one should take advantage of them in any sentiment analysis; 2) a large group of the emoticons conveys complicated sentiment hence they should be treated with extreme caution.

Suicide is one of major public health problems worldwide. Traditionally, suicidal ideation is assessed by surveys or interviews, which lacks of a real-time assessment of personal mental state. Online social networks, with large amount of user-generated data, offer opportunities to gain insights of suicide assessment and prevention. In this paper, we explore potentiality to identify and monitor suicide expressed in microblog on social networks. First, we identify users who have committed suicide and collect millions of microblogs from social networks. Second, we build suicide psychological lexicon by psychological standards and word embedding technique. Third, by leveraging both language styles and online behaviors, we employ Topic Model and other machine learning algorithms to identify suicidal ideation. Our approach achieves the best results on topic-500, yielding F1 − measure of 80.0%, Precision of 87.1%, Recall of 73.9%, and Accuracy of 93.2%. Furthermore, a prototype system for monitoring suicidal ideation on several social networks is deployed.


Question: A sentiment analysis of bearish and bullish trends on financial microblogging platforms.

This paper introduces Opinion Observer, a pioneering framework designed for the analysis and comparison of consumer opinions on competing products available on the Web. With the proliferation of online platforms like customer reviews, forums, discussion groups, and blogs, the Web has emerged as a rich repository for consumer sentiments. Our framework, embodied in the Opinion Observer system, offers two key innovations. Firstly, it presents a novel approach to visually analyze and contrast consumer opinions on different products. Through its intuitive visualization, the system enables users to quickly grasp the perceived strengths and weaknesses of each product based on specific features. This comparative analysis is invaluable to both prospective buyers and product manufacturers. For potential customers, it facilitates an informed decision-making process by providing a side-by-side, feature-specific comparison of consumer opinions. For manufacturers, it serves as a powerful tool for gathering market intelligence and product benchmarking insights. Secondly, the paper proposes a cutting-edge technique rooted in language pattern mining to extract product features from the Pros and Cons sections of reviews. These features are crucial for enabling the aforementioned comparative analysis. Extensive experiments validate the efficacy of our technique, demonstrating a significant outperformance over existing methods.

In this work we examine the characteristics of social activity and patterns of communication on Twitter, a prominent example of the emerging class of communication systems we call "social awareness streams." We use system data and message content from over 350 Twitter users, applying human coding and quantitative analysis to provide a deeper understanding of the activity of individuals on the Twitter network. In particular, we develop a content-based categorization of the type of messages posted by Twitter users, based on which we examine users' activity. Our analysis shows two common types of user behavior in terms of the content of the posted messages, and exposes differences between users in respect to these activities.

Use of data generated through social media for health studies is gradually increasing. Twitter is a short-text message system developed 6 years ago, now with more than 100 million users generating over 300 million Tweets every day. Twitter may be used to gain real-world insights to promote healthy behaviors. The purposes of this paper are to describe a practical approach to analyzing Tweet contents and to illustrate an application of the approach to the topic of physical activity. The approach includes five steps: (1) selecting keywords to gather an initial set of Tweets to analyze; (2) importing data; (3) preparing data; (4) analyzing data (topic, sentiment, and ecologic context); and (5) interpreting data. The steps are implemented using tools that are publically available and free of charge and designed for use by researchers with limited programming skills. Content mining of Tweets can contribute to addressing challenges in health behavior research.

An analysis of the social video sharing platform YouTube reveals a high amount of community feedback through comments for published videos as well as through meta ratings for these comments. In this paper, we present an in-depth study of commenting and comment rating behavior on a sample of more than 6 million comments on 67,000 YouTube videos for which we analyzed dependencies between comments, views, comment ratings and topic categories. In addition, we studied the influence of sentiment expressed in comments on the ratings for these comments using the SentiWordNet thesaurus, a lexical WordNet-based resource containing sentiment annotations. Finally, to predict community acceptance for comments not yet rated, we built different classifiers for the estimation of ratings for these comments. The results of our large-scale evaluations are promising and indicate that community feedback on already rated comments can help to filter new unrated comments or suggest particularly useful but still unrated comments.

People share their opinions about things like products, movies and services using social media channels. The analysis of these textual contents for sentiments is a gold mine for marketing experts, thus automatic sentiment analysis is a popular area of applied artificial intelligence. We propose a latent syntactic structure-based approach for sentiment analysis which requires only sentence-level polarity labels for training. Our experiments on three domains (movie, IT products, restaurant) show that a sentiment analyzer that exploits syntactic parses and has access only to sentence-level polarity annotation for in-domain sentences can outperform state-of-the-art models that were trained on out-domain parse trees with sentiment annotation for each node of the trees. In practice, millions of sentence-level polarity annotations are usually available for a particular domain thus our approach is applicable for training a sentiment analyzer for a new domain while it can exploit the syntactic structure of sentences as well.

Sentiment analysis is the procedure by which information is extracted from the opinions, appraisals and emotions of people in regards to entities, events and their attributes. In decision making, the opinions of others have a significant effect on customers ease in making choices regards to online shopping, choosing events, products, entities. In this paper, a rule based domain independent sentiment analysis method is proposed. The proposed method classifies subjective and objective sentences from reviews and blog comments. The semantic score of subjective sentences is extracted from SentiWordNet to calculate their polarity as positive, negative or neutral based on the contextual sentence structure. The results show the effectiveness of the proposed method and it outperforms the machine learning methods. The proposed method achieves an accuracy of 87% at the feedback level and 83% at the sentence level for comments. [Aurangzeb khan, Muhammad Zubair Asghar, Shakeel Ahmad, Fazal Masud Kundi, Maria Qasim, Furqan. Sentiment Classification through Semantic Orientation Using SentiWordNet. Life Sci J 2014; 11(10):309-315] (ISSN: 1097-8135). http://www.lifesciencesite.com. 44

A growing number of people are changing the way they consume news, replacing the traditional physical newspapers and magazines by their virtual online versions or/and weblogs. The interactivity and immediacy present in online news are changing the way news are being produced and exposed by media corporations. News websites have to create effective strategies to catch people’s attention and attract their clicks. In this paper we investigate possible strategies used by online news corporations in the design of their news headlines. We analyze the content of 69,907 headlines produced by four major global media corporations during a minimum of eight consecutive months in 2014. In order to discover strategies that could be used to attract clicks, we extracted features from the text of the news headlines related to the sentiment polarity of the headline. We discovered that the sentiment of the headline is strongly related to the popularity of the news and also with the dynamics of the posted comments on that particular news.

With the widespread use of email, we now have access to unprecedented amounts of text that we ourselves have written. In this paper, we show how sentiment analysis can be used in tandem with effective visualizations to quantify and track emotions in many types of mail. We create a large word–emotion association lexicon by crowdsourcing, and use it to compare emotions in love letters, hate mail, and suicide notes. We show that there are marked differences across genders in how they use emotion words in work-place email. For example, women use many words from the joy–sadness axis, whereas men prefer terms from the fear–trust axis. Finally, we show visualizations that can help people track emotions in their emails.

We examine sentiment analysis on Twitter data. The contributions of this paper are: (1) We introduce POS-specific prior polarity features. (2) We explore the use of a tree kernel to obviate the need for tedious feature engineering. The new features (in conjunction with previously proposed features) and the tree kernel perform approximately at the same level, both outperforming the state-of-the-art baseline.

In this paper we investigate the use of a multimodal feature learning approach, using neural network based models such as Skip-gram and Denoising Autoencoders, to address sentiment analysis of micro-blogging content, such as Twitter short messages, that are composed by a short text and, possibly, an image. The approach used in this work is motivated by the recent advances in: i) training language models based on neural networks that have proved to be extremely efficient when dealing with web-scale text corpora, and have shown very good performances when dealing with syntactic and semantic word similarities; ii) unsupervised learning, with neural networks, of robust visual features, that are recoverable from partial observations that may be due to occlusions or noisy and heavily modified images. We propose a novel architecture that incorporates these neural networks, testing it on several standard Twitter datasets, and showing that the approach is efficient and obtains good classification results.

Twitter sentiment analysis technology provides the methods to survey public emotion about the events or products related to them. Most of the current researches are focusing on obtaining sentiment features by analyzing lexical and syntactic features. These features are expressed explicitly through sentiment words, emoticons, exclamation marks, and so on. In this paper, we introduce a word embeddings method obtained by unsupervised learning based on large twitter corpora, this method using latent contextual semantic relationships and co-occurrence statistical characteristics between words in tweets. These word embeddings are combined with n-grams features and word sentiment polarity score features to form a sentiment feature set of tweets. The feature set is integrated into a deep convolution neural network for training and predicting sentiment classification labels. We experimentally compare the performance of our model with the baseline model that is a word n-grams model on five Twitter data sets, the results indicate that our model performs better on the accuracy and F1-measure for twitter sentiment classification.


Question: Forecasting defects in SAP Java code: An experiential account

This paper introduces Hipikat, a tool designed to serve as a project memory for software development teams, particularly addressing the challenges faced by new members in noncollocated settings. Often, the lack of informal interactions and shared knowledge can hinder the learning process for newcomers, leading to difficulties in understanding the project's nuances and artifacts. Hipikat addresses this issue by automatically aggregating and providing access to the collective memory of the project, encapsulated in the various artifacts created during development, with minimal disruption to existing work practices. The paper details Hipikat's functionality and presents two empirical studies to evaluate its efficacy. The first study focuses on the tool's effectiveness in assisting with software modification tasks, specifically analyzing its recommendations for 20 tasks performed on the Eclipse Java IDE during the development of release 2.1. The study provides quantitative metrics on Hipikat's performance and delves into three detailed case studies to highlight the range of outcomes and insights gleaned from the results. The second study explores the utility of Hipikat for new team members, assessing whether they can effectively utilize the recommended artifacts from the project memory to enhance their learning and integration into the project. The study offers qualitative observations and discusses the implications of using project memory as a tool for facilitating the onboarding process for newcomers.

Smartphone applications’ performance has a vital impact on user experience. However, many smartphone applications suffer from bugs that cause significant performance degradation, thereby losing their competitive edge. Unfortunately, people have little understanding of these performance bugs. They also lack effective techniques to fight with such bugs. To bridge this gap, we conducted a study of 70 real-world performance bugs collected from eight large-scale and popular Android applications. We studied the characteristics (e.g., bug types and how they manifested) of these bugs and identified their common patterns. These findings can support follow-up research on performance bug avoidance, testing, debugging and analysis for smartphone applications. To demonstrate the usefulness of our findings, we implemented a static code analyzer, PerfChecker, to detect our identified performance bug patterns. We experimentally evaluated PerfChecker by applying it to 29 popular Android applications, which comprise 1.1 million lines of Java code. PerfChecker successfully detected 126 matching instances of our performance bug patterns. Among them, 68 were quickly confirmed by developers as previously-unknown issues that affect application performance, and 20 were fixed soon afterwards by following our optimization suggestions.

To be useful, software must respond to events in a predictable manner. The results can then be used as a window to the interior workings of the code, revealing some of the mechanisms of operations, which may be used to find ways to make it fail in a dangerous way. To some, the window is as clear as a six inch thick pane of lead, but to those with a high level of understanding it can be clear, or at the very least serve as a keyhole. This is an allusion to the old detective stories where someone looks through the keyhole to see what is behind the door. For these reasons, no software that interacts with humans can ever be considered completely secure, and human error in the development of the software can leave the equivalent of keyholes throughout the code.

The importance of using requirements information in the testing phase has been well recognized by the requirements engineering community, but to date, a vast majority of regression testing techniques have primarily relied on software code information. Incorporating requirements information into the current testing practice could help software engineers identify the source of defects more easily, validate the product against requirements, and maintain software products in a holistic way. In this paper, we investigate whether the requirements-based clustering approach that incorporates traditional code analysis information can improve the effectiveness of test case prioritization techniques. To investigate the effectiveness of our approach, we performed an empirical study using two Java programs with multiple versions and requirements documents. Our results indicate that the use of requirements information during the test case prioritization process can be beneficial.

1 Introduction Jackson (Jackson 95) recognises the vital difference between an application's domain and its code: two different worlds, each with its own language, experts, ways of thinking etc. A finished application forms the intersection between these worlds. The difficult job of the software engineer is to build a bridge between these worlds, at the same time as solving problems in both worlds.

The discovery of vulnerabilities in source code is a key for securing computer systems. While specific types of security flaws can be identified automatically, in the general case the process of finding vulnerabilities cannot be automated and vulnerabilities are mainly discovered by manual analysis. In this paper, we propose a method for assisting a security analyst during auditing of source code. Our method proceeds by extracting abstract syntax trees from the code and determining structural patterns in these trees, such that each function in the code can be described as a mixture of these patterns. This representation enables us to decompose a known vulnerability and extrapolate it to a code base, such that functions potentially suffering from the same flaw can be suggested to the analyst. We evaluate our method on the source code of four popular open-source projects: LibTIFF, FFmpeg, Pidgin and Asterisk. For three of these projects, we are able to identify zero-day vulnerabilities by inspecting only a small fraction of the code bases.

Editor: Peter Anderson Rochester Institute of Technology 1 Lomb Memorial Dr. Rochester, NY 14623 pga@cs.rit.edu Each technique receives similar coverage, including a description of its new and prerequisite vocabulary, its model, the bug assumptlons that underlie its effectiveness, typical applications, examples (almost all based on standard IRS tax forms), caveats and limitations, automation and tools for the technique, a self-evaluation quiz, and exercises. Common threads in the presentation show the role

As a traditional dynamic language, Python is increasingly used in various software engineering tasks. However, due to its flexibility and dynamism, Python is a particularly challenging language to write code in and maintain. Consequently, Python programs contain code smells which indicate potential comprehension and maintenance problems. With the aim of supporting refactoring strategies to enhance maintainability, this paper describes how to detect code smells in Python programs. We introduce 11 Python smells and describe the detection strategy. We also implement a smell detection tool named Pysmell and use it to identify code smells in five real world Python systems. The results show that Pysmell can detect 285 code smell instances in total with the average precision of 97.7%. It reveals that Large Class and Large Method are most prevalent. Our experiment also implies Python programs may be suffering code smells further.

Debugging is still among the most common and costly of programming activities. One reason is that current debugging tools do not directly support the inquisitive nature of the activity. Interrogative Debugging is a new debugging paradigm in which programmers can ask why did and even why didn't questions directly about their program's runtime failures. The Whyline is a prototype Interrogative Debugging interface for the Alice programming environment that visualizes answers in terms of runtime events directly relevant to a programmer's question. Comparisons of identical debugging scenarios from user tests with and without the Whyline showed that the Whyline reduced debugging time by nearly a factor of 8, and helped programmers complete 40% more tasks.

Static type systems play an essential role in contemporary programming languages. Despite their importance, whether static type systems impact human software development capabilities remains open. One frequently mentioned argument in favor of static type systems is that they improve the maintainability of software systems—an often-used claim for which there is little empirical evidence. This paper describes an experiment that tests whether static type systems improve the maintainability of software systems, in terms of understanding undocumented code, fixing type errors, and fixing semantic errors. The results show rigorous empirical evidence that static types are indeed beneficial to these activities, except when fixing semantic errors. We further conduct an exploratory analysis of the data in order to understand possible reasons for the effect of type systems on the three kinds of tasks used in this experiment. From the exploratory analysis, we conclude that developers using a dynamic type system tend to look at different files more frequently when doing programming tasks—which is a potential reason for the observed differences in time.

In this study, we attempted to quantify indicators of novice programmer progress in the task of writing programs, and we evaluated the use of these indicators for identifying academically at-risk students. Over the course of nine weeks, students completed five different graded programming exercises in a computer lab. Using an instrumented version of BlueJ, an integrated development environment for Java, we collected novice compilations and explored the errors novices encountered, the locations of these errors, and the frequency with which novices compiled their programs. We identified which frequently encountered errors and which compilation behaviors were characteristic of at-risk students. Based on these findings, we developed linear regression models that allowed prediction of students' scores on a midterm exam. However, the models derived could not accurately predict the at-risk students. Although our goal of identifying at-risk students was not attained, we have gained insights regarding the compilation behavior of our students, which may help us identify students who are in need of intervention.


Question: Forecasting defects in SAP Java programming: A practical case study

In response to the pivotal role of software development in the deployment and utilization of information technology, a growing emphasis on process enhancement is being observed. This trend has led to the emergence and refinement of novel software development methodologies, with object-orientation (OO) standing out as a particularly significant development. Concurrently, the heightened focus on process improvement has amplified the demand for software metrics, or quantitative indicators, to oversee and optimize the software development process. This demand is especially pronounced when an organization is transitioning to a new technology, where established practices are still evolving. This research endeavor addresses these requirements by developing and implementing a novel set of metrics for OO design. Previous efforts in metric development, while enriching our comprehension of software development processes, have often been critiqued for lacking a robust theoretical foundation. Inspired by Wand and Weber, the chosen theoretical underpinning for these metrics is the ontology proposed by Bunge. Six design metrics are formulated and analytically scrutinized against Weyuker’s criteria for measurement principles. Subsequently, an automated data collection mechanism is devised and executed to gather empirical data on these metrics from two operational environments. This data collection aims to substantiate the practicality of these metrics and to propose potential applications for managers seeking to enhance their software development processes through metrics-driven process improvement. "A Metrics Suite for Object-Oriented Design: An Analytical and Empirical Exploration" Shyam R. Chidamber Chris F. Kemerer Keywords: CR Classification

Spreadsheet programs can be found everywhere in organizations and they are used for a variety of purposes, including financial calculations, planning, data aggregation and decision making tasks. A number of research surveys have however shown that such programs are particularly prone to errors. Some reasons for the error-proneness of spreadsheets are that spreadsheets are developed by end users and that standard software quality assurance processes are mostly not applied. Correspondingly, during the last two decades, researchers have proposed a number of techniques and automated tools aimed at supporting the end user in the development of error-free spreadsheets. In this paper, we provide a review of the research literature and develop a classification of automated spreadsheet quality assurance (QA) approaches, which range from spreadsheet visualization, static analysis and quality reports, over testing and support to model-based spreadsheet development. Based on this review, we outline possible opportunities for future work in the area of automated spreadsheet QA.

Support for generic programming was added to the Java language in 2004, representing perhaps the most significant change to one of the most widely used programming languages today. Researchers and language designers anticipated this addition would relieve many long-standing problems plaguing developers, but surprisingly, no one has yet measured how generics have been adopted and used in practice. In this paper, we report on the first empirical investigation into how Java generics have been integrated into open source software by automatically mining the history of 40 popular open source Java programs, traversing more than 650 million lines of code in the process. We evaluate five hypotheses and research questions about how Java developers use generics. For example, our results suggest that generics sometimes reduce the number of type casts and that generics are usually adopted by a single champion in a project, rather than all committers. We also offer insights into why some features may be adopted sooner and others features may be held back.

In the early days of programming the concept of subroutines, and through this software reuse, was invented to spare limited hardware resources. Since then software systems have become increasingly complex and developing them would not have been possible without reusable software elements such as standard libraries and frameworks. Furthermore, other approaches commonly subsumed under the umbrella of software reuse such as product lines and design patterns have become very successful in recent years. However, there are still no software component markets available that would make buying software components as simple as buying parts in a do-it-yourself hardware store and millions of software fragments are still lying un(re)used in configuration management repositories all over the world. The literature primarily blames this on the immense effort required so far to set up and maintain searchable component repositories and the weak mechanisms available for retrieving components from them, resulting in a severe usability problem. In order to address these issues within this thesis, we developed a proactive component reuse recommendation system, naturally integrated into test-first development approaches, which is able to propose semantically appropriate, reusable components according to the specification a developer is just working on. We have implemented an appropriate system as a plugin for the well-known Eclipse IDE and demonstrated its usefulness by carrying out a case study from a popular agile development book. Furthermore, we present a precision analysis for our approach and examples of how components can be retrieved based on a simplified semantics description in terms of standard test cases. Zusammenfassung Zu Zeiten der ersten Programmiersprachen wurde die Idee von Unterprogrammen und damit die Idee der Wiederverwendung von Software zur Einsparung knapper Hardware-Ressourcen erdacht. Seit dieser Zeit wurden Software-Systeme immer komplexer und ihre Entwicklung wäre ohne weitere wiederverwendbare SoftwareElemente wie Bibliotheken und Frameworks schlichtweg nicht mehr handhabbar. Weitere, üblicherweise unter dem Begriff Software Reuse zusammengefasste Ansätze, wie z.B. Produktlinien und Entwurfsmuster waren in den letzten Jahren ebenfalls sehr erfolgreich, gleichzeitig existieren allerdings noch immer keine Marktplätze, die das Kaufen von Software-Komponenten so einfach machen würden, wie den Einkauf von Kleinteilen in einem Heimwerkermarkt. Daher schlummern derzeit Millionen von nicht (wieder) genutzten Software-Fragmenten in Konfigurations-Management-Systemen auf der ganzen Welt. Die Fachliteratur lastet dies primär dem hohen Aufwand, der bisher für Aufbau und Instandhaltung von durchsuchbaren Komponenten-Repositories getrieben werden musste, an. Zusammen mit den ungenauen Algorithmen, wie sie bisher zum Durchsuchen solcher Komponentenspeicher zur Verfügung stehen, macht diese Tatsache die Benutzung dieser Systeme zu kompliziert und damit unattraktiv. Um diese Hürde künftig abzumildern, entwickelten wir in der vorliegenden Arbeit ein proaktives Komponenten-Empfehlungssystem, das eng an testgetriebene Entwicklungsprozesse angelehnt ist und darauf aufbauend wiederverwendbare Komponenten vorschlagen kann, die genau die Funktionalität erbringen, die ein Entwickler gerade benötigt. Wir haben das System als Plugin für die bekannte Eclipse IDE entwickelt und seine Nutzbarkeit unter Beweis gestellt, in dem wir ein Beispiel aus einem bekannten Buch über agile Entwicklung damit nachimplementiert haben. Weiterhin enthält diese Arbeit eine Analyse der Precision unseres Ansatzes sowie zahlreiche Beispiele, wie gewöhnliche Testfälle als vereinfachte semantische Beschreibung einer Komponente und als Ausgangspunkt für die Suche nach wiederverwendbaren Komponenten genutzt werden können.

Bug fixing is generally a manually-intensive task. However, recent work has proposed the idea of automated program repair, which aims to repair (at least a subset of) bugs in different ways such as code mutation, etc. Following in the same line of work as automated bug repair, in this paper we aim to leverage past fixes to propose fixes of current/future bugs. Specifically, we propose Ratchet, a corrective patch generation system using neural machine translation. By learning corresponding pre-correction and post-correction code in past fixes with a neural sequence-to-sequence model, Ratchet is able to generate a fix code for a given bug-prone code query. We perform an empirical study with five open source projects, namely Ambari, Camel, Hadoop, Jetty and Wicket, to evaluate the effectiveness of Ratchet. Our findings show that Ratchet can generate syntactically valid statements 98.7% of the time, and achieve an F1-measure between 0.41-0.83 with respect to the actual fixes adopted in the code base. In addition, we perform a qualitative validation using 20 participants to see whether the generated statements can be helpful in correcting bugs. Our survey showed that Ratchet’s output was considered to be helpful in fixing the bugs on many occasions, even if fix was not 100%

Most IT experts agree that software failures occur far more often than they should despite the fact that, for the most part, they are predictable and avoidable. It is unfortunate that most organizations don't see preventing failure as an urgent matter, even though that view risks harming the organization and maybe even destroying it. Because software failure has tremendous implications for business and society, it is important to understand why this attitude persists.

Successful software maintenance is becoming increasingly critical due to the increasing dependence of our society and economy on software systems. One key problem of software maintenance is the difficulty in understanding the evolving software systems. Program workflows can help system operators and administrators to understand system behaviors and verify system executions so as to greatly facilitate system maintenance. In this paper, we propose an algorithm to automatically discover program workflows from event traces that record system events during system execution. Different from existing workflow mining algorithms, our approach can construct concurrent workflows from traces of interleaved events. Our workflow mining approach is a three-step coarse-to-fine algorithm. At first, we mine temporal dependencies for each pair of events. Then, based on the mined pair-wise tem-poral dependencies, we construct a basic workflow model by a breadth-first path pruning algorithm. After that, we refine the workflow by verifying it with all training event traces. The re-finement algorithm tries to find out a workflow that can interpret all event traces with minimal state transitions and threads. The results of both simulation data and real program data show that our algorithm is highly effective.

We present a new approach for predicting program properties from massive codebases (aka "Big Code"). Our approach first learns a probabilistic model from existing data and then uses this model to predict properties of new, unseen programs.
 The key idea of our work is to transform the input program into a representation which allows us to phrase the problem of inferring program properties as structured prediction in machine learning. This formulation enables us to leverage powerful probabilistic graphical models such as conditional random fields (CRFs) in order to perform joint prediction of program properties.
 As an example of our approach, we built a scalable prediction engine called JSNice for solving two kinds of problems in the context of JavaScript: predicting (syntactic) names of identifiers and predicting (semantic) type annotations of variables. Experimentally, JSNice predicts correct names for 63% of name identifiers and its type annotation predictions are correct in 81% of the cases. In the first week since its release, JSNice was used by more than 30,000 developers and in only few months has become a popular tool in the JavaScript developer community.
 By formulating the problem of inferring program properties as structured prediction and showing how to perform both learning and inference in this context, our work opens up new possibilities for attacking a wide range of difficult problems in the context of "Big Code" including invariant generation, decompilation, synthesis and others.

Software testing plays an important role in early verification of software systems and it enforces quality in the system under development. One of the challenging tasks in the software testing is generation of software test cases. There are many existing approaches to generate test cases like using uses case, activity diagrams and sequence diagrams; they have their own limitations such as inability to capture test cases for non functional requirements and etc. Thus these techniques have restricted use in acceptance testing and are not effective for verification & acceptance of large software system. If software requirements are stated using semi-formal or formal methods then it is difficult for the testers and other third party domain experts to test the system. It also requires much expertise in interpreting requirements and only limited number of persons can understand them. This paper proposes an approach to generate test case from software requirements expressed in natural language using natural language processing technique.

Software development in highly variable domains constrained by tight regulations and with many business concepts involved results in hard to deliver and maintain applications, due to the complexity of dealing with the large number of concepts provided by the different parties and system involved in the process. One way to tackle these problems is thru combining software product lines and model-driven software development supported by ontologies. Software product lines and model-driven approaches would promote reuse on the software artifacts and, if supported by an ontological layer, those artifacts would be domain-validated. We intend to create a new conceptual framework for software development with domain validated models in highly variable domains. To define such a framework we will propose a model that relates several dimensions and areas of software development thru time and abstraction levels. This model would guarantee to the software house traceability of components, domain validated artifacts, easy to maintain and reusable components, due to the relations and mappings we propose to establish in the conceptual framework, between the software artifacts and the ontology.

Pair or collaborative programming is where two programmers develop software side by side at one computer. Using interviews and controlled experiments, the authors investigated the costs and benefits of pair programming. They found that for a development-time cost of about 15%, pair programming improves design quality, reduces defects, reduces staffing risk, enhances technical skills, improves team communications and is considered more enjoyable at statistically significant levels.


Question: Forecasting defects in SAP Java programming: A case study report

This paper explores the dynamics of open source software development through an in-depth analysis of two prominent projects: the Apache web server and the Mozilla browser. Advocates argue that the open source model can not only rival but potentially supplant conventional commercial software development methodologies. To evaluate these assertions, we scrutinize data from these two projects, leveraging email archives that track source code modifications and issue reports. Our examination quantifies key aspects of the development process, including developer involvement, the size of the core team, code stewardship, productivity metrics, defect density, and the timeframes for resolving issues. By juxtaposing the Apache project against several commercial endeavors, we formulate several hypotheses. We then rigorously test and refine these hypotheses using data from the Mozilla project. Our findings offer insights into the viability of high-performance models that blend commercial and open source development practices, shedding light on the future of software engineering paradigms.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design
An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design

This case focuses on the applicability of the Scaled Agile Framework (SAFe) founded by Dean Leffingwell. Modern organizations often work with agile software engineering teams using traditional single team-level methods like Scrum, but multiple teams and the program or portfolio level are not part of methods like Scrum. SAFe tries to apply agile methodologies to the whole organisation. The real world example focuses on a key element of SAFe, the Program Increment (PI) planning meeting and how it can improve multiple team collaboration.

Programming foundations is not an easy subject to be taught — many students have difficulties understanding the abstract concepts of programming and have a wrong view about the programming activity. In order to address these problems, experiences have suggested the integrated teaching of programming concepts and software testing in introductory CS courses. Shortly, the idea is that testing can contribute to enhance the students' capabilities of understanding and analysis. However, such perspective requires tools to provide an adequate feedback to evaluate the students' performance concerning programming and testing activities. In this paper we describe ProgTest — a web-based tool for the submission and automatic evaluation of practical programming assignments based on testing activities. Results from a preliminary validation of ProgTest are also presented. Such results provide evidences on the practical use of ProgTest as a supporting mechanism for the integrated teaching of programming foundations and software testing.

Shared understanding is essential for efficient software engineering when the risk of unsatisfactory out-come and rework of project results shall be low. Today, however, shared understanding is used mostly in an unreflected, ad-hoc way. This affects the quality of the engineered software solutions and generates re-work once the quality problems are discovered. In this article, we investigate the role, value, and usage of shared understanding in software engineering. We contribute a reflected analysis of the problem, in particular of how to rely on shared understanding that is implicit, rather than explicit. After an overview of the state of the art we discuss forms and value of shared understanding in software engineering, survey enablers and obstacles, compile existing practices for dealing with shared understanding, and present a roadmap for improving knowledge and practice in this area.

Real-time analytics over data streams are crucial for a wide range of use cases in industry and research. Today's sensor systems can produce high throughput data streams that have to be analyzed in real-time. One important analytic task is anomaly or outlier detection from the streaming data. In many industry applications, sensing devices produce a data stream that can be monitored to know the correct operation of industry devices and consequently avoid damages by triggering reactions in real-time.
 While anomaly detection is a well-studied topic in data mining, the real-time high-performance anomaly detection from big data streams require special studies and well-optimized implementation. This paper presents our implementation of a real-time anomaly detection system over data streams. We outline details of our two separate implementations using the Java and C++ programming languages, and provide technical details about the data processing pipelines. We report experimental results and describe performance tuning strategies.

School of Computing and Mathematics, Faculty of Engineering, University of Ulster at Jordanstown, Shore Road, Newtownabbey, County Antrim BT37 0QB, Northern Ireland, UK The Applied Software Systems Laboratory (TASSL), Electrical and Computer Engineering, Rutgers, The State University of New Jersey, 94 Brett Road, Piscataway, NJ 08854-8058, USA School of Computing and Mathematical Sciences, The SRIF/SHEFC Centre for Virtual Organization Technology Enabling Research (VOTER), Glasgow Caledonian University, 70 Cowcaddens Road, Glasgow G4 0BA, UK Institute for Computer Science and Business Information Systems (ICB), University of Duisburg-Essen, Schützenbahn 70, 45117 Essen, Germany

Locating bugs is important, difficult, and expensive, particularly for large-scale systems. To address this, natural language information retrieval techniques are increasingly being used to suggest potential faulty source files given bug reports. While these techniques are very scalable, in practice their effectiveness remains low in accurately localizing bugs to a small number of files. Our key insight is that structured information retrieval based on code constructs, such as class and method names, enables more accurate bug localization. We present BLUiR, which embodies this insight, requires only the source code and bug reports, and takes advantage of bug similarity data if available. We build BLUiR on a proven, open source IR toolkit that anyone can use. Our work provides a thorough grounding of IR-based bug localization research in fundamental IR theoretical and empirical knowledge and practice. We evaluate BLUiR on four open source projects with approximately 3,400 bugs. Results show that BLUiR matches or outperforms a current state-of-the-art tool across applications considered, even when BLUiR does not use bug similarity data used by the other tool.

Effectively performing code review increases the quality of software and reduces occurrence of defects. However, this requires reviewers with experiences and deep understandings of system code. Manual selection of such reviewers can be a costly and time-consuming task. To reduce this cost, we propose a reviewer recommendation algorithm determining file path similarity called FPS algorithm. Using three OSS projects as case studies, FPS algorithm was accurate up to 77.97%, which significantly outperformed the previous approach.

Background: In 2004 the concept of evidence-based software engineering (EBSE) was introduced at the ICSE04 conference. Aims: This study assesses the impact of systematic literature reviews (SLRs) which are the recommended EBSE method for aggregating evidence. Method: We used the standard systematic literature review method employing a manual search of 10 journals and 4 conference proceedings. Results: Of 20 relevant studies, eight addressed research trends rather than technique evaluation. Seven SLRs addressed cost estimation. The quality of SLRs was fair with only three scoring less than 2 out of 4. Conclusions: Currently, the topic areas covered by SLRs are limited. European researchers, particularly those at the Simula Laboratory appear to be the leading exponents of systematic literature reviews. The series of cost estimation SLRs demonstrate the potential value of EBSE for synthesising evidence and making it available to practitioners. ! 2008 Elsevier B.V. All rights reserved.

Requirements traceability enables software engineers to trace a requirement from its emergence to its ful ̄llment. In this paper we examine requirements traceability de ̄nitions, challenges, tools and techniques, by the use of a systematic review performing an exhaustive search through the years 1997 2007. We present a number of common de ̄nitions, challenges, available tools and techniques (presenting empirical evidence when found), while complementing the results and analysis with a static validation in industry through a series of interviews.

Code smell is an indication of a software designing problem. The presence of code smells can have a severe impact on the software quality. Smells basically refers to the structure of the code which violates few of the design principals and so has negative effect on the quality of the software. Larger the source code, more is its presence. Software needs to be reliable, robust and easily maintainable so that it can minimize the cost of its development as well as maintenance. Smells may increase the chances of failure of the system during maintenance. A SLR has been performed based on the search of digital libraries that includes the publications since 1999 to 2016. 60 research papers are deeply analyzed that are most relevant. The objective of this paper is to provide an extensive overview of existing research in the field of bad smells, identify the detection techniques and correlation between the detection techniques, in addition to find the name of the code smells that need more attention in detection approaches. This SLR identified that code clone (code smell) receives most research attention. Our findings also show that very few papers report on the impact of code bad smells. Most of the papers focused on the detection techniques and tools. A significant correlation between detection techniques has been calculated. There are four code smells that are not yet detected are Primitive Obsession, Inappropriate Intimacy, Incomplete library class and Comments.


Question: Forecasting defects in SAP Java programming: An empirical study report

An Experience Report on a Line-Oriented File Comparison Utility

This paper discusses a line-oriented file comparison utility designed to highlight discrepancies between two files. The utility operates by identifying lines that need to be inserted, deleted, or moved to transform the first file into the second, focusing solely on line-based differences rather than individual bytes. This approach is particularly effective for text files, where lines are the primary units of comparison. By treating lines as indivisible entities, the comparison process simplifies, allowing files to be represented as strings of symbols, where each symbol corresponds to a line. The file comparison challenge can thus be redefined as computing the minimum number of insertions and deletions required to convert one string into another. The utility's primary goal is to calculate this minimum sequence, often referred to as the edit distance, and generate a concise edit script detailing the specific commands needed for the transformation. Alternative methods, such as Tichy's approach involving the copying and appending of line blocks, have been explored but may not yield the most efficient edit scripts. The utility's effectiveness in generating optimal edit scripts is contingent upon the specific operations permitted within the script. Beyond its use in comparing files, the edit script can serve as a set of text editor instructions, enabling the storage of nearly identical files in a more economical manner by saving the edit commands instead of the entire file content. This approach not only aids in understanding the evolution of programs or documents but also optimizes storage and retrieval processes for similar files.

A number of effective error detection tools have been built in recent years to check if a program conforms to certain design rules. An important class of design rules deals with sequences of events asso-ciated with a set of related objects. This paper presents a language called PQL (Program Query Language) that allows programmers to express such questions easily in an application-specific context. A query looks like a code excerpt corresponding to the shortest amount of code that would violate a design rule. Details of the tar-get application's precise implementation are abstracted away. The programmer may also specify actions to perform when a match is found, such as recording relevant information or even correcting an erroneous execution on the fly.We have developed both static and dynamic techniques to find solutions to PQL queries. Our static analyzer finds all potential matches conservatively using a context-sensitive, flow-insensitive, inclusion-based pointer alias analysis. Static results are also use-ful in reducing the number of instrumentation points for dynamic analysis. Our dynamic analyzer instruments the source program to catch all violations precisely as the program runs and to optionally perform user-specified actions.We have implemented the techniques described in this paper and found 206 errors in 6 large real-world open-source Java applica-tions containing a total of nearly 60,000 classes. These errors are important security flaws, resource leaks, and violations of consis-tency invariants. The combination of static and dynamic analysis proves effective at addressing a wide range of debugging and pro-gram comprehension queries. We have found that dynamic analysis is especially suitable for preventing errors such as security vulner-abilities at runtime.

Bug fixing is generally a manually-intensive task. However, recent work has proposed the idea of automated program repair, which aims to repair (at least a subset of) bugs in different ways such as code mutation, etc. Following in the same line of work as automated bug repair, in this paper we aim to leverage past fixes to propose fixes of current/future bugs. Specifically, we propose Ratchet, a corrective patch generation system using neural machine translation. By learning corresponding pre-correction and post-correction code in past fixes with a neural sequence-to-sequence model, Ratchet is able to generate a fix code for a given bug-prone code query. We perform an empirical study with five open source projects, namely Ambari, Camel, Hadoop, Jetty and Wicket, to evaluate the effectiveness of Ratchet. Our findings show that Ratchet can generate syntactically valid statements 98.7% of the time, and achieve an F1-measure between 0.41-0.83 with respect to the actual fixes adopted in the code base. In addition, we perform a qualitative validation using 20 participants to see whether the generated statements can be helpful in correcting bugs. Our survey showed that Ratchet’s output was considered to be helpful in fixing the bugs on many occasions, even if fix was not 100%

Written development communication (e.g. mailing lists, issue trackers) constitutes a precious source of information to build recommenders for software engineers, for example aimed at suggesting experts, or at redocumenting existing source code. In this paper we propose a novel, semi-supervised approach named DECA (Development Emails Content Analyzer) that uses Natural Language Parsing to classify the content of development emails according to their purpose (e.g. feature request, opinion asking, problem discovery, solution proposal, information giving etc), identifying email elements that can be used for specific tasks. A study based on data from Qt and Ubuntu, highlights a high precision (90%) and recall (70%) of DECA in classifying email content, outperforming traditional machine learning strategies. Moreover, we successfully used DECA for re-documenting source code of Eclipse and Lucene, improving the recall, while keeping high precision, of a previous approach based on ad-hoc heuristics.

Software development teams use test suites to test changes to their source code. In many situations, the test suites are so large that executing every test for every source code change is infeasible, due to time and resource constraints. Development teams need to prioritize their test suite so that as many distinct faults as possible are detected early in the execution of the test suite. We consider the problem of static black-box test case prioritization (TCP), where test suites are prioritized without the availability of the source code of the system under test (SUT). We propose a new static black-box TCP technique that represents test cases using a previously unused data source in the test suite: the linguistic data of the test cases, i.e., their identifier names, comments, and string literals. Our technique applies a text analysis algorithm called topic modeling to the linguistic data to approximate the functionality of each test case, allowing our technique to give high priority to test cases that test different functionalities of the SUT. We compare our proposed technique with existing static black-box TCP techniques in a case study of multiple real-world open source systems: several versions of Apache Ant and Apache Derby. We find that our static black-box TCP technique outperforms existing static black-box TCP techniques, and has comparable or better performance than two existing execution-based TCP techniques. Static black-box TCP methods are widely applicable because the only input they require is the source code of the test cases themselves. This contrasts with other TCP techniques which require access to the SUT runtime behavior, to the SUT specification models, or to the SUT source code.

Context: Software reuse is considered as the key to a successful software development because of its potential to reduce the time to market, increase quality and reduce costs. This increase in demand made the software organizations to envision the use of software reusable assets which can also help in solving recurring problems. Till now, software reuse is confined to reuse of source code in the name of code scavenging. Now a day, software organizations are extending the concepts of software reuse to other life cycle objects as they realized that reuse of source code alone does not save money. The academia has put forward some assets as reusable and presented methods or approaches for reusing them. Also, for a successful software reuse the organizations should assess the value of reuse and keep track on their reuse programs. The other area which is vital for software reuse is the maintenance. Maintenance of reusable software has direct impact on the cost of the software. In this regard, academia has presented a number of techniques, methods, metrics and models for assessing the value of reuse and for maintaining the reusable software. Objectives: In our thesis, we investigate on the reusable assets and the methods/ approaches that are put forward by the academia for reusing those assets. Also a systematic mapping study is performed to investigate what techniques, methods, models and metrics for assessing the value of reuse and for maintaining the reused software are proposed and we also investigate their validation status as well. Methods: The databases like IEEE Xplore, ACM digital library, Inspec, Springer and Google scholar were used to search for the relevant studies for our systematic mapping study. We followed basic inclusion criteria along with detailed inclusion/exclusion criteria for selecting the appropriate article. Results: Through our systematic mapping study, we could summarize the list of 14 reusable assets along with the approaches/methods for reusing them. Taxonomy for assessing the value of reuse and taxonomy for maintaining the reusable software are presented. We also presented the methods/metrics/models/techniques for measuring reuse to assess its value and for maintaining the reusable software along with their validation status and areas in focus. Conclusion: We conclude that, there is a need for defining a standard set of reusable assets that are commonly accepted by the researchers in the field of software reuse. Most metrics/models/methods/approaches presented for assessing the value of reuse and for maintaining the reuse software are academically validated. Efforts have to be put on industrially validating them using the real data.

The data quality directly determines the data further analysis and other applications. With the important of data quality, data is a major asset in software applications and information systems. Getting and ensuring high quality data is critical to the software and related business operations. At present, there are many methods and techniques of data quality assessment or evaluation. Based on the evaluation criteria of data quality, the paper mainly starts from the needs of application software and uses the requirements as a guide in the evaluation process. Define evaluation criteria based on user-defined requirements. In this paper, the method proposed in a practical case has been applied, and achieved good practical results.

Programming languages are the way developers communicate with computers—just like natural languages let us communicate with one another. In both cases multiple languages have evolved. However, the programming language ecosystem changes at a much higher rate compared to natural languages. In fact, programming languages need to constantly evolve in response to user needs, hardware advances and research developments or they are otherwise displaced by newcomers. As a result, existing code written by developers may stop working with a newer language version. Consequently, developers need to search (analyse) and replace (refactor) code in their code bases to support new language versions. Traditionally, tools have focused on the replace aspect (refactoring) to support developers evolving their code bases. This dissertation argues that developers also need machine support focused on the search aspect. This dissertation starts by characterising factors driving programming language evolution based on external versus internal forces. Next, it introduces a classification of changes introduced by language designers that affect developers. It then contributes three techniques to support developers in analysing their code bases. First, we show a source code query system based on graphs that express code queries at a mixture of syntax-tree, type, control-flow graph and data-flow levels. We demonstrate how this system can support developers and language designers in locating various code patterns relevant in evolution. Second, we design an optional run-time type system for Python, that lets developers manually specify contracts to identify semantic incompatibilities between Python 2 and Python 3. Third, recognising that existing codebases do not have such contracts, we describe a dynamic analysis to automatically generate them.

0 7 4 0 7 4 5 9 / 0 3 / $ 1 9 . 0 0 © 2 0 0 3 I E E E All the methodologies that have been proposed recently for choosing software packages compare user requirements with the packages’ capabilities.3–5 There are different types of requirements, such as managerial, political, and, of course, quality requirements. Quality requirements are often difficult to check. This is partly due to their nature, but there is another reason that can be mitigated, namely the lack of structured and widespread descriptions of package domains (that is, categories of software packages such as ERP systems, graphical or data structure libraries, and so on). This absence hampers the accurate description of software packages and the precise statement of quality requirements, and consequently overall package selection and confidence in the result of the process. Our methodology for building structured quality models helps solve this drawback. (See the “Related Work” sidebar for information about other approaches.) A structured quality model for a given package domain provides a taxonomy of software quality features, and metrics for computing their value. Our approach relies on the International Organization for Standardization and International Electrotechnical Commission 9126-1 quality standard,6 which we selected for the following reasons:

Current software cost estimation models, such as the 1981 Constructive Cost Model (COCOMO) for software cost estimation and its 1987 Ada COCOMO update, have been experiencing increasing difficulties in estimating the costs of software developed to new life cycle processes and capabilities. These include non-sequential and rapid-development process models; reuse-driven approaches involving commercial off the shelf (COTS) packages, reengineering, applications composition, and applications generation capabilities; object-oriented approaches supported by distributed middleware; and software process maturity initiatives. This paper summarizes research in deriving a baseline COCOMO 2.0 model tailored to these new forms of software development, including rationales for the model decisions. The major new modeling capabilities of COCOMO 2.0 are a tailorable family of software sizing models, involving Object Points, Function Points, and Source Lines of Code; nonlinear models for software reuse and reengineering; an exponent-driver approach for modeling relative software diseconomies of scale; and several additions, deletions, and updates to previous COCOMO effort-multiplier cost drivers. This model is serving as a framework for an extensive current data collection and analysis effort to further refine and calibrate the model’s estimation capabilities.

Objective: Our objective is to describe how softwareengineering might benefit from an evidence-basedapproach and to identify the potential difficultiesassociated with the approach.Method: We compared the organisation and technicalinfrastructure supporting evidence-based medicine (EBM)with the situation in software engineering. We consideredthe impact that factors peculiar to software engineering(i.e. the skill factor and the lifecycle factor) would haveon our ability to practice evidence-based softwareengineering (EBSE).Results: EBSE promises a number of benefits byencouraging integration of research results with a view tosupporting the needs of many different stakeholdergroups. However, we do not currently have theinfrastructure needed for widespread adoption of EBSE.The skill factor means software engineering experimentsare vulnerable to subject and experimenter bias. Thelifecycle factor means it is difficult to determine howtechnologies will behave once deployed.Conclusions: Software engineering would benefit fromadopting what it can of the evidence approach providedthat it deals with the specific problems that arise from thenature of software engineering.