Question: Regression testing for graphical user interfaces.

This paper delves into the critical role of coverage criteria in the testing of graphical user interfaces (GUIs), highlighting their distinct nature compared to traditional software. Given the significant role GUIs play in modern software applications, tailored testing methods are essential. The focus here is on developing coverage criteria that serve as key benchmarks for assessing test quality in GUIs. We propose novel coverage criteria that enable the evaluation of whether a GUI has undergone sufficient testing, focusing on events and event sequences as a measure of adequacy. Given the astronomical number of possible event sequence permutations in even moderately complex GUIs, we leverage the inherent hierarchical structure of GUIs to pinpoint the critical event sequences for testing. This involves breaking down the GUI into components, each serving as a fundamental testing unit. Event-flow graphs, a representation of these components, capture the interaction of events within each component, facilitating the application of intra-component coverage criteria to assess test adequacy on these events. The hierarchical interplay between components is depicted through an integration tree, where inter-component coverage criteria are employed to evaluate the effectiveness of test sequences that traverse multiple components. The paper provides algorithms for constructing event-flow graphs and integration trees for a given GUI, as well as for assessing the coverage of a test suite according to these new criteria. A case study is presented to demonstrate the practical utility of the coverage report in guiding further testing efforts and to underscore the significant correlation between event-based coverage of a GUI and the statement coverage of the underlying software code.

Cross-device interactions involve input and output on multiple computing devices. Implementing and reasoning about interactions that cover multiple devices with a diversity of form factors and capabilities can be complex. To assist developers in programming cross-device interactions, we created DemoScript, a technique that automatically analyzes a cross-device interaction program while it is being written. DemoScript visually illustrates the step-by-step execution of a selected portion or the entire program with a novel, automatically generated cross-device storyboard visualization. In addition to helping developers understand the behavior of the program, DemoScript also allows developers to revise their program by interactively manipulating the cross-device storyboard. We evaluated DemoScript with 8 professional programmers and found that DemoScript significantly improved development efficiency by helping developers interpret and manage cross-device interaction; it also encourages testing to think through the script in a development process.

Errors in requirements specifications have been identified as a major contributor to costly software project failures. It would be highly beneficial if information systems developers could verify requirements by predicting workplace acceptance of a new system based on user evaluations of its specifications measured during the earliest stages of the development project, ideally before building a working prototype. However, conventional wisdom among system developers asserts that prospective users must have direct hands-on experience with at least a working prototype of a new system before they can provide assessments that accurately reflect future usage behavior after workplace implementation. The present research demonstrates that this assumption is only partially true. Specifically, it is true that stable and predictive assessments of a system's perceived ease of use should be based on direct behavioral experience using the system. However, stable and behaviorally predictive measures of perceived usefulness can be captured from target users who have received information about a system's functionality, but have not had direct hands-on usage experience. This distinction is key because, compared to ease of use, usefulness is generally much more strongly linked to future usage intentions and behaviors in the workplace. Two longitudinal field experiments show that preprototype usefulness measures can closely approximate hands-on based usefulness measures, and are significantly predictive of usage intentions and behavior up to six months after workplace implementation. The present findings open the door toward research on how user acceptance testing may be done much earlier in the system development process than has traditionally been the case. Such preprototype user acceptance tests have greater informational value than their postprototype counterparts because they are captured when only a relatively small proportion of project costs have been incurred and there is greater flexibility to modify a new system's design attributes. Implications are discussed for future research to confirm the robustness of the present findings and to better understand the practical potential and limitations of preprototype user acceptance testing.

This film demonstrates a tool for visualizing the development history of software projects as an interactive animation, showing developers working on the hierarchical file-directory structure of a project over the course of its development.

This paper presents a usage analysis and taxonomy of methods which are used to evaluate the usability of computer systems. To accommodate the analysis and taxonomy, a matrix of strategies which can be used for effective usability evaluation is presented. Such an analysis, taxonomy and strategies support human-computer interaction (HCI) professionals who have the responsibility for ensuring computer system usability. The strategies outlined are named Virtual Engineering, Soft Modelling, Hard Review and Real World. This paper also uses a composite set of existing popular generic evaluation methods which can be used as part of these strategies. The methods used are observation, questionnaire, interview, empirical methods, user groups, cognitive walkthroughs, heuristic methods, review methods and model methods. The paper continues by presenting a Usage Analysis Table of these methods and concludes by grouping them into a Taxonomy of Usability Evaluation Methods. A key emphasis of this paper is the appropriateness of individual methods to lifecycle timing.

A new robot toolbox for dynamics simulation based on MATLAB Graphical User Interface (GUI) is developed for educational purposes. It is built on the previous version named as ROBOLAB performing only the kinematics analysis of robot manipulators. The toolbox presented in this paper provides interactive real-time simulation and visualization of the industrial robot manipulator’s dynamics based on Langrange Euler and Newton Euler formulations. Since most of the industrial robot manipulators are produced with six degrees of freedom (DOF) such as the PUMA 560, the Fanuc ArcMate 120iB and Stanford Arm, the library of the toolbox includes sixteen fundamental 6-DOF robot manipulators with Euler wrist. The software can be used to interactively perform robotics analysis and off-line programming of robot dynamics such as forward and inverse dynamics as well as to interactively teach and simulate the basic principles of robot dynamics in a very realistic way. In order to demonstrate the user-friendly features of the toolbox much better, simulation of the NS robot manipulator (Stanford Arm) is provided as an example. 2009 Wiley Periodicals, Inc. Comput Appl Eng Educ 18: 319 330, 2010; Published online in Wiley InterScience (www.interscience.wiley.com); DOI 10.1002/cae.20262

With the increasing amount of automated tests, we need ways to measure the test e↵ectiveness. The state-of-the-art technique for assessing test e↵ectiveness, mutation testing, is too slow and cumbersome to be used in large scale evolution studies or code audits by external companies. In this paper we investigated two alternatives, namely code coverage and assertion count. We discovered that code coverage outperforms assertion count by showing a relation with test suite e↵ectiveness for all analysed project. Assertion count only displays such a relation in only one of the analysed projects. Further analysing this relationship between assertion count coverage and test e↵ectiveness would allow to circumvent some of the problems of mutation testing.

We describe a new algorithm GUESS-AND-CHECK for computing algebraic equation invariants. These invariants are of the form ∧ifi(x1, . . . , xn) = 0, where each fi is a polynomial over the variables x1, . . . , xn of the program. Two novel features of our algorithm are:(1) it is data driven, that is, invariants are derived from data generated from concrete executions of the program, and (2) it is sound and complete, that is, the algorithm terminates with the correct invariant if it exists. The GUESS-AND-CHECK algorithm proceeds iteratively in two phases. The “guess” phase uses linear algebra techniques to efficiently derive a candidate invariant from data. This candidate invariant is subsequently validated in a “check” phase. If the check phase fails to validate the candidate invariant, then the proof of invalidity is used to generate more data and this is used to find better candidate invariants in a subsequent guess phase. Therefore, GUESS-AND-CHECK iteratively invokes the guess and check phases until it finds the desired invariant. In terms of implementation, off-the-shelf linear algebra solvers are used to implement the guess phase, and off-the-shelf decision procedures are used to implement the check phase. We have evaluated our technique on a number of benchmark programs from recent papers on invariant generation. Our results are encouraging – we are able to efficiently compute algebraic invariants in all cases.

Fuzz testing is an automated black-box testing technique providing random data as input to a software system in the hope to find vulnerability. In order to be effective, the fuzzed input must be common enough to pass elementary consistency checks. Web Browser accepts JavaScript, CSS files as well as the html as input, which must be considered in fuzzing testing, while traditional fuzzing technology generates test cases using simple mutation strategies, ignoring the grammar and code structure. In this article, vulnerability patterns are summarized and a new fuzzing testing method are proposed based on grammar analysis of input data and mutation of code structure. Combining methods of generation and mutation, test cases will be more effective in the fuzzing testing of web browsers. Applied on the Mozilla and IE web browsers, it discovered a total of 36 new severe vulnerabilities(and thus became one of the top security bug bounty collectors within this period).

To mitigate potential misinterpretation and security violations, software developers should use tools that reflect the state of web applications and visualise them as graphical models. Modelling helps to ensure that functionality and access control mechanisms are consistently interconnected. In this paper, we propose an approach to support Web application development using the Spring platform. Our proposal is supported by the Eclipse IDE plugin tool, which recognises Spring Security configuration captures, its notations, and visualises them in the role-based access control (RBAC) models. The RBAC models are represented using SecureUML modelling language. The plugin is validated through survey taken by software developers.

The features of current TV sets is increasing rapidly resulting in more complicated embedded TV software that is also harder to test and verify. Using model checking in verification of embedded software is a widely accepted practice even though it is seriously affected by the exponential increase in the number of states being produced during the verification process. Using fully non-deterministic user agent models is one of the reasons that can result in a drastic increase in the number of states being produced. In order to shrink the state space being observed during the model checking process of TV software, a method is proposed that rely on using previous test logs to generate partly nondeterministic user agent model. Results show that by using partly non-deterministic user agents, the verification time of certain safety and liveness properties can be significantly decreased.


Question: Regression testing for graphical user interfaces.

This paper introduces an innovative and secure algorithm designed for regression test selection, a critical component in software maintenance that aims to ensure modifications haven't introduced defects. The proposed technique offers a selective regression testing approach, where tests for an updated program are carefully chosen from a pre-existing test suite. Unlike conventional methods, our algorithm employs control dependence graphs for both the original and modified program versions. These graphs facilitate the identification of tests that might exhibit altered behavior when executed on the updated program. The algorithm is distinctive in its ability to select every test from the initial suite that has the potential to reveal errors in the modified program, achieving this without requiring any prior knowledge of the specific modifications made. Moreover, it is versatile enough to accommodate all programming constructs and modifications, making it an easily implementable solution in automated testing environments. This approach not only ensures thorough testing but also optimizes resource utilization by focusing on relevant test cases.

Larger display surfaces are becoming increasingly available due to multi-monitor capability built into many systems, in addition to the rapid decrease in their costs. However, little is known about the performance benefits of using these larger surfaces compared to traditional single-monitor displays. In addition, it is not clear that current software designs and interaction techniques have been properly tuned for these larger surfaces. A preliminary user study was carried out to provide some initial evidence about the benefits of large versus small display surfaces for complex, multi-application office work. Significant benefits were observed in the use of a prototype, larger display, in addition to significant positive user preference and satisfaction with its use over a small display. In addition, design guidelines for enhancing user interaction across large display surfaces were identified. User productivity could be significantly enhanced in future graphical user interface designs if developed with these findings in mind.

It is important to make mobile apps accessible, so as not to exclude users with common disabilities such as blindness, low vision, or color blindness. Even when developers are aware of these accessibility needs, the lack of tool support makes the development and assessment of accessible apps challenging. Some accessibility properties can be checked statically, but user interface widgets are often created dynamically and are not amenable to static checking. Some accessibility checking frameworks analyze accessibility properties at runtime, but have to rely on existing thorough test suites. In this paper, we introduce the idea of using automated test generation to explore the accessibility of mobile apps. We present the MATE tool (Mobile Accessibility Testing), which automatically explores apps while applying different checks for accessibility issues related to visual impairment. For each issue, MATE generates a detailed report that supports the developer in fixing the issue. Experiments on a sample of 73 apps demonstrate that MATE detects more basic accessibility problems than static analysis, and many additional types of accessibility problems that cannot be detected statically at all. Comparison with existing accessibility testing frameworks demonstrates that the independence of an existing test suite leads to the identification of many more accessibility problems. Even when enabling Android's assistive features like contrast enhancement, MATE can still find many accessibility issues.

The goal of the SLAM project is to check whether or not a program obeys "API usage rules" that specify what it means to be a good client of an API. The SLAM toolkit statically analyzes a C program to determine whether or not it violates given usage rules. The toolkit has two unique aspects: it does not require the programmer to annotate the source program (invariants are inferred); it minimizes noise (false error messages) through a process known as "counterexample-driven refinement". SLAM exploits and extends results from program analysis, model checking and automated deduction. We have successfully applied the SLAM toolkit to Windows XP device drivers, to both validate behavior and find defects in their usage of kernel APIs.

An end-user questionnaire to measure user experience quickly in a simple and immediate way while covering a preferably comprehensive impression of the product user experience was the goal of the reported construction process. An empirical approach for the item selection was used to ensure practical relevance of items. Usability experts collected terms and statements on user experience and usability, including ‘hard’ as well as ‘soft’ aspects. These statements were consolidated and transformed into a first questionnaire version containing 80 bipolar items. It was used to measure the user experience of software products in several empirical studies. Data were subjected to a factor analysis which resulted in the construction of a 26 item questionnaire including the six factors Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty. Studies conducted for the original German questionnaire and an English version indicate a satisfactory level of reliability and construct validity.

AJAX is a very promising approach for improving rich interactivity and responsiveness of web applications. At the same time, AJAX techniques shatter the metaphor of a web "page" upon which general search crawlers are based. This paper describes a novel technique for crawling AJAX applications through dynamic analysis and reconstruction of user interface state changes. Our method dynamically infers a state-flow graph modeling the various navigation paths and states within an AJAX application. This reconstructed model can be used to generate linked static pages. These pages could be used to expose AJAX sites to general search engines. Moreover, we believe that the crawling techniques that are part of our solution have other applications, such as within general search engines, accessibility improvements, or in automatically exercising all user interface elements and conducting state-based testing of AJAX applications. We present our open source tool called CRAWLJAX which implements the concepts discussed in this paper. Additionally, we report a case study in which we apply our approach to a number of representative AJAX applications and elaborate on the obtained results.

Programming foundations is not an easy subject to be taught — many students have difficulties understanding the abstract concepts of programming and have a wrong view about the programming activity. In order to address these problems, experiences have suggested the integrated teaching of programming concepts and software testing in introductory CS courses. Shortly, the idea is that testing can contribute to enhance the students' capabilities of understanding and analysis. However, such perspective requires tools to provide an adequate feedback to evaluate the students' performance concerning programming and testing activities. In this paper we describe ProgTest — a web-based tool for the submission and automatic evaluation of practical programming assignments based on testing activities. Results from a preliminary validation of ProgTest are also presented. Such results provide evidences on the practical use of ProgTest as a supporting mechanism for the integrated teaching of programming foundations and software testing.

A useful starting point for designing advanced graphical user interfaces is the Visual InformationSeeking Mantra: Overview first, zoom and filter, then details-on-demand. But this is only a starting point in trying to understand the rich and varied set of information visualizations that have been proposed in recent years. This paper offers a task by data type taxonomy with seven data types (1-, 2-, 3-dimensional data, temporal and multi-dimensional data, and tree and network data) and seven tasks (overview, zoom, filter, details-on-demand, relate, history, and extract).

The paper presents, and discusses the rationale behind, a method for structuring complex computing systems by the use of what we term “recovery blocks”, “conversations” and “fault-tolerant interfaces”. The aim is to facilitate the provision of dependable error detection and recovery facilities which can cope with errors caused by residual design inadequacies, particularly in the system software, rather than merely the occasional malfunctioning of hardware components.

R<scp>ANDOOP</scp> for Java generates unit tests for Java code using feedback-directed random test generation. Below we describe R<scp>ANDOOP</scp>'s input, output, and test generation algorithm. We also give an overview of RANDOOP's annotation-based interface for specifying configuration parameters that affect R<scp>ANDOOP</scp>'s behavior and output.

This text is a general, self contained, and tool independent introduction into the Java Modeling Language, JML. It appears in a book about the KeY approach and tool, because JML is the dominating starting point of KeY style Java verification. However, this chapter does not depend on KeY, nor any other specific tool, nor on any specific verification methodology. With this text, the authors aim to provide, for the time being, the definitive, general JML tutorial. Other chapters in this book discuss the particular usage of JML in KeY style verification.1 In this chapter, however, we only refer to KeY in very few places, without relying on it. This introduction is written for all readers with an interest in formal specification of software in general, and anyone who wants to learn about the JML approach to specification in particular. A preliminary version of this chapter appeared as a technical report [Huisman et al., 2014].


Question: Regression testing for graphical user interfaces.

This paper provides an overview of least commitment planning, a method for creating partially ordered, partially specified sequences of actions aimed at achieving an agent's objectives. We trace the evolution of least commitment planners, beginning with a basic version that works with the simple STRIPS representation and culminating with a more advanced system capable of handling actions with disjunctive preconditions, conditional effects, and universal quantification across dynamic domains. We clarify misunderstandings regarding Chapman's interpretation of the Modal Truth Criterion and demonstrate why his NP-completeness finding for reasoning about plans with conditional effects does not apply to our planner. Throughout this work, we are grateful for the insights provided by Franz Amador, Tony Barrett, Darren Cronquist, Denise Draper, Ernie Davis, Oren Etzioni, Nort Fowler, Rao Kambhampati, Craig Knoblock, Nick Kushmerick, Neal Lesh, Karen Lochbaum, Drew McDermott, Ramesh Patil, Kari Pulli, Ying Sun, Austin Tate, and Mike Williamson, while acknowledging sole responsibility for any errors. This research was partly supported by the Office of Naval Research Grant 90-J-1904 and the National Science Foundation Grant IRI-8957302.

**Question**: Regression testing of GUIs

**Documents**: Automated Regression Testing for Graphical User Interfaces: An Overview
Regression testing is an essential part of software maintenance, ensuring that changes or updates to a software system do not adversely affect its existing functionalities. When dealing with Graphical User Interfaces (GUIs), regression testing becomes particularly challenging due to the dynamic and interactive nature of GUIs. This paper provides an overview of automated regression testing techniques specifically designed for GUIs. We discuss various approaches, including model-based testing, visual testing, and script-based testing, and evaluate their effectiveness and limitations. The paper also highlights the role of machine learning and artificial intelligence in enhancing GUI regression testing. We conclude with future research directions and open challenges in the field of GUI regression testing.

**Similar Documents**: Automated Regression Testing for Graphical User Interfaces: A Comprehensive Review
Regression testing plays a critical role in software maintenance by verifying that modifications to a software system do not introduce defects in previously working features, especially in the context of Graphical User Interfaces (GUIs). GUIs, with their interactive and dynamic elements, present unique challenges for regression testing. This review paper examines automated regression testing methods tailored for GUIs, exploring different strategies such as model-based testing, visual testing, and script-based testing

0 7 4 0 7 4 5 9 / 0 3 / $ 1 9 . 0 0 © 2 0 0 3 I E E E All the methodologies that have been proposed recently for choosing software packages compare user requirements with the packages’ capabilities.3–5 There are different types of requirements, such as managerial, political, and, of course, quality requirements. Quality requirements are often difficult to check. This is partly due to their nature, but there is another reason that can be mitigated, namely the lack of structured and widespread descriptions of package domains (that is, categories of software packages such as ERP systems, graphical or data structure libraries, and so on). This absence hampers the accurate description of software packages and the precise statement of quality requirements, and consequently overall package selection and confidence in the result of the process. Our methodology for building structured quality models helps solve this drawback. (See the “Related Work” sidebar for information about other approaches.) A structured quality model for a given package domain provides a taxonomy of software quality features, and metrics for computing their value. Our approach relies on the International Organization for Standardization and International Electrotechnical Commission 9126-1 quality standard,6 which we selected for the following reasons:

The features of current TV sets is increasing rapidly resulting in more complicated embedded TV software that is also harder to test and verify. Using model checking in verification of embedded software is a widely accepted practice even though it is seriously affected by the exponential increase in the number of states being produced during the verification process. Using fully non-deterministic user agent models is one of the reasons that can result in a drastic increase in the number of states being produced. In order to shrink the state space being observed during the model checking process of TV software, a method is proposed that rely on using previous test logs to generate partly nondeterministic user agent model. Results show that by using partly non-deterministic user agents, the verification time of certain safety and liveness properties can be significantly decreased.

Developers often run tests to check that their latest changes to a code repository did not break any previously working functionality. Ideally, any new test failures would indicate regressions caused by the latest changes. However, some test failures may not be due to the latest changes but due to non-determinism in the tests, popularly called flaky tests. The typical way to detect flaky tests is to rerun failing tests repeatedly. Unfortunately, rerunning failing tests can be costly and can slow down the development cycle.
 We present the first extensive evaluation of rerunning failing tests and propose a new technique, called DeFlaker, that detects if a test failure is due to a flaky test without rerunning and with very low runtime overhead. DeFlaker monitors the coverage of latest code changes and marks as flaky any newly failing test that did not execute any of the changes. We deployed DeFlaker live, in the build process of 96 Java projects on TravisCI, and found 87 previously unknown flaky tests in 10 of these projects. We also ran experiments on project histories, where DeFlaker detected 1, 874 flaky tests from 4, 846 failures, with a low false alarm rate (1.5%). DeFlaker had a higher recall (95.5% vs. 23%) of confirmed flaky tests than Maven's default flaky test detector.

There is increasing evidence that the effectiveness of information visualization techniques can be impacted by the particular needs and abilities of each user. This suggests that it is important to investigate information visualization systems that can dynamically adapt to each user. In this paper, we address the question of how to adapt. In particular, we present a study to evaluate a variety of visual prompts, called "interventions", that can be performed on a visualization to help users process it. Our results show that some of the tested interventions perform better than a condition in which no intervention is provided, both in terms of task performance as well as subjective user ratings. We also discuss findings on how intervention effectiveness is influenced by individual differences and task complexity.

In object oriented programming, the functionalities of a system result from the interactions (message exchanges) among the objects allocated by the system. While designing object interactions is far more complex than designing the object structure in forward engineering, the problem of understanding object interactions during code evolution is even harder, because the related information is spread across the code. In this paper, a technique for the automatic extraction of UML interaction diagrams from C++ code is proposed. The algorithm is based on a static, conservative flow analysis, that approximates the behavior of the system in any execution and for any possible input. Applicability of the approach to large software is achieved by means of two mechanisms: partial analysis and focusing. Usage of our method on a real world, large C++ system confirmed its viability.

The use of automatic static analysis has been a software engineering best practice for decades. However, we still do not know a lot about its use in real-world software projects: How prevalent is the use of Automated Static Analysis Tools (ASATs) such as FindBugs and JSHint? How do developers use these tools, and how does their use evolve over time? We research these questions in two studies on nine different ASATs for Java, JavaScript, Ruby, and Python with a population of 122 and 168,214 open-source projects. To compare warnings across the ASATs, we introduce the General Defect Classification (GDC) and provide a grounded-theory-derived mapping of 1,825 ASAT-specific warnings to 16 top-level GDC classes. Our results show that ASAT use is widespread, but not ubiquitous, and that projects typically do not enforce a strict policy on ASAT use. Most ASAT configurations deviate slightly from the default, but hardly any introduce new custom analyses. Only a very small set of default ASAT analyses is widely changed. Finally, most ASAT configurations, once introduced, never change. If they do, the changes are small and have a tendency to occur within one day of the configuration's initial introduction.

User interface (UI) interactions are essential to Android applications, as many Activities require UI interactions to be triggered. This kind of UI interactions could also help malicious apps to hide their sensitive behaviors (e.g., sending SMS or getting the user's device ID) from being detected by dynamic analysis tools such as TaintDroid, because simply running the app, but without proper UI interactions, will not lead to the exposure of sensitive behaviors. In this paper we focus on the challenging task of triggering a certain behavior through automated UI interactions. In particular, we propose a hybrid static and dynamic analysis method to reveal UI-based trigger conditions in Android applications. Our method first uses static analysis to extract expected activity switch paths by analyzing both Activity and Function Call Graphs, and then uses dynamic analysis to traverse each UI elements and explore the UI interaction paths towards the sensitive APIs. We implement a prototype system SmartDroid and show that it can automatically and efficiently detect the UI-based trigger conditions required to expose the sensitive behavior of several Android malwares, which otherwise cannot be detected with existing techniques such as TaintDroid.

Anomaly detection systems largely depend on user profile data to be able to detect deviations from normal activity. Most of this profile data is currently based on command-line instructions/directives executed by users on a system. With the advent and extensive usage of graphical user interfaces (GUIs), command-line data can no longer fully represent user's complete behavior which is essential for effectively detecting the anomalies in these GUI based systems. Collection of user behavior data is a slow and time consuming process. In this paper, we present a new approach to automate the generation of user data by parameterizing user behavior in terms of user intention (malicious/normal), user skill level, set of applications installed on a machine, mouse movement and keyboard activity. The user behavior parameters are used to generate templates, which can be further customized. The framework is called USim which can achieve rapid generation of user behavior data based on these templates for GUI based systems. The data thus generated can be utilized for rapidly training and testing intrusion detection systems (IDSes) and improving their detection precision.

The adoption of smart phones and the usages of mobile applications are increasing rapidly. Consequently, within limited time-range, mobile Internet usages have managed to take over the desktop usages particularly since the first smart phone-touched application released by iPhone in 2007. This paper is proposed to provide solution and answer the most demandable questions related to mobile application automated and manual testing limitations. Moreover, Mobile application testing requires agility and physically testing. Agile testing is to detect bugs through automated tools, whereas the compatibility testing is more to ensure that the apps operates on mobile OS (Operation Systems) as well as on the different real devices. Moreover, we have managed to answer automated or manual questions through two mobile application case studies MES (Mobile Exam System) and MLM (Mobile Lab Mate) by creating test scripts for both case studies and our experiment results have been discussed and evaluated on whether to adopt test on real devices or on emulators? In addition to this, we have introduced new mobile application testing matrix for the testers and some enterprises to obtain knowledge from.

The interplay of multiple objects in object-oriented programming often follows specific protocols, for example certain orders of method calls and/or control structure constraints among them that are parts of the intended object usages. Unfortunately, the information is not always documented. That creates long learning curve, and importantly, leads to subtle problems due to the misuse of objects.
 In this paper, we propose GrouMiner, a novel graph-based approach for mining the usage patterns of one or multiple objects. GrouMiner approach includes a graph-based representation for multiple object usages, a pattern mining algorithm, and an anomaly detection technique that are efficient, accurate, and resilient to software changes. Our experiments on several real-world programs show that our prototype is able to find useful usage patterns with multiple objects and control structures, and to translate them into user-friendly code skeletons to assist developers in programming. It could also detect the usage anomalies that caused yet undiscovered defects and code smells in those programs.


Question: Cross-domain feature extraction for language identification task.

This study undertakes a detailed examination of two prominent event models employed in naive Bayes text classification, aiming to elucidate their distinctions and evaluate their performance. The first model is the multi-variate Bernoulli, which operates under the assumption of no dependencies between words and utilizes binary word features, as demonstrated in the work of Larkey and Croft (1996) and Koller and Sahami (1997). The second model is the multinomial model, akin to a uni-gram language model that incorporates integer word counts, as exemplified by Lewis and Gale (1994) and Mitchell (1997). Our analysis is designed to address the existing confusion around these models by comparing their classification efficacy on five distinct text corpora. The findings reveal that the multi-variate Bernoulli model exhibits superior performance with smaller vocabulary sizes. However, as the vocabulary size increases, the multinomial model consistently outperforms its counterpart, achieving, on average, a 27% reduction in error rate at any given vocabulary size. This comparative assessment provides valuable insights into the strengths and weaknesses of these models under varying conditions, offering guidance for practitioners in selecting the most appropriate model for their specific text classification tasks.

Domain mismatch between training and testing can lead to significant degradation in performance in many machine learning scenarios. Unfortunately, this is not a rare situation for automatic speech recognition deployments in real-world applications. Research on robust speech recognition can be regarded as trying to overcome this domain mismatch issue. In this paper, we address the unsupervised domain adaptation problem for robust speech recognition, where both source and target domain speech are available, but word transcripts are only available for the source domain speech. We present novel augmentation-based methods that transform speech in a way that does not change the transcripts. Specifically, we first train a variational autoencoder on both source and target domain data (without supervision) to learn a latent representation of speech. We then transform nuisance attributes of speech that are irrelevant to recognition by modifying the latent representations, in order to augment labeled training data with additional data whose distribution is more similar to the target domain. The proposed method is evaluated on the CHiME-4 dataset and reduces the absolute word error rate (WER) by as much as 35% compared to the non-adapted baseline.

In this report, the results from my CS224D final project are given and explained. The project was based on the application of relatively new neural network architectures, namely convolutional neural networks over word embeddings, to the task of authorship identification. The problem was posed as a classification task and models were evaluated over two datasets, a baseline of my own collection and a competition dataset with pre-existing, published results. Despite robust results on the baseline dataset, the CNN architecture failed to outmatch the best competition submission, although it did outscore most of the competitors’ submissions.

In this paper, we propose an odor recognition framework for multiple electronic noses (E-noses), machine olfaction odor perception systems. Straight to the point, the proposed transferring odor recognition model is called cross-domain discriminative subspace learning (CDSL). General odor recognition problems with E-nose are single domain oriented, that is, recognition algorithms are often modeled and tested on the same one domain data set (i.e., from only one E-nose system). Different from that, we focus on a more realistic scenario: the recognition model is trained on a prepared source domain data set from a master E-nose system <inline-formula> <tex-math notation="LaTeX">${A}$ </tex-math></inline-formula>, but tested on another target domain data set from a slave system <inline-formula> <tex-math notation="LaTeX">${B}$ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">${C}$ </tex-math></inline-formula> with the same type of the master system <inline-formula> <tex-math notation="LaTeX">${A}$ </tex-math></inline-formula>. The internal device parameter variance between master and slave systems often results in data distribution discrepancy between source domain and target domain, such that single-domain-based odor recognition model may not be adapted to another domain. Therefore, we propose domain-adaptation-based odor recognition for addressing the realistic recognition scenario across systems. Specifically, the proposed CDSL method consists of three merits: 1) an intraclass scatter minimization- and an interclass scatter maximization-based discriminative subspace learning is solved on source domain; 2) a data fidelity and preservation constraint of the subspace is imposed on target domain without distortion; and 3) a minipatch feature weighted domain distance is minimized for closely connecting the source and target domains. Experiments and comparisons on odor recognition tasks in multiple E-noses demonstrate the efficiency of the proposed method.

Script based features are highly discriminative for text segmentation and recognition. Thus they are widely used in Optical Character Recognition(OCR) problems. But usage of script dependent features restricts the adaptation of such architectures directly for another script. With script independent systems, this problem can be solved to a certain extent for monolingual documents. But the problem aggravates in case of multilingual documents as it is very difficult for a single classifier to learn many scripts. Generally a script identification module identifies text segments and accordingly the script-dependent classifier is selected. This paper presents a unified framework of language model and multiple preprocessing hypotheses for word recognition from bilingual document images. Prior to text recognition, preprocessing steps such as binarization and segmentation are required for ease of recognition. But these steps induce huge combinatorial error propagating to final recognition accuracy. In this paper we use multiple preprocessing routines as alternate hypotheses and use a language model to verify each alternative and choose the best recognized sequence. We test this architecture for word recognition of Kannada-English and Telugu-English bilingual documents and achieved better recognition rates than single methods using same classifier.

This paper proposes an algorithm for the recognition and separation of speech signals in non-stationary noise, such as another speaker. We present a method to combine hidden Markov models (HMMs) trained for the speech and noise into a factorial HMM to model the mixture signal. Robustness is obtained by separating the speech and noise signals in a feature domain, which discards unnecessary information. We use mel-cepstral coefficients (MFCCs) as features, and estimate the distribution of mixture MFCCs from the distributions of the target speech and noise. A decoding algorithm is proposed for finding the state transition paths and estimating gains for the speech and noise from a mixture signal. Simulations were carried out using speech material where two speakers were mixed at various levels, and even for high noise level (9 dB above the speech level), the method produced relatively good (60% word recognition accuracy) results. Audio demonstrations are available at www.cs.tut.fi/ ̃tuomasv.

An adaptive statistical language model is described, which successfully integrates long distance linguistic information with other knowledge sources. Most existing statistical language models exploit only the immediate history of a text. To extract information from further back in the document’s history, we propose and use trigger pairs as the basic information bearing elements. This allows the model to adapt its expectations to the topic of discourse. Next, statistical evidence from multiple sources must be combined. Traditionally, linear interpolation and its variants have been used, but these are shown here to be seriously deficient. Instead, we apply the principle of Maximum Entropy (ME). Each information source gives rise to a set of constraints, to be imposed on the combined estimate. The intersection of these constraints is the set of probability functions which are consistent with all the information sources. The function with the highest entropy within that set is the ME solution. Given consistent statistical evidence, a unique ME solution is guaranteed to exist, and an iterative algorithm exists which is guaranteed to converge to it. The ME framework is extremely general: any phenomenon that can be described in terms of statistics of the text can be readily incorporated. An adaptive language model based on the ME approach was trained on the Wall Street Journal corpus, and showed a 32–39% perplexity reduction over the baseline. When interfaced to SPHINX-II, Carnegie Mellon’s speech recognizer, it reduced its error rate by 10–14%. This thus illustrates the feasibility of incorporating many diverse knowledge sources in a single, unified statistical framework.  1996 Academic Press Limited

Engineering automatic speech recognition (ASR) for speech to speech (S2S) translation systems, especially targeting languages and domains that do not have readily available spoken language resources, is immensely challenging due to a number of reasons. In addition to contending with the conventional data-hungry speech acoustic and language modeling needs, these designs have to accommodate varying requirements imposed by the domain needs and characteristics, target device and usage modality (such as phrase-based, or spontaneous free form interactions, with or without visual feedback) and huge spoken language variability arising due to socio-linguistic and cultural differences of the users. This paper, using case studies of creating speech translation systems between English and languages such as Pashto and Farsi, describes some of the practical issues and the solutions that were developed for multilingual ASR development. These include novel acoustic and language modeling strategies such as language adaptive recognition, active-learning based language modeling, class-based language models that can better exploit resource poor language data, efficient search strategies, including N-best and confidence generation to aid multiple hypotheses translation, use of dialog information and clever interface choices to facilitate ASR, and audio interface design for meeting both usability and robustness requirements

of “Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing” by David McClosky , Ph.D., Brown University, May, 2010. Current efforts in syntactic parsing are largely data-driv en. These methods require labeled examples of syntactic structures to learn statistical patterns governing these structures. Labeled data typically requires expert annotators which makes it both time consuming and costly to p roduce. Furthermore, once training data has been created for one textual domain, portability to similar domains is limited. This domain-dependence has inspired a large body of work since syntactic parsing aims to capture syntactic patterns across an entire language rather than just a specific domain. The simplest approach to this task is to assume that the targe t domain is essentially the same as the source domain. No additional knowledge about the target domain is i cluded. A more realistic approach assumes that only raw text from the target domain is available. This a s umption lends itself well to semi-supervised learning methods since these utilize both labeled and unlab eled examples. This dissertation focuses on a family of semi-supervised me thods called self-training. Self-training creates semi-supervised learners from existing supervised learne rs with minimal effort. We first show results on self-training for constituency parsing within a single dom ain. While self-training has failed here in the past, we present a simple modification which allows it to succeed, p roducing state-of-the-art results for English constituency parsing. Next, we show how self-training is be neficial when parsing across domains and helps further when raw text is available from the target domain. On e of the remaining issues is that one must choose a training corpus appropriate for the target domain or perfo rmance may be severely impaired. Humans can do this in some situations, but this strategy becomes less prac tical as we approach larger data sets. We present a technique, Any Domain Parsing, which automatically detect s useful source domains and mixes them together to produce a customized parsing model. The resulting models perform almost as well as the best seen parsing models (oracle) for each target domain. As a result, we have a fully automatic syntactic constituency parser which can produce high-quality parses for all types of text, regardless of domain. Abstract of “Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing” by David McClosky , Ph.D., Brown University, May, 2010.of “Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing” by David McClosky , Ph.D., Brown University, May, 2010. Current efforts in syntactic parsing are largely data-driv en. These methods require labeled examples of syntactic structures to learn statistical patterns governing these structures. Labeled data typically requires expert annotators which makes it both time consuming and costly to p roduce. Furthermore, once training data has been created for one textual domain, portability to similar domains is limited. This domain-dependence has inspired a large body of work since syntactic parsing aims to capture syntactic patterns across an entire language rather than just a specific domain. The simplest approach to this task is to assume that the targe t domain is essentially the same as the source domain. No additional knowledge about the target domain is i cluded. A more realistic approach assumes that only raw text from the target domain is available. This a s umption lends itself well to semi-supervised learning methods since these utilize both labeled and unlab eled examples. This dissertation focuses on a family of semi-supervised me thods called self-training. Self-training creates semi-supervised learners from existing supervised learne rs with minimal effort. We first show results on self-training for constituency parsing within a single dom ain. While self-training has failed here in the past, we present a simple modification which allows it to succeed, p roducing state-of-the-art results for English constituency parsing. Next, we show how self-training is be neficial when parsing across domains and helps further when raw text is available from the target domain. On e of the remaining issues is that one must choose a training corpus appropriate for the target domain or perfo rmance may be severely impaired. Humans can do this in some situations, but this strategy becomes less prac tical as we approach larger data sets. We present a technique, Any Domain Parsing, which automatically detect s useful source domains and mixes them together to produce a customized parsing model. The resulting models perform almost as well as the best seen parsing models (oracle) for each target domain. As a result, we have a fully automatic syntactic constituency parser which can produce high-quality parses for all types of text, regardless of domain. Any Domain Parsing: Automatic Domain Adaptation for Natural Language Parsing by David McClosky B. S., University of Rochester, 2004 Sc. M., Brown University, 2006 A dissertation submitted in partial fulfillment of the requirements for the Degree of Doctor of Philosophy in the Department of Computer Science at Brown University. Providence, Rhode Island May, 2010 c © Copyright 2010 by David McClosky This dissertation by David McClosky is accepted in its prese nt form by the Department of Computer Science as satisfying the disser tation requirement for the degree of Doctor of Philosophy. Date Eugene Charniak, Director Recommended to the Graduate Council Date Mark Johnson, Reader Cognitive and Linguistic Sciences Date Dan Klein, Reader University of California at Berkeley Approved by the Graduate Council Date Sheila Bond Dean of the Graduate School

The extraction and selection of acoustic features are crucial steps in the development of a system for classifying emotions in speech. Most works in the field use some kind of prosodic features, often in combination with spectral and glottal features, and select appropriate features in classifying emotions. In the methods, feature choices are mostly made regardless of existing relationships and structures between features. However, considering them can be beneficial, potentially both for interpretability and to improve classification performance. To this end, a structured sparse logistic regression model incorporated with the hierarchical structure of features derived from prosody, spectral envelope, and glottal information is proposed in this paper. The proposed model simultaneously addresses tree-structured sparse feature selection and emotion classification. Evaluation of the proposed model on Berlin emotional database showed substantial improvement over the conventional sparse logistic regression model.

State-of-the-art methods for learning cross-lingual word embeddings have relied on bilingual dictionaries or parallel corpora. Recent studies showed that the need for parallel data supervision can be alleviated with character-level information. While these methods showed encouraging results, they are not on par with their supervised counterparts and are limited to pairs of languages sharing a common alphabet. In this work, we show that we can build a bilingual dictionary between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way. Without using any character information, our model even outperforms existing supervised methods on cross-lingual tasks for some language pairs. Our experiments demonstrate that our method works very well also for distant language pairs, like English-Russian or EnglishChinese. We finally describe experiments on the English-Esperanto low-resource language pair, on which there only exists a limited amount of parallel data, to show the potential impact of our method in fully unsupervised machine translation. Our code, embeddings and dictionaries are publicly available1.


Question: Cross-domain feature selection technique for language identification tasks.

This comprehensive study delves into the comparative evaluation of twelve feature selection methods, including Information Gain, for text classification tasks. Central to document categorization, news filtering, document routing, and personalization systems, machine learning for text classification significantly benefits from effective feature selection to enhance efficiency and accuracy. Using a benchmark of 229 text classification problem instances sourced from reputable datasets such as Reuters, TREC, and OHSUMED, this paper scrutinizes these methods through multiple lenses—accuracy, F-measure, precision, and recall—to cater to diverse situational needs. Among the methods, a novel feature selection metric, 'Bi-Normal Separation' (BNS), emerges as a standout performer, consistently outdoing others in most scenarios, particularly in tasks with pronounced class skew, a common challenge in text classification that often taxes induction algorithms. The study introduces a novel evaluation approach tailored to the practical needs of data mining practitioners working with a singular dataset and seeking optimal metrics. Under this perspective, BNS proves to be the superior single choice for all performance goals except precision, where Information Gain excels. The analysis also uncovers that Information Gain and Chi-Squared exhibit correlated failures, rendering them a suboptimal combination. When selecting optimal pairs of metrics for each of the four performance criteria, BNS invariably forms one half of the optimal pair—illustratively, for maximizing recall, the BNS + F1-measure pairing delivers the best performance on the highest number of tasks by a significant margin.

This paper describes discriminative language modeling for a large vocabulary speech recognition task. We contrast two parameter estimation methods: the perceptron algorithm, and a method based on maximizing the regularized conditional log-likelihood. The models are encoded as deterministic weighted finite state automata, and are applied by intersecting the automata with word-lattices that are the output from a baseline recognizer. The perceptron algorithm has the benefit of automatically selecting a relatively small feature set in just a couple of passes over the training data. We describe a method based on regularized likelihood that makes use of the feature set given by the perceptron algorithm, and initialization with the perceptron’s weights; this method gives an additional 0.5% reduction in word error rate (WER) over training with the perceptron alone. The final system achieves a 1.8% absolute reduction in WER for a baseline first-pass recognition system (from 39.2% to 37.4%), and a 0.9% absolute reduction in WER for a multi-pass recognition system (from 28.9% to 28.0%). 2006 Elsevier Ltd. All rights reserved.

In this paper we describe a work in progress on designing the continuous vector space word representations able to map unseen data adequately. We propose a LSTM-based feature extraction layer that reads in a sequence of characters corresponding to a word and outputs a single fixed-length real-valued vector. We then test our model on a POS tagging task on four typologically different languages. The results of the experiments suggest that the model can offer a solution to the out-of-vocabulary words problem, as in a comparable setting its OOV accuracy improves over that of a state of the art tagger.

We explore efficient domain adaptation for the task of statistical machine translation based on extracting sentences from a large generaldomain parallel corpus that are most relevant to the target domain. These sentences may be selected with simple cross-entropy based methods, of which we present three. As these sentences are not themselves identical to the in-domain data, we call them pseudo in-domain subcorpora. These subcorpora – 1% the size of the original – can then used to train small domain-adapted Statistical Machine Translation (SMT) systems which outperform systems trained on the entire corpus. Performance is further improved when we use these domain-adapted models in combination with a true in-domain model. The results show that more training data is not always better, and that best results are attained via proper domain-relevant data selection, as well as combining inand general-domain systems during decoding.

Recently there has been signi cant interest in supervised learning algorithms that combine labeled and unlabeled data for text learning tasks. The co-training setting [1] applies to datasets that have a natural separation of their features into two disjoint sets. We demonstrate that when learning from labeled and unlabeled data, algorithms explicitly leveraging a natural independent split of the features outperform algorithms that do not. When a natural split does not exist, co-training algorithms that manufacture a feature split may out-perform algorithms not using a split. These results help explain why co-training algorithms are both discriminative in nature and robust to the assumptions of their embedded classi ers.

While several methods for automatic essay scoring (AES) for the English language have been proposed, systems for other languages are unusual. To this end, we propose in this paper a multi-aspect AES system for Brazilian Portuguese which we apply to a collection of essays, which human experts evaluated according to the five aspects defined by the Brazilian Government for the National High School Exam (ENEM). These aspects are skills that student must master and every skill is assessed separately from one another. In addition to prediction, we also performed feature analysis for each aspect. The proposed AES system employs several features already used by AES systems for the English language. Our results show that predictions for some aspects performed well with the employed features, while predictions for other aspects performed poorly. Furthermore, the detailed feature analysis we performed made it possible to note their independent impacts on each of the five aspects. Finally, aside from these contributions, our work reveals some challenges and directions for future research, related, for instance, to the fact that the ENEM has over eight million yearly enrollments.

We present a paradigm for the automatic assessment of pronunciation quality by machine. In this scoring paradigm, both native and nonnative speech data is collected and a database of human-expert ratings is created to enable the development of a variety of machine scores. We ®rst discuss issues related to the design of speech databases and the reliability of human ratings. We then address pronunciation evaluation as a prediction problem, trying to predict the grade a human expert would assign to a particular skill. Using the speech and the expert-ratings databases, we build statistical models and introduce dierent machine scores that can be used as predictor variables. We validate these machine scores on the Voice Interactive Language Training System (VILTS) corpus, evaluating the pronunciation of American speakers speaking French and we show that certain machine scores, like the log-posterior and the normalized duration, achieve a correlation with the targeted human grades that is comparable to the human-to-human correlation when a sucient amount of speech data is available. Ó 2000 Elsevier Science B.V. All rights reserved.

In this paper, we show that local features computed from the derivations of tree substitution grammars — such as the identify of particular fragments, and a count of large and small fragments — are useful in binary grammatical classification tasks. Such features outperform n-gram features and various model scores by a wide margin. Although they fall short of the performance of the hand-crafted feature set of Charniak and Johnson (2005) developed for parse tree reranking, they do so with an order of magnitude fewer features. Furthermore, since the TSGs employed are learned in a Bayesian setting, the use of their derivations can be viewed as the automatic discovery of tree patterns useful for classification. On the BLLIP dataset, we achieve an accuracy of 89.9% in discriminating between grammatical text and samples from an n-gram language model.

The paper systematically compares two feature extraction algorithms to mine product features commented on in customer reviews. The first approach [17] identifies candidate features by applying a set of POS patterns and pruning the candidate set based on the log likelihood ratio test. The second approach [11] applies association rule mining for identifying frequent features and a heuristic based on the presence of sentiment terms for identifying infrequent features. We evaluate the performance of the algorithms on five product specific document collections regarding consumer electronic devices. We perform an analysis of errors and discuss advantages and limitations of the algorithms.

The efficacy of syntactic features for topic-independent authorship attribution is evaluated, taking a feature set of frequencies of words and punctuation marks as baseline. The features are ‘deep’ in the sense that they are derived by parsing the subject texts, in contrast to ‘shallow’ syntactic features for which a part-of-speech analysis is enough. The experiments are conducted on a corpus of novels written around the year 1900 by 20 different authors, and cover two tasks. In the first task, text samples are taken from books by one author, and the goal is to pair samples from the same book. In the second task, text samples are taken from several authors, but only one sample from each book, and the goal is to pair samples from the same author. In the first task, the baseline feature set outperformed the syntax-based feature set, but for the second task, the outcome was the opposite. This suggests that, compared to lexical features such as vocabulary and punctuation, syntactic features are more robust to changes in topic.

Speech datasets and corpora are crucial for both developing and evaluating accurate Natural Language Processing systems. While Modern Standard Arabic has received more attention, dialects are drastically underestimated, even they are the most used in our daily life and the social media, recently. In this paper, we present the methodology of building an Arabic Speech Corpus for Algerian dialects, and the preliminary version of that dataset of dialectal arabic speeches uttered by Algerian native speakers selected from different Algeria’s departments. In fact, by means of a direct recording way, we have taken into acount numerous aspects that foster the richness of the corpus and that provide a representation of phonetic, prosodic and orthographic varieties of Algerian dialects. Among these considerations, we have designed a rich speech topics and content. The annotations provided are some useful information related to the speakers, time-aligned orthographic word transcription. Many potential uses can be considered such as speaker/dialect identification and computational linguistic for Algerian sub-dialects. In its preliminary version, our corpus encompasses 17 sub-dialects with 109 speakers and more than 6 K utterances.


Question: A feature selection method for cross-domain language identification.

In the realm of language identification, a pivotal study has delved into the learnability of languages, specifically focusing on a scenario where a learner is tasked with identifying an unknown language from a predefined class, based on the information provided. This investigation explores whether the given information is adequate to ascertain the identity of the unknown language among the possible candidates. The study employs a specific definition of learnability, characterized by discrete time intervals with a clear starting point. At each time interval, the learner is presented with a unit of information and is required to conjecture the identity of the unknown language, based on the cumulative information received. This iterative process continues indefinitely. A class of languages is deemed learnable with respect to the outlined information presentation method if there exists an algorithm that the learner can employ to formulate guesses, with the algorithm possessing the critical property that, for any language within the class, there exists a finite time threshold after which all subsequent guesses are identical and correct.

In this foundational exploration, a language is conceived as a collection of strings over a finite alphabet, with the alphabet being consistent across all languages within the class. The investigation scrutinizes two fundamental methods of information presentation, along with their variants: the text method and the informant method. Under the text method, a text for a language generates the strings of the language in any order, ensuring that every string appears at least once. Conversely, under the informant method, an informant reveals whether a string is part of the language, selecting the strings in a particular order to guarantee that every string is encountered at least once. The study reveals that the class of context-sensitive languages is learnable from an informant, whereas even the class of regular languages is not learnable from a text.

This seminal work in the domain of language identification underscores the nuanced relationship between the type of information presentation and the learnability of language classes, paving the way for further advancements in the field.

This paper examines the impact of multilingual (ML) acoustic representations on Automatic Speech Recognition (ASR) and keyword search (KWS) for low resource languages in the context of the OpenKWS15 evaluation of the IARPA Babel program. The task is to develop Swahili ASR and KWS systems within two weeks using as little as 3 hours of transcribed data. Multilingual acoustic representations proved to be crucial for building these systems under strict time constraints. The paper discusses several key insights on how these representations are derived and used. First, we present a data sampling strategy that can speed up the training of multilingual representations without appreciable loss in ASR performance. Second, we show that fusion of diverse multilingual representations developed at different LORELEI sites yields substantial ASR and KWS gains. Speaker adaptation and data augmentation of these representations improves both ASR and KWS performance (up to 8.7% relative). Third, incorporating un-transcribed data through semi-supervised learning, improves WER and KWS performance. Finally, we show that these multilingual representations significantly improve ASR and KWS performance (relative 9% for WER and 5% for MTWV) even when forty hours of transcribed audio in the target language is available. Multilingual representations significantly contributed to the LORELEI KWS systems winning the OpenKWS15 evaluation.

In this paper, we show that local features computed from the derivations of tree substitution grammars — such as the identify of particular fragments, and a count of large and small fragments — are useful in binary grammatical classification tasks. Such features outperform n-gram features and various model scores by a wide margin. Although they fall short of the performance of the hand-crafted feature set of Charniak and Johnson (2005) developed for parse tree reranking, they do so with an order of magnitude fewer features. Furthermore, since the TSGs employed are learned in a Bayesian setting, the use of their derivations can be viewed as the automatic discovery of tree patterns useful for classification. On the BLLIP dataset, we achieve an accuracy of 89.9% in discriminating between grammatical text and samples from an n-gram language model.

We investigate the potential of Context-Dependent Deep-Neural-Network HMMs, or CD-DNN-HMMs, from a feature-engineering perspective. Recently, we had shown that for speaker-independent transcription of phone calls (NIST RT03S Fisher data), CD-DNN-HMMs reduced the word error rate by as much as one third—from 27.4%, obtained by discriminatively trained Gaussian-mixture HMMs with HLDA features, to 18.5%—using 300+ hours of training data (Switchboard), 9000+ tied triphone states, and up to 9 hidden network layers.

In this paper, we propose a new framework called domain adaptation machine (DAM) for the multiple source domain adaption problem. Under this framework, we learn a robust decision function (referred to as target classifier) for label prediction of instances from the target domain by leveraging a set of base classifiers which are prelearned by using labeled instances either from the source domains or from the source domains and the target domain. With the base classifiers, we propose a new domain-dependent regularizer based on smoothness assumption, which enforces that the target classifier shares similar decision values with the relevant base classifiers on the unlabeled instances from the target domain. This newly proposed regularizer can be readily incorporated into many kernel methods (e.g., support vector machines (SVM), support vector regression, and least-squares SVM (LS-SVM)). For domain adaptation, we also develop two new domain adaptation methods referred to as FastDAM and UniverDAM. In FastDAM, we introduce our proposed domain-dependent regularizer into LS-SVM as well as employ a sparsity regularizer to learn a sparse target classifier with the support vectors only from the target domain, which thus makes the label prediction on any test instance very fast. In UniverDAM, we additionally make use of the instances from the source domains as Universum to further enhance the generalization ability of the target classifier. We evaluate our two methods on the challenging TRECIVD 2005 dataset for the large-scale video concept detection task as well as on the 20 newsgroups and email spam datasets for document retrieval. Comprehensive experiments demonstrate that FastDAM and UniverDAM outperform the existing multiple source domain adaptation methods for the two applications.

We describe a series o,f five statistical models o,f the translation process and give algorithms,for estimating the parameters o,f these models given a set o,f pairs o,f sentences that are translations o,f one another. We define a concept o,f word-by-word alignment between such pairs o,f sentences. For any given pair of such sentences each o,f our models assigns a probability to each of the possible word-by-word alignments. We give an algorithm for seeking the most probable o,f these alignments. Although the algorithm is suboptimal, the alignment thus obtained accounts well for the word-by-word relationships in the pair o,f sentences. We have a great deal o,f data in French and English from the proceedings o,f the Canadian Parliament. Accordingly, we have restricted our work to these two languages; but we,feel that because our algorithms have minimal linguistic content they would work well on other pairs o,f languages. We also ,feel, again because of the minimal linguistic content o,f our algorithms, that it is reasonable to argue that word-by-word alignments are inherent in any sufficiently large bilingual corpus.

Research in Social Science is usually based on survey data where individual research questions relate to observable concepts (variables). However, due to a lack of standards for data citations a reliable identification of the variables used is often difficult. In this paper, we present a work-in-progress study that seeks to provide a solution to the variable detection task based on supervised machine learning algorithms, using a linguistic analysis pipeline to extract a rich feature set, including terminological concepts and similarity metric scores. Further, we present preliminary results on a small dataset that has been specifically designed for this task, yielding a significant increase in performance over the random baseline.

Microblogs such as Twitter, Facebook, and Sina Weibo (China's equivalent of Twitter) are a remarkable linguistic resource. In contrast to content from edited genres such as newswire, microblogs contain discussions of virtually every topic by numerous individuals in different languages and dialects and in different styles. In this work, we show that some microblog users post “self-translated” messages targeting audiences who speak different languages, either by writing the same message in multiple languages or by retweeting translations of their original posts in a second language. We introduce a method for finding and extracting this naturally occurring parallel data. Identifying the parallel content requires solving an alignment problem, and we give an optimally efficient dynamic programming algorithm for this. Using our method, we extract nearly 3M Chinese–English parallel segments from Sina Weibo using a targeted crawl of Weibo users who post in multiple languages. Additionally, from a random sample of Twitter, we obtain substantial amounts of parallel data in multiple language pairs. Evaluation is performed by assessing the accuracy of our extraction approach relative to a manual annotation as well as in terms of utility as training data for a Chinese–English machine translation system. Relative to traditional parallel data resources, the automatically extracted parallel data yield substantial translation quality improvements in translating microblog text and modest improvements in translating edited news content.

This paper proposes an accurate method for partof-speech (POS) tagging that is highly domain-adaptable. The method is based on an assumption that the POS transition tendencies do not depend on domains, and has the following three characteristics: 1) it is trainable from partially annotated data, 2) it uses efficiently trainable pointwise POS taggers to allow for active learning, and 3) is more accurate than the pointwise or sequence-based POS taggers. The proposed method estimates POS tags by stacking pointwise and sequence-based predictors. In the experiments we deal with the joint problem of word segmentation and POS tagging in Japanese. We show that our proposed stacking process improves over pointwise and sequence-based methods (hidden Markov models and conditional random fields) both in the general domain and the target domain. In addition we show the learning curve in a domain adaptation scenario. The result shows that our method outperforms state-of-the-art methods in the same domain as the training data and is better than them in domain adaptation situations as well. Keywords-Active learning; Reranking; Word segmentation; Part-of-speech tagging; Pointwise prediction;

Script based features are highly discriminative for text segmentation and recognition. Thus they are widely used in Optical Character Recognition(OCR) problems. But usage of script dependent features restricts the adaptation of such architectures directly for another script. With script independent systems, this problem can be solved to a certain extent for monolingual documents. But the problem aggravates in case of multilingual documents as it is very difficult for a single classifier to learn many scripts. Generally a script identification module identifies text segments and accordingly the script-dependent classifier is selected. This paper presents a unified framework of language model and multiple preprocessing hypotheses for word recognition from bilingual document images. Prior to text recognition, preprocessing steps such as binarization and segmentation are required for ease of recognition. But these steps induce huge combinatorial error propagating to final recognition accuracy. In this paper we use multiple preprocessing routines as alternate hypotheses and use a language model to verify each alternative and choose the best recognized sequence. We test this architecture for word recognition of Kannada-English and Telugu-English bilingual documents and achieved better recognition rates than single methods using same classifier.

Entity linking (EL) is the task of disambiguating mentions in text by associating them with entries in a predefined database of mentions (persons, organizations, etc). Most previous EL research has focused mainly on one language, English, with less attention being paid to other languages, such as Spanish or Chinese. In this paper, we introduce LIEL, a Language Independent Entity Linking system, which provides an EL framework which, once trained on one language, works remarkably well on a number of different languages without change. LIEL makes a joint global prediction over the entire document, employing a discriminative reranking framework with many domain and language-independent feature functions. Experiments on numerous benchmark datasets, show that the proposed system, once trained on one language, English, outperforms several state-of-the-art systems in English (by 4 points) and the trained model also works very well on Spanish (14 points better than a competitor system), demonstrating the viability of the approach.


Question: Cross-domain characteristic selection for language recognition.

An Overview of Information Retrieval: Cataloging Techniques

This paper provides a comprehensive guide to the methodologies employed in information retrieval, with a particular focus on cataloging techniques. Unlike traditional retrieval systems that rely solely on keyword matching, modern approaches emphasize the importance of indexing strategies for efficient and effective data retrieval. The document delves into the intricacies of cataloging, detailing how information is systematically organized and accessed. It discusses various cataloging techniques, including those that leverage metadata, controlled vocabularies, and semantic analysis. The paper also explores the role of cataloging in cross-domain applications, highlighting its significance in language identification tasks. By presenting a thorough introduction to information retrieval and cataloging, the document aims to equip readers with the knowledge necessary to navigate the complexities of data management and retrieval in diverse linguistic contexts. This comprehensive guide is essential for anyone seeking to understand the foundational principles of information retrieval and their application in language identification across different domains.

In emotion classification of speech signals, the popular features employed are statistics of fundamental frequency, energy contour, duration of silence and voice quality. However, the performance of systems employing these features degrades substantially when more than two categories of emotion are to be classified. In this paper, a text independent method of emotion classification of speech is proposed. The proposed method makes use of short time log frequency power coefficients (LFPC) to represent the speech signals and a discrete hidden Markov model (HMM) as the classifier. The emotions are classified into six categories. The category labels used are, the archetypal emotions of Anger, Disgust, Fear, Joy, Sadness and Surprise. Adatabase consisting of 60 emotional utterances, each from twelve speakers is constructed and used to train and test the proposed system. Performance of the LFPC feature parameters is compared with that of the linear prediction Cepstral coefficients (LPCC) and mel-frequency Cepstral coefficients (MFCC) feature parameters commonly used in speech recognition systems. Results show that the proposed system yields an average accuracy of 78% and the best accuracy of 96% in the classification of six emotions. This is beyond the 17% chances by a random hit for a sample set of 6 categories. Results also reveal that LFPC is a better choice as feature parameters for emotion classification than the traditional feature parameters. 2003 Elsevier B.V. All rights reserved.

Large Vocabulary Continuous Speech Recognition (LVCSR), which is characterized by a high variability of the speech, is the most challenging task in automatic speech recognition (ASR). Believing that the evaluation of ASR systems on relevant and common speech corpora is one of the key factors that help accelerating research, we present, in this paper, a benchmark comparison of the performances of the current state-of-the-art LVCSR systems over different speech recognition tasks. Furthermore, we put objectively into evidence the best performing technologies and the best accuracy achieved so far in each task. The benchmarks have shown that the Deep Neural Networks and Convolutional Neural Networks have proven their efficiency on several LVCSR tasks by outperforming the traditional Hidden Markov Models and Guaussian Mixture Models. They have also shown that despite the satisfying performances in some LVCSR tasks, the problem of large-vocabulary speech recognition is far from being solved in some others, where more research efforts are still needed.

We present the AP16-OL7 database which was released as the training and test data for the oriental language recognition (OLR) challenge on APSIPA 2016. Based on the database, a baseline system was constructed on the basis of the i-vector model. We report the baseline results evaluated in various metrics defined by the AP16-OLR evaluation plan and demonstrate that AP16-OL7 is a reasonable data resource for multilingual research.

Making no claim of being exhaustive, a review of the most popular MFCC (Mel Frequency Cepstral Coefficients) implementations is made. These differ mainly in the particular approximation of the nonlinear pitch perception of human, the filter bank design, and the compression of the filter bank output. Then, a comparative evaluation of the presented implementations is performed on the task of text-independent speaker verification, by means of the well-known 2001 NIST SRE (speaker recognition evaluation) one-speaker detection database.

Recently, deep learning techniques have been successfully applied to automatic speech recognition tasks -first to phonetic recognition with context-independent deep belief network (DBN) hidden Markov models (HMMs) and later to large vocabulary continuous speech recognition using context-dependent (CD) DBN-HMMs. In this paper, we report our most recent experiments designed to understand the roles of the two main phases of the DBN learning -pre-training and fine tuning -in the recognition performance of a CD-DBN-HMM based large-vocabulary speech recognizer. As expected, we show that pre-training can initialize weights to a point in the space where fine-tuning can be effective and thus is crucial in training deep structured models. However, a moderate increase of the amount of unlabeled pre-training data has an insignificant effect on the final recognition results as long as the original training size is sufficiently large to initialize the DBN weights. On the other hand, with additional labeled training data, the fine-tuning phase of DBN training can significantly improve the recognition accuracy.

Mainly for the sake of solving the lack of keyword-specific data, we propose one Keyword Spotting (KWS) system using Deep Neural Network (DNN) and Connectionist Temporal Classifier (CTC) on power-constrained small-footprint mobile devices, taking full advantage of general corpus from continuous speech recognition which is of great amount. DNN is to directly predict the posterior of phoneme units of any personally customized key-phrase, and CTC to produce a confidence score of the given phoneme sequence as responsive decision-making mechanism. The CTC-KWS has competitive performance in comparison with purely DNN based keyword specific KWS, but not increasing any computational complexity.

In this paper, we propose a novel method to adapt context-dependent deep neural network hidden Markov model (CD-DNN-HMM) with only limited number of parameters by taking into account the underlying factors that contribute to the distorted speech signal. We derive this factorized adaptation method from the perspectives of joint factor analysis and vector Taylor series expansion, respectively. Evaluated on Aurora 4, the proposed method can get 19.0% and 10.6% relative word error rate reduction on test set B and D with only 20 adaptation utterances, and can have decent improvement with as few as two adaptation utterances. We also show that the proposed method is better than feature discriminative linear regression (fDLR), an existing DNN adaptation method. Its small number of parameters and short training time offer an attractive solution to low-footprint speech applications.

This paper presents an entropy-based algorithm for accurate and robust endpoint detection for speech recognition under noisy environments. Instead of using the conventional energy-based features, the spectral entropy is developed to identify the speech segments accurately. Experimental results show that this algorithm outperforms the energy-based algorithms in both detection accuracy and recognition performance under noisy environments, with an average error rate reduction of more than 16%.

In Singapore and Malaysia, people often speak a mixture of Mandarin and English within a single sentence. We call such sentences intra-sentential code-switch sentences. In this paper, we report on the development of a Mandarin-English codeswitching spontaneous speech corpus: SEAME. The corpus is developed as part of a multilingual speech recognition project and will be used to examine how Mandarin-English codeswitch speech occurs in the spoken language in South-East Asia. Additionally, it can provide insights into the development of large vocabulary continuous speech recognition (LVCSR) for code-switching speech. The corpus collected consists of intra-sentential code-switching utterances that are recorded under both interview and conversational settings. This paper describes the corpus design and the analysis of collected corpus.

Large quantities of audio data with associated text such as audiobooks are nowadays available. These data are attractive for a range of research areas as they include features that go beyond the level of single sentences. The proposed approach allows high quality transcriptions and associated alignments of this form of data to be automatically generated. It combines information from lightly supervised recognition and the original text to yield the final transcription. The scheme is fully automatic and has been successfully applied to a number of audiobooks. Performance measurements show low word/sentence error rates as well as high sentence boundary accuracy.