Question: Tools for facilitating systematic reviews in software engineering: an analysis of features.

Introducing SLR-Tool: An Instrument for Conducting Systematic Literature Reviews in Software Engineering and Beyond

Systematic literature reviews (SLRs) have rapidly gained traction in the Software Engineering research community since their emergence in 2004. Recognized as an innovative research methodology, SLRs enable the collection of evidence concerning the efficacy of technologies proposed within Software Engineering for software development and maintenance. This trend is evidenced by the proliferation of SLR-related publications in recent times. Despite the existence of tools that facilitate various stages of the SLR process outlined by Kitchenham & Charters (2007), these are often commercially available and not free of cost. This paper introduces SLR-Tool, a free-to-use instrument accessible at http://alarcosj.esi.uclm.es/SLRTool/, designed for the benefit of researchers across all disciplines, not exclusively Software Engineering.

SLR-Tool not only aligns with the SLR process proposed by Kitchenham & Charters (2007) but also offers a range of advanced features. These include the capability to refine searches within documents through the application of text mining techniques, the establishment of a classification schema to streamline data synthesis, and the generation of results in the form of tables and charts for enhanced data visualization. Additionally, the tool supports the export of references from primary studies in formats compatible with popular bibliographic packages such as EndNote, BibTeX, or Ris.

The SLR-Tool has been utilized by members of the Alarcos Research Group and PhD students, who have found it to be indispensable and beneficial in their research endeavors. Our current objective is to promote the widespread adoption of SLR-Tool among the research community to gather feedback and further enhance its functionality.

Writing documentation about software internals is rarely considered a rewarding activity. It is highly time-consuming and the resulting documentation is fragile when the software is continuously evolving in a multi-developer setting. Unfortunately, traditional programming environments poorly support the writing and maintenance of documentation. Consequences are severe as the lack of documentation on software structure negatively impacts the overall quality of the software product. We show that using a controlled natural language with a reasoner and a query engine is a viable technique for verifying the consistency and accuracy of documentation and source code. Using ACE, a state-of-the-art controlled natural language, we present positive results on the comprehensibility and the general feasibility of creating and verifying documentation. As a case study, we used automatic documentation verification to identify and fix severe flaws in the architecture of a non-trivial piece of software. Moreover, a user experiment shows that our language is faster and easier to learn and understand than other formal languages for software documentation.

There exists a large body of scienti c literature devoted to ontology matching, aligning, mapping translating and merging. With it, comes a long list (90+) of tools that support various aspects of these operations. We have approached such tools from the perspective of the INTER-IoT project, in which one of the goals is to facilitate semantic interoperability of Internet of Things platforms. Thus, we had to answer a question: what is actually available when one needs to align/merge ontologies. Here, we summarize our ndings.

As modern systems become increasingly complex, current security practices lack effective methodologies to adequately address the system security. This paper proposes a repeatable and tailorable framework to assist in the application of systems security engineering (SSE) processes, activities, and tasks as defined in the recently released National Institute of Standards and Technology (NIST) Special Publication 800–160. First, a brief survey of systems-oriented security methodologies is provided. Next, an examination of the relationships between the NIST-defined SSE processes is conducted to provide context for the engineering problem space. These findings inform a mapping of the NIST SSE processes to seven system-agnostic security domains which enable prioritization for three types of systems (conventional IT, cyber-physical, and defense). These concrete examples provide further understanding for applying and prioritizing the SSE effort. The goal of this paper is assist practitioners by informing the efficient application of the 30 processes, 111 activities, and 428 tasks defined in NIST SP 800–160. The customizable framework tool is available online for developers to employ, modify, and tailor to meet their needs.

Systematic reviews are increasingly used to inform health care decisions, but are expensive to produce. We explore the use of crowdsourcing (distributing tasks to untrained workers via the web) to reduce the cost of screening citations. We used Amazon Mechanical Turk as our platform and 4 previously conducted systematic reviews as examples. For each citation, workers answered 4 or 5 questions that were equivalent to the eligibility criteria. We aggregated responses from multiple workers into an overall decision to include or exclude the citation using 1 of 9 algorithms and compared the performance of these algorithms to the corresponding decisions of trained experts. The most inclusive algorithm (designating a citation as relevant if any worker did) identified 95% to 99% of the citations that were ultimately included in the reviews while excluding 68% to 82% of irrelevant citations. Other algorithms increased the fraction of irrelevant articles excluded at some cost to the inclusion of relevant studies. Crowdworkers completed screening in 4 to 17 days, costing $460 to $2220, a cost reduction of up to 88% compared to trained experts. Crowdsourcing may represent a useful approach to reducing the cost of identifying literature for systematic reviews.

Adequate, timely and effective consultation of relevant stakeholders is of paramount importance in the requirements engineering process. However, the thorny issue of making sure that all relevant stakeholders are consulted has received less attention than other areas which depend on it, such as scenario-based requirements, involving users in development, negotiating between different viewpoints and so on. The literature suggests examples of stakeholders, and categories of stakeholder, but does not provide help in identifying stakeholders for a specific system. In this paper, we discuss current work in stakeholder identification, propose an approach to identifying relevant stakeholders for a specific system, and propose future directions for the work. 1. What is a ÔstakeholderÕ? There is a large body of literature in the strategic management area which discusses organisations in terms of a stakeholder model. Stakeholder analysis, it is claimed, can be used to analyse an organisationÕs performance and determine its future strategic direction. An oft-quoted definition of ÔstakeholderÕ, taken from a key reference in this literature is: ÔA stakeholder in an organisation is (by definition) any group or individual who can affect or is affected by the achievement of the organisationÕs objectives.Õ [10] A much broader definition, which has also been attributed to Freeman, is that a stakeholder is Ôanything influencing or influenced byÕ the firm, but it has been claimed that this definition is problematic because it leads to the identification of a very broad set of stakeholders. It is important to distinguish between influencers and stakeholders because while some potential stakeholders may indeed be both stakeholders and influencers, some who have a real stake in an enterprise may have no influence, e.g. a job applicant, while some influencers may have no stake, e.g. the media [8]. Information systems (IS) researchers have also taken up the idea of stakeholders: ÔWe define stakeholders as these participants <in the development process> together with any other individuals, groups or organisations whose actions can influence or be influenced by the development and use of the system whether directly or indirectly.Õ [22] In software engineering, stakeholders have been defined as: ÔThe people and organisations affected by the applicationÕ [3] ÔSystem stakeholders are people or organisations who will be affected by the system and who have a direct or indirect influence on the system requirementsÕ [16] ÔStakeholders are people who have a stake or interest in the projectÕ [4] A more explicit refinement of this definition is: ÔÉ anyone whose jobs will be altered, who supplies or gains information from it, or whose power or influence within the organisation will increase or decrease.Õ [7] They go on to say that ÔIt will frequently be the case that the formal ÔclientÕ who orders the system falls very low on the list of those affected. Be very wary of changes which take power, influence or control from some stakeholders without returning something tangible in its place.Õ [7] When faced with the practical problem of how to identify the set of stakeholders relevant to a specific project, these definitions are not particularly helpful. The main concern is that, although such definitions are usually accompanied by example groups of stakeholders, they are vague and may lead to consideration of inappropriate or incomplete groups of stakeholders. Categories of stakeholder include end-users, managers and others involved in the organisational processes influenced by the system, engineers responsible for system development and maintenance, customers of the Sharp, Finkelstein & Galal 11/12/99 2 organisation who will use the system to provide a service, external bodies such as regulators, domain experts, and so on. They will each have different goals, and will try to satisfy their own without recourse to others [16]. Cotterell and Hughes [4] suggest that stakeholders might be in one of three categories: internal to the project team; external to the project team, but internal to the organisation; and external to both the project team and the organisation. Newman and Lamming [20] suggest a different division: into those who will use the system directly or indirectly, and those who will be involved in developing the system. This distinction is also taken up with respect to the development of knowledge-based systems [22]. The set of stakeholders in the knowledge acquisition process and the set of stakeholders in the use of a knowledge-based system, are not necessarily identical; they are likely to vary in membership, and for those members in common, the type and level of stake they have is likely to vary. The IS literature suggests a different division again, into ÔhubsÕ or ÔsponsorsÕ and ÔspokesÕ or ÔadaptorsÕ, where the former are those initiating and sustaining the system, while the latter are those participating [21]. Macaulay [19] identifies four categories of stakeholder in any computer system: 1. Those responsible for design and development; 2. Those with a financial interest, responsible for its sale or purchase; 3. Those responsible for introduction and maintenance; 4. Those who have an interest in its use. But again, she offers no guidelines for identifying specific stakeholders for a given system. So far, we have not distinguished between individuals or groups and roles. As with other situations, the mapping between stakeholders and individuals or groups is not oneto-one. It is therefore more appropriate to think of stakeholders as roles rather than as specific people. In subsequent discussion, we shall use ÔstakeholderÕ to mean Ôstakeholder roleÕ. 2. Identifying stakeholders There has been little focus on the participants of the requirements engineering (RE) process, for example in terms of how to trace participants in the RE process and how to identify stakeholders [13]. All of the references cited above emphasise the importance of identifying stakeholders, and although they provide examples, or broad guidance for identifying them, none describes a model or a concrete approach for identifying stakeholders for a specific project. This deficiency has been noted in the management literature, and in the IS literature [21], where the approaches have been criticised for either assuming that stakeholders are ÔobviousÕ, or for providing broad categories which are too generic to be of practical use. Expert identification for knowledge-based systems development has similarities with stakeholder identification in RE, although here too there has been an assumption that experts are readily identifiable [22]. Pouloudi and Whitley [21] suggest four principles of stakeholder identification, and describe an approach which is based on these principles, and which they used to identify stakeholders in the drug use management domain. Lyytinen and Hirschheim [17] also provides some guidance for stakeholder identification for IS, while acknowledging that Ôthe identification of the set of all stakeholders is far from trivialÕ. Methods for requirements engineering, e.g. KAOS [5], do not directly support stakeholder identification; it seems to be assumed that it is straightforward. The task of identifying actors for use case development [15] has similarities with stakeholder identification, but is targeted only at a fraction of system stakeholders. Stakeholders are related to each other and interact with each other [22,11,17]. Interactions between them include: exchanging information, products, or instructions, or providing supporting tasks. Information about the stakeholders and the nature of their relationships and interactions needs to be identified and recorded. Dimensions of importance are: relationships between stakeholders, the relationship of each stakeholder to the system, and the priority to be given to each stakeholderÕs view. This information is needed to manage, interpret, balance and process stakeholder input. Reconciling different stakeholder views is beyond this paper, but is considered in [12]. 3. An approach to identifying stakeholders for requirements engineering Here, we propose an approach to discovering all relevant stakeholders of a specific system that we believe is domain-independent, effective and pragmatic. We draw on the literature cited above, and on aspects of ethical decision-making in software systems [18]. Our starting point is a set of stakeholders that we refer to as ÔbaselineÕ stakeholders. From these, we can recognise ÔsupplierÕ stakeholders and ÔclientÕ stakeholders: the former provides information or supporting tasks to the baseline, and the latter processes or inspects the products of the baseline. Other stakeholders that we call ÔsatelliteÕs interact with the baseline in a variety of ways. ÔInteractionÕ may involve communicating, reading a set of rules or guidelines, searching for information and so on. Our approach focuses on interactions between stakeholders rather than relationships between the system and the stakeholder, Sharp, Finkelstein & Galal 11/12/99 3 because they are easier to follow. Figure 1 illustrates the main elements of stakeholder identification. Figure 1. The main elements of stakeholder identification addressed by our approach 3.1 Baseline stakeholders We have identified four groups of baseline stakeholder: users, developers, legislators and decision-makers. We have dubbed these ÔbaselineÕ because the web of stakeholders and their relationships can be identified from them. The nature of each group is explored below. 3.1.1 Users. The term ÔuserÕ has many interpretations. For example, Holtzblatt and Jones [14] include in their definition of ÔusersÕ those who manage direct users, those who receive products from the system, those who test the system, those who make the purchasing decision, and those who use competitive products. Eason [9] identifies three categories of user: primary, secondary and te

Shared understanding is essential for efficient software engineering when the risk of unsatisfactory out-come and rework of project results shall be low. Today, however, shared understanding is used mostly in an unreflected, ad-hoc way. This affects the quality of the engineered software solutions and generates re-work once the quality problems are discovered. In this article, we investigate the role, value, and usage of shared understanding in software engineering. We contribute a reflected analysis of the problem, in particular of how to rely on shared understanding that is implicit, rather than explicit. After an overview of the state of the art we discuss forms and value of shared understanding in software engineering, survey enablers and obstacles, compile existing practices for dealing with shared understanding, and present a roadmap for improving knowledge and practice in this area.

D istributed systems pose unique challenges for software developers. Reasoning about concurrent activities of system nodes and even understanding the system’s communication topology can be difficult. A standard approach to gaining insight into system activity is to analyze system logs. Unfortunately, this can be a tedious and complex process. This article looks at several key features and debugging challenges that differentiate distributed systems from other kinds of software. The article presents several promising tools and ongoing research to help resolve these challenges.

We present four datasets that are focused on the general roles of OSS peer review members. With data mined from both an integrated peer review system and code source repositories, our rich datasets comprise of peer review data that was automatically recorded. Using the Android project as a case study, we describe our extraction methodology, the datasets and their application used for three separate studies. Our datasets are available online at http://sdlab.naist.jp/reviewmining/.

Software quality is a major concern in software engineering, particularly when we are dealing with software services that must be available 24 hours a day, please the customer and be kept permanently. In software engineering, requirements management is one of the most important tasks that can influence the success of a software project. Maintain traceability information is a fundamental aspect in software engineering and can facilitate the process of software development and maintenance. This information can be used to support various activities such as analyzing the impact of changes and software verification and validation. However, traceability techniques are mainly used between requirements and software test cases. This paper presents a prototype of a tool that supports the mapping of functional requirements with the pages and HTML elements of a web site. This tool helps maintaining requirements traceability information updated and, ultimately, increasing the efficiency of requirements change management, which may contribute to the overall quality of the software service.

This survey paper presents the current state of model comparison as it applies to Model-Driven Engineering. We look specifically at how model matching is accomplished, the application of the approaches, and the types of models that approaches are intended to work with. Our paper also indicates future trends and directions. We find that many of the latest model comparison techniques are geared towards facilitating arbitrary meta models and use similarity-based matching. Thus far, model versioning is the most prevalent application of model comparison. Recently, however, work on comparison for versioning has begun to stagnate, giving way to other applications. Lastly, there is wide variance among the tools in the amount of user effort required to perform model comparison, as some require more effort to facilitate more generality and expressive power.


Question: A feature analysis of tools supporting systematic reviews in software engineering.

This study addresses the scarcity of valid measurement scales for assessing user acceptance of information technology, particularly in the context of computer systems. Many subjective measures currently employed lack validation and their correlation with system usage is unclear. To address this gap, the research introduces and validates two new scales for evaluating perceived usefulness and perceived ease of use, two critical factors believed to significantly influence user acceptance. The scales were developed based on precise definitions for these variables, ensuring content validity through pretesting. Two studies, involving 152 participants and four application programs, were conducted to test the reliability and construct validity of these scales. Following a refinement process, the final scales consisted of six items each, achieving high reliabilities of .98 for usefulness and .94 for ease of use. The scales demonstrated strong convergent, discriminant, and factorial validity. Perceived usefulness was found to have a significant correlation with both current (r = .63) and predicted future usage (r = .85). Perceived ease of use also showed significant correlations with current (r = .45) and future usage (r = .59). Notably, usefulness had a stronger correlation with usage behavior than ease of use in both studies. The regression analysis indicates that perceived ease of use might be a precursor to perceived usefulness, rather than a parallel determinant of system usage. The findings have important implications for future research on user acceptance, particularly in the development of user-friendly software engineering tools.

**Question**: A Hybrid Approach for Improving Software Defect Prediction

**Documents**: A Hybrid Genetic Algorithm-Particle Swarm Optimization for the Design of Recurrent Neural Networks
In this paper, a hybrid evolutionary algorithm combining genetic algorithm (GA) and particle swarm optimization (PSO) is proposed for the design of recurrent neural networks (RNNs). The hybrid algorithm, named HGAPSO, utilizes the strengths of both GA and PSO to overcome the limitations of each individual algorithm. In HGAPSO, the initial population is generated randomly. Then, the algorithm proceeds in two phases: the GA phase and the PSO phase. In the GA phase, crossover and mutation operations are applied to generate new individuals. In the PSO phase, the velocity and position of each particle (individual) are updated based on its own best position and the best position of the swarm. The algorithm alternates between these two phases until a stopping criterion is met. The proposed HGAPSO algorithm is applied to the design of RNNs for time series prediction. The performance of HGAPSO is compared

A designer often has to evaluate alternative designs during the development of a system. A multitude of Design-Space Exploration (DSE) techniques exist in the literature. Integration of these techniques into the modelling paradigm is needed when a model-driven engineering approach is used for designing systems. To a greater or lesser extent, the integration of those different DSE techniques share characteristics with each other. Inspired by software design patterns, we introduce an initial pattern catalogue to categorise the embedding of different DSE techniques in an MDE context. We demonstrate their use by a literature survey and discuss the consequences of each pattern.

Software evolves with continuous source-code changes. These code changes usually need to be understood by software engineers when performing their daily development and maintenance tasks. However, despite its high importance, such change-understanding practice has not been systematically studied. Such lack of empirical knowledge hinders attempts to evaluate this fundamental practice and improve the corresponding tool support.
 To address this issue, in this paper, we present a large-scale quantitative and qualitative study at Microsoft. The study investigates the role of understanding code changes during software-development process, explores engineers' information needs for understanding changes and their requirements for the corresponding tool support. The study results reinforce our beliefs that understanding code changes is an indispensable task performed by engineers in software-development process. A number of insufficiencies in the current practice also emerge from the study results. For example, it is difficult to acquire important information needs such as a change's completeness, consistency, and especially the risk imposed by it on other software components. In addition, for understanding a composite change, it is valuable to decompose it into sub-changes that are aligned with individual development issues; however, currently such decomposition lacks tool support.

With the increase in popularity of online review sites comes a corresponding need for tools capable of extracting the information most important to the user from the plain text data. Due to the diversity in products and services being reviewed, supervised methods are often not practical. We present an unsupervised system for extracting aspects and determining sentiment in review text. The method is simple and flexible with regard to domain and language, and takes into account the influence of aspect on sentiment polarity, an issue largely ignored in previous literature. We demonstrate its effectiveness on both component tasks, where it achieves similar results to more complex semi-supervised methods that are restricted by their reliance on manual annotation and extensive knowledge sources.

We introduce SoftGUESS, a code clone exploration system. SoftGUESS is built on the more general GUESS system which provides users with a mechanism to interactively explore graph structures both through direct manipulation as well as a domain-specific language. We demonstrate SoftGUESS through a number of mini-applications to analyze evolutionary code-clone behavior in software systems. The mini-applications of SoftGUESS represent a novel way of looking at code-clones in the context of many system features. It is our hope that SoftGUESS will form the basis for other analysis tools in the software-engineering domain.

Patterns have shown to be an effective means of capturing and communicating software design experience. However, there is more to patterns than software design patterns: We believe that patterns work for software development on several levels. In this paper we explore what we have come to understand as crucial aspects of the pattern concept, relate patterns to the different models built during software design, discuss pattern forms and how we think that patterns can form larger wholes like pattern handbooks.

The increased availability of data created as part of the sof tware development process allows us to apply novel analysis techniques on the data and use the results to guide t he process’s optimization. In this paper we describe variou s data sources and discuss the principles and techniques of data mi ning as applied on software engineering data. Data that can b e mined is generated by most parts of the development process: requi rements elicitation, development analysis, testing, debu gging, and maintenance. Based on this classification we survey the mini ng approaches that have been used and categorize them accord ing to the corresponding parts of the development process and th e task they assist. Thus the survey provides researchers wit h a concise overview of data mining techniques applied to softw are engineering data, and aids practitioners on the selecti on of appropriate data mining techniques for their work.

N-fold inspection uses traditional inspections of the user requirements document (URD) but replicates the inspection activities using N independent teams. A pilot study was conducted to explore the usefulness of N-fold inspection during requirements analysis. A comparison of N-fold inspection with other development techniques reveals that N-fold inspection is a cost-effective method for finding faults in the URD and may be a valid technique in the development of mission-critical software systems.

This paper discusses a model-based approach to software development. It argues that an approach using models as central development artifact needs to be added to the portfolio of software engineering techniques, to further increase efficiency and flexibility of the development as well as quality and reusability of the results. Two major and strongly related techniques are identified and discussed: Test case modeling and an evolutionary approach to model transformation.

The gathering of stakeholder requirements comprises an early, but continuous and highly critical stage in system development. This phase in development is subject to a large degree of error, influenced by key factors rooted in communication problems. This pilot study builds upon an existing theory-based categorisation of these problems through presentation of a four-dimensional framework on communication. Its structure is validated through a content analysis of interview data, from which themes emerge, that can be assigned to the dimensional categories, highlighting any problematic areas. The paper concludes with a discussion on the utilisation of the framework for requirements elicitation exercises.

Value of software log file analysis has been constantly increasing with the value of information to organizations. Log management tools still have a lot to deliver in order to empower their customers with the true strength of log information. In addition to the traditional uses such as testing software functional conformance, troubleshooting and performance benchmarking, log analysis has proven its capabilities in fields like intrusion detection and compliance evaluation. This is verified by the emphasis on log analysis in regulations like PCI DSS, FISMA, HIPAA and frameworks such as ISO 27001 and COBIT. In this paper we present an in depth analysis into current log analysis domains and common problems. A practical guide to the use of few popular log analysis tools is also included. Lack of proper support for structured analysis is identified as one major flaw in existing tools. After that, we describe a framework we developed for structured log analysis with the view of providing a solution to open problems in the domain. The core strength of the framework is its ability to handle many log file formats that are not well served by existing tools and providing sophisticated infrastructure for automating recurring log analysis procedures. We prove the usefulness of the framework with a simple experiment.


Question: Tools for facilitating systematic reviews in software engineering: an analysis of features.

This paper explores the potential advantages and challenges of implementing an evidence-based approach in software engineering, drawing parallels with evidence-based medicine (EBM). The aim is to illustrate how integrating research findings could better serve the diverse stakeholder groups in software engineering. However, the current infrastructure is inadequate for the widespread implementation of evidence-based software engineering (EBSE). Two key factors unique to software engineering—the skill factor and the lifecycle factor—pose significant obstacles. The skill factor indicates that software engineering experiments are susceptible to both subject and experimenter bias, affecting the reliability of results. The lifecycle factor complicates the prediction of technology behavior once deployed in real-world scenarios. Despite these challenges, adopting an evidence-based approach could be highly beneficial for software engineering, provided that the field addresses these specific issues inherent to its nature. To achieve this, the software engineering community must develop and refine the necessary infrastructure and methodologies to mitigate biases and enhance the predictability of technology performance over its lifecycle.

A tutorial introduction to designs in unifying theories of programming p. 40 An integration of program analysis and automated theorem proving p. 67 Verifying controlled components p. 87 Efficient CSP[subscript Z] data abstraction p. 108 State/event-based software model checking p. 128 Formalising behaviour trees with CSP p. 148 Generating MSCs from an integrated formal specification language p. 168 UML to B : formal verification of object-oriented models p. 187 Software verification with integrated data type refinement for integer arithmetic p. 207

Shared understanding is essential for efficient software engineering when the risk of unsatisfactory out-come and rework of project results shall be low. Today, however, shared understanding is used mostly in an unreflected, ad-hoc way. This affects the quality of the engineered software solutions and generates re-work once the quality problems are discovered. In this article, we investigate the role, value, and usage of shared understanding in software engineering. We contribute a reflected analysis of the problem, in particular of how to rely on shared understanding that is implicit, rather than explicit. After an overview of the state of the art we discuss forms and value of shared understanding in software engineering, survey enablers and obstacles, compile existing practices for dealing with shared understanding, and present a roadmap for improving knowledge and practice in this area.

A designer often has to evaluate alternative designs during the development of a system. A multitude of Design-Space Exploration (DSE) techniques exist in the literature. Integration of these techniques into the modelling paradigm is needed when a model-driven engineering approach is used for designing systems. To a greater or lesser extent, the integration of those different DSE techniques share characteristics with each other. Inspired by software design patterns, we introduce an initial pattern catalogue to categorise the embedding of different DSE techniques in an MDE context. We demonstrate their use by a literature survey and discuss the consequences of each pattern.

Using automatic tools for the quality analysis of Natural Language (NL) requirements is recognized as a key factor for achieving software quality. Unfortunately few tools and techniques for the NL requirements analysis are currently available. This paper presents a methodology and a tool (called QuARS Quality Analyzer for Requirement Specifications) for analyzing NL requirements in a systematic and automatic way. QuARS allows requirements engineers to perform an initial parsing of the requirements in order to automatically detect potential linguistic defects that could cause interpretation problems at subsequent stages in developing the software. This tool is also able to partially support the consistency and completeness analysis by clustering the requirements according to specific topics.

The automated and semi-automated analysis of source code has remained a topic of intense research for more than thirty years. During this period, algorithms and techniques for source-code analysis have changed, sometimes dramatically. The abilities of the tools that implement them have also expanded to meet new and diverse challenges. This paper surveys current work on source-code analysis. It also provides a road map for future work over the next five-year period and speculates on the development of source-code analysis applications, techniques, and challenges over the next 10, 20, and 50 years.

In the context of open source development or software evolution, developers often face test suites which have been developed with no apparent rationale and which may need to be augmented or refined to ensure sufficient dependability, or even reduced to meet tight deadlines. We refer to this process as the re-engineering of test suites. It is important to provide both methodological and tool support to help people understand the limitations of test suites and their possible redundancies, so as to be able to refine them in a cost effective manner. To address this problem in the case of black-box testing, we propose a methodology based on machine learning that has shown promising results on a case study.

Software integration is still a challenging and time consuming task and therefore a major part of the development of industrial and domestic service robot applications. The presented toolchain BRIDE is able to streamline this process by the separation of user roles and the separation of developer concerns of software components to ensure a frame-work independent implementation. The impact of the BRIDE toolchain in the development process is demonstrated in a case study on the SyncMM mobile manipulation control framework.

Software visualization studies techniques and methods for graphically representing different aspects of software. Its main goal is to enhance, simplify and clarify the mental representation a software engineer has of a computer system. During many years, visualization in 2D space has been actively studied, but in the last decade, researchers have begun to explore new 3D representations for visualizing software. In this article, we present an overview of current research in the area, describing several major aspects like: visual representations, interaction issues, evaluation methods and development tools. We also perform a survey of some representative tools to support different tasks, i.e., software maintenance and comprehension, requirements validation and algorithm animation for educational purposes, among others. Finally, we conclude identifying future research directions.

Software development in highly variable domains constrained by tight regulations and with many business concepts involved results in hard to deliver and maintain applications, due to the complexity of dealing with the large number of concepts provided by the different parties and system involved in the process. One way to tackle these problems is thru combining software product lines and model-driven software development supported by ontologies. Software product lines and model-driven approaches would promote reuse on the software artifacts and, if supported by an ontological layer, those artifacts would be domain-validated. We intend to create a new conceptual framework for software development with domain validated models in highly variable domains. To define such a framework we will propose a model that relates several dimensions and areas of software development thru time and abstraction levels. This model would guarantee to the software house traceability of components, domain validated artifacts, easy to maintain and reusable components, due to the relations and mappings we propose to establish in the conceptual framework, between the software artifacts and the ontology.

For many years, the IT industry has sought to accelerate the software development process by assembling new applications from existing software assets. However, true component-based reuse of the form Douglas Mcllroy envisaged in the 1960s is still the exception rather than the rule, and most of the systematic software reuse practiced today uses heavyweight approaches such as product-line engineering or domain-specific frameworks. By component, we mean any cohesive and compact unit of software functionality with a well-defined interface - from simple programming language classes to more complex artifacts such as Web services and Enterprise JavaBeans.


Question: A feature analysis of tools supporting systematic reviews in software engineering.

This paper delves into an empirical study that expands the Technology Acceptance Model (TAM) to better understand the factors influencing individual acceptance and usage of websites. We explore the effects of perceived ease-of-use, usefulness, enjoyment, and their interplay on the attitude towards using, intention to use, and actual use of websites. A novel concept, "perceived visual attractiveness," is introduced and hypothesized to affect usefulness, enjoyment, and ease-of-use. Our research was conducted in collaboration with a popular generic portal in the Netherlands, boasting over 300,000 subscribers. The web survey yielded a significant sample size of 825 participants. The findings supported all 12 hypotheses put forth. Notably, attitude was the strongest influencer of intention (β = 0.51). Ease-of-use, enjoyment, and usefulness had a similar impact on the attitude towards using (β = 0.23, 0.23, and 0.17, respectively). Visual attractiveness significantly contributed to ease-of-use, enjoyment, and usefulness (β = 0.41, 0.35, and 0.21). Our study makes three key contributions: (1) it focuses on a single website as the unit of analysis, (2) it introduces visual attractiveness as a critical factor, and (3) it utilizes a sample of genuine website users rather than students. Future research could explore the link between specific website features and usage, with the TAM framework serving as a valuable analytical tool.

**Question**: A review of software engineering techniques for developing web applications

**Documents**: The Role of Social Media in Software Development: A Systematic Literature Review
The paper presents a systematic literature review of studies that investigate the role of social media in software development. The review was conducted by searching databases of scientific literature for studies published between 2009 and 2019. A total of 122 papers were included in the review. The results of the review show that social media is used in various ways in software development, including for communication, collaboration, knowledge sharing, and problem-solving. The review also identifies several challenges associated with the use of social media in software development, such as information overload, lack of control over information, and the potential for distraction. The review concludes that social media can be a valuable tool for software development, but that its use should be carefully managed to avoid potential negative effects. The paper also suggests several areas for future research, including

Patents are of growing importance in current economic markets. Analyzing patent information has, therefore, become a common task for many interest groups. As a prerequisite for patent analysis, extensive search for relevant patent information is essential. Unfortunately, the complexity of patent material inhibits a straightforward retrieval of all relevant patent documents and leads to iterative, time-consuming approaches in practice. Already the amount of patent data to be analyzed poses challenges with respect to scalability. Further scalability issues arise concerning the diversity of users and the large variety of analysis tasks. With "PatViz”, a system for interactive analysis of patent information has been developed addressing scalability at various levels. PatViz provides a visual environment allowing for interactive reintegration of insights into subsequent search iterations, thereby bridging the gap between search and analytic processes. Because of its extensibility, we expect that the approach we have taken can be employed in different problem domains that require high quality of search results regarding their completeness.

In this paper, we present FLUORITE, a publicly available event logging plug-in for Eclipse which captures all of the low-level events when using the Eclipse code editor. FLUORITE captures not only what types of events occurred in the code editor, but also more detailed information such as the inserted and deleted text and the specific parameters for each command. This enables the detection of many usage patterns that could otherwise not be recognized, such as "typo correction" that requires knowing that the entered text is immediately deleted and replaced. Moreover, the snapshots of each source code file that has been opened during the session can be completely reproduced using the collected information. We also provide analysis and visualization tools which report various statistics about usage patterns, and we provide the logs in an XML format so others can write their own analyzers. FLUORITE can be used for not only evaluating existing tools, but also for discovering issues that motivate new tools.

This paper presents the Multiword Expression Toolkit (mwetoolkit), an environment for type and language-independent MWE identification from corpora. The mwetoolkit provides a targeted list of MWE candidates, extracted and filtered according to a number of user-defined criteria and a set of standard statistical association measures. For generating corpus counts, the toolkit provides both a corpus indexation facility and a tool for integration with web search engines, while for evaluation, it provides validation and annotation facilities. The mwetoolkit also allows easy integration with a machine learning tool for the creation and application of supervised MWE extraction models if annotated data is available. In our experiments, the mwetoolkit was tested and evaluated in the context of MWE extraction in the biomedical domain. Our preliminary results show that the toolkit performs better than other approaches, especially concerning recall. Moreover, this first version can be extended in several ways in order to improve the quality of the results.

Online shoppers are generally highly task-driven: they have a certain goal in mind, and they are looking for a product with features that are consistent with that goal. Unfortunately, finding a product with specific features is extremely time-consuming using the search functionality provided by existing web sites.In this paper, we present a new search system called Red Opal that enables users to locate products rapidly based on features. Our fully automatic system examines prior customer reviews, identifies product features, and scores each product on each feature. Red Opal uses these scores to determine which products to show when a user specifies a desired product feature. We evaluate our system on four dimensions: precision of feature extraction, efficiency of feature extraction, precision of product scores, and estimated time savings to customers. On each dimension, Red Opal performs better than a comparison system.

The KeY system offers a platform of software analysis tools for sequential Java. Foremost, this includes full functional verification against contracts written in the Java Modeling Language. But the approach is general enough to provide a basis for other methods and purposes: (i) complementary validation techniques to formal verification such as testing and debugging, (ii) methods that reduce the complexity of verification such as modularization and abstract interpretation, (iii) analyses of non-functional properties such as information flow security, and (iv) sound program transformation and code generation. We show that deductive technology that has been developed for full functional verification can be used as a basis and framework for other purposes than pure functional verification. We use the current release of the KeY system as an example to explain and prove this claim.

The success of Web 2.0 applications has made online social media websites tremendous assets for supporting critical business intelligence applications. The knowledge gained from social media can potentially lead to the development of novel services that are better tailored to users’ needs and at the same time meet the objectives of businesses offering them. Online consumer reviews are one of the critical social media contents. Proper analysis of consumer reviews not only provides valuable information to facilitate the purchase decisions of customers but also helps merchants or product manufacturers better understand general responses of customers on their products for marketing campaign improvement. This study aims at designing an approach for supporting the effective analysis of the huge volume of online consumer reviews and, at the same time, settling the major limitations of existing approaches. Specifically, the proposed rule-based sentiment analysis (R-SA) technique employs the class association rule mining algorithm to automatically discover interesting and effective rules capable of extracting product features or opinion sentences for a specific product feature interested. According to our preliminary evaluation results, the R-SA technique performs well in comparison with its benchmark technique.

The present thesis is concerned with enterprise software systems of companies in the Swedish electricity industry, an industry that for the past few years has been exposed to a fairly tumultuous change process as a consequence of legislative reforms. Previously, the business operations of electric utilities – as well as those of most companies in the computerized world – were supported by a number of isolated software systems performing specific tasks. In recent years, these systems have been extended, and more importantly, integrated into a company-wide system in its own right, in this thesis referred to as the enterprise software system. As enterprise software systems have evolved, so has a need for strategies, methods and techniques for their management. Enterprise software system management involves a number of concerns; of these concerns, software integration is one of the most prominent. The discipline of software architecture is concerned with the modeling of large-scale structures of software systems. While the generated models are employed for a number of purposes , their perhaps most significant function is to serve as a base for reasoning about the represented system. Several methods for analysis of architectures have been proposed in recent years, and although software architecture analysis has displayed considerable success for a number of systems, enterprise software systems have to a large extent been ignored. In the empirical context of the Scandinavian electricity industry, this thesis explores the applicability of software architecture analysis to enterprise software system integration. Two conceptually different architectural analysis methods – deduction-and induction-based approaches – are considered, as well as the engineering process in which architectural analysis is performed. As a result of the investigations, the thesis proposes a modified process for architectural analysis, presents an evaluation of deduction-based analysis methods , and proposes an adaptation of induction-based analysis methods to the enterprise software system context.

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. The data to be collected are based on the changes made to the software during development, and are obtained when the changes are made. To ensure accuracy of the data, validation is performed concurrently with software development and data collection. Validation is based on interviews with those people supplying the data. Results from using the methodology show that data validation is a necessary part of change data collection. Without it, as much as 50 percent of the data may be erroneous. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments. The application showed that the methodology was both feasible and useful.

The design of software has been a focus of software engineering research since the field's beginning. This paper explores key aspects of this research focus and shows why design will remain a principal focus. The intrinsic elements of software design, both process and product, are discussed: concept formation, use of experience, and means for representation, reasoning, and directing the design activity. Design is presented as being an activity engaged by a wide range of stakeholders, acting throughout most of a system's lifecycle, making a set of key choices which constitute the application's architecture. Directions for design research are outlined, including: (a) drawing lessons, inspiration, and techniques from design fields outside of computer science, (b) emphasizing the design of application "character" (functionality and style) as well as the application's structure, and (c) expanding the notion of software to encompass the design of additional kinds of intangible complex artifacts.

In most cases, development of Web-based systems has been ad hoc, lacking systematic approach and quality control and assurance procedures. Hence, there is now legitimate and growing concern about the manner in which Web-based systems are developed and their long-term quality and integrity. Web Engineering, an emerging new discipline, advocates a process and a systematic approach to development of high quality Web-based systems. It promotes the establishment and use of sound scientific, engineering and management principles, and disciplined and systematic approaches to development, deployment and maintenance of Web-based systems. This paper gives an introductory overview on Web Engineering. It presents the principles and roles of Web Engineering, assesses the similarities and differences between development of traditional software and Web-based systems, identifies key Web engineering activities and reviews some of the ongoing work in this area. It also highlights the prospects of Web engineering and the areas that need further study.


Question: Visual recognition across domains through adaptive domain dictionary learning.

This paper presents an innovative technique, CORrelation ALignment (CORAL), for tackling unsupervised domain adaptation in machine learning. Unlike human cognition, machine learning systems can struggle with changes in input distributions between training (source) and testing (target) data, a challenge known as domain shift. This discrepancy often leads to degraded performance of conventional machine learning algorithms. While supervised domain adaptation methods, which require labeled target data, have been devised to address this issue, they are not always applicable as target labels are often unavailable in real-world scenarios. To bridge this gap, CORAL offers a straightforward yet potent solution for unsupervised domain adaptation. By minimizing domain shift through the alignment of second-order statistics between source and target distributions, CORAL effectively mitigates the impact of distributional differences without the need for target labels. Despite its simplicity—its implementation can be achieved in a mere four lines of Matlab code—CORAL exhibits exceptional performance on standard benchmark datasets. The philosophy guiding the development of CORAL is encapsulated in the maxim: "Everything should be made as simple as possible, but not simpler." This approach underscores the effectiveness of straightforward solutions in complex problems.

Multiple modalities can provide more valuable information than single one by describing the same contents in various ways. Hence, it is highly expected to learn effective joint representation by fusing the features of different modalities. However, previous methods mainly focus on fusing the shallow features or high-level representations generated by unimodal deep networks, which only capture part of the hierarchical correlations across modalities. In this paper, we propose to densely integrate the representations by greedily stacking multiple shared layers between different modality-specific networks, which is named as Dense Multimodal Fusion (DMF). The joint representations in different shared layers can capture the correlations in different levels, and the connection between shared layers also provides an efficient way to learn the dependence among hierarchical correlations. These two properties jointly contribute to the multiple learning paths in DMF, which results in faster convergence, lower training loss, and better performance. We evaluate our model on three typical multimodal learning tasks, including audiovisual speech recognition, cross-modal retrieval, and multimodal classification. The noticeable performance in the experiments demonstrates that our model can learn more effective joint representation.

Human visual object recognition is typically rapid and seemingly effortless, as well as largely independent of viewpoint and object orientation. Until very recently, animate visual systems were the only ones capable of this remarkable computational feat. This has changed with the rise of a class of computer vision algorithms called deep neural networks (DNNs) that achieve human-level classification performance on object recognition tasks. Furthermore, a growing number of studies report similarities in the way DNNs and the human visual system process objects, suggesting that current DNNs may be good models of human visual object recognition. Yet there clearly exist important architectural and processing differences between stateof-the-art DNNs and the primate visual system. The potential behavioural consequences of these differences are not well understood. We aim to address this issue by comparing human and DNN generalisation abilities towards image degradations. We find the human visual system to be more robust to image manipulations like contrast reduction, additive noise or novel eidolon-distortions. In addition, we find progressively diverging classification error-patterns between man and DNNs when the signal gets weaker, indicating that there may still be marked differences in the way humans and current DNNs perform visual object recognition. We envision that our findings as well as our carefully measured and freely available behavioural datasets1 provide a new useful benchmark for the computer vision community to improve the robustness of DNNs and a motivation for neuroscientists to search for mechanisms in the brain that could facilitate this robustness. Data and materials available at https://github.com/rgeirhos/object-recognition 1 ar X iv :1 70 6. 06 96 9v 1 [ cs .C V ] 2 1 Ju n 20 17

Cross-domain mapping has been a very active topic in recent years. Given one image, its main purpose is to translate it to the desired target domain, or multiple domains in the case of multiple labels. This problem is highly challenging due to three main reasons: (i) unpaired datasets, (ii) multiple attributes, and (iii) the multimodality associated with the translation. Most of the existing state-of-theart has focused only on two reasons, i.e. producing disentangled representations from unpaired datasets in a one-toone domain translation or producing multiple unimodal attributes from unpaired datasets. In this work, we propose a joint framework of diversity and multi-mapping image-toimage translations, using a single generator to conditionally produce countless and unique fake images that hold the underlying characteristics of the source image. Extensive experiments over different datasets demonstrate the effectiveness of our proposed approach with comparisons to the state-of-the-art in both multi-label and multimodal problems. Additionally, our method is able to generalize under different scenarios: continuous style interpolation, continuous label interpolation, and multi-label mapping.

This paper describes a machine learning approach for visual object detection which is capable of processing images extremely rapidly and achieving high detection rates. This work is distinguished by three key contributions. The first is the introduction of a new image representation called the “Integral Image” which allows the features used by our detector to be computed very quickly. The second is a learning algorithm, based on AdaBoost, which selects a small number of critical visual features from a larger set and yields extremely efficient classifiers[6]. The third contribution is a method for combining increasingly more complex classifiers in a “cascade” which allows background regions of the image to be quickly discarded while spending more computation on promising object-like regions. The cascade can be viewed as an object specific focus-of-attention mechanism which unlike previous approaches provides statistical guarantees that discarded regions are unlikely to contain the object of interest. In the domain of face detection the system yields detection rates comparable to the best previous systems. Used in real-time applications, the detector runs at 15 frames per second without resorting to image differencing or skin color detection.

In this paper, we propose an odor recognition framework for multiple electronic noses (E-noses), machine olfaction odor perception systems. Straight to the point, the proposed transferring odor recognition model is called cross-domain discriminative subspace learning (CDSL). General odor recognition problems with E-nose are single domain oriented, that is, recognition algorithms are often modeled and tested on the same one domain data set (i.e., from only one E-nose system). Different from that, we focus on a more realistic scenario: the recognition model is trained on a prepared source domain data set from a master E-nose system <inline-formula> <tex-math notation="LaTeX">${A}$ </tex-math></inline-formula>, but tested on another target domain data set from a slave system <inline-formula> <tex-math notation="LaTeX">${B}$ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">${C}$ </tex-math></inline-formula> with the same type of the master system <inline-formula> <tex-math notation="LaTeX">${A}$ </tex-math></inline-formula>. The internal device parameter variance between master and slave systems often results in data distribution discrepancy between source domain and target domain, such that single-domain-based odor recognition model may not be adapted to another domain. Therefore, we propose domain-adaptation-based odor recognition for addressing the realistic recognition scenario across systems. Specifically, the proposed CDSL method consists of three merits: 1) an intraclass scatter minimization- and an interclass scatter maximization-based discriminative subspace learning is solved on source domain; 2) a data fidelity and preservation constraint of the subspace is imposed on target domain without distortion; and 3) a minipatch feature weighted domain distance is minimized for closely connecting the source and target domains. Experiments and comparisons on odor recognition tasks in multiple E-noses demonstrate the efficiency of the proposed method.

Domain mismatch between training and testing can lead to significant degradation in performance in many machine learning scenarios. Unfortunately, this is not a rare situation for automatic speech recognition deployments in real-world applications. Research on robust speech recognition can be regarded as trying to overcome this domain mismatch issue. In this paper, we address the unsupervised domain adaptation problem for robust speech recognition, where both source and target domain speech are available, but word transcripts are only available for the source domain speech. We present novel augmentation-based methods that transform speech in a way that does not change the transcripts. Specifically, we first train a variational autoencoder on both source and target domain data (without supervision) to learn a latent representation of speech. We then transform nuisance attributes of speech that are irrelevant to recognition by modifying the latent representations, in order to augment labeled training data with additional data whose distribution is more similar to the target domain. The proposed method is evaluated on the CHiME-4 dataset and reduces the absolute word error rate (WER) by as much as 35% compared to the non-adapted baseline.

This paper introduces the visually informed embedding of word (VIEW), a continuous vector representation for a word extracted from a deep neural model trained using the Microsoft COCO data set to forecast the spatial arrangements between visual objects, given a textual description. The model is composed of a deep multilayer perceptron (MLP) stacked on the top of a Long Short Term Memory (LSTM) network, the latter being preceded by an embedding layer. The VIEW is applied to transferring multimodal background knowledge to Spatial Role Labeling (SpRL) algorithms, which recognize spatial relations between objects mentioned in the text. This work also contributes with a new method to select complementary features and a fine-tuning method for MLP that improves the F1 measure in classifying the words into spatial roles. The VIEW is evaluated with the Task 3 of SemEval-2013 benchmark data set, SpaceEval.

The success and impact of activity recognition algorithms largely depends on the availability of the labeled training samples and adaptability of activity recognition models across various domains. In a new environment, the pre-trained activity recognition models face challenges in presence of sensing bias- ness, device heterogeneities, and inherent variabilities in human behaviors and activities. Activity Recognition (AR) system built in one environment does not scale well in another environment, if it has to learn new activities and the annotated activity samples are scarce. Indeed building a new activity recognition model and training the model with large annotated samples often help overcome this challenging problem. However, collecting annotated samples is cost-sensitive and learning activity model at wild is computationally expensive. In this work, we propose an activity recognition framework, UnTran that utilizes source domains' pre-trained autoencoder enabled activity model that transfers two layers of this network to generate a common feature space for both source and target domain activities. We postulate a hybrid AR framework that helps fuse the decisions from a trained model in source domain and two activity models (raw and deep-feature based activity model) in target domain reducing the demand of annotated activity samples to help recognize unseen activities. We evaluated our framework with three real-world data traces consisting of 41 users and 26 activities in total. Our proposed UnTran AR framework achieves ≈ 75% F1 score in recognizing unseen new activities using only 10% labeled activity data in the target domain. UnTran attains ≈ 98% F1 score while recognizing seen activities in presence of only 2-3% of labeled activity samples.

Unsupervised feature learning is an important problem in computer vision and machine learning. Various unsupervised learning methods, e.g., PCA, KMeans, autoencoder, have been applied for visual recognition tasks. Especially, autoencoder has superior image recognition performance, which is usually used as a global feature extractor. To make better use of autoencoder for image recognition, we propose unsupervised local deep feature (ULDF). We collect patches from training images and use autoencoder to transform vectorized patches into short codes. Then these codes are encoded using Fisher Vector with spatial pyramid. Image recognition with ULDF only requires an efficient linear SVM classifier. Finally, we perform multi-scale voting to achieve more accurate classification performance. In the experiments, we evaluate ULDF on the handwritten digit recognition task and the shape classification task, and show that it achieves highly competitive performance comparing to the state-of-the-art image recognition methods. The results suggest that the proposed local deep feature is superior than traditional global autoencoder. © 2016 Elsevier Inc. All rights reserved.

This paper addresses the problem of multi-domain spoken language understanding (SLU) where domain detection and domaindependent semantic tagging problems are combined. We present a transfer learning approach to the multi-domain SLU problem in which multiple domain-specific data sources can be incorporated. To implement multi-domain SLU with transfer learning, we introduce a triangular-chain structured model. This model effectively learns multiple domains in parallel, and allows use of domain-independent patterns among domains to create a better model for the target domain. We demonstrate that the proposed method outperforms baseline models on dialog data for multi-domain SLU problems. 2009 Elsevier B.V. All rights reserved.


Question: Visual recognition across domains through adaptive domain dictionary learning.

This paper presents a novel strategy for cross-domain video concept detection, focusing on the adaptation of existing classifiers to new datasets with distinct distributions. Specifically, the work addresses two critical issues: (1) the transformation of existing classifiers into efficient models for datasets with scarce labeled examples, and (2) the identification of the most suitable existing classifiers for adaptation. To tackle the first challenge, the paper introduces Adaptive Support Vector Machines (A-SVMs), a versatile method for adjusting any type of classifier to a new dataset. This is achieved by learning the "delta function," a representation of the difference between the original and the adapted classifier, through an objective function akin to SVMs. For the second issue, the performance of each existing classifier on the sparsely labeled new dataset is estimated by scrutinizing the score distribution and other meta features. The classifiers with the highest estimated performance are then selected for adaptation. The proposed methodology demonstrates superior performance compared to various baseline and competitive methods in terms of classification accuracy and efficiency, as tested on cross-domain concept detection tasks within the TRECVID corpus.

Discriminative deep learning approaches have shown impressive results for problems where human-labeled ground truth is plentiful, but what about tasks where labels are difficult or impossible to obtain? This paper tackles one such problem: establishing dense visual correspondence across different object instances. For this task, although we do not know what the ground-truth is, we know it should be consistent across instances of that category. We exploit this consistency as a supervisory signal to train a convolutional neural network to predict cross-instance correspondences between pairs of images depicting objects of the same category. For each pair of training images we find an appropriate 3D CAD model and render two synthetic views to link in with the pair, establishing a correspondence flow 4-cycle. We use ground-truth synthetic-to-synthetic correspondences, provided by the rendering engine, to train a ConvNet to predict synthetic-to-real, real-to-real and real-to-synthetic correspondences that are cycle-consistent with the ground-truth. At test time, no CAD models are required. We demonstrate that our end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.

In reinforcement learning, we often define goals by specifying rewards within desirable states. One problem with this approach is that we typically need to redefine the rewards each time the goal changes, which often requires some understanding of the solution in the agent’s environment. When humans are learning to complete tasks, we regularly utilize alternative sources that guide our understanding of the problem. Such task representations allow one to specify goals on their own terms, thus providing specifications that can be appropriately interpreted across various environments. This motivates our own work, in which we represent goals in environments that are different from the agent’s. We introduce Cross-Domain Perceptual Reward (CDPR) functions, learned rewards that represent the visual similarity between an agent’s state and a cross-domain goal image. We report results for learning the CDPRs with a deep neural network and using them to solve two tasks with deep reinforcement learning.

Cross-domain mapping has been a very active topic in recent years. Given one image, its main purpose is to translate it to the desired target domain, or multiple domains in the case of multiple labels. This problem is highly challenging due to three main reasons: (i) unpaired datasets, (ii) multiple attributes, and (iii) the multimodality associated with the translation. Most of the existing state-of-theart has focused only on two reasons, i.e. producing disentangled representations from unpaired datasets in a one-toone domain translation or producing multiple unimodal attributes from unpaired datasets. In this work, we propose a joint framework of diversity and multi-mapping image-toimage translations, using a single generator to conditionally produce countless and unique fake images that hold the underlying characteristics of the source image. Extensive experiments over different datasets demonstrate the effectiveness of our proposed approach with comparisons to the state-of-the-art in both multi-label and multimodal problems. Additionally, our method is able to generalize under different scenarios: continuous style interpolation, continuous label interpolation, and multi-label mapping.

This paper summarizes our ongoing project on developing an architecture for a robot that can acquire new words and their meanings while engaging in multidomain dialogues. These two functions are crucial in making conversational service robots work in real tasks in the real world. Household robots and office robots need to be able to work in multiple task domains and they also need to engage in dialogues in multiple domains corresponding to those task domains. Lexical acquisition is necessary because speech understanding cannot be done without enough knowledge on words that are possibly spoken in the task domain. Our architecture is based on a multi-expert model in which multiple domain experts are employed and one of them is selected based on the user utterance and the situation to engage in the control of the dialogue and physical behaviors. We incorporate experts that have an ability to acquire new lexical entries and their meanings grounded on the physical world through spoken interactions. By appropriately selecting those experts, lexical acquisition in multi-domain dialogues becomes possible. An example robotic system based on this architecture that can acquire object names and location names demonstrates the viability of the architecture.

We propose simple and effective models for the image annotation that make use of Convolutional Neural Network (CNN) features extracted from an image and word embedding vectors to represent their associated tags. Our first set of models is based on the Canonical Correlation Analysis (CCA) framework that helps in modeling both views - visual features (CNN feature) and textual features (word embedding vectors) of the data. Results on all three variants of the CCA models, namely linear CCA, kernel CCA and CCA with k-nearest neighbor (CCA-KNN) clustering, are reported. The best results are obtained using CCA-KNN which outperforms previous results on the Corel-5k and the ESP-Game datasets and achieves comparable results on the IAPRTC-12 dataset. In our experiments we evaluate CNN features in the existing models which bring out the advantages of it over dozens of handcrafted features. We also demonstrate that word embedding vectors perform better than binary vectors as a representation of the tags associated with an image. In addition we compare the CCA model to a simple CNN based linear regression model, which allows the CNN layers to be trained using back-propagation.

Domain generalization (DG) is the challenging and topical problem of learning models that generalize to novel testing domain with different statistics than a set of known training domains. The simple approach of aggregating data from all source domains and training a single deep neural network end-to-end on all the data provides a surprisingly strong baseline that surpasses many prior published methods. In this paper we build on this strong baseline by designing an episodic training procedure that trains a single deep network in a way that exposes it to the domain shift that characterises a novel domain at runtime. Specifically, we decompose a deep network into feature extractor and classifier components, and then train each component by simulating it interacting with a partner who is badly tuned for the current domain. This makes both components more robust, ultimately leading to our networks producing state-of-the-art performance on three DG benchmarks. As a demonstration, we consider the pervasive workflow of using an ImageNet trained CNN as a fixed feature extractor for downstream recognition tasks. Using the Visual Decathlon benchmark, we demonstrate that our episodic-DG training improves the performance of such a general purpose feature extractor by explicitly training it for robustness to novel problems. This provides the largest-scale demonstration of heterogeneous DG to date.

Most supervised dictionary learning methods optimize the combinations of reconstruction error, sparsity prior, and discriminative terms. Thus, the learnt dictionaries may not be optimal for recognition tasks. Also, the sparse codes learning models in the training and the testing phases are inconsistent. Besides, without utilizing the intrinsic data structure, many dictionary learning methods only employ the  $\ell _{0}$  or  $\ell _{1}$  norm to encode each datum independently, limiting the performance of the learnt dictionaries. We present a novel bilevel model-based discriminative dictionary learning method for recognition tasks. The upper level directly minimizes the classification error, while the lower level uses the sparsity term and the Laplacian term to characterize the intrinsic data structure. The lower level is subordinate to the upper level. Therefore, our model achieves an overall optimality for recognition in that the learnt dictionary is directly tailored for recognition. Moreover, the sparse codes learning models in the training and the testing phases can be the same. We further propose a novel method to solve our bilevel optimization problem. It first replaces the lower level with its Karush-Kuhn–Tucker conditions and then applies the alternating direction method of multipliers to solve the equivalent problem. Extensive experiments demonstrate the effectiveness and robustness of our method.

A visual word lexicon can be constructed by clustering primitive visual features, and a visual object can be described by a set of visual words. Such a "bag-of-words" representation has led to many significant results in various vision tasks including object recognition and categorization. However, in practice, the clustering of primitive visual features tends to result in synonymous visual words that over-represent visual patterns, as well as polysemous visual words that bring large uncertainties and ambiguities in the representation. This paper aims at generating a higher-level lexicon, i.e. visual phrase lexicon, where a visual phrase is a meaningful spatially co-occurrent pattern of visual words. This higher-level lexicon is much less ambiguous than the lower-level one. The contributions of this paper include: (1) a fast and principled solution to the discovery of significant spatial co-occurrent patterns using frequent itemset mining; (2) a pattern summarization method that deals with the compositional uncertainties in visual phrases; and (3) a top-down refinement scheme of the visual word lexicon by feeding back discovered phrases to tune the similarity measure through metric learning.

We present an unsupervised technique for visual learning, which is based on density estimation in high-dimensional spaces using an eigenspace decomposition. Two types of density estimates are derived for modeling the training data: a multivariate Gaussian (for unimodal distributions) and a Mixture-of-Gaussians model (for multimodal distributions). These probability densities are then used to formulate a maximum-likelihood estimation framework for visual search and target detection for automatic object recognition and coding. Our learning technique is applied to the probabilistic visual modeling, detection, recognition, and coding of human faces and nonrigid objects, such as hands.

In the Object Recognition task, there exists a dichotomy between the categorization of objects and estimating object pose, where the former necessitates a view-invariant representation, while the latter requires a representation capable of capturing pose information over different categories of objects. With the rise of deep architectures, the prime focus has been on object category recognition. Deep learning methods have achieved wide success in this task. In contrast, object pose estimation using these approaches has received relatively less attention. In this work, we study how Convolutional Neural Networks (CNN) architectures can be adapted to the task of simultaneous object recognition and pose estimation. We investigate and analyze the layers of various CNN models and extensively compare between them with the goal of discovering how the layers of distributed representations within CNNs represent object pose information and how this contradicts with object category representations. We extensively experiment on two recent large and challenging multi-view datasets and we achieve better than the state-of-the-art.


Question: Cross-domain visual recognition through dictionary learning with domain adaptation.

This paper presents a significant advancement in the field of signal processing, showcasing that the Orthogonal Matching Pursuit (OMP) algorithm can effectively reconstruct a signal with m non-zero components in a d-dimensional space from merely O(m ln d) random linear measurements of the signal. This is a substantial enhancement compared to previous findings, which necessitated O(m^2) measurements. The new outcomes for OMP are on par with the recent discoveries for the Basis Pursuit (BP) method. In certain scenarios, OMP outshines BP in terms of computational speed and ease of implementation, making it a compelling alternative for tackling signal recovery problems. The theoretical and empirical evidence provided in this paper supports the notion that OMP can achieve comparable, if not superior, performance to BP with fewer measurements, thereby offering a more efficient solution for signal recovery tasks.

We assess the applicability of several popular learning methods for the problem of recognizing generic visual categories with invariance to pose, lighting, and surrounding clutter. A large dataset comprising stereo image pairs of 50 uniform-colored toys under 36 azimuths, 9 elevations, and 6 lighting conditions was collected (for a total of 194,400 individual images). The objects were 10 instances of 5 generic categories: four-legged animals, human figures, airplanes, trucks, and cars. Five instances of each category were used for training, and the other five for testing. Low-resolution grayscale images of the objects with various amounts of variability and surrounding clutter were used for training and testing. Nearest neighbor methods, support vector machines, and convolutional networks, operating on raw pixels or on PCA-derived features were tested. Test error rates for unseen object instances placed on uniform backgrounds were around 13% for SVM and 7% for convolutional nets. On a segmentation/recognition task with highly cluttered images, SVM proved impractical, while convolutional nets yielded 16/7% error. A real-time version of the system was implemented that can detect and classify objects in natural scenes at around 10 frames per second.

Sparse representation based classification has led to interesting image recognition results, while the dictionary used for sparse coding plays a key role in it. This paper presents a novel dictionary learning (DL) method to improve the pattern classification performance. Based on the Fisher discrimination criterion, a structured dictionary, whose dictionary atoms have correspondence to the class labels, is learned so that the reconstruction error after sparse coding can be used for pattern classification. Meanwhile, the Fisher discrimination criterion is imposed on the coding coefficients so that they have small within-class scatter but big between-class scatter. A new classification scheme associated with the proposed Fisher discrimination DL (FDDL) method is then presented by using both the discriminative information in the reconstruction error and sparse coding coefficients. The proposed FDDL is extensively evaluated on benchmark image databases in comparison with existing sparse representation and DL based classification methods.

Object recognition is a key function in both human and machine vision. While brain decoding of seen and imagined objects has been achieved, the prediction is limited to training examples. We present a decoding approach for arbitrary objects using the machine vision principle that an object category is represented by a set of features rendered invariant through hierarchical processing. We show that visual features, including those derived from a deep convolutional neural network, can be predicted from fMRI patterns, and that greater accuracy is achieved for low-/high-level features with lower-/higher-level visual areas, respectively. Predicted features are used to identify seen/imagined object categories (extending beyond decoder training) from a set of computed features for numerous object images. Furthermore, decoding of imagined objects reveals progressive recruitment of higher-to-lower visual representations. Our results demonstrate a homology between human and machine vision and its utility for brain-based information retrieval.

We propose simple and effective models for the image annotation that make use of Convolutional Neural Network (CNN) features extracted from an image and word embedding vectors to represent their associated tags. Our first set of models is based on the Canonical Correlation Analysis (CCA) framework that helps in modeling both views - visual features (CNN feature) and textual features (word embedding vectors) of the data. Results on all three variants of the CCA models, namely linear CCA, kernel CCA and CCA with k-nearest neighbor (CCA-KNN) clustering, are reported. The best results are obtained using CCA-KNN which outperforms previous results on the Corel-5k and the ESP-Game datasets and achieves comparable results on the IAPRTC-12 dataset. In our experiments we evaluate CNN features in the existing models which bring out the advantages of it over dozens of handcrafted features. We also demonstrate that word embedding vectors perform better than binary vectors as a representation of the tags associated with an image. In addition we compare the CCA model to a simple CNN based linear regression model, which allows the CNN layers to be trained using back-propagation.

With the advent of the Internet, billions of images are now freely available online and constitute a dense sampling of the visual world. Using a variety of non-parametric methods, we explore this world with the aid of a large dataset of 79,302,017 images collected from the Internet. Motivated by psychophysical results showing the remarkable tolerance of the human visual system to degradations in image resolution, the images in the dataset are stored as 32 x 32 color images. Each image is loosely labeled with one of the 75,062 non-abstract nouns in English, as listed in the Wordnet lexical database. Hence the image database gives a comprehensive coverage of all object categories and scenes. The semantic information from Wordnet can be used in conjunction with nearest-neighbor methods to perform object classification over a range of semantic levels minimizing the effects of labeling noise. For certain classes that are particularly prevalent in the dataset, such as people, we are able to demonstrate a recognition performance comparable to class-specific Viola-Jones style detectors.

In Proc. of IEEE Conf. on CVPR’2000, Vol.II, pp.88-94, Hilton Head Island, SC, 2000 Since human hand is highly articulated and deformable, hand posture recognition is a challenging example in the research of view-independent object recognition. Due to the difficulties of the modelbased approach, the appearance-based learning approach is promising to handle large variation in visual inputs. However, the generalization of many proposed supervised learning methods to this problem often suffers from the insufficiency of labeled training data. This paper describes an approach to alleviate this difficulty by adding a large unlabeled training set. Combining supervised and unsupervised learning paradigms, a novel and powerful learning approach, the Discriminant-EM (D-EM) algorithm, is proposed in this paper to handle the case of small labeled training set. Experiments show that D-EM outperforms many other learning methods. Based on this approach, we implement a gesture interface to recognize a set of predefined gesture commands, and it is also extended to hand detection. This algorithm can also apply to other object recognition tasks.

Two sets of items can share the same underlying conceptual structure, while appearing unrelated at a surface level. Humans excel at recognizing and using alignments between such underlying structures in many domains of cognition, most notably in analogical reasoning. Here we show that structural alignment reveals how different people's neural representations of word meaning are preserved across different languages, such that patterns of brain activation can be used to translate words from one language to another. Groups of Chinese and English speakers underwent fMRI scanning while reading words in their respective native languages. Simply by aligning structures representing the two groups' neural semantic spaces, we successfully infer all seven Chinese–English word translations. Beyond language translation, conceptual structural alignment underlies many aspects of high-level cognition, and this work opens the door to deriving many such alignments directly from neural representational content.

We consider the problem of image representation for visual analysis. When representing images as vectors, the feature space is of very high dimensionality, which makes it difficult for applying statistical techniques for visual analysis. To tackle this problem, matrix factorization techniques, such as Singular Vector Decomposition (SVD) and Non-negative Matrix Factorization (NMF), received an increasing amount of interest in recent years. Matrix factorization is an unsupervised learning technique, which finds a basis set capturing high-level semantics in the data and learns coordinates in terms of the basis set. However, the representations obtained by them are highly dense and can not capture the intrinsic geometric structure in the data. In this paper, we propose a novel method, called Sparse Concept Coding (SCC), for image representation and analysis. Inspired from the recent developments on manifold learning and sparse coding, SCC provides a sparse representation which can capture the intrinsic geometric structure of the image space. Extensive experimental results on image clustering have shown that the proposed approach provides a better representation with respect to the semantic structure.

Transfer learning is generally exploited to adapt well-established source knowledge for learning tasks in weakly labeled or unlabeled target domain. Nowadays, it is common to see multiple sources available for knowledge transfer, each of which, however, may not include complete classes information of the target domain. Naively merging multiple sources together would lead to inferior results due to the large divergence among multiple sources. In this paper, we attempt to utilize incomplete multiple sources for effective knowledge transfer to facilitate the learning task in target domain. To this end, we propose an incomplete multisource transfer learning through two directional knowledge transfer, i.e., cross-domain transfer from each source to target, and cross-source transfer. In particular, in cross-domain direction, we deploy latent low-rank transfer learning guided by iterative structure learning to transfer knowledge from each single source to target domain. This practice reinforces to compensate for any missing data in each source by the complete target data. While in cross-source direction, unsupervised manifold regularizer and effective multisource alignment are explored to jointly compensate for missing data from one portion of source to another. In this way, both marginal and conditional distribution discrepancy in two directions would be mitigated. Experimental results on standard cross-domain benchmarks and synthetic data sets demonstrate the effectiveness of our proposed model in knowledge transfer from incomplete multiple sources.

We present a novel method for generic visual categorization: the problem of identifying the object content of natural images while generalizing across variations inherent to the object class. This bag of keypoints method is based on vector quantization of affine invariant descriptors of image patches. We propose and compare two alternative implementations using different classifiers: Naïve Bayes and SVM. The main advantages of the method are that it is simple, computationally efficient and intrinsically invariant. We present results for simultaneously classifying seven semantic visual categories. These results clearly demonstrate that the method is robust to background clutter and produces good categorization accuracy even without exploiting geometric information.