Question: The design and life-cycle phases of a data warehouse system.

This paper introduces a formal approach to conceptual modeling for ETL (Extraction, Transformation, Loading) processes in data warehouse design. ETL tools are crucial software components tasked with extracting data from diverse sources, cleansing it, customizing it, and loading it into a data warehouse. The focus here is on defining ETL activities and providing a conceptual framework for their representation. The proposed model is tailored to track inter-attribute relationships and ETL activities from the early stages of a data warehouse project. It is augmented with a predefined set of common ETL operations, such as the assignment of surrogate keys and null value checks, encapsulated in a 'palette'. Moreover, the model is designed to be both customizable and extensible, allowing the designer to incorporate their own recurring patterns for ETL activities, thereby enhancing the model's adaptability and utility in various data warehouse design scenarios.

I To cope with issues related to optimization, rationalization, risk management, economic value of technology and information assets, the implementation of appropriate IT governance seems an important need in public and private organizations. It’s one of these concepts that suddenly emerged and became an important issue in the information technology area. Many organizations started with the implementation of IT governance to achieve a better alignment between business and IT, however, there is no method to apply the IT governance principles in companies. This paper proposes a new approach to implement IT governance based on five iterative phases. This approach is a critical business process that ensures that the business meets its strategic objectives and depending on IT resources for execution. Keywords—Information technology, IT governance, lifecycle, strategic alignment, value

This paper focuses on the influence of BI on manufacturing and distribution organizations, although much of what’s covered in it can apply to businesses in non-related industries, as well. To kick it off, the chart below illustrates the primary value propositions of business intelligence and the types of business analytics (e.g., Sales, Inventory, Supplier Performance analysis) that support each value area. The effect of BI on each of these business value areas are then discussed in more detail in the following sections.

0 7 4 0 7 4 5 9 / 0 3 / $ 1 9 . 0 0 © 2 0 0 3 I E E E All the methodologies that have been proposed recently for choosing software packages compare user requirements with the packages’ capabilities.3–5 There are different types of requirements, such as managerial, political, and, of course, quality requirements. Quality requirements are often difficult to check. This is partly due to their nature, but there is another reason that can be mitigated, namely the lack of structured and widespread descriptions of package domains (that is, categories of software packages such as ERP systems, graphical or data structure libraries, and so on). This absence hampers the accurate description of software packages and the precise statement of quality requirements, and consequently overall package selection and confidence in the result of the process. Our methodology for building structured quality models helps solve this drawback. (See the “Related Work” sidebar for information about other approaches.) A structured quality model for a given package domain provides a taxonomy of software quality features, and metrics for computing their value. Our approach relies on the International Organization for Standardization and International Electrotechnical Commission 9126-1 quality standard,6 which we selected for the following reasons:

In the future many people in industrialized countries will manage their personal health data electronically in centralized, reliable and trusted repositories - so-called personal health record systems (PHR). At this stage PHR systems still fail to satisfy the individual medical information needs of their users. Personalized recommendations could solve this problem.
 A first approach of integrating recommender system (RS) methodology into personal health records - termed health recommender system (HRS) - is presented. By exploitation of existing semantic networks like Wikipedia a health graph data structure is obtained. The data kept within such a graph represent health related concepts and are used to compute semantic distances among pairs of such concepts.
 A ranking procedure based on the health graph is outlined which enables a match between entries of a PHR system and health information artifacts. This way a PHR user will obtain individualized health information he might be interested in.

The use of data mining techniques in banking domain is suitable due to the nature and sensitivity of bank data and due to the real time complex decision processes. The main concern for a bank's manager is to take good decisions in order to minimize the risks level associated to bank's activities. It is very important for a bank to have knowledge of causes which generate the financial crises or imbalances. Lending is one of the most risky activities in banking area and adequate methods to support the decision making process are necessary. In this paper the authors present a prototype decision support system based on data mining techniques used in lending process. The proposed system was designed to assist a customer who applies for a credit and it may represent an extension for e-banking activities.

Early phases of information systems engineering include the understanding of the enterprise’s context and the construction of models at different levels of decomposition, required to design the system architecture. These time-consuming activities are usually conducted by relatively large teams, composed of groups of non-technical stakeholders playing mostly an informative role (i.e. not involved in documentation and even less in modelling), led by few experienced technical consultants performing most of the documenting and modelling effort. This paper evaluates the ability of non-technical stakeholders to create strategic dependency diagrams written with the i* language in the design of the context model of a system architecture, and find out which difficulties they may encounter and what the quality of the models they build is. A case study involving non-technical stakeholders from 11 organizational areas in an Ecuadorian university held under the supervision and coordination of the two authors acting as consultants. The non-technical stakeholders identified the majority of the dependencies that should appear in the case study’s context model, although they experienced some difficulties in declaring the type of dependency, representing such dependencies graphically and applying the description guidelines provided in the training. Managers were observed to make more mistakes than other more operational roles. From the observations of these results, a set of methodological advices were compiled for their use in future, similar endeavours. It is concluded that non-technical stakeholders can take an active role in the construction of the context model. This conclusion is relevant for both researchers and practitioners involved in technology transfer actions with use of i*.

Understanding user requirements is an integral part of information systems design and is critical to the success of interactive systems. However specifying these requirements is not so simple to achieve. This paper describes general methods to support user requirements analysis that can be adapted to a range of situations. Some brief case studies are described to illustrate how these methods have been applied in practice.

S ystems are now being designed that can scale to very large configurations. Examples include parallel architectures with thousands of processors, wireless networks connecting tens of thousands of computers, and parallel database systems capable of processing millions of transactions a second. In some cases, systems have grown considerably larger than envisaged by their designers. Design and development costs for such systems could be significantly reduced if only there were efficient techniques for evaluating design alternatives and predicting their impact on overall system performance metrics. Due to the systems' analytical intractability, simulation is the most common performance evaluation technique for such systems. However, the long execution times needed for sequential simulation models often hamper evaluation. For instance, it can easily take weeks to simulate wireless networks with thousands of mobile nodes that are subject to the interference and signal-fading effects of a realistic environment. Also, the simulation of parallel programs typically suffers from slowdown factors of 30 to 50 per processor. 1 This means that simulation of a parallel program executing for five to 15 minutes on a 128-node multi-processor can take anywhere from a few weeks to a few months—even on state-of-the-art sequential workstations. As the size of the physical system increases, the models' memory requirements can easily exceed the workstations' capacity. The limitations of sequential model execution have led to growing interest in the use of parallel execution for simulating large-scale systems. Widespread use of parallel simulation, however, has been significantly hindered by a lack of tools for integrating parallel model execution into the overall framework of system simulation. Although a number of algorithmic alternatives exist for parallel execution of discrete-event simulation models, performance analysts not expert in parallel simulation have relatively few tools giving them flexibility to experiment with multiple algorithmic or architectural alternatives for model execution. Another drawback to widespread use of simulations is the cost of model design and maintenance. The design and development costs for detailed simulation models for complex systems can easily rival the costs for the physical systems themselves. The simulation environment we developed at UCLA attempts to address some of these issues by providing these features: • An easy path for the migration of simulation models to operational software prototypes. • Implementation on both distributed-and shared-memory platforms and support for a diverse set of parallel simulation protocols. • Support for visual and hierarchical model design. Our environment consists of three primary components: …

Spreadsheets contain a huge amount of high-value data but do not observe a standard data model and thus are difficult to integrate. A large number of data integration tools exist, but they generally can only work on relational data. Existing systems for extracting relational data from spreadsheets are too labor intensive to support ad-hoc integration tasks, in which the correct extraction target is only learned during the course of user interaction.
 This paper introduces a system that automatically extracts relational data from spreadsheets, thereby enabling relational spreadsheet integration. The resulting integrated relational data can be queried directly or can be translated into RDF triples. When compared to standard techniques for spreadsheet data extraction on a set of 100 random Web spreadsheets, the system reduces the amount of human labor by 72% to 92%. In addition to the system design, we present the results of a general survey of more than 400,000 spreadsheets we downloaded from the Web, giving a novel view of how users organize their data in spreadsheets.

We live life in the network. We check our e-mails regularly, make mobile phone calls from almost any location ... make purchases with credit cards ... [and] maintain friendships through online social networks. ... These transactions leave digital traces that can be compiled into comprehensive pictures of both individual and group behavior, with the potential to transform our understanding of our lives, organizations, and societies.


Question: The design and life-cycle phases of a data warehouse system.

This paper explores the evolution of business intelligence (BI) beyond traditional data warehousing, highlighting the shift in corporate management strategies over the past decade. Companies have recognized the significance of aligning their strategic objectives with metrics-driven decision-making. While data warehousing (DW) facilitates the bottom-up analysis of data for information extraction, it falls short in directly supporting top-down strategy enforcement. A new BI paradigm, termed Business Performance Management (BPM), is gaining traction. BPM encompasses DW but also necessitates a reactive component for monitoring critical operational processes in real-time. This enables tactical and operational decision-makers to adjust their actions in line with the company's strategic goals. The objective of this paper is to stimulate academic interest in the advent of a new era in BI, to outline a comprehensive architecture for BPM, and to set the groundwork for exploring the most pressing challenges associated with this transition.

**Question**: An Overview of Quantum Computing

**Documents**: Quantum Computing for Computer Scientists
Quantum computing is an exciting new field that combines the principles of quantum physics with computer science to solve problems that are intractable for classical computers. Quantum computers use qubits, which can be in a superposition of states, allowing them to process a vast amount of data simultaneously. Additionally, quantum computers leverage quantum entanglement, a phenomenon where qubits become correlated in such a way that the state of one qubit can depend on the state of another, even over long distances. This book provides an introduction to quantum computing for computer scientists, presenting the fundamental concepts and algorithms in a way that is accessible to those with a background in computer science. It covers topics such as quantum mechanics, quantum gates, quantum circuits, and quantum algorithms, including Shor's algorithm for factoring large numbers and Grover's algorithm for searching unsorted databases.

**Similar Documents**: This text offers a comprehensive guide to the emerging field of quantum computing, a groundbreaking area that merges the fundamentals of quantum physics with computer science to tackle complex problems beyond the reach of classical computing systems. Quantum computers, powered by qubits that can exist in multiple states simultaneously, are capable of processing extensive datasets concurrently. They also capitalize on quantum entanglement, a unique property where qubits can become interconnected in such a manner that the state of one qubit is dependent on the state of another, regardless of spatial separation. This book serves as an introductory resource for computer scientists, elucidating the core principles and algorithms of quantum computing in an accessible format. It delves into essential topics such as quantum mechanics, quantum gate operations,

This paper gives an overview about the development of the field of Knowledge Engineering over the last 15 years. We discuss the paradigm shift from a transfer view to a modeling view and describe two approaches which considerably shaped research in Knowledge Engineering: Role-limiting Methods and Generic Tasks. To illustrate various concepts and methods which evolved in the last years we describe three modeling frameworks: CommonKADS, MIKE, and PROTÉGÉ-II. This description is supplemented by discussing some important methodological developments in more detail: specification languages for knowledge-based systems, problem-solving methods, and ontologies. We conclude with outlining the relationship of Knowledge Engineering to Software Engineering, Information Integration and Knowledge Management.

— To classify data mining problems and algorithms we used two dimensions: data type and type of mining operations. One of the main issue that arise during the data mining process is treating data that contains temporal information. The area of temporal data mining has very much attention in the last decade because from the time related feature of the data, one can extract much significant information which can not be extracted by the general methods of data mining. Many interesting techniques of temporal data mining were proposed and shown to be useful in many applications. Since temporal data mining brings together techniques from different fields such as databases, statistics and machine learning the literature is scattered among many different sources. In this paper, we present a survey on techniques of temporal data mining.

0 7 4 0 7 4 5 9 / 0 3 / $ 1 9 . 0 0 © 2 0 0 3 I E E E All the methodologies that have been proposed recently for choosing software packages compare user requirements with the packages’ capabilities.3–5 There are different types of requirements, such as managerial, political, and, of course, quality requirements. Quality requirements are often difficult to check. This is partly due to their nature, but there is another reason that can be mitigated, namely the lack of structured and widespread descriptions of package domains (that is, categories of software packages such as ERP systems, graphical or data structure libraries, and so on). This absence hampers the accurate description of software packages and the precise statement of quality requirements, and consequently overall package selection and confidence in the result of the process. Our methodology for building structured quality models helps solve this drawback. (See the “Related Work” sidebar for information about other approaches.) A structured quality model for a given package domain provides a taxonomy of software quality features, and metrics for computing their value. Our approach relies on the International Organization for Standardization and International Electrotechnical Commission 9126-1 quality standard,6 which we selected for the following reasons:

Many of the problems arjd failures of Management Information Systems (MIS) and Management Science/Operations Research (MS/OR) projects have been attributed to organizational behavioral problems. The millions of dollars organizations spend on MIS and MS/OR development are of little benefit because systems continue to fail. Steps can be taken to understand and solve these behavioral probiems. This article argues that in most oases these behavioral problems are the result of inadequate designs. These bad designs are attributed to the way MIS systems designers view organizations, their members, and the function of an MIS within them. I.e., systems designers' frames of reference. These frames of reference cause faulty design choices and failures to perceive better design alternatives. Seven conditions are discussed which reflect current systems designers' points of view. The discussion of these conditions demonstrates the need to refrawe MIS design methodology within the Socio-Technicai Systems (STS) design approach and change systems designers' perspectives. The STS approach is introduced as a realistic view of organizations and a way to change them. This article is the first of two to appear in consecutive issues of the MIS Quarterly. The purpose of this first article is to demonstrate the need for the srs approaj:h. The second will present the basic concepts and principles of the STS methodology and how it can be utilized in the design of an MIS.

In today’s business where speed is of essence, an iterative development approach that allows the functionality to be delivered in parts has become a necessity and an effective way to manage risks. In an iterative process, the development of a software system is done in increments, each increment forming of an iteration and resulting in a working system. A common iterative approach is to decide what should be developed in an iteration and then plan the iteration accordingly. A somewhat different iterative is approach is to time box different iterations. In this approach, the length of an iteration is fixed and what should be developed in an iteration is adjusted to fit the time box. Generally, the time boxed iterations are executed in sequence, with some overlap where feasible. In this paper we propose the timeboxing process model that takes the concept of time boxed iterations further by adding pipelining concepts to it for permitting overlapped execution of different iterations. In the timeboxing process model, each time boxed iteration is divided into equal length stages, each stage having a defined function and resulting in a clear work product that is handed over to the next stage. With this division into stages, pipelining concepts are employed to have multiple time boxes executing concurrently, leading to a reduction in the delivery time for product releases. We illustrate the use of this process model through an example of a commercial project that was successfully executed using the proposed model.

Micro finance organizations play an important role since they facilitate integration of all social classes to sustained economic growth. Against this background, exponential growth of data, resulting from transactions and operations carried out with these companies on a daily basis, becomes imminent. Appropriate management of this data is therefore necessary because, otherwise, it will result in a competitive disadvantage due to the lack of valuable and quality information for decision-making and process improvement. Data Governance provides a different approach to data management, as seen from the perspective of business assets. In this regard, it is necessary that the organization have the ability to assess the extent to which that management is correct or is generating expected results. This paper proposes a data governance maturity model for micro finance organizations, which frames a series of formal requirements and criteria providing an objective diagnosis. This model was implemented based on the information of a Peruvian micro finance organization. Four domains, out of the seven listed in the model, were evaluated. Finally, after validation of the proposed model, it was evidenced that it serves as a means for identifying the gap between data management and objectives set.

The problems in building a transaction processing system are discussed, and it is shown that the difficulties are a function of specific attributes of the underlying database system. A model of a transaction processing system is presented, and five system dimensions important in classifying transaction processing systems-the process, machine, heterogeneity, data, and site components-are introduced. The specific problems posed by various combinations of system characteristics are analyzed. The evolution of transaction processing systems are described in terms of the framework.<<ETX>>

Surveying a suite of algorithms that offer a solution to managing large document archives.

T he articles in this issue ofDATA BASE were chosen b y Anthony G . Hopwood, who is a professor of accounting and financial reporting at the London Graduate Schoo l of Business Studies . The articles contain important ideas , Professor Hopwood wrote, of significance to all intereste d in information systems, be they practitioners or academics . The authors, with their professional affiliations at th e time, were Chris Argyris, Graduate School of Education , Harvard University; Bo Hedberg and Sten Jonsson, Department of Business Administration, University o f Gothenburg; J . Frisco den Hertog, N . V. Philips' Gloeilampenfabrieken, The Netherlands, and Michael J . Earl, Oxford Centre for Management Studies . The articles appeared originally in Accounting, Organizations and Society, a publication of which Professor Hopwood is editor-in-chief. AOS exists to monitor emergin g developments and to actively encourage new approaches and perspectives .

Educational Data Mining (EDM) is an emerging multidisciplinary research area, in which methods and techniques for exploring data originating from various educational information systems have been developed. EDM is both a learning science, as well as a rich application area for data mining, due to the growing availability of educational data. EDM contributes to the study of how students learn, and the settings in which they learn. It enables data-driven decision making for improving the current educational practice and learning material. We present a brief overview of EDM and introduce four selected EDM papers representing a crosscut of different application areas for data mining in education.


Question: The design process and life-cycle stages of a data warehouse.

This paper introduces a novel methodology for the modeling of ETL (Extraction-Transformation-Loading) processes in data warehouse (DW) systems, utilizing the Unified Modeling Language (UML). DWs serve a critical role in supporting the decision-making capabilities of knowledge workers by consolidating data from diverse operational sources. The accuracy and reliability of ETL processes are paramount, as they extract, transform, and load data into DWs. A flawed ETL design can lead to incorrect data, resulting in suboptimal business decisions. Despite the significance of ETL processes, there has been a notable scarcity of research dedicated to their modeling. Our proposed approach fills this gap by offering an integrated UML-based framework for the conceptual modeling of ETL processes alongside the DW's conceptual schema. This unified perspective provides a streamlined mechanism for specifying common ETL operations, including data integration from multiple sources, attribute transformations, and surrogate key generation. The methodology also supports comprehensive documentation and tracking of ETL processes, greatly simplifying maintenance. A key benefit of our approach is the leveraging of UML's standardization, user-friendliness, and functional richness, which facilitates a seamless integration of ETL process design with the DW's conceptual schema. To illustrate the practical application of our integrated approach, we demonstrate its use with a popular modeling tool like Rational Rose.

Knowledge discovery in databases is a rapidly growing field, whose development is driven by strong research interests as well as urgent practical, social, and economical needs. While the last few years knowledge discovery tools have been used mainly in research environments, sophisticated software products are now rapidly emerging. In this paper, we provide an overview of common knowledge discovery tasks and approaches to solve these tasks. We propose a feature classification scheme that can be used to study knowledge and data mining software. This scheme is based on the software's general characteristics, database connectivity, and data mining characteristics. We then apply our feature classification scheme to investigate 43 software products, which are either research prototypes or commercially available. Finally, we specify features that we consider important for knowledge discovery software to possess in order to accommodate its users effectively, as well as issues that are either not addressed or insufficiently solved yet.

Today web is the best medium of communication in modern business. Many companies are redefining their business strategies to improve the business output. Business over internet provides the opportunity to customers and partners where their products and specific business can be found. Nowadays online business breaks the barrier of time and space as compared to the physical office. Big companies around the world are realizing that e-commerce is not just buying and selling over Internet, rather it improves the efficiency to compete with other giants in the market. For this purpose data mining sometimes called as knowledge discovery is used. Web mining is data mining technique that is applied to the WWW. There are vast quantities of information available over the Internet.

This chapter covers the fundamentals of spatial database analysis and design, It begins by defining the most important concepts: ‘spatial database’, ‘analysis’, ‘design’, and ‘model’; and continues with a presentation of the rationale supporting the use of formal methods for analysis and design. The basic elements and approaches of such methods are described, in addition to the processes used. Emphasis is placed on the particularities of spatial databases and the improvements needed for non-spatial methods and tools in order to enhance their efficiency. Finally, the chapter presents a set of tools, called CASE (computer-assisted software engineering), which are built to support the formal analysis and design methods.

Data integration is the problem of combining data residing at different sources, and providing the user with a unified view of these data. The problem of designing data integration systems is important in current real world applications, and is characterized by a number of issues that are interesting from a theoretical point of view. This document presents on overview of the material to be presented in a tutorial on data integration. The tutorial is focused on some of the theoretical issues that are relevant for data integration. Special attention will be devoted to the following aspects: modeling a data integration application, processing queries in data integration, dealing with inconsistent data sources, and reasoning on queries.

In this paper we present the data manipulation facility for a structured English query language (SEQUEL) which can be used for accessing data in an integrated relational data base. Without resorting to the concepts of bound variables and quantifiers SEQUEL identifies a set of simple operations on tabular structures, which can be shown to be of equivalent power to the first order predicate calculus. A SEQUEL user is presented with a consistent set of keyword English templates which reflect how people use tables to obtain information. Moreover, the SEQUEL user is able to compose these basic templates in a structured manner in order to form more complex queries. SEQUEL is intended as a data base sublanguage for both the professional programmer and the more infrequent data base user.

— To classify data mining problems and algorithms we used two dimensions: data type and type of mining operations. One of the main issue that arise during the data mining process is treating data that contains temporal information. The area of temporal data mining has very much attention in the last decade because from the time related feature of the data, one can extract much significant information which can not be extracted by the general methods of data mining. Many interesting techniques of temporal data mining were proposed and shown to be useful in many applications. Since temporal data mining brings together techniques from different fields such as databases, statistics and machine learning the literature is scattered among many different sources. In this paper, we present a survey on techniques of temporal data mining.

Ž . Design management and process management are two important elements of total quality management TQM implementation. They are drastically different in their targets of improvement, visibility, and techniques. In this paper, we establish a framework for identifying the synergistic linkages of design and process management to the operational quality Ž . Ž . outcomes during the manufacturing process internal quality and upon the field usage of the products external quality . Through a study of quality practices in 418 manufacturing plants from multiple industries, we empirically demonstrate that both design and process management efforts have an equal positive impact on internal quality outcomes such as scrap, rework, defects, performance, and external quality outcomes such as complaints, warranty, litigation, market share. A detailed contingency analysis shows that the proposed model of synergies between design and process management holds true for large and small firms; for firms with different levels of TQM experience; and in different industries with varying levels of competition, logistical complexity of production, or production process characteristics. Finally, the results also suggest that organizational learning enables mature TQM firms to implement both design and process efforts more rigorously and their synergy helps these firms to attain better quality outcomes. These findings indicate that, to attain superior quality outcomes, firms need to balance their design and process management efforts and persevere with long-term Ž . implementation of these efforts. Because the study spans all of the manufacturing sectors SIC 20 through 39 , these conclusions should help firms in any industry revisit their priorities in terms of the relative efforts in design management and process management. q 2000 Elsevier Science B.V. All rights reserved.

Measuring and improving data quality in an organization or in a group of interacting organizations is a complex task. Several methodologies have been developed in the past providing a basis for the definition of a complete data quality program applying assessment and improvement techniques in order to guarantee high data quality levels. Since the main limitation of existing approaches is their specialization on specific issues or contexts, this paper presents the comprehensive data quality (CDQ) methodology that aims at integrating and enhancing the phases, techniques and tools proposed by previous approaches. CDQ methodology is conceived to be at the same time complete, flexible and simple to apply. Completeness is achieved by considering existing techniques and tools and integrating them in a framework that can work in both intra and inter organizational contexts, and can be applied to all types of data. The methodology is flexible since it supports the user in the selection of the most suitable techniques and tools within each phase and in any context. Finally, CDQ is simple since it is organized in phases and each phase is characterized by a specific goal and techniques to apply. The methodology is explained by means of a running example.

This paper presents a framework for process mining in critical care. The framework uses the CRISP-DM model, extended to incorporate temporal and multidimensional aspects (CRISP-TDM), combined with the Patient Journey Modeling Architecture (PaJMa), to provide a structured approach to knowledge discovery of new condition onset pathophysiologies in physiological data streams. The approach is based on temporal abstraction and mining of physiological data streams to develop process flow mappings that can be used to update patient journeys; instantiated in critical care within clinical practice guidelines. We demonstrate the framework within the neonatal intensive care setting, where we are performing clinical research in relation to pathophysiology within physiological streams of patients diagnosed with late onset neonatal sepsis. We present an instantiation of the framework for late onset neonatal sepsis, using CRISP-TDM for the process mining model and PaJMa for the knowledge representation.

Knowledge Discovery in Databases (KDD) has grown a lot during the last years. But providing user support for constructing workflows is still problematic. The large number of operators available in current KDD systems makes it difficult for a user to successfully solve her task. Also, workflows can easily reach a huge number of operators(hundreds) and parts of the workflows are applied several times. Therefore, it becomes hard for the user to construct them manually. In addition, workflows are not checked for correctness before execution. Hence, it frequently happens that the execution of the workflow stops with an error after several hours runtime. In this paper we present a solution to these problems. We introduce a knowledge-based representation of Data Mining (DM) workflows as a basis for cooperative interactive planning. Moreover, we discuss workflow templates, i.e. abstract workflows that can mix executable operators and tasks to be refined later into sub-workflows. This new representation helps users to structure and handle workflows, as it constrains the number of operators that need to be considered. Finally, workflows can be grouped in templates which foster re-use further simplifying DM workflow construction. Posted at the Zurich Open Repository and Archive, University of Zurich ZORA URL: https://doi.org/10.5167/uzh-44851 Originally published at: Kietz, Jörg-Uwe; Serban, Floarea; Bernstein, Abraham; Fischer, Simon (2010). Data mining workflow templates for intelligent discovery assistance and auto-experimentation. In: Proc of the ECML/PKDD’10 Workshop on Third Generation Data Mining: Towards Service-oriented Knowledge Discovery (SoKD’10), Barcelona, Spain, 20 September 2010 24 September 2010, 1-12. Data Mining Workflow Templates for Intelligent Discovery Assistance and Auto-Experimentation Jörg-Uwe Kietz, Floarea Serban, Abraham Bernstein, and Simon Fischer 1 University of Zurich, Department of Informatics, Dynamic and Distributed Information Systems Group, Binzmühlestrasse 14, CH-8050 Zurich, Switzerland {kietz|serban|bernstein}@ifi.uzh.ch 2 Rapid-I GmbH, Stockumer Str. 475, 44227 Dortmund, Germany fischer@rapid-i.com Abstract. Knowledge Discovery in Databases (KDD) has grown a lot during the last years. But providing user support for constructing workflows is still problematic. The large number of operators available in current KDD systems makes it difficult for a user to successfully solve her task. Also, workflows can easily reach a huge number of operators (hundreds) and parts of the workflows are applied several times. Therefore, it becomes hard for the user to construct them manually. In addition, workflows are not checked for correctness before execution. Hence, it frequently happens that the execution of the workflow stops with an error after several hours runtime. In this paper we present a solution to these problems. We introduce a knowledge-based representation of Data Mining (DM) workflows as a basis for cooperative-interactive planning. Moreover, we discuss workflow templates, i.e. abstract workflows that can mix executable operators and tasks to be refined later into sub-workflows. This new representation helps users to structure and handle workflows, as it constrains the number of operators that need to be considered. Finally, workflows can be grouped in templates which foster re-use further simplifying DM workflow construction. Knowledge Discovery in Databases (KDD) has grown a lot during the last years. But providing user support for constructing workflows is still problematic. The large number of operators available in current KDD systems makes it difficult for a user to successfully solve her task. Also, workflows can easily reach a huge number of operators (hundreds) and parts of the workflows are applied several times. Therefore, it becomes hard for the user to construct them manually. In addition, workflows are not checked for correctness before execution. Hence, it frequently happens that the execution of the workflow stops with an error after several hours runtime. In this paper we present a solution to these problems. We introduce a knowledge-based representation of Data Mining (DM) workflows as a basis for cooperative-interactive planning. Moreover, we discuss workflow templates, i.e. abstract workflows that can mix executable operators and tasks to be refined later into sub-workflows. This new representation helps users to structure and handle workflows, as it constrains the number of operators that need to be considered. Finally, workflows can be grouped in templates which foster re-use further simplifying DM workflow construction.


Question: The design process and life-cycle stages of a data warehouse.

Introducing Potter's Vase: An Enhanced Interactive Data Cleaning System for Data Warehouse Optimization

Ensuring the accuracy and integrity of data, particularly in the context of data warehousing and integration, is crucial for effective decision-making processes. Existing methodologies for data cleansing typically involve a series of repetitive audits to identify errors, followed by time-consuming transformations to rectify them. This often leads to significant delays and requires users to possess advanced scripting skills to execute complex transformations. In response to these challenges, we unveil Potter's Vase, an advanced interactive data cleaning system designed to streamline the process of data cleansing by integrating discrepancy detection and transformation capabilities seamlessly.

Potter's Vase offers a user-friendly interface that mirrors a spreadsheet, enabling users to incrementally construct transformations to cleanse their data. Users can intuitively add or reverse transformations in real-time, observing the immediate impact on the displayed records. These modifications can be executed through straightforward graphical commands or by demonstrating the desired outcomes on a selection of data values. Behind the scenes, Potter's Vase autonomously discerns the structure of data values, guided by user-defined domains, to monitor for any violations of established constraints. This allows users to progressively refine their transformations as discrepancies are discovered, effectively cleansing the data without the need for intricate programming or enduring lengthy processing times.

Potter's Vase represents a significant advancement in the field of data cleaning, offering a more efficient and user-friendly alternative to traditional data cleansing techniques. By simplifying the process of identifying and rectifying errors in data structure and content, it enhances the overall quality of data in data warehousing environments, thereby improving the accuracy of subsequent data analysis and decision-making.

Cleaning data of errors in structure and content is important for data warehousing and integration. Current solutions for data cleaning involve many iterations of data “auditing” to find errors, and long-running transformations to fix them. Users need to endure long waits, and often write complex transformation scripts. We present Potter’s Wheel, an interactive data cleaning system that tightly integrates transformation and discrepancy detection. Users gradually build transformations to clean the data by adding or undoing transforms on a spreadsheet-like interface; the effect of a transform is shown at once on records visible on screen. These transforms are specified either through simple graphical operations, or by showing the desired effects on example data values. In the background, Potter’s Wheel automatically infers structures for data values in terms of user-defined domains, and accordingly checks for constraint violations. Thus users can gradually build a transformation as discrepancies are found, and clean the data without writing complex programs or enduring long delays.

Currently, the IoT discussion is focused primarily on the operational phase. This includes how a IoT device behaves, operates, communicates, and interacts with other IoT devices during operation. However, IoT devices and systems have other lifecycle phases before and after operation. This extended abstract provides an overview of how other IoT lifecycle phases (e.g., design and service) can be improved with information feedback and feedforward flows between them. Digital Twins are a new mechanism to manage IoT devices and IoT systems-of-systems throughout their lifecycle. We present our vision on the industrial IoT lifecycle managed and optimized at scale via Digital Twins.

In today’s business where speed is of essence, an iterative development approach that allows the functionality to be delivered in parts has become a necessity and an effective way to manage risks. In an iterative process, the development of a software system is done in increments, each increment forming of an iteration and resulting in a working system. A common iterative approach is to decide what should be developed in an iteration and then plan the iteration accordingly. A somewhat different iterative is approach is to time box different iterations. In this approach, the length of an iteration is fixed and what should be developed in an iteration is adjusted to fit the time box. Generally, the time boxed iterations are executed in sequence, with some overlap where feasible. In this paper we propose the timeboxing process model that takes the concept of time boxed iterations further by adding pipelining concepts to it for permitting overlapped execution of different iterations. In the timeboxing process model, each time boxed iteration is divided into equal length stages, each stage having a defined function and resulting in a clear work product that is handed over to the next stage. With this division into stages, pipelining concepts are employed to have multiple time boxes executing concurrently, leading to a reduction in the delivery time for product releases. We illustrate the use of this process model through an example of a commercial project that was successfully executed using the proposed model.

This paper extends the Function-Behaviour-Structure (FBS) framework, which proposed eight fundamental processes involved in designing. This framework did not explicitly account for the dynamic character of the context in which designing takes place, described by the notion of situatedness. This paper describes this concept as a recursive interrelationship between different environments, which, together with a model of constructive memory, provides the foundation of a situated FBS framework. The eight fundamental processes are then reconstructed within this new framework to represent designing in a dynamic world.

Discusses how the VisDB system supports the query specification process by representing the result visually. The main idea behind the system stems from the view of relational database tables as sets of multidimensional data where the number of attributes corresponds to the number of dimensions. In such a view, it is often unclear. In this system, each display pixel represents one database item. Pixels are arranged and colored to indicate the item's relevance to a user query and to give a visual impression of the resulting data set.<<ETX>>

This paper takes an initial step forward in bringing to life the certification mechanisms according to Art. 42 of the General Data Protection Regulation (GDPR). These newly established methods of legal specification act not only as a central vehicle for overcoming widely articulated and discussed legal challenges, but also as a sandbox for the much needed close collaboration between computer sciences and legal studies. In order to illustrate, for example, what data protection seals could look like in the future, the authors propose a methodology for "translating" legal requirements into technical guidelines: architectural blueprints designed using legal requirements. The purpose of these blueprints is to show developers how their solutions might comply with the principle of Privacy by Design (Art. 25 GDPR). To demonstrate this methodology, the authors propose an architectural blueprint that embodies the legal concept of the data subject’s consent (Art. 6 sec. 1 lit. a GDPR) and elevates best practice to a high standard of Privacy by Design. Finally, the authors highlight further legal problems concerning blockchain technology under the GDPR that will have to be addressed in order to achieve a comprehensive certification mechanism for Privacy by Blockchain Design in the future. ACM Classification

In this paper, we describe a novel analytics system that enables query processing and predictive analytics over streams of aviation data. As part of an Internal Research and Development project, Boeing Research and Technology (BR&T) Advanced Air Traffic Management (AATM) built a system that makes predictions based upon descriptive patterns of archived aviation data. Boeing AATM has been receiving live Aircraft Situation Display to Industry (ASDI) data and archiving it for over two years. At the present time, there is not an easy mechanism to perform analytics on the data. The incoming ASDI data is large, compressed, and requires correlation with other flight data before it can be analyzed.
 The service exposes this data once it has been uncompressed, correlated, and stored in a data warehouse for further analysis using a variety of descriptive, predictive, and possibly prescriptive analytics tools. The service is being built partially in response to requests from Boeing Commercial Aviation (BCA) for analysis of capacity and flow in the US National Airspace System (NAS). The service utilizes a custom tool for correlating the raw ASDI feed, IBM Warehouse with DB2 for data management, WebSphere Message Broker for real-time message brokering, SPSS Modeler for statistical analysis, and Cognos BI for front-end business intelligence (BI) visualization. This paper describes a scalable service architecture, implementation and the value it adds to the aviation domain.

Designations used by companies to distinguish their products are often claimed as trademarks or registered trademarks. In all instances in which Morgan Kaufmann Publishers is aware of a claim, the product names appear in initial capital or all capital letters. Readers, however, should contact the appropriate companies for more complete information regarding trademarks and registration. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means—electronic, mechanical, photocopying, scanning, or otherwise—without prior written permission of the publisher. may also complete your request on-line via the Elsevier homepage (http://elsevier.com), by selecting " Support & Contact " then " Copyright and Permission " and then " Obtaining Permissions. " Page 197: " Make a new plan Stan. .. and get yourself free. Physical database design : the database professional's guide to exploiting indexes, views, storage, and more / Sam Lightstone, Toby Teorey, and Tom Nadeau. p. cm.-(The Morgan Kaufmann series in database management systems) Includes bibliographical references and index.

Collecting and processing provenance, i.e., information describing the production process of some end product, is important in various applications, e.g., to assess quality, to ensure reproducibility, or to reinforce trust in the end product. In the past, different types of provenance meta-data have been proposed, each with a different scope. The first part of the proposed tutorial provides an overview and comparison of these different types of provenance. To put provenance to good use, it is essential to be able to interact with and present provenance data in a user-friendly way. Often, users interested in provenance are not necessarily experts in databases or query languages, as they are typically domain experts of the product and production process for which provenance is collected (biologists, journalists, etc.). Furthermore, in some scenarios, it is difficult to use solely queries for analyzing and exploring provenance data. The second part of this tutorial therefore focuses on enabling users to leverage provenance through adapted visualizations. To this end, we will present some fundamental concepts of visualization before we discuss possible visualizations for provenance.

We identify top management leadership, a sophisticated management infrastructure, process management efficacy, and stakeholder participation as important elements of a quality-oriented organizational system for software development. A model interrelating these constructs and quality performance is proposed. Data collected through a national survey of IS executives in Fortune 1000 companies and government agencies was used to Robert Zmud was the accepting senior editor for this paper. test the model using a Partial Least Squares analysis methodology. Our results suggest that software quality goals are best attained when top management creates a management infrastructure that promotes improvements in process design and encourages stakeholders to evolve the design of the development processes. Our results also suggest that all elements of the organizational system need to be developed in order to attain quality goals and that piecemeal adoption of select quality management practices are unlikely to be effective. Implications of this research for IS theory and practice are discussed.


Question: The design and life-cycle stages of a data warehouse.

This paper presents a novel approach to conceptualizing and optimizing the Extraction-Transformation-Loading (ETL) processes within a data warehouse environment. ETL tools play a pivotal role in gathering data from diverse sources, cleansing and modifying it, and then integrating it into a data warehouse. The focus here is on the logical design of the ETL process, specifically by representing it as a graph structure. A formal logical model is employed, encompassing data stores, activities, and their constituent components. This model is then transformed into an Architecture Graph, where all entities are depicted as nodes, and four distinct types of relationships—instance-of, part-of, regulator, and provider—are illustrated as edges. To simplify the graph and enhance its clarity, basic graph transformations are introduced to reduce complexity. Moreover, to facilitate the design and evolution of the data warehouse, the paper introduces two critical metrics: dependence and responsibility. These metrics serve to quantify the interdependence and interconnectivity between different entities, thereby providing valuable insights into the system's architecture and guiding its development and maintenance.

Data lineage and data provenance are key to the management of scientific data. Not knowing the exact provenance and processing pipeline used to produce a derived data set often renders the data set useless from a scientific point of view. On the positive side, capturing provenance information is facilitated by the widespread use of workflow tools for processing scientific data. The workflow process describes all the steps involved in producing a given data set and, hence, captures its lineage. On the negative side, efficiently storing and querying workflow based data lineage is not trivial. All existing solutions use recursive queries and even recursive tables to represent the workflows. Such solutions do not scale and are rather inefficient. In this paper we propose an alternative approach to storing lineage information captured as a workflow process. We use a space and query efficient interval representation for dependency graphs and show how to transform arbitrary workflow processes into graphs that can be stored using such representation. We also characterize the problem in terms of its overall complexity and provide a comprehensive performance evaluation of the approach.

This paper gives an overview about the development of the field of Knowledge Engineering over the last 15 years. We discuss the paradigm shift from a transfer view to a modeling view and describe two approaches which considerably shaped research in Knowledge Engineering: Role-limiting Methods and Generic Tasks. To illustrate various concepts and methods which evolved in the last years we describe three modeling frameworks: CommonKADS, MIKE, and PROTÉGÉ-II. This description is supplemented by discussing some important methodological developments in more detail: specification languages for knowledge-based systems, problem-solving methods, and ontologies. We conclude with outlining the relationship of Knowledge Engineering to Software Engineering, Information Integration and Knowledge Management.

In previous papers [SC05, SBC+07], some of us predicted the end of “one size fits all” as a commercial relational DBMS paradigm. These papers presented reasons and experimental evidence that showed that the major RDBMS vendors can be outperformed by 1-2 orders of magnitude by specialized engines in the data warehouse, stream processing, text, and scientific database markets. Assuming that specialized engines dominate these markets over time, the current relational DBMS code lines will be left with the business data processing (OLTP) market and hybrid markets where more than one kind of capability is required. In this paper we show that current RDBMSs can be beaten by nearly two orders of magnitude in the OLTP market as well. The experimental evidence comes from comparing a new OLTP prototype, H-Store, which we have built at M.I.T., to a popular RDBMS on the standard transactional benchmark, TPC-C. We conclude that the current RDBMS code lines, while attempting to be a “one size fits all” solution, in fact, excel at nothing. Hence, they are 25 year old legacy code lines that should be retired in favor of a collection of “from scratch” specialized engines. The DBMS vendors (and the research community) should start with a clean sheet of paper and design systems for tomorrow’s requirements, not continue to push code lines and architectures designed for yesterday’s needs.

So far there have never been systematic evaluation criteria and entire assessment and evaluation system in Data quality internationally. Based on the research on the relative content of both international and domestic data quality, this article analyzes the requests of data quality for the large enterprises. First of all, this paper raises and builds a complete data quality assessment system. Second, the definitions and specific algorithms of data quality assessment indicators and poses data quality analysis are built to evaluate the architecture and processes. A frame structure of data quality meta-model is presented in this paper. In addition, this paper also designs an evaluation system. This system includes the classification and definition of data quality and the algorithm in evaluation index and the system and process of data quality evaluation. This paper provides credibility basis for enterprises in evaluation of data quality.

Ž . Design management and process management are two important elements of total quality management TQM implementation. They are drastically different in their targets of improvement, visibility, and techniques. In this paper, we establish a framework for identifying the synergistic linkages of design and process management to the operational quality Ž . Ž . outcomes during the manufacturing process internal quality and upon the field usage of the products external quality . Through a study of quality practices in 418 manufacturing plants from multiple industries, we empirically demonstrate that both design and process management efforts have an equal positive impact on internal quality outcomes such as scrap, rework, defects, performance, and external quality outcomes such as complaints, warranty, litigation, market share. A detailed contingency analysis shows that the proposed model of synergies between design and process management holds true for large and small firms; for firms with different levels of TQM experience; and in different industries with varying levels of competition, logistical complexity of production, or production process characteristics. Finally, the results also suggest that organizational learning enables mature TQM firms to implement both design and process efforts more rigorously and their synergy helps these firms to attain better quality outcomes. These findings indicate that, to attain superior quality outcomes, firms need to balance their design and process management efforts and persevere with long-term Ž . implementation of these efforts. Because the study spans all of the manufacturing sectors SIC 20 through 39 , these conclusions should help firms in any industry revisit their priorities in terms of the relative efforts in design management and process management. q 2000 Elsevier Science B.V. All rights reserved.

Surveying a suite of algorithms that offer a solution to managing large document archives.

ion, Databases and Conceptual Modeling (Pingree Park, Colo., June), pp. 112-114; ACM SZGMOD Rec. 11, 2 (Feb.). CODD, E. F. 1982. Relational database: A practical foundation for productivity. Commun. ACM 25, 2 (Feb.), 109-117. COLMERAUER, A. 1973. Un systeme de communication homme-machine en francais. Rapport, Groupe Intelligence Artificielle, Universite d’AixMarseille-Luminy, Marseilles, France. COLMERAUER, A., AND PIQUE, J. F. 1981. About natural logic. In Advances in Database Theory vol. 1, H. Gallaire, J. Minker, and J.-M. Nicolas, Eds. Plenum, New York, pp. 343-365. COOPER, E. C. 1980. On the expressive power of query languages for relational databases. Tech. Rep. 14-80, Computer Science Dept., Harvard Univ., Cambridge, Mass. DAHL, V. 1982. On database systems development through logic. ACM Trans. Database Syst. 7, 1 (Mar.), 102-123. DATE, C. J. 1977. An Introduction to Database Systems. Addison-Wesley, Reading, Mass. DATE, C. J. 1981. An Introduction to Database Systems, 3rd ed. Addison-Wesley, Reading, Mass. DELIYANNI, A., AND KOWALSKI, R. A. 1979. Logic and semantic networks. Commun. ACM 22, 3

Many healthcare leaders find themselves overwhelmed with data, but lack the information they need to make right decisions. Knowledge Discovery in Databases (KDD) can help organizations turn their data into information. Organizations that take advantage of KDD techniques will find that they can lower the healthcare costs while improving healthcare quality by using fast and better clinical decision making. In this paper, a review study is done on existing data mining and knowledge discovery techniques, applications and process models that are applicable to healthcare environments. The challenges for applying data mining techniques in healthcare environment will also be discussed.

Life Cycle Assessment (LCA) evaluates the environmental impact of a product through its entire life cycle, from material extraction to final disposal or recycling. The environmental impacts of an activity depend on both the activity’s direct emissions to the environment as well as indirect emissions caused by activities elsewhere in the supply chain. Both the impacts of direct emissions and the provisioning of supply chain inputs to an activity depend on the activity’s spatiotemporal scope. When accounting for spatiotemporal dynamics, LCA often faces significant data interoperability challenges. Ontologies and Semantic technologies can foster interoperability between diverse data sets from a variety of domains. Thus, this paper presents an ontology for modeling spatiotemporal scopes, i.e., the contexts in which impact estimates are valid. We discuss selected axioms and illustrate the use of the ontology by providing an example from LCA practice. The ontology enables practitioners to address key competency questions regarding the effect of spatiotemporal scopes on environmental impact estimation.

WHAT IS DATA MINING? WHY DATA MINING? NEED FOR HUMAN DIRECTION OF DATA MINING CROSS-INDUSTRY STANDARD PROCESS: CRISP–DM CASE STUDY 1: ANALYZING AUTOMOBILE WARRANTY CLAIMS: EXAMPLE OF THE CRISP–DM INDUSTRY STANDARD PROCESS IN ACTION FALLACIES OF DATA MINING WHAT TASKS CAN DATA MINING ACCOMPLISH? CASE STUDY 2: PREDICTING ABNORMAL STOCK MARKET RETURNS USING NEURAL NETWORKS CASE STUDY 3: MINING ASSOCIATION RULES FROM LEGAL DATABASES CASE STUDY 4: PREDICTING CORPORATE BANKRUPTCIES USING DECISION TREES CASE STUDY 5: PROFILING THE TOURISM MARKET USING k-MEANS CLUSTERING ANALYSIS


Question: Topic-relevance mapping: a visualization technique for enhancing the understanding of search results.

This paper discusses the design and evaluation of Exploration Wall, a novel touch-based search interface tailored for exploratory search tasks on touch devices. Exploratory search poses unique challenges for users, as it necessitates a deep understanding of search intents, iterative query refinement, and the ability to navigate through large volumes of information. The Exploration Wall interface is designed to facilitate incremental exploration and sense-making in extensive information spaces by integrating entity search capabilities, allowing for the flexible use of search results as query parameters, and visualizing search streams in a spatial configuration for interactive manipulation.

The interface's design allows users to dynamically reuse entities to modify or create new search streams, and it enables the inspection of relationships between entities through intuitive touch-based interactions. A series of task-based experiments comparing the Exploration Wall interface with conventional search interfaces have shown that Exploration Wall significantly improves recall for exploratory search tasks, maintaining precision levels. The subjective feedback from participants validates the design decisions, highlighting improved user satisfaction and engagement with the Exploration Wall interface.

These findings offer valuable insights for the design of user interfaces that effectively support exploratory search tasks on touch devices, emphasizing the importance of visual and interactive elements in enhancing the user experience and search efficiency. The Exploration Wall's design principles and performance outcomes can guide the development of future search interfaces that cater to the complex nature of exploratory search, particularly in the context of touch devices.

In order to adopt deep learning for information retrieval, models are needed that can capture all relevant information required to assess the relevance of a document to a given user query. While previous works have successfully captured unigram term matches, how to fully employ position-dependent information such as proximity and term dependencies has been insufficiently explored. In this work, we propose a novel neural IR model named PACRR aiming at better modeling position-dependent interactions between a query and a document. Extensive experiments on six years’ TREC Web Track data confirm that the proposed model yields better results under multiple benchmarks.

Statistical topic models can help analysts discover patterns in large text corpora by identifying recurring sets of words and enabling exploration by topical concepts. However, understanding and validating the output of these models can itself be a challenging analysis task. In this paper, we offer two design considerations - interpretation and trust - for designing visualizations based on data-driven models. Interpretation refers to the facility with which an analyst makes inferences about the data through the lens of a model abstraction. Trust refers to the actual and perceived accuracy of an analyst's inferences. These considerations derive from our experiences developing the Stanford Dissertation Browser, a tool for exploring over 9,000 Ph.D. theses by topical similarity, and a subsequent review of existing literature. We contribute a novel similarity measure for text collections based on a notion of "word-borrowing" that arose from an iterative design process. Based on our experiences and a literature review, we distill a set of design recommendations and describe how they promote interpretable and trustworthy visual analysis tools.

The English WordNet is a lexical database containing more than 206,900 wordconcept pairs and more than 377,500 semantic and lexical links. Trying to visualize them all at once as a node-link diagram results in a representation which can be too big and complex for the user to grasp, and which cannot be easily processed by current web technologies. We propose a visualization technique based on the concept of semantic zooming, usually found in popular web map applications. This technique makes it feasible to handle large and complex graphs, and display them in an interactive representation. By zooming, it is possible to switch from an overview visualization, quickly and clearly presenting the global characteristics of the structure, to a detailed one, displaying a classic node-link diagram. WordNet Atlas is a web application that leverages this method to aid the user in the exploration of the WordNet data set, from its global taxonomy structure to the detail of words and synonym sets.

Recently, statistical topic modeling has been widely applied in text mining and knowledge management due to its powerful ability. A topic, as a probability distribution over words, is usually difficult to be understood. A common, major challenge in applying such topic models to other knowledge management problem is to accurately interpret the meaning of each topic. Topic labeling, as a major interpreting method, has attracted significant attention recently. However, previous works simply treat topics individually without considering the hierarchical relation among topics, and less attention has been paid to creating a good hierarchical topic descriptors for a hierarchy of topics. In this paper, we propose two effective algorithms that automatically assign concise labels to each topic in a hierarchy by exploiting sibling and parent-child relations among topics. The experimental results show that the inter-topic relation is effective in boosting topic labeling accuracy and the proposed algorithms can generate meaningful topic labels that are useful for interpreting the hierarchical topics.

Query expansion methods have been studied for a long time - with debatable success in many instances. In this paper we present a probabilistic query expansion model based on a similarity thesaurus which was constructed automatically. A similarity thesaurus reflects domain knowledge about the particular collection from which it is constructed. We address the two important issues with query expansion: the selection and the weighting of additional search terms. In contrast to earlier methods, our queries are expanded by adding those terms that are most similar to the concept of the query, rather than selecting terms that are similar to the query terms. Our experiments show that this kind of query expansion results in a notable improvement in the retrieval effectiveness when measured using both recall-precision and usefulness.

The research objective is to establish an approach for supporting the classification of text documents referring to a specified domain. The focus is on the preliminary topic assignment to the documents used for training the model. The method implements domain ontology as background knowledge. The idea consists in extracting the preliminary topics for training the classifier by means of unsupervised machine learning on a text corpus and further alignment of the document vectors to concepts of the ontology. The results obtained by classification of new documents supervised by e-governance ontology with several machine learning algorithms showed sufficient match of their content to the ontology concepts. A conclusion is drawn that the approach can support the automatic extraction of documents relevant to any domain described by ontology.

The probabilistic formalism of quantum physics is said to provide a sound basis for building a principled information retrieval framework. Such a framework can be based on the notion of information need vector spaces where events, such as document relevance or observed user interactions, correspond to subspaces. As in quantum theory, a probability distribution over these subspaces is defined through weighted sets of state vectors (density operators), and used to represent the current view of the retrieval system on the user information need. Tensor spaces can be used to capture different aspects of information needs. Our evaluation shows that the framework can lead to acceptable performance in an ad-hoc retrieval task. Going beyond this, we discuss the potential of the framework for three active challenges in information retrieval, namely, interaction, novelty and diversity.

This article reports on a detailed investigation of PubMed users' needs and behavior as a step toward improving biomedical information retrieval. PubMed is providing free service to researchers with access to more than 19 million citations for biomedical articles from MEDLINE and life science journals. It is accessed by millions of users each day. Efficient search tools are crucial for biomedical researchers to keep abreast of the biomedical literature relating to their own research. This study provides insight into PubMed users' needs and their behavior. This investigation was conducted through the analysis of one month of log data, consisting of more than 23 million user sessions and more than 58 million user queries. Multiple aspects of users' interactions with PubMed are characterized in detail with evidence from these logs. Despite having many features in common with general Web searches, biomedical information searches have unique characteristics that are made evident in this study. PubMed users are more persistent in seeking information and they reformulate queries often. The three most frequent types of search are search by author name, search by gene/protein, and search by disease. Use of abbreviation in queries is very frequent. Factors such as result set size influence users' decisions. Analysis of characteristics such as these plays a critical role in identifying users' information needs and their search habits. In turn, such an analysis also provides useful insight for improving biomedical information retrieval.Database URL:http://www.ncbi.nlm.nih.gov/PubMed.

The prediction of query performance is an interesting and important issue in Information Retrieval (IR). Current predictors involve the use of relevance scores, which are time-consuming to compute. Therefore, current predictors are not very suitable for practical applications. In this paper, we study a set of predictors of query performance, which can be generated prior to the retrieval process. The linear and non-parametric correlations of the predictors with query performance are thoroughly assessed on the TREC disk4 and disk5 (minus CR) collections. According to the results, some of the proposed predictors have significant correlation with query performance, showing that these predictors can be useful to infer query performance in practical applications.

This paper proposes a novel topic model, Citation-Author-Topic (CAT) model that addresses a semantic search task we define as expert search – given a research area as a query, it returns names of experts in this area. For example, Michael Collins would be one of the top names retrieved given the querySyntactic Parsing. Our contribution in this paper is two-fold. First, we model the cited author information together with words and paper authors. Such extra contextual information directly models linkage among authors and enhances the author-topic association, thus produces more coherent author-topic distribution. Second, we provide a preliminary solution to the task of expert search when the learning repository contains exclusively research related documents authored by the experts. When compared with a previous proposed model (Johri et al., 2010), the proposed model produces high quality author topic linkage and achieves over 33% error reduction evaluated by the standard MAP measurement.


Question: Topic-relevance mapping: a visualization technique for enhancing understanding of search outcomes.

This paper introduces a systematic framework for categorizing information visualizations, focusing on a task by data type taxonomy. This approach aims to provide a structured understanding of the diverse array of visualizations that have emerged in recent times, offering a more nuanced perspective beyond the initial Visual Information Seeking Mantra: Overview first, zoom and filter, then details-on-demand. The proposed taxonomy encompasses seven distinct data types: 1-dimensional, 2-dimensional, 3-dimensional, temporal, multi-dimensional, tree, and network data. Corresponding to these data types are seven key tasks: overview, zoom, filter, details-on-demand, relate, history, and extract. This taxonomy serves as a valuable tool for designers and researchers in the field of information visualization, enabling them to better comprehend and develop interfaces that cater to the specific needs of various data types and tasks.

As with any application of machine learning, web search ranking requires labeled data. The labels usually come in the form of relevance assessments made by editors. Click logs can also provide an important source of implicit feedback and can be used as a cheap proxy for editorial labels. The main difficulty however comes from the so called position bias - urls appearing in lower positions are less likely to be clicked even if they are relevant. In this paper, we propose a Dynamic Bayesian Network which aims at providing us with unbiased estimation of the relevance from the click logs. Experiments show that the proposed click model outperforms other existing click models in predicting both click-through rate and relevance.

Web search engines help users find useful information on the World Wide Web (WWW). However, when the same query is submitted by different users, typical search engines return the same result regardless of who submitted the query. Generally, each user has different information needs for his/her query. Therefore, the search result should be adapted to users with different information needs. In this paper, we first propose several approaches to adapting search results according to each user's need for relevant information without any user effort, and then verify the effectiveness of our proposed approaches. Experimental results show that search systems that adapt to each user's preferences can be achieved by constructing user profiles based on modified collaborative filtering with detailed analysis of user's browsing history in one day.

This paper presents strategies and lessons learned from the use of natural language annotations to facilitate question answering in the START information access system.

This paper aims to tackle on Linked Open Data (LOD) customised visualisations. The work is part of an ongoing research on interfaces and tools for helping non-programmers to easily present, analyse and support sense making over large semantic dataset. The customisation is a key aspect of our work. Producing effective customised visualisations is still difficult due to the complexity of the existing tools and the limited set of options they offer, especially to those with little knowledge in LOD and semantic data models. How can we give users full control on the primitives of the visualisation and their properties, without requiring them to master Semantic Web technologies or programming languages? The paper presents a conceptual model that can be used as a reference to build tools for generating customisable infoviews. We used it to conduct a survey on existing tools in terms of customisation capabilities. Starting from the feedback collected in this phase, we will implement these customisation features into some prototypes we are currently experimenting on.

Search engine logs are an emerging new type of data that offers interesting opportunities for data mining. Existing work on mining such data has mostly attempted to discover knowledge at the level of queries (e.g., query clusters). In this paper, we propose to mine search engine logs for patterns at the level of terms through analyzing the relations of terms inside a query. We define two novel term association patterns (i.e., context-sensitive term substitutions and term additions) and propose new methods for mining such patterns from search engine logs. These two patterns can be used to address the mis-specification and under-specification problems of ineffective queries. Experiment results on real search engine logs show that the mined context-sensitive term substitutions can be used to effectively reword queries and improve their accuracy, while the mined context-sensitive term addition patterns can be used to support query refinement in a more effective way.

Purpose – This paper aims to demonstrate the role that the Google general search engine plays in the information-seeking behaviour of scientists, particularly physicists and astronomers. Design/methodology/approach – The paper is based on a mixed-methods study including 56 semi-structured interviews, a questionnaire survey of 114 respondents (47 per cent response rate) and the use of information-event cards to collect critical incident data. The study was conducted at the Department of Physics and Astronomy at University College, London. Findings – The results show that Google is the tool most used for problem-specific information seeking. The results also show the growing reliance of scientists on general search engines, particularly Google, for finding scholarly articles. Initially, finding scholarly articles was a by-product of general searching for information rather than focused searches for papers. However, a growing number of articles read by scientists are identified through the Google general search engine and, as scientists are becoming more aware of the quantity of scholarly papers searchable by Google, they are increasingly relying on Google for finding scholarly literature. Research limitations/implications – As the only fields covered in the study were physics and astronomy, and the research participants were sourced from just one department of one institution, caution should be taken in generalising the findings. Originality/value – The data are based on a mixed-methods in-depth study of scientists’ information-seeking behaviour which sheds some light on a question raised in past studies relating to the reason for the high number of articles identified through Google.

Information retrieval systems traditionally rely on textual keywords to index and retrieve documents. Keyword-based retrieval may return inaccurate and incomplete results when different keywords are used to describe the same concept in the documents and in the queries. Furthermore, the relationship between these related keywords may be semantic rather than syntactic, and capturing it thus requires access to comprehensive human world knowledge. Concept-based retrieval methods have attempted to tackle these difficulties by using manually built thesauri, by relying on term cooccurrence data, or by extracting latent word relationships and concepts from a corpus. In this article we introduce a new concept-based retrieval approach based on Explicit Semantic Analysis (ESA), a recently proposed method that augments keyword-based text representation with concept-based features, automatically extracted from massive human knowledge repositories such as Wikipedia. Our approach generates new text features automatically, and we have found that high-quality feature selection becomes crucial in this setting to make the retrieval more focused. However, due to the lack of labeled data, traditional feature selection methods cannot be used, hence we propose new methods that use self-generated labeled training data. The resulting system is evaluated on several TREC datasets, showing superior performance over previous state-of-the-art results.

Domain-specific search engines are growing in popularity because they offer increased accuracy and extra functionality not possible with the general, Web-wide search engines. For example, www.campsearch.com allows complex queries by age-group, size, location and cost over .summer camps. Unfortunately these domain-specific search engines are difficult and timeconsuming to maintain. This paper proposes the use of machine learning techniques to greatly automate the creation and maintenance of domain-specific search engines. We describe new research in reinforcement learning, information extraction and text classification that enables efficient spidering, identifying informative text segments, and populating topic hierarchies. Using these techniques, we have built a demonstration system: a search engine for computer science research papers. It already contaius over 50,000 papers and is publicly available at ~w. cora.justres earch, com.

The English WordNet is a lexical database containing more than 206,900 wordconcept pairs and more than 377,500 semantic and lexical links. Trying to visualize them all at once as a node-link diagram results in a representation which can be too big and complex for the user to grasp, and which cannot be easily processed by current web technologies. We propose a visualization technique based on the concept of semantic zooming, usually found in popular web map applications. This technique makes it feasible to handle large and complex graphs, and display them in an interactive representation. By zooming, it is possible to switch from an overview visualization, quickly and clearly presenting the global characteristics of the structure, to a detailed one, displaying a classic node-link diagram. WordNet Atlas is a web application that leverages this method to aid the user in the exploration of the WordNet data set, from its global taxonomy structure to the detail of words and synonym sets.

We explore the relation between classical probabilistic models of information retrieval and the emerging language modeling approaches. It has long been recognized that the primary obstacle to effective performance of classical models is the need to estimate a relevance model: probabilities of words in the relevant class. We propose a novel technique for estimating these probabilities using the query alone. We demonstrate that our technique can produce highly accurate relevance models, addressing important notions of synonymy and polysemy. Our experiments show relevance models outperforming baseline language modeling systems on TREC retrieval and TDT tracking tasks. The main contribution of this work is an effective formal method for estimating a relevance model with no training data.


Question: Topic-relevance mapping: a visualization technique for enhancing understanding of search results.

Introducing TopicPanorama: A Visual Analytics Framework for Comprehensive Topic Exploration
This paper unveils TopicPanorama, a visual analytics framework designed to offer a comprehensive view of topics discussed across diverse sources, encompassing news, blogs, and micro-blogs. TopicPanorama aims to provide not just a snapshot, but a full picture, of the landscape of topics, including those commonly addressed by multiple sources as well as those unique to each. The framework models each text corpus as a topic graph, which are subsequently aligned through a sophisticated graph matching technique. To ensure the visualization is both readable and stable, TopicPanorama employs a level-of-detail (LOD) approach, facilitating multi-perspective analysis of the matched graphs. By integrating metric learning and feature selection into the graph matching algorithm, the framework enables users to dynamically adjust the matching outcomes according to their specific information requirements. TopicPanorama's versatility has been demonstrated through its application to a range of data types, including news articles, tweets, and blog entries. Both quantitative evaluations and practical case studies affirm the effectiveness of TopicPanorama, particularly in facilitating detailed exploration of topic graphs at various levels of granularity.

Keyword queries over structured databases are notoriously ambiguous. No single interpretation of a keyword query can satisfy all users, and multiple interpretations may yield overlapping results. This paper proposes a scheme to balance the relevance and novelty of keyword search results over structured databases. Firstly, we present a probabilistic model which effectively ranks the possible interpretations of a keyword query over structured data. Then, we introduce a scheme to diversify the search results by re-ranking query interpretations, taking into account redundancy of query results. Finally, we propose α-nDCG-W and WS-recall, an adaptation of α-nDCG and S-recall metrics, taking into account graded relevance of subtopics. Our evaluation on two real-world datasets demonstrates that search results obtained using the proposed diversification algorithms better characterize possible answers available in the database than the results of the initial relevance ranking.

We propose a novel probabilistic technique for modeling and extracting salient structure from large document collections. As in clustering and topic modeling, our goal is to provide an organizing perspective into otherwise overwhelming amounts of information. We are particularly interested in revealing and exploiting relationships between documents. To this end, we focus on extracting diverse sets of threads—singlylinked, coherent chains of important documents. To illustrate, we extract research threads from citation graphs and construct timelines from news articles. Our method is highly scalable, running on a corpus of over 30 million words in about four minutes, more than 75 times faster than a dynamic topic model. Finally, the results from our model more closely resemble human news summaries according to several metrics and are also preferred by human judges.

Statistical topic models can help analysts discover patterns in large text corpora by identifying recurring sets of words and enabling exploration by topical concepts. However, understanding and validating the output of these models can itself be a challenging analysis task. In this paper, we offer two design considerations - interpretation and trust - for designing visualizations based on data-driven models. Interpretation refers to the facility with which an analyst makes inferences about the data through the lens of a model abstraction. Trust refers to the actual and perceived accuracy of an analyst's inferences. These considerations derive from our experiences developing the Stanford Dissertation Browser, a tool for exploring over 9,000 Ph.D. theses by topical similarity, and a subsequent review of existing literature. We contribute a novel similarity measure for text collections based on a notion of "word-borrowing" that arose from an iterative design process. Based on our experiences and a literature review, we distill a set of design recommendations and describe how they promote interpretable and trustworthy visual analysis tools.

Information visualization as a field is growing rapidly in popularity since the first information visualization conference in 1995. However, as a consequence of its growth, it is increasingly difficult to follow the growing body of literature within the field. Survey papers and literature reviews are valuable tools for managing the great volume of previously published research papers, and the quantity of survey papers in visualization has reached a critical mass. To this end, this survey paper takes a quantum step forward by surveying and classifying literature survey papers in order to help researchers understand the current landscape of Information Visualization. It is, to our knowledge, the first survey of survey papers (SoS) in Information Visualization. This paper classifies survey papers into natural topic clusters which enables readers to find relevant literature and develops the first classification of classifications. The paper also enables researchers to identify both mature and less developed research directions as well as identify future directions. It is a valuable resource for both newcomers and experienced researchers in and outside the field of Information Visualization and Visual Analytics.

While generative models such as Latent Dirichlet Allocation (LDA) have proven fruitful in topic modeling, they often require detailed assumptions and careful specification of hyperparameters. Such model complexity issues only compound when trying to generalize generative models to incorporate human input. We introduce Correlation Explanation (CorEx), an alternative approach to topic modeling that does not assume an underlying generative model, and instead learns maximally informative topics through an information-theoretic framework. This framework naturally generalizes to hierarchical and semi-supervised extensions with no additional modeling assumptions. In particular, word-level domain knowledge can be flexibly incorporated within CorEx through anchor words, allowing topic separability and representation to be promoted with minimal human intervention. Across a variety of datasets, metrics, and experiments, we demonstrate that CorEx produces topics that are comparable in quality to those produced by unsupervised and semi-supervised variants of LDA.

The alignment of terminologies can be considered as a kind of “translation” between two or more terminologies that aims to enhance the communication among people coming from different domains or expertise. In this paper we introduce the Visual Terminology Alignment tool (VisTA) that enables the exact alignment for RDF/SKOS-like terminologies, suitable for integrating knowledge bases, rather than information retrieval systems. The tool provides a simple and friendly web-based user interface for the alignment between two terminologies, while it visualizes the terminology hierarchies, enables the interactive alignment process, and presents the alignment result. The latter is a native RDF/SKOS graph that interconnects the two terminology graphs, supporting interoperability and extending the search capabilities over the integrated semantic graph, using the broader and exact match properties.

Many applications require finding objects closest to a specified location that contains a set of keywords. For example, online yellow pages allow users to specify an address and a set of keywords. In return, the user obtains a list of businesses whose description contains these keywords, ordered by their distance from the specified address. The problems of nearest neighbor search on spatial data and keyword search on text data have been extensively studied separately. However, to the best of our knowledge there is no efficient method to answer spatial keyword queries, that is, queries that specify both a location and a set of keywords. In this work, we present an efficient method to answer top-k spatial keyword queries. To do so, we introduce an indexing structure called IR2-Tree (Information Retrieval R-Tree) which combines an R-Tree with superimposed text signatures. We present algorithms that construct and maintain an IR2-Tree, and use it to answer top-k spatial keyword queries. Our algorithms are experimentally compared to current methods and are shown to have superior performance and excellent scalability.

Rich multimedia content including images, audio and text are frequently used to describe the same semantics in E-Learning and Ebusiness web pages, instructive slides, multimedia cyclopedias, and so on. In this paper, we present a framework for cross-media retrieval, where the query example and the retrieved result(s) can be of different media types. We first construct Multimedia Correlation Space (MMCS) by exploring the semantic correlation of different multimedia modalities, during which multimedia content and co-occurrence information is utilized. We propose a novel ranking algorithm, namely ranking with Local Regression and Global Alignment (LRGA), which learns a robust Laplacian matrix for data ranking. In LRGA, for each data point, a local linear regression model is used to predict the ranking values of its neighboring points. We propose a unified objective function to globally align the local models from all the data points so that an optimal ranking value can be assigned to each data point. LRGA is insensitive to parameters, making it particularly suitable for data ranking. A relevance feedback algorithm is proposed to improve the retrieval performance. Comprehensive experiments have demonstrated the effectiveness of our methods.

Personalized recommendations have become a common feature of many modern online services. In particular on ecommerce sites, one value of such recommendations is that they help consumers find items of interest in large product assortments more quickly. Many of today’s sites take advantage of modern recommendation technologies to create personalized item suggestions for consumers navigating the site. However, limited research exists on the use of personalization and recommendation technology when consumers rely on the site’s catalog search functionality to discover relevant items. In this work we explore the value of personalizing search results on e-commerce sites using recommendation technology. We design and evaluate different personalization strategies using log data of an online retail site. Our results show that considering several item relevance signals within the recommendation process in parallel leads to the best ranking of the search results. Specifically, the factors taken into account include the users’ general interests, their most recent browsing behavior, as well as the consideration of current sales trends.

The presence (and, sometimes, prominence) of incorrect and misleading content on the Web can have serious consequences for people who increasingly rely on the internet as their information source for topics such as health, politics, and financial advice. In this paper, we identify and collect several page features (such as popularity among specialized user groups) that are currently difficult or impossible for end users to assess, yet provide valuable signals regarding credibility. We then present visualizations designed to augment search results and Web pages with the most promising of these features. Our lab evaluation finds that our augmented search results are particularly effective at increasing the accuracy of users'" credibility assessments, highlighting the potential of data aggregation and simple interventions to help people make more informed decisions as they search for information online.