Question: Two methods for preserving privacy in data publishing while maintaining identity reservation.

This paper presents a method for anonymizing data to ensure privacy when disclosing information (Abstract)
With the rise of internet accessibility and the widespread availability of powerful computers with extensive storage capabilities, the amount of personal data collected and distributed has surged. The presence of additional data sources significantly simplifies the process of re-identifying individuals whose supposedly anonymous information is made public. Concurrently, there is a growing pressure on organizations to release detailed, individual-level data instead of merely aggregated statistics. Even after removing or encrypting explicit identifiers such as names and phone numbers, other types of data, referred to as quasi-identifiers, can still enable the recipient to deduce the identities of the individuals associated with the disclosed data. In this paper, we propose a computational disclosure control technique for disseminating data from a confidential database in a manner that ensures the identities of the individuals cannot be conclusively determined. Our method safeguards against potential linkage with other data sources. It is grounded in the principles of data generalization, where stored values are replaced with semantically equivalent but less specific alternatives, and the concept of k-anonymity. A table is said to offer k-anonymity when the released data does not permit the recipient to link the information to any group smaller than k individuals. We introduce the concepts of a generalized table and the minimal generalization of a table to meet a k-anonymity requirement. As an optimization challenge, the goal is to minimally alter the data while providing sufficient privacy protection. We outline an algorithm that, given a table, efficiently calculates a preferred minimal generalization to ensure anonymity.

Companies require data from multiple sources to develop new information systems, such as social networking, e-commerce and location-based services. Systems rely on complex, multi-stakeholder data supply-chains to deliver value. These data supply-chains have complex privacy requirements: privacy policies affecting multiple stakeholders (e.g. user, developer, company, government) regulate the collection, use and sharing of data over multiple jurisdictions (e.g. California, United States, Europe). Increasingly, regulators expect companies to ensure consistency between company privacy policies and company data practices. To address this problem, we propose a methodology to map policy requirements in natural language to a formal representation in Description Logic. Using the formal representation, we reason about conflicting requirements within a single policy and among multiple policies in a data supply chain. Further, we enable tracing data flows within the supply-chain. We derive our methodology from an exploratory case study of Facebook platform policy. We demonstrate the feasibility of our approach in an evaluation involving Facebook, Zynga and AOL-Advertising policies. Our results identify three conflicts that exist between Facebook and Zynga policies, and one conflict within the AOL Advertising policy.

Cloud-based multimedia systems are becoming increasingly common. These systems offer not only storage facility, but also high-end computing infrastructure which can be used to process data for various analysis tasks ranging from low-level data quality enhancement to high-level activity and behavior identification operations. However, cloud data centers, being third party servers, are often prone to information leakage, raising security and privacy concerns. In this article, we present a Shamir's secret sharing based method to enhance the quality of encrypted image data over cloud. Using the proposed method we show that several image enhancement operations such as noise removal, antialiasing, edge and contrast enhancement, and dehazing can be performed in encrypted domain with near-zero loss in accuracy and minimal computation and data overhead. Moreover, the proposed method is proven to be information theoretically secure.

Personal health record (PHR) service is an emerging model for health information exchange. In PHR systems, patient’s health records and information are maintained by the patient himself through the Web. In reality, PHRs are often outsourced to be stored at the third parties like cloud service providers. However, there have been serious privacy concerns about cloud service as it may expose user’s sensitive data like PHRs to those cloud service providers or unauthorized users. Using attribute-based encryption (ABE) to encrypt patient’s PHRs in cloud environment, secure and flexible access control can be achieved. Yet, problems like scalability in key management, fine-grained access control, and efficient user revocation remain to be addressed. In this paper, we propose a privacy-preserving PHR, which supports fine-grained access control and efficient revocation. To be specific, our scheme achieves the goals (1) scalable and fine-grained access control for PHRs by using multi-authority ABE scheme, and (2) efficient on-demand user/attribute revocation and dynamic policy update. In our scheme, we consider the situation that multiple data owners exist, and patient’s PHRs are encrypted and stored in semi-trust servers. The access structure in our scheme is expressive access tree structure, and the security of our scheme can be reduced to the standard decisional bilinear Diffie–Hellman assumption.

Privacy protection from unwanted video recordings is an important societal challenge. We desire a computer vision system (e.g., a robot) that can recognize human activities and assist our daily life, yet ensure that it is not recording video that may invade our privacy. This paper presents a fundamental approach to address such contradicting objectives: human activity recognition while only using extreme low-resolution (e.g., 16x12) anonymized videos. We introduce the paradigm of inverse super resolution (ISR), the concept of learning the optimal set of image transformations to generate multiple low-resolution (LR) training videos from a single video. Our ISR learns different types of sub-pixel transformations optimized for the activity classification, allowing the classifier to best take advantage of existing high-resolution videos (e.g., YouTube videos) by creating multiple LR training videos tailored for the problem. We experimentally confirm that the paradigm of inverse super resolution is able to benefit activity recognition from extreme low-resolution videos.

Protecting social norms as confidentiality wanes.

In recent years, uncertain data has become ubiquitous because of new technologies for collecting data which can only measure and collect the data in an imprecise way. Furthermore, many technologies such as privacy-preserving data mining create data which is inherently uncertain in nature. As a result there is a need for tools and techniques for mining and managing uncertain data. This chapter discusses the broad outline of the book and the methods used for various uncertain data applications.

Machine learning algorithms, such as neural networks, create better predictive models when having access to larger datasets. In many domains, such as medicine and finance, each institute has only access to limited amounts of data, and creating larger datasets typically requires collaboration. However, there are privacy related constraints on these collaborations for legal, ethical, and competitive reasons. In this work, we present a feasible protocol for learning neural networks in a collaborative way while preserving the privacy of each record. This is achieved by combining Differential Privacy and Secure Multi-Party Computation with Machine Learning.

1 Qatar Computing Research Institute (QCRI), Qatar Foundation, Doha, Qatar; E-Mail: ikhalil@uaeu.ac.ae 2 Department of Electrical and Computer Engineering, Newark College of Engineering, New Jersey Institute of Technology, University Heights, Newark, NJ 07102, USA; E-Mail: abdallah@njit.edu 3 College of Information Technology, United Arab Emirates University, PO Box 15551, Al Ain, United Arab Emirates; E-Mail: azeemsamma@hotmail.com

Modern mobile phones and PDAs are equipped with positioning capabilities (e.g., GPS). Users can access public location-based services (e.g., Google Maps) and ask spatial queries. Although communication is encrypted, privacy and confidentiality remain major concerns, since the queries may disclose the location and identity of the user. Commonly, spatial K-anonymity is employed to hide the query initiator among a group of K users. However, existing work either fails to guarantee privacy, or exhibits unacceptably long response time. In this paper we propose MobiHide, a Peer-to-Peer system for anonymous location-based queries, which addresses these problems. MobiHide employs the Hilbert space-filling curve to map the 2-D locations of mobile users to 1-D space. The transformed locations are indexed by a Chord-based distributed hash table, which is formed by the mobile devices. The resulting Peer-to-Peer system is used to anonymize a query by mapping it to a random group of K users that are consecutive in the 1-D space. Compared to existing state-of-the-art, MobiHide does not provide theoretical anonymity guarantees for skewed query distributions. Nevertheless, it achieves strong anonymity in practice, and it eliminates system hotspots. Our experimental evaluation shows that MobiHide has good load balancing and fault tolerance properties, and is applicable to real-life scenarios with numerous mobile users.

We consider a statistical database in which a trusted administrator introduces noise to the query responses with the goal of maintaining privacy of individual database entries. In such a database, a query consists of a pair (S, f) where S is a set of rows in the database and f is a function mapping database rows to {0, 1}. The true answer is ΣiεS f(di), and a noisy version is released as the response to the query. Results of Dinur, Dwork, and Nissim show that a strong form of privacy can be maintained using a surprisingly small amount of noise -- much less than the sampling error -- provided the total number of queries is sublinear in the number of database rows. We call this query and (slightly) noisy reply the SuLQ (Sub-Linear Queries) primitive. The assumption of sublinearity becomes reasonable as databases grow increasingly large.We extend this work in two ways. First, we modify the privacy analysis to real-valued functions f and arbitrary row types, as a consequence greatly improving the bounds on noise required for privacy. Second, we examine the computational power of the SuLQ primitive. We show that it is very powerful indeed, in that slightly noisy versions of the following computations can be carried out with very few invocations of the primitive: principal component analysis, k means clustering, the Perceptron Algorithm, the ID3 algorithm, and (apparently!) all algorithms that operate in the in the statistical query learning model [11].


Question: Two methods for privacy-preserving data publishing while preserving identity information.

This research explores the use of bottom-up generalization as a method for protecting privacy in data publishing, particularly focusing on maintaining the utility of the data for classification tasks while preventing the inference of sensitive information through linkage to external sources. The technique, termed data mining based privacy protection, integrates the requirements of targeted data mining tasks into the data masking process, ensuring that the fundamental structure of the data is preserved even after generalization. The paper addresses a specific privacy concern where a data holder seeks to distribute a modified version of their dataset for classification model development but wishes to safeguard against the identification of individual records by linking the released data to other sources. To achieve this, an iterative bottom-up generalization approach, derived from data mining techniques, is employed to abstract the data. This process ensures that the generalized data remains useful for classification purposes but is significantly less susceptible to linkage attacks. The generalization hierarchy is defined by a pre-established structure of generalizations, and the challenge lies in selecting the most appropriate level of generalization at each step of the iteration. This method offers a practical solution for data holders looking to balance the utility of their data for analytical purposes with the need for privacy protection.

In several experimental auctions, participants put a dollar value on private information before revealing it to a group. An analysis of results show that a trait's desirability in relation to the group played a key role in the amount people demanded to publicize private information. Because people can easily obtain, aggregate, and disperse personal data electronically, privacy is a central concern in the information age. This concern is clear in relation to financial data and genetic information, both of which can lead to identity abuse and discrimination. However, other relatively harmless information can also be abused, including a person's gender, salary, age, marital status, or shopping preferences. What's unclear is whether it's the fear of such abuse that actually causes people's stated hesitance to reveal their data. Our hypothesis - and the motivation for our study - is that people reveal information when they feel that they're somewhat typical or positively atypical compared to the target group. To test this hypothesis, we conducted experiments that elicit the value people place on their private data. We found, with great significance (more than 95 percent statistical confidence) that a linear relationship exists between an individual's belief about a trait and the value he or she places on it. That is, the less desirable the trait, the greater the price a person demands for releasing the information. Furthermore, we found that small deviations in a socially positive direction are associated with a lower asking price.

Attribute-based proxy re-encryption (ABPRE) scheme is one of the proxy cryptography, which can delegate the reencryption capability to the proxy and re-encrypt the encrypted data by using the re-encryption key. ABPRE extending the traditional proxy cryptography and attributes plays an important role. In ABPRE, users are identified by attributes, and the access policy is designed to control the user’s access. Using ABPRE can have these advantages: (i) The proxy can be delegated to execute the re-encryption operation, which reduces the computation overhead of the data owner; (ii) The authorized user just uses his own secret key to decrypt the encrypted data, and he doesn’t need to store an additional decryption key for deciphering; (iii) The sensitive information cannot be revealed to the proxy in re-encryption, and the proxy only complies to the data owner’s command. In this paper, we survey two various access policy attribute-based proxy re-encryption schemes and analyze these schemes. Thereafter, we list the comparisons of them by some criteria.

The large-scale automated transaction systems of the near future can be designed to protect the privacy and maintain the security of both individuals and organizations.

Location-based services, which employ data from smartphones, vehicles, etc., are growing in popularity. To reduce the threat that shared location data poses to a user's privacy, some services anonymize or obfuscate this data. In this paper, we show these methods can be effectively defeated: a set of location traces can be deanonymized given an easily obtained social network graph. The key idea of our approach is that a user may be identified by those she meets: a "contact graph" identifying meetings between anonymized users in a set of traces can be structurally correlated with a social network graph, thereby identifying anonymized users. We demonstrate the effectiveness of our approach using three real world datasets: University of St Andrews mobility trace and social network (27 nodes each), SmallBlue contact trace and Facebook social network (125 nodes), and Infocom 2006 bluetooth contact traces and conference attendees' DBLP social network (78 nodes). Our experiments show that 80% of users are identified precisely, while only 8% are identified incorrectly, with the remainder mapped to a small set of users.

An enormous quantity of personal health information is available in recent decades and tampering of any part of this information imposes a great risk to the health care field. Existing anonymization methods are only apt for single sensitive and low dimensional data to keep up with privacy specifically like generalization and bucketization. In this paper, an anonymization technique is proposed that is a combination of the benefits of anatomization, and enhanced slicing approach adhering to the principle of k-anonymity and l-diversity for the purpose of dealing with high dimensional data along with multiple sensitive data. The anatomization approach dissociates the correlation observed between the quasi identifier attributes and sensitive attributes (SA) and yields two separate tables with non-overlapping attributes. In the enhanced slicing algorithm, vertical partitioning does the grouping of the correlated SA in ST together and thereby minimizes the dimensionality by employing the advanced clustering algorithm. In order to get the optimal size of buckets, tuple partitioning is conducted by MFA. The experimental outcomes indicate that the proposed method can preserve privacy of data with numerous SA. The anatomization approach minimizes the loss of information and slicing algorithm helps in the preservation of correlation and utility which in turn results in reducing the data dimensionality and information loss. The advanced clustering algorithms prove its efficiency by minimizing the time and complexity. Furthermore, this work sticks to the principle of k-anonymity, l-diversity and thus avoids privacy threats like membership, identity and attributes disclosure.

Identity-based encryption (IBE) is an exciting alternative to public-key encryption, as IBE eliminates the need for a Public Key Infrastructure (PKI). The senders using an IBE do not need to look up the public keys and the corresponding certificates of the receivers, the identities (e.g. emails or IP addresses) of the latter are sufficient to encrypt. Any setting, PKI- or identity-based, must provide a means to revoke users from the system. Efficient revocation is a well-studied problem in the traditional PKI setting. However in the setting of IBE, there has been little work on studying the revocation mechanisms. The most practical solution requires the senders to also use time periods when encrypting, and all the receivers (regardless of whether their keys have been compromised or not) to update their private keys regularly by contacting the trusted authority. We note that this solution does not scale well -- as the number of users increases, the work on key updates becomes a bottleneck. We propose an IBE scheme that significantly improves key-update efficiency on the side of the trusted party (from linear to logarithmic in the number of users), while staying efficient for the users. Our scheme builds on the ideas of the Fuzzy IBE primitive and binary tree data structure, and is provably secure.

More and more people go online today and share their personal images using popular web services like Picasa. While enjoying the convenience brought by advanced technology, people also become aware of the privacy issues of data being shared. Recent studies have highlighted that people expect more tools to allow them to regain control over their privacy. In this work, we propose an Adaptive Privacy Policy Prediction (A3P) system to help users compose privacy settings for their images. In particular, we examine the role of image content and metadata as possible indicators of users' privacy preferences. We propose a two-level image classification framework to obtain image categories which may be associated with similar policies. Then, we develop a policy prediction algorithm to automatically generate a policy for each newly uploaded image. Most importantly, the generated policy will follow the trend of the user's privacy concerns evolved with time. We have conducted an extensive user study and the results demonstrate effectiveness of our system with the prediction accuracy around 90%.

Suppose that you want to delegate the ability to process your data, without giving away access to it. We show that this separation is possible: we describe a "fully homomorphic" encryption scheme that keeps data private, but that allows a worker that does not have the secret decryption key to compute any (still encrypted) result of the data, even when the function of the data is very complex. In short, a third party can perform complicated processing of data without being able to see it. Among other things, this helps make cloud computing compatible with privacy.

In this paper, we consider the problem of preserving privacy in the context of online learning. Online learning involves learning from data in real-time, due to which the learned model as well as its predictions are continuously changing. This makes preserving privacy of each data point significantly more challenging as its effect on the learned model can be easily tracked by observing changes in the subsequent predictions. Furthermore, with more and more online systems (e.g. search engines like Bing, Google etc.) trying to learn their customers’ behavior by leveraging their access to sensitive customer data (through cookies etc.), the problem of privacy preserving online learning has become critical. We study the problem in the framework of online convex programming (OCP)—a popular online learning setting with several theoretical and practical implications—while using differential privacy as the formal measure of privacy. For this problem, we provide a generic framework that can be used to convert any given OCP algorithm into a private OCP algorithm with provable privacy as well as regret guarantees (utility), provided that the given OCP algorithm satisfies the following two criteria: 1) linearly decreasing sensitivity, i.e., the effect of the new data points on the learned model decreases linearly, 2) sub-linear regret. We then illustrate our approach by converting two popular OCP algorithms into corresponding differentially private algorithms while guaranteeing Õ( √ T ) regret for strongly convex functions. Next, we consider the practically important class of online linear regression problems, for which we generalize the approach by Dwork et al. (2010a) to provide a differentially private algorithm with just poly-log regret. Finally, we show that our online learning framework can be used to provide differentially private algorithms for the offline learning problem as well. For the offline learning problem, our approach guarantees better error bounds and is more practical than the existing state-of-the-art methods (Chaudhuri et al., 2011; Rubinstein et al., 2009).

Can health entities collaboratively train deep learning models without sharing sensitive raw data? This paper proposes several configurations of a distributed deep learning method called SplitNN to facilitate such collaborations. SplitNN does not share raw data or model details with collaborating institutions. The proposed configurations of splitNN cater to practical settings of i) entities holding different modalities of patient data, ii) centralized and local health entities collaborating on multiple tasks and iii) learning without sharing labels. We compare performance and resource efficiency trade-offs of splitNN and other distributed deep learning methods like federated learning, large batch synchronous stochastic gradient descent and show highly encouraging results for splitNN.


Question: Two methods for preserving privacy in data publishing while maintaining identity reservation.

This paper introduces a significant advancement in privacy-preserving data publication by addressing the issue of dynamic dataset re-publication. Traditionally, privacy preservation techniques have been designed for single-time data releases, neglecting the need for updating datasets with new insertions and deletions. This oversight has hindered publishers from providing researchers with the most current and accurate data on a continuous basis.
To tackle this challenge, we first expose the limitations of existing approaches based on k-anonymity and l-diversity when applied to re-publication scenarios. Through comprehensive theoretical analysis, we propose a novel concept: m-invariance. This principle is specifically designed to mitigate privacy risks associated with the re-publication of updated datasets. Accompanying this principle is an algorithm that generates privacy-protected datasets, enabling the retrieval of precise aggregate information from the original data while ensuring confidentiality.
Our theoretical findings are substantiated through extensive experimentation using real-world datasets, demonstrating the effectiveness of the m-invariance principle in safeguarding privacy during the re-publication of dynamic datasets. This approach not only enhances data privacy but also supports the continuous provision of the most recent data to researchers, bridging a critical gap in the field of privacy-preserving data publication.

The privacy protection of the biometric data is an important research topic, especially in the case of distributed biometric systems. In this scenario, it is very important to guarantee that biometric data cannot be steeled by anyone, and that the biometric clients are unable to gather any information different from the single user verification/identification. In a biométrie system with high level of privacy compliance, also the server that processes the biométrie matching should not learn anything on the database and it should be impossible for the server to exploit the resulting matching values in order to extract any knowledge about the user presence or behavior. Within this conceptual framework, in this paper we propose a novel complete demonstrator based on a distributed biométrie system that is capable to protect the privacy of the individuals by exploiting cryptosystems. The implemented system computes the matching task in the encrypted domain by exploiting homomorphic encryption and using Fingercode templates. The paper describes the design methodology of the demonstrator and the obtained results. The demonstrator has been fully implemented and tested in real applicative conditions. Experimental results show that this method is feasible in the cases where the privacy of the data is more important than the accuracy of the system and the obtained computational time is satisfactory.

MONOMI is a system for securely executing analytical workloads over sensitive data on an untrusted database server. MONOMI works by encrypting the entire database and running queries over the encrypted data. MONOMI introduces split client/server query execution, which can execute arbitrarily complex queries over encrypted data, as well as several techniques that improve performance for such workloads, including per-row precomputation, space-efficient encryption, grouped homomorphic addition, and pre-filtering. Since these optimizations are good for some queries but not others, MONOMI introduces a designer for choosing an efficient physical design at the server for a given workload, and a planner to choose an efficient execution plan for a given query at runtime. A prototype of MONOMI running on top of Postgres can execute most of the queries from the TPC-H benchmark with a median overhead of only 1.24× (ranging from 1.03× to 2.33×) compared to an un-encrypted Postgres database where a compromised server would reveal all data.

We propose a novel scheme for selective distribution of content, encoded as documents, that preserves the privacy of the users to whom the documents are delivered and is based on an efficient and novel group key management scheme. Our document broadcasting approach is based on access control policies specifying which users can access which documents, or subdocuments. Based on such policies, a broadcast document is segmented into multiple subdocuments, each encrypted with a different key. In line with modern attribute-based access control, policies are specified against identity attributes of users. However our broadcasting approach is privacy-preserving in that users are granted access to a specific document, or subdocument, according to the policies without the need of providing in clear information about their identity attributes to the document publisher. Under our approach, not only does the document publisher not learn the values of the identity attributes of users, but it also does not learn which policy conditions are verified by which users, thus inferences about the values of identity attributes are prevented. Moreover, our key management scheme on which the proposed broadcasting approach is based is efficient in that it does not require to send the decryption keys to the users along with the encrypted document. Users are able to reconstruct the keys to decrypt the authorized portions of a document based on subscription information they have received from the document publisher. The scheme also efficiently handles new subscription of users and revocation of subscriptions.

We introduce new theoretical measures for the qualitative and quantitative assessment of encryption schemes designed for broadcast transmissions. The goal is to allow a central broadcast site to broadcast secure transmissions to an arbitrary set of recipients while minimizing key management related transmissions. We present several schemes that allow a center to broadcast a secret to any subset of privileged users out of a universe of size so that coalitions of users not in the privileged set cannot learn the secret. The most interesting scheme requires every user to store keys and the center to broadcast messages regardless of the size of the privileged set. This scheme is resilient to any coalition of users. We also present a scheme that is resilient with probability against a random subset of users. This scheme requires every user to store keys and the center to broadcast messages. Preliminary version appeared in Advances in Cryptology CRYPTO ’93 Proceedings, Lecture Notes in Computer Science, Vol. 773, 1994, pp. 480–491. Dept. of Computer Science, School of Mathematics, Tel Aviv University, Tel Aviv, Israel, and Algorithmic Research Ltd. E-mail fiat@math.tau.ac.il. Incumbent of the Morris and Rose Goldman Career Development Chair, Dept. of Applied Mathematics and Computer Science, Weizmann Institute of Science, Rehovot 76100, Israel. Research supported by an Alon Fellowship and a grant from the Israel Science Foundation administered by the Israeli Academy of Sciences. E-mail: naor@wisdom.weizmann.ac.il.

In cloud computing, searchable encryption scheme over outsourced data is a hot research field. However, most existing works on encrypted search over outsourced cloud data follow the model of “one size fits all” and ignore personalized search intention. Moreover, most of them support only exact keyword search, which greatly affects data usability and user experience. So how to design a searchable encryption scheme that supports personalized search and improves user search experience remains a very challenging task. In this paper, for the first time, we study and solve the problem of personalized multi-keyword ranked search over encrypted data (PRSE) while preserving privacy in cloud computing. With the help of semantic ontology WordNet, we build a user interest model for individual user by analyzing the user's search history, and adopt a scoring mechanism to express user interest smartly. To address the limitations of the model of “one size fit all” and keyword exact search, we propose two PRSE schemes for different search intentions. Extensive experiments on real-world dataset validate our analysis and show that our proposed solution is very efficient and effective.

It is well-known that cloud computing has many potential advantages and many enterprise applications and data are migrating to public or hybrid cloud. But regarding some business-critical applications, the organizations, especially large enterprises, still wouldn't move them to cloud. The market size the cloud computing shared is still far behind the one expected. From the consumers' perspective, cloud computing security concerns, especially data security and privacy protection issues, remain the primary inhibitor for adoption of cloud computing services. This paper provides a concise but all-round analysis on data security and privacy protection issues associated with cloud computing across all stages of data life cycle. Then this paper discusses some current solutions. Finally, this paper describes future research work about data security and privacy protection issues in cloud.

Functional encryption supports restricted decryption keys that allow users to learn specific functions of the encrypted messages. Although the vast majority of research on functional encryption has so far focused on the privacy of the encrypted messages, in many realistic scenarios it is crucial to offer privacy also for the functions for which decryption keys are provided. Whereas function privacy is inherently limited in the public-key setting, in the private-key setting it has a tremendous potential. Specifically, one can hope to construct schemes where encryptions of messages $$\mathsf{m}_1, \ldots , \mathsf{m}_T$$ m 1 , … , m T together with decryption keys corresponding to functions $$f_1, \ldots , f_T$$ f 1 , … , f T , reveal essentially no information other than the values $$\{ f_i(\mathsf{m}_j)\}_{i,j\in [T]}$$ { f i ( m j ) } i , j ∈ [ T ] . Despite its great potential, the known function-private private-key schemes either support rather limited families of functions (such as inner products) or offer somewhat weak notions of function privacy. We present a generic transformation that yields a function-private functional encryption scheme, starting with any non-function-private scheme for a sufficiently rich function class. Our transformation preserves the message privacy of the underlying scheme and can be instantiated using a variety of existing schemes. Plugging in known constructions of functional encryption schemes, we obtain function-private schemes based either on the learning with errors assumption, on obfuscation assumptions, on simple multilinear-maps assumptions, and even on the existence of any one-way function (offering various trade-offs between security and efficiency).

Freenet is a popular peer to peer anonymous content-sharing network, with the objective to provide the anonymity of both content publishers and retrievers. Despite more than a decade of active development and deployment and the adoption of well-established cryptographic algorithms in Freenet, it remains unanswered how well the anonymity objective of the initial Freenet design has been met. In this paper we develop a traceback attack on Freenet, and show that the originating machine of a content request message in Freenet can be identified; that is, the anonymity of a content retriever can be broken, even if a single request message has been issued by the retriever. We present the design of the traceback attack, and perform both experimental and simulation studies to confirm the feasibility and effectiveness of the attack. For example, with randomly chosen content requesters (and random contents stored in the Freenet testbed), Emulab-based experiments show that, for <inline-formula><tex-math notation="LaTeX">$24$</tex-math> <alternatives><inline-graphic xlink:href="duan-ieq1-2453983.gif"/></alternatives></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$43$</tex-math><alternatives><inline-graphic xlink:href="duan-ieq2-2453983.gif"/> </alternatives></inline-formula> percent of the content request messages, we can identify their originating machines. We also briefly discuss potential solutions to address the developed traceback attack. Despite being developed specifically on Freenet, the basic principles of the traceback attack and solutions have important security implications for similar anonymous content-sharing systems.

Privacy and security of Big Data is gaining momentum in the research community, also due to emerging technologies like Cloud Computing, analytics engines and social networks. In response of this novel research challenge, several privacy and security of big data models, techniques and algorithms have been proposed recently, mostly adhering to algorithmic paradigms or model-oriented paradigms. Following this major trend, in this paper we provide an overview of state-of-the-art research issues and achievements in the field of privacy and security of big data, by highlighting open problems and actual research trends, and drawing novel research directions in this field.

We introduce a new, generic framework for private data analysis.The goal of private data analysis is to release aggregate information about a data set while protecting the privacy of the individuals whose information the data set contains.Our framework allows one to release functions f of the data withinstance-based additive noise. That is, the noise magnitude is determined not only by the function we want to release, but also bythe database itself. One of the challenges is to ensure that the noise magnitude does not leak information about the database. To address that, we calibrate the noise magnitude to the smoothsensitivity of f on the database x --- a measure of variabilityof f in the neighborhood of the instance x. The new frameworkgreatly expands the applicability of output perturbation, a technique for protecting individuals' privacy by adding a smallamount of random noise to the released statistics. To our knowledge, this is the first formal analysis of the effect of instance-basednoise in the context of data privacy.
 Our framework raises many interesting algorithmic questions. Namely,to apply the framework one must compute or approximate the smoothsensitivity of f on x. We show how to do this efficiently for several different functions, including the median and the cost ofthe minimum spanning tree. We also give a generic procedure based on sampling that allows one to release f(x) accurately on manydatabases x. This procedure is applicable even when no efficient algorithm for approximating smooth sensitivity of f is known orwhen f is given as a black box. We illustrate the procedure by applying it to k-SED (k-means) clustering and learning mixtures of Gaussians.


Question: An integrated methodology for analyzing log files in computing system administration.

This paper explores the burgeoning field of automated text categorization, a critical process for organizing documents into predefined categories, particularly relevant in the digital age where the volume of electronic documents has surged. The primary methodology for addressing this challenge is through machine learning, which automates the construction of classifiers by learning from a dataset of documents that have been pre-assigned to categories. This approach offers significant advantages over traditional knowledge engineering methods, which rely on domain experts manually defining classifiers. Machine learning-based categorization boasts higher effectiveness, reduces the need for expert labor, and is easily adaptable to new domains. The paper delves into the central themes of document representation, classifier construction, and evaluation, providing a comprehensive review of the machine learning techniques employed in text categorization. It highlights the evolution of these techniques over the past decade, reflecting the growing importance of automated text categorization in managing and organizing vast digital document collections.

Based on research and analysis of the Ceph file system, the log cache backup scheme is proposed. The log cache backup scheme reduces metadata access delays by caching logs and reducing the time of storing logs to server clusters. In order to prevent the loss of cached data in metadata servers, a cached data backup scheme is proposed. Compared to the metadata management subsystem of the Ceph, the log cache backup scheme can effectively improve the access performance of metadata. Finally, based on open source codes of the Ceph file system, the log cache backup scheme is implemented. Compared to the performance of the Ceph metadata management subsystem, experiment results show that performance improvements of the log cache backup scheme are up to 11.5%.

The Web holds valuable, vast, and unstructured information about public opinion. Here, the history, current use, and future of opinion mining and sentiment analysis are discussed, along with relevant techniques and tools.

The LogicBlox system aims to reduce the complexity of software development for modern applications which enhance and automate decision-making and enable their users to evolve their capabilities via a ``self-service'' model. Our perspective in this area is informed by over twenty years of experience building dozens of mission-critical enterprise applications that are in use by hundreds of large enterprises across industries such as retail, telecommunications, banking, and government. We designed and built LogicBlox to be the system we wished we had when developing those applications.
 In this paper, we discuss the design considerations behind the LogicBlox system and give an overview of its implementation, highlighting innovative aspects. These include: LogiQL, a unified and declarative language based on Datalog; the use of purely functional data structures; novel join processing strategies; advanced incremental maintenance and live programming facilities; a novel concurrency control scheme; and built-in support for prescriptive and predictive analytics.

Big data is a field that has traditionally been dominated by disciplines such as computer science and business, where mainly data-driven analyses have been performed. Psychology, a discipline in which a strong emphasis is placed on behavioral theories and empirical research, has the potential to contribute greatly to the big data movement. However, one challenge to psychologists-and probably the most crucial one-is that most researchers may not have the necessary programming and computational skills to analyze big data. In this study we argue that psychologists can also conduct big data research and that, rather than trying to acquire new programming and computational skills, they should focus on their strengths, such as performing psychometric analyses and testing theories using multivariate analyses to explain phenomena. We propose a split/analyze/meta-analyze approach that allows psychologists to easily analyze big data. Two real datasets are used to demonstrate the proposed procedures in R. A new research agenda related to the analysis of big data in psychology is outlined at the end of the study.

Large enterprise IT (Information Technology) infrastructure components generate large volumes of alerts and incident tickets. These are manually screened, but it is otherwise difficult to extract information automatically from them to gain insights in order to improve operational efficiency. We propose a framework to cluster alerts and incident tickets based on the text in them, using unsupervised machine learning. This would be a step towards eliminating manual classification of the alerts and incidents, which is very labor intense and costly. Our framework can handle the semi-structured text in alerts generated by IT infrastructure components such as storage devices, network devices, servers etc., as well as the unstructured text in incident tickets created manually by operations support personnel. After text pre-processing and application of appropriate distance metrics, we apply different graph-theoretic approaches to cluster the alerts and incident tickets, based on their semi-structured and unstructured text respectively. For automated interpretation and read-ability on semi-structured text clusters, we propose a method to visualize clusters that preserves the structure and human-readability of the text data as compared to traditional word clouds where the text structure is not preserved; for unstructured text clusters, we find a simple way to define prototypes of clusters for easy interpretation. This framework for clustering and visualization will enable enterprises to prioritize the issues in their IT infrastructure and improve the reliability and availability of their services.

Cyber-attacks have increased in severity and complexity. That requires, that the CERT/CSIRT research and develops new security tools. Therefore, our study focuses on the design of an integral model based on Business Intelligence (BI), which provides reactive and proactive services in a CSIRT, in order to alert and reduce any suspicious or malicious activity on information systems and data networks. To achieve this purpose, a solution has been assembled, that generates information stores, being compiled from a continuous network transmission of several internal and external sources of an organization. However, it contemplates a data warehouse, which is focused like a correlator of logs, being formed by the information of feeds with diverse formats. Furthermore, it analyzed attack detection and port scanning, obtained from sensors such as Snort and Passive Vulnerability Scanner, which are stored in a database, where the logs have been generated by the systems. With such inputs, we designed and implemented BI systems using the phases of the Ralph Kimball methodology, ETL and OLAP processes. In addition, a software application has been implemented using the SCRUM methodology, which allowed to link the obtained logs to the BI system for visualization in dynamic dashboards, with the purpose of generating early alerts and constructing complex queries using the user interface through objects structures. The results demonstrate, that this solution has generated early warnings based on the level of criticality and level of sensitivity of malware and vulnerabilities as well as monitoring efficiency, increasing the level of security of member institutions.

Intrusion detection systems are most popular de-fence mechanisms used to provide security to IT infrastructures. Organisation need best performance, so it uses multiple IDSs from different vendors. Different vendors are using different formats and protocols. Difficulty imposed by this is the generation of several false alarms. Major part of this work concentrates on the collection of alerts from different intrusion detection systems to represent them in IDMEF(Intrusion Detection Message Exchange Format) format. Alerts were collected from intrusion detection systems like snort, ossec, suricata etc. Later classification is attempted using machine learning technique, which helps to mitigate generation of false positives.

A practitioner's guide Measuring computer performance sets out the fundamental techniques used in analyzing and understanding the performance of computer systems. Throughout the book, the emphasis is on practical methods of measurement, simulation and analytical modeling. The author discusses performance metrics and provides detailed coverage of the strategies used in benchmark programs. He gives intuitive explanations of the key statistical tools needed to interpret measured performance data. He also describes the generaìdesign of experiments' technique, and shows how the maximum amount of information can be obtained for the minimum effort. The book closes with a chapter on the technique of queueing analysis. Appendices listing common probability distributions and statistical tables are included, along with a glossary of important technical terms. This practically oriented book will be of great interest to anyone who wants a detailed, yet intuitive, understanding of computer systems performance analysis.

Crime analysis and prevention is a systematic approach for identifying and analyzing patterns and trends in crime. Our system can predict regions which have high probability for crime occurrence and can visualize crime prone areas. With the increasing advent of computerized systems, crime data analysts can help the Law enforcement officers to speed up the process of solving crimes. Using the concept of data mining we can extract previously unknown, useful information from an unstructured data. Here we have an approach between computer science and criminal justice to develop a data mining procedure that can help solve crimes faster. Instead of focusing on causes of crime occurrence like criminal background of offender, political enmity etc we are focusing mainly on crime factors of each day.

The Large Hadron Collider (LHC) is one of the most complex machines ever build. It is composed of many components which constitute a large system. The tunnel and the accelerator is just one of a very critical fraction of the whole LHC infrastructure. Hardware comissioning as one of the critical processes before running the LHC is implemented during the Long Shutdown (LS) states of the macine, where Electrical Quality Assurance (ELQA) is one of its key components. Here a huge data is collected when implementing various ELQA electrical tests. In this paper we present a conceptual framework for supporting a rapid design of web applications for ELQA data analysis. We show a framework’s main components, their possible integration with other systems and machine learning algorithms and a simple use case of prototyping an application for Electrical Quality Assurance of the LHC.


Question: An integrated methodology for analyzing log files in computing system administration.

This paper introduces a methodology for identifying significant patterns in event data, crucial for enhancing applications in system management and intrusion detection. These applications often rely on automated real-time systems that collect and process sensor data on the fly, significantly minimizing the dependency on human operators. However, they necessitate the creation and maintenance of correlation rules, a task that traditionally demands expert knowledge to pinpoint problematic patterns, a process that is both time-intensive and prone to inaccuracies. To alleviate this challenge, we propose leveraging historical data, which is readily accessible, to streamline the process. Initially, we outline efficient algorithms designed to uncover three pivotal pattern types within historical event datasets: event surges, periodicity, and mutual dependencies. Subsequently, we delve into a systematic approach for mining events with multiple attributes in an efficient manner. Finally, we unveil Event Correlation Constructor, a software tool aimed at verifying and expanding the existing correlation knowledge base. This tool serves as a pivotal component in our framework, aimed at facilitating the discovery and utilization of actionable patterns in event data, thereby enhancing the effectiveness of automated systems in various applications.

Analysing text-based data has become increasingly important due to the importance of text from sources such as social media, web contents, web searches. The growing volume of such data creates challenges for data analysis including efficient and scalable algorithm, effective computing platforms and energy efficiency. Compression is a standard method for reducing data size but current standard compression algorithms are destructive to the organisation of data contents. This work introduces Content-aware, Partial Compression (CaPC) for text using a dictionary-based approach. We simply use shorter codes to replace strings while maintaining the original data format and structure, so that the compressed contents can be directly consumed by analytic platforms. We evaluate our approach with a set of real-world datasets and several classical MapReduce jobs on Hadoop. We also provide a supplementary utility library for Hadoop, hence, existing MapReduce programs can be used directly on the compressed datasets with little or no modification. In evaluation, we demonstrate that CaPC works well with a wide variety of data analysis scenarios, experimental results show ~30% average data size reduction, and up to ~32% performance increase on some I/O intensive jobs on an in-house Hadoop cluster. While the gains may seem modest, the point is that these gains are 'for free' and act as supplementary to all other optimizations.

In this paper we describe a method to discover frequent behavioral patterns in event logs. We express these patterns as local process models. Local process model mining can be positioned in-between process discovery and episode / sequential pattern mining. The technique presented in this paper is able to learn behavioral patterns involving sequential composition, concurrency, choice and loop, like in process mining. However, we do not look at start-to-end models, which distinguishes our approach from process discovery and creates a link to episode / sequential pattern mining. We propose an incremental procedure for building local process models capturing frequent patterns based on so-called process trees. We propose five quality dimensions and corresponding metrics for local process models, given an event log. We show monotonicity properties for some quality dimensions, enabling a speedup of local process model discovery through pruning. We demonstrate through a real life case study that mining local patterns allows us to get insights in processes where regular start-to-end process discovery techniques are only able to learn unstructured, flower-like, models.

High speed computer and telephone networks carry large amounts of data and signalling traffic. The engineers who build and maintain these networks use a combination of hardware and software tools to monitor the stream of network traffic. Some of these tools operate directly on the live network; others record data on magnetic tape for later offline analysis by software. Most analysis tasks require tens to hundreds of gigabytes of data. Traffic analysis applications include protocol performance analysis, conformance testing, error monitoring and fraud detection.

The increasingly large number of available writings describing technical and scientific progress, calls for advanced analytic tools for their efficient analysis. This is true for many application scenarios in science and industry and for different types of writings, comprising patents and scientific articles. Despite important differences between patents and scientific articles, both have a variety of common characteristics that lead to similar search and analysis tasks. However, the analysis and visualization of these documents is not a trivial task due to the complexity of the documents as well as the large number of possible relations between their multivariate attributes. In this survey, we review interactive analysis and visualization approaches of patents and scientific articles, ranging from exploration tools to sophisticated mining methods. In a bottom-up approach, we categorize them according to two aspects: (a) data type (text, citations, authors, metadata, and combinations thereof), and (b) task (finding and comparing single entities, seeking elementary relations, finding complex patterns, and in particular temporal patterns, and investigating connections between multiple behaviours). Finally, we identify challenges and research directions in this area that ask for future investigations.

Research and industrial experience reveal that code reviews as a part of software inspection might be the most cost-effective technique a team can use to reduce defects. Tools that automate code inspection mostly focus on the detection of a priori known defect patterns and security vulnerabilities. Automated detection of logical errors, due to a faulty implementation of applications’ functionality is a relatively uncharted territory. Automation can be based on profiling the intended behavior behind the source code. In this paper, we present a new code profiling method that combines an information flow analysis, the crosschecking of dynamic invariants with symbolic execution, and the use of fuzzy logic. Our goal is to detect logical errors and exploitable vulnerabilities. The theoretical underpinnings and the practical implementation of our approach are discussed. We test the APP_LogGIC tool that implements the proposed analysis on two realworld applications. The results show that profiling the intended program behavior is feasible in diverse applications. We discuss the heuristics used to overcome the problem of state space explosion and of the large data sets. Code metrics and test results are provided to demonstrate the effectiveness of the approach.

The architecture of existing – first generation – computer forensic tools, including the widely used EnCase and FTK products, is rapidly becoming outdated. Tools are not keeping pace with increased complexity and data volumes of modern investigations. This paper discuses the limitations of first generation computer forensic tools. Several metrics for measuring the efficacy and performance of computer forensic tools are introduced. A set of requirements for second generation tools are proposed. A high-level design for a (work in progress) second generation computer forensic analysis system is presented. a 2009 Digital Forensic Research workshop. Published by Elsevier Ltd. All rights reserved.

We present a method for detecting exploitations of vulnerabilities in privileged programs by monitoring their execution using audit trials, where the monitoring is with respect to specifications of the security-relevant behavior of the programs. Our work is motivated by the intrusion detection paradigm, but is a n attempt to avoid a d hoc approaches to codifying misuse behavior. Our approach as based on the observation that although privileged programs can be exploited (due to errors) to cause security compromise in systems because of the privileges accorded to them, the intended behavior of privileged programs is, of course, limited a n d benign. The key, then i s to specify the intended behavior (Le., the program policy) and to detect any action by privileged program that is outside the intended behavior a n d that imperils security. We describe a program policy specification language, which is based on simple predicate logic and regular expressions. I n addition, we present specifications of privileged programs in Unix, and Q prototype execution monitor for analyzing audit trails with respect to these specifications. The program policies are surprisingly concise and clear, a n d an addition, capable of detecting exploitations of known vulnerabilities in these programs. Although our work has been motivated by the known vulnerabilities in Unix, we believe that by tightly restricting the behavior of all privileged programs, exploitations of unknown vulnerabilities can be detected. A s Q check on the specifications, work is in progress on verifying them with respect to a n abstract security policy. *This work is funded in part by the National Security Agency University Research Program under Contract No. DOD-MDA904-93-C4083 and by ARPA under Contract No. USNN00014-94-1-0065.

Operating systems are complex and their behavior depends on many factors. Source code, if available, does not directly help one to understand the OS's behavior, as the behavior depends on actual workloads and external inputs. Runtime profiling is a key technique to prove new concepts, debug problems, and optimize performance. Unfortunately, existing profiling methods are lacking in important areas---they do not provide enough information about the OS's behavior, they require OS modification and therefore are not portable, or they incur high overheads thus perturbing the profiled OS.
 We developed OSprof: a versatile, portable, and efficient OS profiling method based on latency distributions analysis. OSprof automatically selects important profiles for subsequent visual analysis. We have demonstrated that a suitable workload can be used to profile virtually any OS component. OSprof is portable because it can intercept operations and measure OS behavior from user-level or from inside the kernel without requiring source code. OSprof has typical CPU time overheads below 4%. In this paper we describe our techniques and demonstrate their usefulness through a series of profiles conducted on Linux, FreeBSD, and Windows, including client/server scenarios. We discovered and investigated a number of interesting interactions, including scheduler behavior, multi-modal I/O distributions, and a previously unknown lock contention, which we fixed.

The goal of the Quest project at the IBM Almaden Research center is to develop technology to enable a new breed of da&intensive decision-support applications. This paper is a capsule summary of the current functionality and architecture of the Quest data mining System. Our overall approach has been to identify basic data mining operations that cut across applications and develop fast, scalable algorithms for their execution (Agrawal, Imielinski, & Swami 1993a). We wanted our algorithms to:

Faulty changes to the IT infrastructure can lead to critical system and application outages, and therefore cause serious economical losses. In this paper, we describe a change planning support tool that aims at assisting the change requesters in leveraging aggregated information associated with the change, like past failure reasons or best implementation practices. The thus gained knowledge can be used in the subsequent planning and implementation steps of the change. Optimal matching of change requests with the aggregated information is achieved through the classification of the change request into about 200 fine-grained activities. We propose to automatically classify the incoming change requests using various information retrieval and machine learning techniques. The cost of building the classifiers is reduced by employing active learning techniques or by leveraging labeled features. Historical tickets from two customers were used to empirically assess and compare the accuracy of the different classification approaches (Lucene index, multinomial logistic regression, and generalized expectation criteria).


Question: An integrated methodology for analyzing log files in computing system administration.

This document provides a comprehensive guide to hidden Markov models (HMMs) and their various applications, particularly focusing on speech recognition systems. Unlike conventional models, HMMs offer a probabilistic framework for sequence analysis, making them particularly suited for tasks that involve temporal or sequential data. The tutorial begins by introducing the basic concepts of HMMs, including their mathematical formulation and the three fundamental problems associated with them: evaluation, decoding, and learning. It then delves into the algorithms used to address these problems, such as the forward-backward algorithm for evaluation, the Viterbi algorithm for decoding, and the Baum-Welch algorithm for learning. The document further discusses the application of HMMs in speech recognition, where they are used to model the acoustic signals produced by human speech. This involves converting the analog speech signal into a digital form, segmenting it into phonemes, and using HMMs to recognize these phonemes and assemble them into words. The tutorial also covers advanced topics such as continuous density HMMs, which can handle continuous acoustic features, and the integration of HMMs with other machine learning techniques for improved performance. The document concludes by exploring recent developments and challenges in the field, including the adaptation of HMMs for non-speech applications and the integration of deep learning techniques to enhance their capabilities.

**Question**: An integrated framework on mining logs files for computing system management

**Similar Documents**: An advanced tutorial on the application of hidden Markov models in computing system management through log file analysis

This detailed tutorial explores the innovative use of hidden Markov models (HMMs) in the realm of computing system management, specifically focusing on the analysis of log files. Unlike traditional monitoring techniques, HMMs offer a probabilistic approach to sequence analysis, making them ideal for interpreting the temporal patterns found in system logs. The tutorial starts by outlining the foundational principles of HMMs, including their probabilistic structure and the three primary challenges: evaluation, decoding, and learning. It then proceeds to explain the algorithms used to tackle these issues, such as the forward-backward algorithm for evaluation, the Viterbi algorithm for decoding, and the Baum-Welch algorithm for learning. The document delves into the application of HMMs in log file analysis, where they are employed to model the sequences of events recorded in system logs. This involves preprocessing the log data to extract meaningful features, applying HMMs to recognize patterns, and using the results to predict anomalies or system failures. The tutorial also discusses advanced topics like the

It is a well-known problem that intrusion detection systems overload their human operators by triggering thousands of alarms per day. This paper presents a new approach for handling intrusion detection alarms more efficiently. Central to this approach is the notion that each alarm occurs for a reason, which is referred to as the alarm's root causes. This paper observes that a few dozens of rather persistent root causes generally account for over 90% of the alarms that an intrusion detection system triggers. Therefore, we argue that alarms should be handled by identifying and removing the most predominant and persistent root causes. To make this paradigm practicable, we propose a novel alarm-clustering method that supports the human analyst in identifying root causes. We present experiments with real-world intrusion detection alarms to show how alarm clustering helped us identify root causes. Moreover, we show that the alarm load decreases quite substantially if the identified root causes are eliminated so that they can no longer trigger alarms in the future.

Predicting knowledge levels from user’s implicit interactions with an adaptive system is a difficult task, particularly in learning systems that are used in the context of daily work tasks. We have collected interactions of six persons working with the adaptive work-integrated learning system APOSDLE over a period of two months to find out whether naturally occurring interactions with the system can be used to predict their level of expertise. One set of interactions is based on the tasks they performed, the other on a number of additional Knowledge Indicating Events (KIE). We find that the addition of KIE significantly improves the prediction as compared to using tasks only. Both approaches are superior to a model that uses only the frequencies of events.

In this work we describe, design and analyze the security of a tamper-evident, append-only data structure for maintaining secure data sequences in a loosely coupled distributed system where individual system components may be mutually distrust-ful. The resulting data structure, called an Authenticated Append-Only Skip List (AASL), allows its maintainers to produce one-way digests of the entire data sequence, which they can publish to others as a commitment on the contents and order of the sequence. The maintainer can produce efficiently succinct proofs that authenticate a particular datum in a particular position of the data sequence against a published digest. AASLs are secure against tampering even by malicious data structure maintain-ers. First, we show that a maintainer cannot " invent " and authenticate data elements for the AASL after he has committed to the structure. Second, he cannot equivocate by being able to prove conflicting facts about a particular position of the data sequence. This is the case even when the data sequence grows with time and its maintainer publishes successive commitments at times of his own choosing. AASLs can be invaluable in reasoning about the integrity of system logs maintained by untrusted components of a loosely-coupled distributed system .

Increasingly in the 21st century, our daily lives leave behind a detailed digital record: our shifting thoughts and opinions shared on Twitter, our social relationships, our purchasing habits, our information seeking, our photos and videos—even the movements of our bodies and cars. Naturally, for those interested in human behavior, this bounty of personal data is irresistible. Decision makers of all kinds, from company executives to government agencies to researchers and scientists, would like to base their decisions and actions on this data. In response, a new discipline of big data analytics is forming. Fundamentally, big data analytics is a workflow that distills terabytes of low-value data (e.g., every tweet) down to, in some cases, a single bit of high-value data (Should Company X acquire Company Y? Can we reject the null hypothesis?). The goal is to see the big picture from the minutia of our digital lives. It is no surprise today that big data is useful for HCI researchers and user interface design. As one example, A/B testing is a standard practice in the usability community to help determine relative differences in user performance using different interfaces. For many years, we have used strict laboratory conditions to evaluate interfaces, but more recently we have seen the ability to implement those tests quickly and on a large population by running controlled Interactions with Big Data Analytics

Workload generation is commonly employed for performance characterization, testing and benchmarking of computer systems and networks. Workload generation typically aims at simulating or emulating traffic generated by different types of applications, protocols and activities, such as web browsing, email, chat, as well as stream multimedia traffic. We present a Scalable WORkloaD generator (SWORD) that we have developed for the testing and benchmarking of high-volume data processing systems. The tool is not only scalable but is also flexible and extensible allowing the generation of workload of a variety of types of applications and of contents.

The present thesis is concerned with enterprise software systems of companies in the Swedish electricity industry, an industry that for the past few years has been exposed to a fairly tumultuous change process as a consequence of legislative reforms. Previously, the business operations of electric utilities – as well as those of most companies in the computerized world – were supported by a number of isolated software systems performing specific tasks. In recent years, these systems have been extended, and more importantly, integrated into a company-wide system in its own right, in this thesis referred to as the enterprise software system. As enterprise software systems have evolved, so has a need for strategies, methods and techniques for their management. Enterprise software system management involves a number of concerns; of these concerns, software integration is one of the most prominent. The discipline of software architecture is concerned with the modeling of large-scale structures of software systems. While the generated models are employed for a number of purposes , their perhaps most significant function is to serve as a base for reasoning about the represented system. Several methods for analysis of architectures have been proposed in recent years, and although software architecture analysis has displayed considerable success for a number of systems, enterprise software systems have to a large extent been ignored. In the empirical context of the Scandinavian electricity industry, this thesis explores the applicability of software architecture analysis to enterprise software system integration. Two conceptually different architectural analysis methods – deduction-and induction-based approaches – are considered, as well as the engineering process in which architectural analysis is performed. As a result of the investigations, the thesis proposes a modified process for architectural analysis, presents an evaluation of deduction-based analysis methods , and proposes an adaptation of induction-based analysis methods to the enterprise software system context.

Networked systems still suffer from poor firewall configuration and monitoring. VisualFirewall seeks to aid in the configuration of firewalls and monitoring of networks by providing four simultaneous views that display varying levels of detail and time-scales as well as correctly visualizing firewall reactions to individual packets. The four implemented views, real-time traffic, visual signature, statistics, and IDS alarm, provide the levels of detail and temporality that system administrators need to properly monitor their systems in a passive or an active manner. We have visualized several attacks, and we feel that even individuals unfamiliar with networking concepts can quickly distinguish between benign and malignant traffic patterns with a minimal amount of introduction.

Intrusion-detection systems aim at detecting attacks against computer systems and networks, or against information systems in general, as it is difficult to provide provably secure information systems and maintain them in such a secure state for their entire lifetime and for every utilization. Sometimes, legacy or operational constraints do not even allow a fully secure information system to be realized at all. Therefore, the task of intrusion-detection systems is to monitor the usage of such systems and to detect the apparition of insecure states. They detect attempts and active misuse by legitimate users of the information systems or external parties to abuse their privileges or exploit security vulnerabilities. In this paper, we introduce a taxonomy of intrusion-detection systems that highlights the various aspects of this area. This taxonomy defines families of intrusion-detection systems according to their properties. It is illustrated by numerous examples from past and current projects. q 1999 Elsevier Science B.V. All rights reserved.

Data retrieval systems process data from many sources, each subject to its own data use policy. Ensuring compliance with these policies despite bugs, misconfiguration, or operator error in a large, complex, and fast evolving system is a major challenge. Thoth provides an efficient, kernel-level compliance layer for data use policies. Declarative policies are attached to the systems’ input and output files, key-value tuples, and network connections, and specify the data’s integrity and confidentiality requirements. Thoth tracks the flow of data through the system, and enforces policy regardless of bugs, misconfigurations, compromises in application code, or actions by unprivileged operators. Thoth requires minimal changes to an existing system and has modest overhead, as we show using a prototype Thoth-enabled data retrieval system based on the popular Apache Lucene.

With the rapid expansion of computer networks during the past few years, security has become a crucial issue for modern computer systems. A good way to detect illegitimate use is through monitoring unusual user activity. Methods of intrusion detection based on hand-coded rule sets or predicting commands on-line are laborous to build or not very reliable. This paper proposes a new way of applying neural networks to detect intrusions. We believe that a user leaves a ’print’ when using the system; a neural network can be used to learn this print and identify each user much like detectives use thumbprints to place people at crime scenes. If a user’s behavior does not match his/her print, the system administrator can be alerted of a possible security breech. A backpropagation neural network called NNID (Neural Network Intrusion Detector) was trained in the identification task and tested experimentally on a system of 10 users. The system was 96% accurate in detecting unusual activity, with 7% false alarm rate. These results suggest that learning user profiles is an effective way for detecting intrusions.


Question: An integrated methodology for analyzing log files in computing system administration.

This paper introduces an innovative Bayesian Inference Network (BIN) model designed for automated indexing, utilizing a controlled vocabulary of index terms, or descriptors. Essential for the model's operation is an indexing dictionary that establishes rules for mapping subject field terms onto descriptors, alongside inverted lists for tracking term occurrences within a corpus of documents related to the subject field, as well as descriptors manually allocated to these documents. Notably, this indexing dictionary can be automatically generated from a collection of documents that have been manually indexed. The paper proceeds to detail an implementation of the BIN model, accompanied by a practical indexing example and a presentation of experimental outcomes that gauge the model's indexing effectiveness.

**Question**: A Comparative Study of Metaheuristic Algorithms for the Economic Dispatch Problem

**Documents**: A Comparative Study of Some Metaheuristic Algorithms for Solving the Unit Commitment Problem
This paper presents a comparative study of the performance of some metaheuristic algorithms for solving the unit commitment problem. The algorithms considered are: the genetic algorithm (GA), the particle swarm optimization (PSO), the differential evolution (DE), the simulated annealing (SA), and the tabu search (TS). The performance of these algorithms is evaluated on the IEEE 118-bus test system. The results show that the DE algorithm outperforms the other algorithms in terms of solution quality and computational efficiency.

**Similar Documents**: This study offers a comprehensive evaluation of various metaheuristic algorithms applied to the unit commitment problem, focusing on their comparative efficacy. The algorithms scrutinized include the genetic algorithm (GA), particle swarm optimization (PSO), differential evolution (DE), simulated annealing (SA), and tabu search (TS). Their performance is rigorously tested using the IEEE 118-bus benchmark system. The findings highlight the superiority of the DE algorithm over its counterparts, both in terms of solution quality and computational efficiency.

The Large Hadron Collider (LHC) is one of the most complex machines ever build. It is composed of many components which constitute a large system. The tunnel and the accelerator is just one of a very critical fraction of the whole LHC infrastructure. Hardware comissioning as one of the critical processes before running the LHC is implemented during the Long Shutdown (LS) states of the macine, where Electrical Quality Assurance (ELQA) is one of its key components. Here a huge data is collected when implementing various ELQA electrical tests. In this paper we present a conceptual framework for supporting a rapid design of web applications for ELQA data analysis. We show a framework’s main components, their possible integration with other systems and machine learning algorithms and a simple use case of prototyping an application for Electrical Quality Assurance of the LHC.

Contemporary workflow management systems are driven by explicit process models, i.e., a completely specified workflow design is required in order to enact a given workflow process. Creating a workflow design is a complicated time-consuming process and, typically, there are discrepancies between the actual workflow processes and the processes as perceived by the management. Therefore, we have developed techniques for discovering workflow models. The starting point for such techniques is a so-called "workflow log" containing information about the workflow process as it is actually being executed. We present a new algorithm to extract a process model from such a log and represent it in terms of a Petri net. However, we also demonstrate that it is not possible to discover arbitrary workflow processes. We explore a class of workflow processes that can be discovered. We show that the /spl alpha/-algorithm can successfully mine any workflow represented by a so-called SWF-net.

We present a method for detecting exploitations of vulnerabilities in privileged programs by monitoring their execution using audit trials, where the monitoring is with respect to specifications of the security-relevant behavior of the programs. Our work is motivated by the intrusion detection paradigm, but is a n attempt to avoid a d hoc approaches to codifying misuse behavior. Our approach as based on the observation that although privileged programs can be exploited (due to errors) to cause security compromise in systems because of the privileges accorded to them, the intended behavior of privileged programs is, of course, limited a n d benign. The key, then i s to specify the intended behavior (Le., the program policy) and to detect any action by privileged program that is outside the intended behavior a n d that imperils security. We describe a program policy specification language, which is based on simple predicate logic and regular expressions. I n addition, we present specifications of privileged programs in Unix, and Q prototype execution monitor for analyzing audit trails with respect to these specifications. The program policies are surprisingly concise and clear, a n d an addition, capable of detecting exploitations of known vulnerabilities in these programs. Although our work has been motivated by the known vulnerabilities in Unix, we believe that by tightly restricting the behavior of all privileged programs, exploitations of unknown vulnerabilities can be detected. A s Q check on the specifications, work is in progress on verifying them with respect to a n abstract security policy. *This work is funded in part by the National Security Agency University Research Program under Contract No. DOD-MDA904-93-C4083 and by ARPA under Contract No. USNN00014-94-1-0065.

This paper presents strategies and lessons learned from the use of natural language annotations to facilitate question answering in the START information access system.

Computer-based learning systems can now keep detailed logs of user-system interactions, including key clicks, eye-tracking, and video data, opening up new opportunities to study how students learn with technology. Educational Data Mining (EDM; Romero, Ventura, Pechenizkiy, & Baker, 2010) is concerned with developing, researching, and applying computerized methods to detect patterns in large collections of educational data – patterns that would otherwise be hard or impossible to analyze due to the enormous volume of data they exist within. Data of interest is not restricted to interactions of individual students with an educational system (e.g., navigation behavior, input to quizzes and interactive exercises) but might also include data from collaborating students (e.g., text chat), administrative data (e.g., school, school district, teacher), and demographic data (e.g., gender, age, school grades). Data on student affect (e.g., motivation, emotional states) has also been a focus, which can be inferred from physiological sensors (e.g., facial expression, seat posture and perspiration). EDM uses methods and tools from the broader field of Data Mining (Witten & Frank, 2005), a sub-field of Computer Science and Artificial Intelligence that has been used for purposes as diverse as credit card fraud detection, analysis of gene sequences in bioinformatics, or the analysis of purchasing behaviors of customers. Distinguishing EDM features are its particular focus on educational data and problems, both theoretical (e.g., investigating a learning hypothesis) and practical (e.g., improving a learning tool). Furthermore, EDM makes a methodological contribution by developing and researching data mining techniques for educational applications. Typical steps in an EDM project include data acquisition, data preprocessing (e.g., data “cleaning”), data mining, and validation of results.

Systematic endeavors to take computer science (CS) and computational thinking (CT) to scale in middle and high school classrooms are underway with curricula that emphasize the enactment of authentic CT skills, especially in the context of programming in block-based programming environments. There is, therefore, a growing need to measure students’ learning of CT in the context of programming and also support all learners through this process of learning computational problem solving. The goal of this research is to explore hypothesis-driven approaches that can be combined with data-driven ones to better interpret student actions and processes in log data captured from block-based programming environments with the goal of measuring and assessing students’ CT skills. Informed by past literature and based on our empirical work examining a dataset from the use of the Fairy Assessment in the Alice programming environment in middle schools, we present a framework that formalizes a process where a hypothesis-driven approach informed by Evidence-Centered Design effectively complements data-driven learning analytics in interpreting students’ programming process and assessing CT in block-based programming environments. We apply the framework to the design of Alice tasks for high school CS to be used for measuring CT during programming.

In a decade of process mining research, several algorithms have been proposed to solve particular process mining tasks. At the same pace, tools have appeared both in the academic and the commercial domains. These tools have enabled the use of process mining practices to a rather limited extent. In this paper we advocate for a change in the mentality: process mining may be an exploratory discipline, and PMLAB – a Python-based scripting environment supporting this – is proposed. This demo presents the main features of the PMLAB environment

Large enterprise IT (Information Technology) infrastructure components generate large volumes of alerts and incident tickets. These are manually screened, but it is otherwise difficult to extract information automatically from them to gain insights in order to improve operational efficiency. We propose a framework to cluster alerts and incident tickets based on the text in them, using unsupervised machine learning. This would be a step towards eliminating manual classification of the alerts and incidents, which is very labor intense and costly. Our framework can handle the semi-structured text in alerts generated by IT infrastructure components such as storage devices, network devices, servers etc., as well as the unstructured text in incident tickets created manually by operations support personnel. After text pre-processing and application of appropriate distance metrics, we apply different graph-theoretic approaches to cluster the alerts and incident tickets, based on their semi-structured and unstructured text respectively. For automated interpretation and read-ability on semi-structured text clusters, we propose a method to visualize clusters that preserves the structure and human-readability of the text data as compared to traditional word clouds where the text structure is not preserved; for unstructured text clusters, we find a simple way to define prototypes of clusters for easy interpretation. This framework for clustering and visualization will enable enterprises to prioritize the issues in their IT infrastructure and improve the reliability and availability of their services.

In this paper, the use of systems development as a methodology in information systems (is) research is described and defended. A framework to explain the nature of systems development as a research methodology in is research is proposed. Use of this methodology in the engineering field in general is compared with its use specifically in computer science and computer engineering. An integrated program for conducting IS research that incorporates theory building, systems development, experimentation, and observation is proposed. Progress in several application domains is reviewed to provide a basis upon which to argue that systems development is a valid research methodology. A systems development research process is presented from a methodological perspective. Software engineering, which is the basic method of applying the systems development research methodology, is then discussed. It is the authors' belief that systems development and other research methodologies are complementary and that an integrated multi-dimensional and multimethodological approach will generate fruitful is research results. The premise is that research contributions can result from systems development, experimentation, observation, and performance testing of the systems under development and that all of these research approaches are needed to investigate different aspects of the research question. An earlier version of Ihis paper was originally published in the Proceedings of ihe Twenty-Third Hawaii International Conference on System Sciences (IEEE Computer Society Press, 1990). Journal of Management Information Systems I Winter 1990-91, Vol. 7, No, 3, pp. 89-106. Copyright © M. E. Sharpe, Inc., 1991. 90 NUNAMAKER, CHEN. AND PURDIN

Program analysis is the process of gathering deeper insights about a source code and analysing them to resolve software problems of arbitrary complexity. The key challenge in program analysis is to keep it fast, precise and straightforward. This research focuses on three key objectives to achieve an efficient program analysis: (i) expressive data representation, (ii) optimised data structure and (iii) fast data processing mechanisms. State of the art technologies such as Resource Description Framework (RDF) as data representation format, triplestores as the storage & processing layer, and datalog to represent program analysis rules are considered in our research. diagram(BDD) to be embedded in the triplestore. Additionally, an ontology is being designed to standardise the definitions of concepts and representation of the knowledge in the program analysis domain.