Question: A polynomial chaos method using collocation and least-squares for solving stochastic problems.

This paper delves into the exploration of the Homogeneous Chaos theory, a fundamental concept in the realm of stochastic processes and their representations. The author(s) of this work, while respecting the terms and conditions of use as outlined by JSTOR, which can be accessed at http://www.jstor.org/page/info/about/policies/terms.jsp, emphasize that users must adhere to the stipulated guidelines. These terms clarify that without prior permission, users are prohibited from downloading entire journal issues or multiple copies of articles. Instead, the content within the JSTOR archive is intended for personal, non-commercial use, allowing individuals to engage with the material for educational and research purposes within the bounds of fair use. In the context of the Homogeneous Chaos theory, this restriction ensures that scholarly work remains accessible to a wide audience while safeguarding the rights of authors and publishers. The theory itself, when applied to the Collocation least-squares polynomial chaos method, provides a powerful framework for analyzing and modeling stochastic systems, particularly in engineering and scientific applications where uncertainty quantification is crucial. By respecting the terms of use, researchers and students can effectively leverage the Homogeneous Chaos theory to advance their understanding of complex stochastic phenomena without infringing on copyright laws or ethical guidelines.

Consider a power series f ∈ R[[z]], which is obtained by a precise mathematical construction. For instance, f might be the solution to some differential or functional initial value problem or the diagonal of the solution to a partial differential equation. In cases when no suitable method is beforehand for determining the asymptotics of the coefficients fn, but when many such coefficients can be computed with high accuracy, it would be useful if a plausible asymptotic expansion for fn could be guessed automatically. In this paper, we will present a general scheme for the design of such “asymptotic extrapolation algorithms”. Roughly speaking, using discrete differentiation and techniques from automatic asymptotics, we strip off the terms of the asymptotic expansion one by one. The knowledge of more terms of the asymptotic expansion will then allow us to approximate the coefficients in the expansion with high accuracy.

We extend the herding algorithm to continuous spaces by using the kernel trick. The resulting “kernel herding” algorithm is an infinite memory deterministic process that learns to approximate a PDF with a collection of samples. We show that kernel herding decreases the error of expectations of functions in the Hilbert space at a rateO(1/T )which is much faster than the usual O(1/ √ T ) for iid random samples. We illustrate kernel herding by approximating Bayesian predictive distributions.

In this paper we study distributionally robust optimization (DRO) problems where the ambiguity set of the probability distribution is defined by the Kullback-Leibler (KL) divergence. We consider DRO problems where the ambiguity is in the objective function, which takes a form of an expectation, and show that the resulted minimax DRO problems can be formulated as a one-layer convex minimization problem. We also consider DRO problems where the ambiguity is in the constraint. We show that ambiguous expectation-constrained programs may be reformulated as a one-layer convex optimization problem that takes the form of the Benstein approximation of Nemirovski and Shapiro (2006). We further consider distributionally robust probabilistic programs. We show that the optimal solution of a probability minimization problem is also optimal for the distributionally robust version of the same problem, and also show that the ambiguous chance-constrained programs (CCPs) may be reformulated as the original CCP with an adjusted confidence level. A number of examples and special cases are also discussed in the paper to show that the reformulated problems may take simple forms that can be solved easily. The main contribution of the paper is to show that the KL divergence constrained DRO problems are often of the same complexity as their original stochastic programming problems and, thus, KL divergence appears a good candidate in modeling distribution ambiguities in mathematical programming.

Sometimes we need to solve the linear equation Ax = b for a very big and very sparse A. For example, the Poisson equation where only 5 entries of each row of the matrix A are non-zero. Standard methods such as inverting the matrix A (numerically unstable) or Guass elimination do not take advantage of the sparsity of A. In this lesson we will discuss two iterative methods suitable for sparse linear systems: Jacobi and Gauss-Seidel.

We review in this paper several methods from Statistical Learning Theory (SLT) for the performance assessment and uncertainty quantification of predictive models. Computational issues are addressed so to allow the scaling to large datasets and the application of SLT to Big Data analytics. The effectiveness of the application of SLT to manufacturing systems is exemplified by targeting the derivation of a predictive model for quality forecasting of products on an assembly line.

In this paper, we propose a method for robust kernel density estimation. We interpret a KDE with Gaussian kernel as the inner product between a mapped test point and the centroid of mapped training points in kernel feature space. Our robust KDE replaces the centroid with a robust estimate based on M-estimation (P. Huber, 1981), The iteratively re-weighted least squares (IRWLS) algorithm for M-estimation depends only on inner products, and can therefore be implemented using the kernel trick. We prove the IRWLS method monotonically decreases its objective value at every iteration for a broad class of robust loss functions. Our proposed method is applied to synthetic data and network traffic volumes, and the results compare favorably to the standard KDE.

A combination of distributed computation, positive feedback and constructive greedy heuristic is proposed as a new approach to stochastic optimization and problem solving. Positive feedback accounts for rapid discovery of very good solutions, distributed computation avoids premature convergence, and greedy heuristic helps the procedure to find acceptable solutions in the early stages of the search process. An application of the proposed methodology to the classical travelling salesman problem shows that the system can rapidly provide very good, if not optimal, solutions. We report on many simulation results and discuss the working of the algorithm. Some hints about how this approach can be applied to a variety of optimization problems are also given.

Numerical solution of ill-posed problems is often accomplished by discretization (projection onto a finite dimensional subspace) followed by regularization. If the discrete problem has high dimension, though, typically we compute an approximate solution by projecting the discrete problem onto an even smaller dimensional space, via iterative methods based on Krylov subspaces. In this work we present a common framework for efficient algorithms that regularize after this second projection rather than before it. We show that determining regularization parameters based on the final projected problem rather than on the original discretization has firmer justification and often involves less computational expense. We prove some results on the approximate equivalence of this approach to other forms of regularization, and we present numerical examples.

A simple and efficient randomized algorithm is presented for solving single-query path planning problems in high-dimensional configuration spaces. The method works by incrementally building two Rapidly-exploring Random Trees (RRTs) rooted at the start and the goal configurations. The trees each explore space around them and also advance towards each other through the use of a simple greedy heuristic. Although originally designed to plan motions for a human arm (modeled as a 7-DOF kinematic chain) for the automatic graphic animation of collision-free grasping and manipulation tasks, the algorithm has been successfully applied to a variety of path planning problems. Computed examples include generating collision-free motions for rigid objects in 2D and 3D, and collision-free manipulation motions for a 6-DOF PUMA arm in a 3D workspace. Some basic theoretical analysis is also presented.

Numerical errors and rounding errors in continuous collision detection (CCD) can easily cause collision detection failures if they are not handled properly. A simple and effective approach is to use error tolerances, as shown in many existing CCD systems. Unfortunately, finding the optimal tolerance values is a difficult problem for users. Larger tolerance values will introduce false positive artifacts, while smaller tolerance values may cause collisions to be undetected. The biggest issue here is that we do not know whether or when CCD will fail, even though failures are extremely rare. In this paper, we demonstrate a set of simple modifications to make a basic CCD implementation failure-proof. Using error analysis, we prove the safety of this method and we formulate suggested tolerance values to reduce false positives. The resulting algorithms are safe, automatic, efficient, and easy to implement.


Question: Collocation-based least-squares approach using polynomial chaos expansion.

This investigation delves into the application of generalized polynomial chaos theory for modeling intricate, nonlinear multibody dynamic systems under the influence of both parametric and external uncertainties. The theoretical foundations and computational aspects of this method are comprehensively covered in the corresponding paper titled "Modeling Multibody Dynamic Systems With Uncertainties: Part I—Theoretical and Computational Aspects." Here, we focus on demonstrating the practical utility of the approach through a series of test cases. We analyze the cumulative impact of parametric and forcing uncertainties on a quarter car model, validating the uncertainty distributions in system response across both temporal and spectral domains against Monte Carlo simulations. Our findings underscore the superior efficiency of polynomial chaos over Monte Carlo methods and its greater precision compared to statistical linearization. Notably, the outcomes from the direct collocation technique align closely with those derived from the Galerkin approach. To simulate stochastic terrain, a truncated Karhunen-Loeve expansion is employed to construct a stochastic terrain model. The applicability of polynomial chaos to differential-algebraic systems is showcased through the constrained pendulum problem. The inherent constraints of the polynomial chaos methodology are explored using two distinct test scenarios: one featuring multiple attractor points, and the other involving chaotic evolution with a nonlinear attractor set. Despite recognizing its limitations, we conclude that generalized polynomial chaos stands as a potent tool for simulating multibody dynamic systems under uncertainty.

Signal processing and pattern recognition algorithms make extensive use of convolution. In many cases, computational accuracy is not as important as computational speed. In feature extraction, for instance, the features of interest in a signal are usually quite distorted. This form of noise justi es some level of quantization in order to achieve faster feature extraction. Our approach consists of approximating regions of the signal with low degree polynomials, and then di erentiating the resulting signals in order to obtain impulse functions (or derivatives of impulse functions). With this representation, convolution becomes extremely simple and can be implemented quite e ectively. The true convolution can be recovered by integrating the result of the convolution. This method yields substantial speed up in feature extraction and is applicable to convolutional neural networks.

Managing and hedging the risks associated with Variable Annuity (VA) products requires intraday valuation of key risk metrics for these products. The complex structure of VA products and computational complexity of their accurate evaluation has compelled insurance companies to adopt Monte Carlo (MC) simulations to value their large portfolios of VA products. Because the MC simulations are computationally demanding, especially for intraday valuations, insurance companies need more efficient valuation techniques. Recently, a framework based on traditional spatial interpolation techniques has been proposed that can significantly decrease the computational complexity of MC simulation (Gan and Lin, 2015). However, traditional interpolation techniques require the definition of a distance function that can significantly impact their accuracy. Moreover, none of the traditional spatial interpolation techniques provide all of the key properties of accuracy, efficiency, and granularity (Hejazi et al., 2015). In this paper, we present a neural network approach for the spatial interpolation framework that affords an efficient way to find an effective distance function. The proposed approach is accurate, efficient, and provides an accurate granular view of the input portfolio. Our numerical experiments illustrate the superiority of the performance of the proposed neural network approach compared to the traditional spatial interpolation schemes.

Inverse problems correspond to a certain type of optimization problems formulated over appropriate input distributions. Recently, there has been a growing interest in understanding the computational hardness of these optimization problems, not only in the worst case, but in an average-complexity sense under this same input distribution.In this revised note, we are interested in studying another aspect of hardness, related to the ability to learn how to solve a problem by simply observing a collection of previously solved instances. These ‘planted solutions’ are used to supervise the training of an appropriate predictive model that parametrizes a broad class of algorithms, with the hope that the resulting model will provide good accuracy-complexity tradeoffs in the average sense.We illustrate this setup on the Quadratic Assignment Problem, a fundamental problem in Network Science. We observe that data-driven models based on Graph Neural Networks offer intriguingly good performance, even in regimes where standard relaxation based techniques appear to suffer.

The authors show that a multiple-input, single-output, single-hidden-layer feedforward network with (known) hardwired connections from input to hidden layer, monotone squashing at the hidden layer and no squashing at the output embeds as a special case a so-called Fourier network, which yields a Fourier series approximation properties of Fourier series representations. In particular, approximation to any desired accuracy of any square integrable function can be achieved by such a network, using sufficiently many hidden units. In this sense, such networks do not make avoidable mistakes.<<ETX>>

Short documents are typically represented by very sparse vectors, in the space of terms. In this case, traditional techniques for calculating text similarity results in measures which are very close to zero, since documents even the very similar ones have a very few or mostly no terms in common. In order to alleviate this limitation, the representation of short-text segments should be enriched by incorporating information about correlation between terms. In other words, if two short segments do not have any common words, but terms from the first segment appear frequently with terms from the second segment in other documents, this means that these segments are semantically related, and their similarity measure should be high. Towards achieving this goal, we employ a method for enhancing document clustering using statistical semantics. However, the problem of high computation time arises when calculating correlation between all terms. In this work, we propose the selection of a few terms, and using these terms with the Nystr\"om method to approximate the term-term correlation matrix. The selection of the terms for the Nystr\"om method is performed by randomly sampling terms with probabilities proportional to the lengths of their vectors in the document space. This allows more important terms to have more influence on the approximation of the term-term correlation matrix and accordingly achieves better accuracy.

This paper advocates a new ML-based programming framework, called Programming by Feedback (PF), which involves a sequence of interactions between the active computer and the user. The latter only provides preference judgments on pairs of solutions supplied by the active computer. The active computer involves two components: the learning component estimates the user’s utility function and accounts for the user’s (possibly limited) competence; the optimization component explores the search space and returns the most appropriate candidate solution. A proof of principle of the approach is proposed, showing that PF requires a handful of interactions in order to solve some discrete and continuous benchmark problems.

In this paper Görges polygons are used in conjunction with masses geometry to find an easy and affordable way to compute the differential leakage factor of non symmetrical full and dead coil winding. By following the traditional way, the use of the Ossanna's infinite series which has to be obviously truncated under the bound of a predetermined accuracy is mandatory. In the presented method no infinite series is instead required. An example is then shown and discussed to demonstrate practically the effectiveness of the proposed method.

We offer an improved method for using a nuclear-magnetic-resonance quantum computer (NMRQC) to solve the Deutsch-Jozsa problem. Two known obstacles to the application of the NMRQC are exponential diminishment of density-matrix elements with the number of bits, threatening weak signal levels, and the high cost of preparing a suitable starting state. A third obstacle is a heretofore unnoticed restriction on measurement operators available for use by an NMRQC. Variations on the function classes of the Deutsch-Jozsa problem are introduced, both to extend the range of problems advantageous for quantum computation and to escape all three obstacles to use of an NMRQC. By adapting it to one such function class, the Deutsch-Jozsa problem is made solvable without exponential loss of signal. The method involves an extra work bit and a polynomially more involved Oracle; it uses the thermal-equilibrium density matrix systematically for an arbitrary number of spins, thereby avoiding both the preparation of a pseudopure state and temporal averaging.

In this paper, a probabilistic fuzzy logic system (PFLS) is proposed for the modeling and control problems. Similar to the ordinary fuzzy logic system (FLS), the PFLS consists of the fuzzification, inference engine and defuzzification operation to process the fuzzy information. Different to the FLS, it uses the probabilistic modeling method to improve the stochastic modeling capability. By using a three-dimensional membership function (MF), the PFLS is able to handle the effect of random noise and stochastic uncertainties existing in the process. A unique defuzzification method is proposed to simplify the complex operation. Finally, the proposed PFLS is applied to a function approximation problem and a robotic system. It shows a better performance than an ordinary FLS in stochastic circumstance.

Hamiltonian Monte Carlo (HMC) sampling methods provide a mechanism for defining distant proposals with high acceptance probabilities in a Metropolis-Hastings framework, enabling more efficient exploration of the state space than standard random-walk proposals. The popularity of such methods has grown significantly in recent years. However, a limitation of HMC methods is the required gradient computation for simulation of the Hamiltonian dynamical system—such computation is infeasible in problems involving a large sample size or streaming data. Instead, we must rely on a noisy gradient estimate computed from a subset of the data. In this paper, we explore the properties of such a stochastic gradient HMC approach. Surprisingly, the natural implementation of the stochastic approximation can be arbitrarily bad. To address this problem we introduce a variant that uses second-order Langevin dynamics with a friction term that counteracts the effects of the noisy gradient, maintaining the desired target distribution as the invariant distribution. Results on simulated data validate our theory. We also provide an application of our methods to a classification task using neural networks and to online Bayesian matrix factorization.


Question: Collocation-based least-squares polynomial chaos technique for stochastic problems.

This paper introduces an innovative technique for addressing stochastic differential equations (SDEs) utilizing Galerkin projections and an advanced version of Wiener's polynomial chaos theory. The method employs an optimal trial basis derived from the Askey family of orthogonal polynomials, which significantly reduces the complexity of the system by minimizing the dimensionality. This leads to an exponential decrease in error, enhancing the solution's accuracy. The approach is applied to various continuous and discrete stochastic processes, showcasing its versatility. Comparative studies with Monte Carlo simulations highlight the method's efficiency, particularly for problems with a limited number of stochastic inputs, demonstrating a significant reduction in computational time and resources required.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A novel method for economic dispatch problem with valve-point effect
In this paper, a novel method, the Hybrid Bat Algorithm (HBA), is proposed to solve the economic dispatch problem considering valve-point effects. The valve-point effect, which is a nonlinear function, is a significant issue in the operation of thermal power plants. The proposed HBA is a combination of the Bat Algorithm (BA) and the Particle Swarm Optimization (PSO) algorithm. The BA is used for the global search, while the PSO is employed for the local search. The proposed method is tested on three different systems with valve-point effects, and the results are compared with those obtained by other methods. The results show that the proposed HBA is capable of finding the global optimum solution with high accuracy and efficiency.

**Similar Documents**: This paper introduces a new technique, the Hybrid Bat Algorithm (HBA), designed to tackle the economic dispatch problem in the presence of valve-point effects, a critical factor in the operation of thermal power plants due to its nonlinearity. The HBA integrates the strengths of the Bat Algorithm (BA) for global exploration with the Particle Swarm Optimization (PSO) for local exploitation. The proposed method is evaluated on three distinct systems incorporating valve-point effects, with outcomes compared against those from alternative methods. The comparative analysis demonstrates the superior performance of HBA in locating the global optimum solution with remarkable precision and computational efficiency, establishing its effectiveness in solving complex economic dispatch problems.

A new method to fit specific types of conics to scattered data points is introduced. Direct, specific fitting of ellipses and hyperbolae is achieved by imposing a quadratic constraint on the conic coefficients, whereby an improved partitioning of the design matrix is devised so as to improve computational efficiency and numerical stability by eliminating redundant aspects of the fitting procedure. Fitting of parabolas is achieved by determining an orthogonal basis vector set in the Grassmannian space of the quadratic terms’ coefficients. The linear combination of the basis vectors that fulfills the parabolic condition and has a minimum residual norm is determined using Lagrange multipliers. This is the first known direct solution for parabola specific fitting. Furthermore, the inherent bias of a linear conic fit is addressed. We propose a linear method of correcting this bias, producing better geometric fits which are still constrained to specific conic type. 2007 Published by Elsevier B.V.

Function approximation is viewed from the perspective of numerical optimization in function space, rather than parameter space. A connection is made between stagewise additive expansions and steepest{descent minimization. A general gradient{descent \boosting" paradigm is developed for additive expansions based on any tting criterion. Speci c algorithms are presented for least{squares, least{absolute{deviation, and Huber{M loss functions for regression, and multi{class logistic likelihood for classi cation. Special enhancements are derived for the particular case where the individual additive components are decision trees, and tools for interpreting such \TreeBoost" models are presented. Gradient boosting of decision trees produces competitive, highly robust, interpretable procedures for regression and classi cation, especially appropriate for mining less than clean data. Connections between this approach and the boosting methods of Freund and Shapire 1996, and Friedman, Hastie, and Tibshirani 1998 are discussed. 1 Function estimation In the function estimation problem one has a system consisting of a random \output" or \response" variable y and a set of random \input" or \explanatory" variables x = fx1; ; xng. Given a \training" sample fyi;xig N 1 of known (y;x){values, the goal is to nd a function F (x) that maps x to y, such that over the joint distribution of all (y;x){values, the expected value of some speci ed loss function (y; F (x)) is minimized F (x) = argmin F (x) Ey;x (y; F (x)) = argmin F (x) Ex [Ey( (y; F (x)) jx] : (1) Frequently employed loss functions (y; F ) include squared{error (y F ) and absolute error jy F j for y 2 R (regression), and negative binomial log{likelihood, log(1 + e 2yF ), when y 2 f 1; 1g (classi cation). A common procedure is to take F (x) to be a member of a parameterized class of functions F (x;P), where P = fP1; P2; g is a set of parameters. In this paper we focus on \additive" expansions of the form

BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization is sample efficient by building a posterior distribution to capture the evidence and prior knowledge for the target function. Built in standard C++, the library is extremely efficient while being portable and flexible. It includes a common interface for C, C++, Python, Matlab and Octave.

This paper addresses the problem of parametric representation and estimation of complex planar curves in 2-D, surfaces in 3-D and nonplanar space curves in 3-D. Curves and surfaces can be defined either parametrically or implicitly, and we use the latter representation. A planar curve is the set of zeros of a smooth function of two variables X-Y, a surface is the set of zeros of a smooth function of three variables X-~-Z, and a space curve is the intersection of two surfaces, which are the set of zeros of two linearly independent smooth functions of three variables X-!/-Z. For example, the surface of a complex object in 3D can be represented as a subset of a single implicit surface, with similar results for planar and space curves. We show how this unified representation can be used for object recognition, object position estimation, and segmentation of objects into meaningful subobjects, that is, the detection of “interest regions” that are more complex than high curvature regions and, hence, more useful as features for object recognition. Fitting implicit curves and surfaces to data would be ideally based on minimizing the mean square distance from the data points to the curve or surface. Since the distance from a point to a curve or surface cannot be computed exactly by direct methods, the approximate distance, which is a first-order approximation of the real distance, is introduced, generalizing and unifying previous results. We fit implicit curves and surfaces to data minimizing the approximate mean square distance, which is a nonlinear least squares problem. We show that in certain cases, this problem reduces to the generalized eigenvector fit, which is the minimization of the sum of squares of the values of the functions that define the curves or surfaces under a quadratic constraint function of the data. This fit is computationally reasonable to compute, is readily parallelizable, and, hence, is easily computed in real time. In general, the generalized eigenvector lb provides a very good initial estimate for the iterative minimization of the approximate mean square distance. Although we are primarily interested in the 2-D and 3-D cases, the methods developed herein are dimension independent. We show that in the case of algebraic curves and surfaces, i.e., those defined by sets of zeros of polynomials, the minimizers of the approximate mean square distance and the generalized eigenvector fit are invariant with respect to similarity transformations. Thus, the generalized eigenvector lit is independent of the choice of coordinate system, which is a very desirable property for object recognition, position estimation, and the stereo matching problem. Finally, as applications of the previous techniques, we illustrate the concept of “interest regions” Manuscript received April 13, 1988; revised April 30, 1991. This work was supported by an IBM Fellowship and NSF Grant IRI-8715774. The author was with the Laboratory for Engineering Man/Machine Systems, Department of Engineering, Brown University, Providence, RI 02912. He is now with the Exploratory Computer Vision Group, IBM T. J. Watson Research Center, Yorktown Heights, NY 10598. IEEE Log Number 9102093. for object recognition and describe a variable-order segmentation algorithm that applies to the three cases of interest.

We propose a new approach to collision and self– collision detection of dynamically deforming objects that consist of tetrahedrons. Tetrahedral meshes are commonly used to represent volumetric deformable models and the presented algorithm is integrated in a physically–based environment, which can be used in game engines and surgical simulators. The proposed algorithm employs a hash function for compressing a potentially infinite regular spatial grid. Although the hash function does not always provide a unique mapping of grid cells, it can be generated very efficiently and does not require complex data structures, such as octrees or BSPs. We have investigated and optimized the parameters of the collision detection algorithm, such as hash function, hash table size and spatial cell size. The algorithm can detect collisions and self– collisions in environments of up to 20k tetrahedrons in real–time. Although the algorithm works with tetrahedral meshes, it can be easily adapted to other object primitives, such as triangles. Figure 1: Environment with dynamically deforming objects, that consist of tetrahedrons.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features method gains much popularity due to its simplicity and direct reduction of nonlinear problem to a linear one. Different random feature functions have since been proposed to approximate a variety of kernel functions. Among them the Random Binning (RB) feature, proposed in the first random-feature paper [21], has drawn much less attention than the Random Fourier (RF) feature proposed also in [21]. In this work, we observe that the RB features, with right choice of optimization solver, could be orders-of-magnitude more efficient than other random features and kernel approximation methods under the same requirement of accuracy. We thus propose the first analysis of RB from the perspective of optimization, which by interpreting RB as a Randomized Block Coordinate Descent in the infinite-dimensional space, gives a faster convergence rate compared to that of other random features. In particular, we show that by drawing R random grids with at least κ number of non-empty bins per grid in expectation, RB method achieves a convergence rate of O(1/κ R)), which not only sharpens its O(1/√R) rate from Monte Carlo analysis, but also shows a κ times speedup over other random features under the same analysis framework. In addition, we demonstrate another advantage of RB in the L1-regularized setting, where unlike other random features, a RB-based Coordinate Descent solver can be parallelized with guaranteed speedup proportional to κ. Our extensive experiments demonstrate the superior performance of the RB features over other random features and kernel approximation methods.

The problem of retrieving phase information from amplitude measurements alone has appeared in many scientific disciplines over the last century. PhaseLift is a recently introduced algorithm for phase recovery that is computationally efficient, numerically stable, and comes with rigorous performance guarantees. PhaseLift is optimal in the sense that the number of amplitude measurements required for phase reconstruction scales linearly with the dimension of the signal. However, it specifically demands Gaussian random measurement vectors — a limitation that restricts practical utility and obscures the specific properties of measurement ensembles that enable phase retrieval. Here we present a partial derandomization of PhaseLift that only requires sampling from certain polynomial size vector configurations, called t-designs. Such configurations have been studied in algebraic combinatorics, coding theory, and quantum information. We prove reconstruction guarantees for a number of measurements that depends on the degree t of the design. If the degree is allowed to to grow logarithmically with the dimension, the bounds become tight up to polylog-factors. Beyond the specific case of PhaseLift, this work highlights the utility of spherical designs for the derandomization of data recovery schemes.

Problem statement: The identification of collocations is very importa n part in natural language processing applications that require some degree of semantic interpretation such as, machine translation, information retrieval and text summari z tion. Because of the complexities of Arabic, the collocations undergo some variations such as, morph ological, graphical, syntactic variation that constitutes the difficulties of identifying the col location. Approach: We used the hybrid method for extracting the collocations from Arabic corpus that is based on linguistic information and association measures. Results: This method extracted the bi-gram candidates of Ar abic collocation from corpus and evaluated the association measures by using the n-best evaluation method. We reported the precision values for each association measure in ea ch n-best list. Conclusion: The experimental results showed that the log-likelihood ratio is the best as ociation measure that achieved highest precision.

We provide a new unifying view, including all existing prope r probabilistic sparse approximations for Gaussian process regression. Our approach relies on exp ressing theeffective priorwhich the methods are using. This allows new insights to be gained, and highlights the relationship between existing methods. It also allows for a clear theoretically j ustified ranking of the closeness of the known approximations to the corresponding full GPs. Finall y we point directly to designs of new better sparse approximations, combining the best of the exi sting strategies, within attractive computational constraints.

In this paper a comparison is made between four frequently encountered resampling algorithms for particle filters. A theoretical framework is introduced to be able to understand and explain the differences between the resampling algorithms. This facilitates a comparison of the algorithms with respect to their resampling quality and computational complexity. Using extensive Monte Carlo simulations the theoretical results are verified. It is found that systematic resampling is favourable, both in terms of resampling quality and computational complexity.


Question: Collocation least-squares method using polynomial chaos expansion.

This investigation delves into the application of generalized polynomial chaos theory for the modeling of intricate, nonlinear multibody dynamic systems under conditions of parametric and external uncertainty. The choice of the polynomial chaos framework is driven by its computational efficiency in handling the substantial, nonlinear multibody models typical of engineering systems, particularly when the number of uncertain parameters is limited, but the scale of uncertainties can be substantial (for instance, in vehicle-soil interaction scenarios). The methodology proposed herein facilitates the quantification of uncertainty distributions across both the time and frequency domains, enabling simulations of multibody systems that yield results accompanied by "error bars". The initial part of this study is dedicated to elucidating the theoretical and computational facets of the polynomial chaos approach. Both unconstrained and constrained formulations of multibody dynamics are taken into account. A key proposal is the use of direct stochastic collocation as a more cost-effective alternative to the conventional Galerkin method. It is demonstrated that stochastic collocation is tantamount to a stochastic response surface methodology. We also explore the construction of multidimensional basis functions as tensor products of their one-dimensional counterparts, alongside the handling of polynomial and trigonometric nonlinearities. Parametric uncertainties are modeled using finite-support probability densities, and stochastic forces are discretized through truncated Karhunen-Loeve expansions. The companion paper, "Modeling Multibody Dynamic Systems With Uncertainties. Part II: Numerical Applications", provides practical illustrations of the proposed methodology on a series of test cases. The overarching conclusion is that, notwithstanding its limitations, the polynomial chaos approach constitutes a potent tool for the simulation of multibody systems under uncertain conditions.

It is widely believed that in order to solve large scale global optimization problems an appropriate mixture of local approximation and global exploration is necessary. Local approximation, if first order information on the objective function is available, is efficiently performed by means of local optimization methods. Unfortunately, global exploration, in absence of some kind of global information on the problem, is a “blind” procedure, aimed at placing observations as evenly as possible in the search domain. Often this procedure reduces to uniform random sampling (like in Multistart algorithms, or in techniques based on clustering). In this paper we propose a new framework for global exploration which tries to guide random exploration towards the region of attraction of low-level local optima. The main idea originated by the use of smoothing techniques (based on gaussian convolutions): the possibility of applying a smoothing transformation not to the objective function but to the result of local searches seems to have never been explored yet. Although an exact smoothing of the results of local searches is impossible to implement, in this paper we propose a computational approximation scheme which has proven to be very efficient and (maybe more important) extremely robust in solving large scale global optimization problems with huge numbers of local optima. ∗Email: b.addis@ing.unifi.it Dip. Sistemi e Informatica Università di Firenze †Email: locatell@di.unito.it Dip. Informatica Università di Torino ‡Email: schoen@ing.unifi.it Dip. Sistemi e Informatica Università di Firenze

In practical applications of optimization it is common to have several conflicting objective functions to optimize. Frequently, these functions are subject to noise or can be of black-box type, preventing the use of derivative-based techniques. We propose a novel multiobjective derivative-free methodology, calling it direct multisearch (DMS), which does not aggregate any of the objective functions. Our framework is inspired by the search/poll paradigm of direct-search methods of directional type and uses the concept of Pareto dominance to maintain a list of nondominated points (from which the new iterates or poll centers are chosen). The aim of our method is to generate as many points in the Pareto front as possible from the polling procedure itself, while keeping the whole framework general enough to accommodate other disseminating strategies, in particular when using the (here also) optional search step. DMS generalizes to multiobjective optimization (MOO) all direct-search methods of directional type. We prove under the common assumptions used in direct search for single optimization that at least one limit point of the sequence of iterates generated by DMS lies in (a stationary form of) the Pareto front. However, extensive computational experience has shown that our methodology has an impressive capability of generating the whole Pareto front, even without using a search step. Two by-products of this paper are (i) the development of a collection of test problems for MOO and (ii) the extension of performance and data profiles to MOO, allowing a comparison of several solvers on a large set of test problems, in terms of their efficiency and robustness to determine Pareto fronts.

Gaussian processes allow for flexible specification of prior assumptions of unknown dynamics in state space models. We present a procedure for efficient Bayesian learning in Gaussian process state space models, where the representation is formed by projecting the problem onto a set of approximate eigenfunctions derived from the prior covariance structure. Learning under this family of models can be conducted using a carefully crafted particle MCMC algorithm. This scheme is computationally efficient and yet allows for a fully Bayesian treatment of the problem. Compared to conventional system identification tools or existing learning methods, we show competitive performance and reliable quantification of uncertainties in the model.

This paper presents an algorithm to estimate simultaneously both mean and variance of a non parametric regression problem. The key point is that we are able to estimate variance locally unlike standard Gaussian Process regression or SVMs. This means that our estimator adapts to the local noise. The problem is cast in the setting of maximum a posteriori estimation in exponential families. Unlike previous work, we obtain a convex optimization problem which can be solved via Newton's method.

In this paper we explore acceleration techniques for large scale nonconvex optimization problems with special focuses on deep neural networks. The extrapolation scheme is a classical approach for accelerating stochastic gradient descent for convex optimization, but it does not work well for nonconvex optimization typically. Alternatively, we propose an interpolation scheme to accelerate nonconvex optimization and call the method Interpolatron. We explain motivation behind Interpolatron and conduct a thorough empirical analysis. Empirical results on DNNs of great depths (e.g., 98-layer ResNet and 200-layer ResNet) on CIFAR-10 and ImageNet show that Interpolatron can converge much faster than the stateof-the-art methods such as the SGD with momentum and Adam. Furthermore, Anderson’s acceleration, in which mixing coefficients are computed by least-squares estimation, can also be used to improve the performance. Both Interpolatron and Anderson’s acceleration are easy to implement and tune. We also show that Interpolatron has linear convergence rate under certain regularity assumptions.

We introduce a new regression framework, Gaussian process regression networks (GPRN), which combines the structural properties of Bayesian neural networks with the nonparametric flexibility of Gaussian processes. GPRN accommodates input (predictor) dependent signal and noise correlations between multiple output (response) variables, input dependent length-scales and amplitudes, and heavy-tailed predictive distributions. We derive both elliptical slice sampling and variational Bayes inference procedures for GPRN. We apply GPRN as a multiple output regression and multivariate volatility model, demonstrating substantially improved performance over eight popular multiple output (multi-task) Gaussian process models and three multivariate volatility models on real datasets, including a 1000 dimensional gene expression dataset.

BayesOpt is a library with state-of-the-art Bayesian optimization methods to solve nonlinear optimization, stochastic bandits or sequential experimental design problems. Bayesian optimization is sample efficient by building a posterior distribution to capture the evidence and prior knowledge for the target function. Built in standard C++, the library is extremely efficient while being portable and flexible. It includes a common interface for C, C++, Python, Matlab and Octave.

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems.

HOMPACK90 is a Fortran 90 version of the Fortran 77 package HOMPACK (Algorithm 652), a collection of codes for finding zeros or fixed points of nonlinear systems using globally convergent probability-one homotopy algorithms. Three qualitatively different algorithms— ordinary differential equation based, normal flow, quasi-Newton augmented Jacobian matrix—are provided for tracking homotopy zero curves, as well as separate routines for dense and sparse Jacobian matrices. A high level driver for the special case of polynomial systems is also provided. Changes to HOMPACK include numerous minor improvements, simpler and more elegant interfaces, use of modules, new end games, support for several sparse matrix data structures, and new iterative algorithms for large sparse Jacobian matrices.

CANDECOMP/PARAFAC (CP) has found numerous applications in wide variety of areas such as in chemometrics, telecommunication, data mining, neuroscience, separated representations. For an order- N tensor, most CP algorithms can be computationally demanding due to computation of gradients which are related to products between tensor unfoldings and Khatri-Rao products of all factor matrices except one. These products have the largest workload in most CP algorithms. In this paper, we propose a fast method to deal with this issue. The method also reduces the extra memory requirements of CP algorithms. As a result, we can accelerate the standard alternating CP algorithms 20-30 times for order-5 and order-6 tensors, and even higher ratios can be obtained for higher order tensors (e.g., N ≥ 10). The proposed method is more efficient than the state-of-the-art ALS algorithm which operates two modes at a time (ALSo2) in the Eigenvector PLS toolbox, especially for tensors with order N ≥ 5 and high rank.


Question: A least-squares collocation method using polynomial chaos expansion.

This paper delves into polynomial interpolation within a high-dimensional cube, focusing on scenarios where the dimensionality, d, is significantly large. We propose an innovative approach, utilizing the least squares solution at sparse grids, specifically employing the extrema of Chebyshev polynomials as interpolation nodes. This method demonstrates near-optimal polynomial exactness, making it a highly efficient technique. Our analysis reveals that the proposed method exhibits a level of universality, meaning it performs nearly optimally across a wide range of function spaces. To substantiate these theoretical findings, we have conducted numerical experiments for a 10-dimensional case, employing up to 652,065 interpolation points, which further validates the efficacy of our method.

Blind identification of the feedback polynomial in a linear scrambler is an important component in the wireless communication systems which have the function of automatic standard recognition and parameter estimation. The following research contains three parts. In the first part, promotion model is deduced based on Cluzeau’s model. Comparative analysis shows the advantage of computational complexity by use of the promotion model. In the second part, the improved method of estimation for the normalised standard deviation is proposed, which plays an important role for the computation time and precision. In the final part, simulation results explain the better practical application of my algorithm compared with Cluzeau’s algorithm and XiaoBei-Liu’s algorithm.

This paper studies an adaptive tracking control for a class of nonlinear stochastic systems with unknown functions. The considered systems are in the nonaffine pure-feedback form, and it is the first to control this class of systems with stochastic disturbances. The fuzzy-neural networks are used to approximate unknown functions. Based on the backstepping design technique, the controllers and the adaptation laws are obtained. Compared to most of the existing stochastic systems, the proposed control algorithm has fewer adjustable parameters and thus, it can reduce online computation load. By using Lyapunov analysis, it is proven that all the signals of the closed-loop system are semiglobally uniformly ultimately bounded in probability and the system output tracks the reference signal to a bounded compact set. The simulation example is given to illustrate the effectiveness of the proposed control algorithm.

The Arbitrary Lagrangian-Eulerian (ALE) method is used in a variety of engineering and scientific applications for enabling multi-physics simulations. Unfortunately, the ALE method can suffer from simulation failures that require users to adjust parameters iteratively in order to complete a simulation. In this paper, we present a supervised learning framework for predicting conditions leading to simulation failures. To our knowledge, this is the first time machine learning has been applied to ALE simulations. We propose a novel learning representation for mapping the ALE domain onto a supervised learning formulation. We analyze the predictability of these failures and evaluate our framework using well-known test problems.

Sometimes we need to solve the linear equation Ax = b for a very big and very sparse A. For example, the Poisson equation where only 5 entries of each row of the matrix A are non-zero. Standard methods such as inverting the matrix A (numerically unstable) or Guass elimination do not take advantage of the sparsity of A. In this lesson we will discuss two iterative methods suitable for sparse linear systems: Jacobi and Gauss-Seidel.

In this paper, we treat nonlinear active noise control (NANC) with a linear secondary path (LSP) and with a nonlinear secondary path (NSP) in a unified structure by introducing a new virtual secondary path filter concept and using a general function expansion nonlinear filter. We discover that using the filtered-error structure results in greatly reducing the computational complexity of NANC. As a result, we extend the available filtered-error-based algorithms to solve NANC/LSP problems and, furthermore, develop our adjoint filtered-error-based algorithms for NANC/NSP. This family of algorithms is computationally efficient and possesses a simple structure. We also find that the computational complexity of NANC/NSP can be reduced even more using block-oriented nonlinear models, such as the Wiener, Hammerstein, or linear-nonlinear-linear (LNL) models for the NSP. Finally, we use the statistical properties of the virtual secondary path and the robustness of our proposed methods to further reduce the computational complexity and simplify the implementation structure of NANC/NSP when the NSP satisfies certain conditions. Computational complexity and simulation results are given to confirm the efficiency and effectiveness of all of our proposed methods

Managing and hedging the risks associated with Variable Annuity (VA) products requires intraday valuation of key risk metrics for these products. The complex structure of VA products and computational complexity of their accurate evaluation has compelled insurance companies to adopt Monte Carlo (MC) simulations to value their large portfolios of VA products. Because the MC simulations are computationally demanding, especially for intraday valuations, insurance companies need more efficient valuation techniques. Recently, a framework based on traditional spatial interpolation techniques has been proposed that can significantly decrease the computational complexity of MC simulation (Gan and Lin, 2015). However, traditional interpolation techniques require the definition of a distance function that can significantly impact their accuracy. Moreover, none of the traditional spatial interpolation techniques provide all of the key properties of accuracy, efficiency, and granularity (Hejazi et al., 2015). In this paper, we present a neural network approach for the spatial interpolation framework that affords an efficient way to find an effective distance function. The proposed approach is accurate, efficient, and provides an accurate granular view of the input portfolio. Our numerical experiments illustrate the superiority of the performance of the proposed neural network approach compared to the traditional spatial interpolation schemes.

We define distances between geometric curves by the square root of the minimal energy required to transform one curve into the other. The energy is formally defined from a left invariant Riemannian distance on an infinite dimensional group acting on the curves, which can be explicitly computed. The obtained distance boils down to a variational problem for which an optimal matching between the curves has to be computed. An analysis of the distance when the curves are polygonal leads to a numerical procedure for the solution of the variational problem, which can efficiently be implemented, as illustrated by experiments.

The most successful approaches for filtering Monte Carlo noise use feature-based filters (e.g., cross-bilateral and cross non-local means filters) that exploit additional scene features such as world positions and shading normals. However, their main challenge is finding the optimal weights for each feature in the filter to reduce noise but preserve scene detail. In this paper, we observe there is a complex relationship between the noisy scene data and the ideal filter parameters, and propose to learn this relationship using a nonlinear regression model. To do this, we use a multilayer perceptron neural network and combine it with a matching filter during both training and testing. To use our framework, we first train it in an offline process on a set of noisy images of scenes with a variety of distributed effects. Then at run-time, the trained network can be used to drive the filter parameters for new scenes to produce filtered images that approximate the ground truth. We demonstrate that our trained network can generate filtered images in only a few seconds that are superior to previous approaches on a wide range of distributed effects such as depth of field, motion blur, area lighting, glossy reflections, and global illumination.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features method gains much popularity due to its simplicity and direct reduction of nonlinear problem to a linear one. Different random feature functions have since been proposed to approximate a variety of kernel functions. Among them the Random Binning (RB) feature, proposed in the first random-feature paper [21], has drawn much less attention than the Random Fourier (RF) feature proposed also in [21]. In this work, we observe that the RB features, with right choice of optimization solver, could be orders-of-magnitude more efficient than other random features and kernel approximation methods under the same requirement of accuracy. We thus propose the first analysis of RB from the perspective of optimization, which by interpreting RB as a Randomized Block Coordinate Descent in the infinite-dimensional space, gives a faster convergence rate compared to that of other random features. In particular, we show that by drawing R random grids with at least κ number of non-empty bins per grid in expectation, RB method achieves a convergence rate of O(1/κ R)), which not only sharpens its O(1/√R) rate from Monte Carlo analysis, but also shows a κ times speedup over other random features under the same analysis framework. In addition, we demonstrate another advantage of RB in the L1-regularized setting, where unlike other random features, a RB-based Coordinate Descent solver can be parallelized with guaranteed speedup proportional to κ. Our extensive experiments demonstrate the superior performance of the RB features over other random features and kernel approximation methods.

Kernel approximation via nonlinear random feature maps is w idely used in speeding up kernel machines. There are two main challenges for the conventional kernel ap proximation methods. First, before performing kernel approximation, a good kernel has to be chosen. Pickin g a good kernel is a very challenging problem in itself. Second, high-dimensional maps are often required i n order to achieve good performance. This leads to high computational cost in both generating the nonlinear ma ps, nd in the subsequent learning and prediction process. In this work, we propose to optimize the nonlinear m ps directly with respect to the classification objective in a data-dependent fashion. The proposed approa ch achieves kernel approximation and kernel learning in a joint framework. This leads to much more compact maps wit hout hurting the performance. As a by-product, the same framework can also be used to achieve more compact ke rnel maps to approximate a known kernel. We also introduce Circulant Nonlinear Maps, which uses a circu lant-structured projection matrix to speed up the nonlinear maps for high-dimensional data.


Question: Recognition of human characters through handwriting analysis using fuzzy logic methods.

This paper explores the feasibility of identifying personality traits, with a particular focus on those that may lead to violent behavior, through automated handwriting analysis. The practice of handwriting analysis has a long history, yet its efficacy in assessing an individual's behavior and personality remains a topic of debate. Can we ascertain an individual's potential for deviant behavior and personality traits by analyzing their handwriting? Two primary approaches to handwriting analysis exist: graphology, which is centered on psychological assessment, and forensic document examination (also known as handwriting identification), which involves the comparison of documents and writing samples to a known source. In this research, we have reviewed the latest technologies employed in analyzing behavior based on handwriting and assessed their effectiveness in predicting character and personality. Additionally, we aimed to determine whether handedness, authorship, and gender can be discerned through analysis. Our study not only delved into the Lewinson-Zubin method for handwriting analysis but also explored various online tools designed for this purpose, including NEURO SCRIPT, WANDA, CEDAR-FOX, and Gaussian Mixture Model, to evaluate their accuracy and reliability in character recognition.

The ability of communicating easily with everyone is a blessing people with hearing impairment do not have. They completely rely on their vision around healthy individuals to difficultly read lips. This paper proposes a solution for this problem, ISWYS (I See What You Say) is a research-oriented speech recognition system for the Arabic language that interprets lips movements into readable text. It is accomplished by analyzing a video of lips movements that resemble utterances, and then converting it to readable characters using video analysis and motion estimation. Our algorithm involves dividing the video into n number of frames to generate n-1 image frame which is produced by taking the difference between consecutive frames. Then, video features are extracted to be used by our error function which provided recognition of approximately 70%.

In this paper, a large vocabulary sign language interpreter is presented with real-time continuous gesture recognition of sign language using a DataGlove . The most critical problem, end-point detection in a stream of gesture input is first solved and then statistical analysis is done according to 4 parameters in a gesture : posture, position, orientation, and motion. We have implemented a prototype system with a lexicon of 250 vocabularies in Taiwanese Sign Language (TWL). This system uses hidden Markov models (HMMs) for 51 fundamental postures, 6 orientations, and 8 motion primitives. In a signerdependent way, a sentence of gestures based on these vocabularies can be continuously recognized in real-time and the average recognition rate is 80.4%.

We introduce a (semi-)automatic OCR post-processing system that utilizes web-scale linguistic corpora in providing high-quality correction. This paper is a comprehensive system overview with the focus on the computational procedures, applied linguistic analysis, and processing optimization.

This study proposes a modified convolutional neural network (CNN) algorithm that is based on dropout and the stochastic gradient descent (SGD) optimizer (MCNN-DS), after analyzing the problems of CNNs in extracting the convolution features, to improve the feature recognition rate and reduce the time-cost of CNNs. The MCNN-DS has a quadratic CNN structure and adopts the rectified linear unit as the activation function to avoid the gradient problem and accelerate convergence. To address the overfitting problem, the algorithm uses an SGD optimizer, which is implemented by inserting a dropout layer into the all-connected and output layers, to minimize cross entropy. This study used the datasets MNIST, HCL2000, and EnglishHand as the benchmark data, analyzed the performance of the SGD optimizer under different learning parameters, and found that the proposed algorithm exhibited good recognition performance when the learning rate was set to [0.05, 0.07]. The performances of WCNN, MLP-CNN, SVM-ELM, and MCNN-DS were compared. Statistical results showed the following: (1) For the benchmark MNIST, the MCNN-DS exhibited a high recognition rate of 99.97%, and the time-cost of the proposed algorithm was merely 21.95% of MLP-CNN, and 10.02% of SVM-ELM; (2) Compared with SVM-ELM, the average improvement in the recognition rate of MCNN-DS was 2.35% for the benchmark HCL2000, and the time-cost of MCNN-DS was only 15.41%; (3) For the EnglishHand test set, the lowest recognition rate of the algorithm was 84.93%, the highest recognition rate was 95.29%, and the average recognition rate was 89.77%.

This paper describes a new method to segment printed mathematical documents precisely and extract formulas automatically from their images. Unlike classical methods, it is more directed towards segmentation rather than recognition, isolating mathematical formulas outside and inside text-lines. Our ultimate goal is to delimit parts of text that could disturb OCR applications, not yet trained for formula recognition and restructuring. The method is based on a global and a local segmentation. The global segmentation separates isolated formulas from the text lines using a primary labeling. The local segmentation propagates the context around the mathematical operators met to discard embedded formulas from plain text. The primary labeling identifies some mathematical symbols by models created at a learning step using fuzzy logic. The secondary labeling reinforces the results of the primary labeling and locates the subscripts and the superscripts inside the text. A heuristic has been defined that guides this automatic process. In this paper, the different modules making up the automated segmentation of mathematical document system are presented with examples of results. Experiments carried out on some commonly seen mathematical documents show that our proposed method can achieve quite satisfactory rates, making mathematical formula extraction more feasible for real-world applications. The average rate of primary labeling of mathematical operators is about 95.3% and their secondary labeling can improve the rate by about 4%. The formula extraction rate, evaluated with 300 formulas and 100 mathematical documents having variable complexity, is close to 93%

The perceptual computer (Per-C) is an architecture that makes subjective judgments by computing with words (CWWs). This paper applies the Per-C to hierarchical decision making, which means decision making based on comparing the performance of competing alternatives, where each alternative is first evaluated based on hierarchical criteria and subcriteria, and then, these alternatives are compared to arrive at either a single winner or a subset of winners. What can make this challenging is that the inputs to the subcriteria and criteria can be numbers, intervals, type-1 fuzzy sets, or even words modeled by interval type-2 fuzzy sets. Novel weighted averages are proposed in this paper as a CWW engine in the Per-C to aggregate these diverse inputs. A missile-evaluation problem is used to illustrate it. The main advantages of our approaches are that diverse inputs can be aggregated, and uncertainties associated with these inputs can be preserved and are propagated into the final evaluation.

There is a large collection of Handwritten English paper documents of Historical and Scientific importance. But paper documents are not recognised directly by computer. Hence the closest way of indexing these documents is by storing their document digital image. Hence a large database of document images can replace the paper documents. But the document and data corresponding to each image cannot be directly recognised by the computer. This paper applies the technique of word spotting using Modified Character Shape Code to Handwritten English document images for quick and efficient query search of words on a database of document images. It is different from other Word Spotting techniques as it implements two level of selection for word segments to match search query. First based on word size and then based on character shape code of query. It makes the process faster and more efficient and reduces the need of multiple pre-processing.

In last decade lot of efforts had been made by research community to create sign language recognition system which provide a medium of communication for differently-abled people and their machine translations help others having trouble in understanding such sign languages. Computer vision and machine learning can be collectively applied to create such systems. In this paper, we present a sign language recognition system which makes use of depth images that were captured using a Microsoft Kinect® camera. Using computer vision algorithms, we develop a characteristic depth and motion profile for each sign language gesture. The feature matrix thus generated was trained using a multi-class SVM classifier and the final results were compared with existing techniques. The dataset used is of sign language gestures for the digits 0-9.

In this paper, we demonstrate, through the multicategory classification of battlefield ground vehicles using acoustic features, how it is straightforward to directly exploit the information inherent in a problem to determine the number of rules, and subsequently the architecture, of fuzzy logic rule-based classifiers (FLRBC). We propose three FLRBC architectures, one non-hierarchical and two hierarchical (HFLRBC), conduct experiments to evaluate the performances of these architectures, and compare them to a Bayesian classifier. Our experimental results show that: 1) for each classifier the performance in the adaptive mode that uses simple majority voting is much better than in the non-adaptive mode; 2) all FLRBCs perform substantially better than the Bayesian classifier; 3) interval type-2 (T2) FLRBCs perform better than their competing type-1 (T1) FLRBCs, although sometimes not by much; 4) the interval T2 nonhierarchical and HFLRBC-series architectures perform the best; and 5) all FLRBCs achieve higher than the acceptable 80% classification accuracy

We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1 % error rate and about a 9% reject rate on zipcode digits provided by the U.S. Postal Service.


Question: Human character identification through handwriting analysis utilizing fuzzy logic methods.

This paper presents an automated approach for predicting human personality through handwriting analysis using a Support Vector Machine (SVM) with a Segmentation Method. The technique aims to offer insights into the author's personality and provide a deeper understanding of the writer's characteristics. The method involves analyzing allographs and allograph combinations to identify and evaluate behavioral traits. To automate this process, the study focuses on six key features: (i) letter size, (ii) letter and word slant, (iii) baseline patterns, (iv) pen pressure, (v) inter-letter spacing, and (vi) inter-word spacing within a document. These features are extracted through segmentation from digital handwriting samples and then used to train the SVM model. The model employs an RBF kernel and was tested on 100 distinct writers, each providing multiple handwriting samples. The proposed system achieved an accuracy rate of approximately 94%, demonstrating its effectiveness in predicting the psychological personality of the writer. The system's performance was evaluated under two distinct conditions using the same sample set, showcasing the reliability and robustness of the proposed methodology.

Though designing of classifies for Indic script handwriting recognition has been researched with enough attention, use of language model has so far received little exposure. This paper attempts to develop a weighted finite-state transducer (WFST) based language model for improving the current recognition accuracy. Both the recognition hypothesis (i.e. the segmentation lattice) and the lexicon are modeled as two WFSTs. Concatenation of these two FSTs accept a valid word(s) which is (are) present in the recognition lattice. A third FST called error FST is also introduced to retrieve certain words which were missing in the previous concatenation operation. The proposed model has been tested for online Bangla handwriting recognition though the underlying principle can equally be applied for recognition of offline or printed words. Experiment on a part of ISI-Bangla handwriting database shows that while the present classifiers (without using any language model) can recognize about 73% word, use of recognition and lexicon FSTs improve this result by about 9% giving an average word-level accuracy of 82%. Introduction of error FST further improves this accuracy to 93%. This remarkable improvement in word recognition accuracy by using FST-based language model would serve as a significant revelation for the research in handwriting recognition, in general and Indic script handwriting recognition, in particular.

For the natural and social interaction it is necessary to understand human behavior. Personality is one of the fundamental aspects, by which we can understand behavioral dispositions. It is evident that there is a strong correlation between users’ personality and the way they behave on online social network (e.g., Facebook). This paper presents automatic recognition of Big-5 personality traits on social network (Facebook) using users’ status text. For the automatic recognition we studied different classification methods such as SMO (Sequential Minimal Optimization for Support Vector Machine), Bayesian Logistic Regression (BLR) and Multinomial Naïve Bayes (MNB) sparse modeling. Performance of the systems had been measured using macro-averaged precision, recall and F1; weighted average accuracy (WA) and un-weighted average accuracy (UA). Our comparative study shows that MNB performs better than BLR and SMO for personality traits recognition on the social network data.

The ability of communicating easily with everyone is a blessing people with hearing impairment do not have. They completely rely on their vision around healthy individuals to difficultly read lips. This paper proposes a solution for this problem, ISWYS (I See What You Say) is a research-oriented speech recognition system for the Arabic language that interprets lips movements into readable text. It is accomplished by analyzing a video of lips movements that resemble utterances, and then converting it to readable characters using video analysis and motion estimation. Our algorithm involves dividing the video into n number of frames to generate n-1 image frame which is produced by taking the difference between consecutive frames. Then, video features are extracted to be used by our error function which provided recognition of approximately 70%.

Biometrics identification methods proved to be very efficient, more natural and easy for users than traditional methods of human identification. In fact, only biometrics methods truly identify humans, not keys and cards they posses or passwords they should remember. The future of biometrics will surely lead to systems based on image analysis as the data acquisition is very simple and requires only cameras, scanners or sensors. More importantly such methods could be passive, which means that the user does not have to take active part in the whole process or, in fact, would not even know that the process of identification takes place. There are many possible data sources for human identification systems, but the physiological biometrics seem to have many advantages over methods based on human behaviour. The most interesting human anatomical parts for such passive, physiological biometrics systems based on images acquired from cameras are face and ear. Both of those methods contain large volume of unique features that allow to distinctively identify many users and will be surely implemented into efficient biometrics systems for many applications. The article introduces to ear biometrics and presents its advantages over face biometrics in passive human identification systems. Then the geometrical method of feature extraction from human ear images in order to perform human identification is presented.

Fingerprint verification is one of the most reliable personal identification methods. However, manual fingerprint verification is so tedious, time-consuming, and expensive that it is incapable of meeting today’s increasing performance requirements. An automatic fingerprint identification system (AFIS) is widely needed. It plays a very important role in forensic and civilian applications such as criminal identification, access control, and ATM card verification. This paper describes the design and implementation of an on-line fingerprint verification system which operates in two stages: minutia extraction and minutia matching. An improved version of the minutia extraction algorithm proposed by Ratha et al., which is much faster and more reliable, is implemented for extracting features from an input fingerprint image captured with an on-line inkless scanner. For minutia matching, an alignment-based elastic matching algorithm has been developed. This algorithm is capable of finding the correspondences between minutiae in the input image and the stored template without resorting to exhaustive search and has the ability of adaptively compensating for the nonlinear deformations and inexact pose transformations between fingerprints. The system has been tested on two sets of fingerprint images captured with inkless scanners. The verification accuracy is found to be acceptable. Typically, a complete fingerprint verification procedure takes, on an average, about eight seconds on a SPARC 20 workstation. These experimental results show that our system meets the response time requirements of on-line verification with high accuracy. Index Terms —Biometrics, fingerprints, matching, verification, minutia, orientation field, ridge extraction. —————————— ✦ ——————————

In this paper, we propose a fuzzy local search (FLS) method for intrusion detection. The FLS system is a fuzzy classifier, whose knowledge base is modeled as a fuzzy rule such as "if-then" and improved by a local search metaheuristic. The proposed method is implemented and tested on the benchmark KDD'99 intrusion dataset. The results are encouraging and demonstrate the benefits of our approach.

With the rapid expansion of computer networks during the past few years, security has become a crucial issue for modern computer systems. A good way to detect illegitimate use is through monitoring unusual user activity. Methods of intrusion detection based on hand-coded rule sets or predicting commands on-line are laborous to build or not very reliable. This paper proposes a new way of applying neural networks to detect intrusions. We believe that a user leaves a ’print’ when using the system; a neural network can be used to learn this print and identify each user much like detectives use thumbprints to place people at crime scenes. If a user’s behavior does not match his/her print, the system administrator can be alerted of a possible security breech. A backpropagation neural network called NNID (Neural Network Intrusion Detector) was trained in the identification task and tested experimentally on a system of 10 users. The system was 96% accurate in detecting unusual activity, with 7% false alarm rate. These results suggest that learning user profiles is an effective way for detecting intrusions.

Keystroke Dynamics is an important biometric solution for person authentication. Based upon keystroke dynamics, this paper designs an embedded password protection device, develops an online system, collects two public databases for promoting the research on keystroke authentication, exploits the Gabor filter bank to characterize keystroke dynamics, and provides benchmark results of three popular classification algorithms, one-class support vector machine, Gaussian classifier, and nearest neighbour classifier.

A novel minutiae-based fingerprint matching algorithm is proposed. A minutiae matching algorithm has to solve two problems: correspondence and similarity computation. For the correspondence problem, we assign each minutia two descriptors: texture-based and minutiae-based descriptors, and use an alignment-based greedy matching algorithm to establish the correspondences between minutiae. For the similarity computation, we extract a 17-D feature vector from the matching result, and convert the feature vector into a matching score using support vector classifier. The proposed algorithm is tested on FVC2002 databases and compared to all participators in FVC2002. According to equal error rate, the proposed algorithm ranks 1st on DB3, the most difficult database in FVC2002, and on the average ranks 2nd on all 4 databases. 2007 Pattern Recognition Society. Published by Elsevier Ltd. All rights reserved.

This paper presents an OCR method for degraded character recognition applied to a reference number (RN) of 15 printed characters of an invoice document produced by dot-matrix printer. First, the paper deals with the problem of the reference number localization and extraction, in which the characters tops or bottoms are or not touched with a printed reference line of the electrical bill. In case of touched RN, the extracted characters are severely degraded leading to missing parts in the characters tops or bottoms. Secondly, a combined recognition method based on the complementary similarity measure (CSM) method and MLP-based classifier is used. The CSM is used to accept or reject an incoming character. In case of acceptation, the CSM acts as a feature extractor and produces a feature vector of ten component features. The MLP is then trained using these feature vectors. The use of the CSM as a feature extractor tends to make the MLP very powerful and very well suited for rejection. Experimental results on electrical bills show the ability of the model to yield relevant and robust recognition on severely degraded printed characters.


Question: Human character identification through handwriting analysis utilizing fuzzy logic methods.

This research explores the capability of automated text categorization methods to accurately identify the gender of a document's author, a task that might seem less straightforward than classifying by subject or attributing authorship. Surprisingly, it is demonstrated that by harnessing a blend of basic lexical and syntactic characteristics, these techniques can deduce the author's gender in an unseen, formally written document with an impressive accuracy rate of around 80%. Moreover, the same methodologies can be applied to distinguish between fiction and non-fiction texts, achieving an accuracy level of nearly 98%.

**Question**: A Comparative Study of Deep Learning Models for Image Classification.

**Documents**: Comparative Study of Deep Learning Models for Image Recognition
A comprehensive comparative study of deep learning models for image recognition is presented in this paper. The study includes a detailed analysis of the performance of various deep learning architectures such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM) networks. The evaluation is carried out using standard benchmark datasets like ImageNet, CIFAR-10, and MNIST. The results show that CNNs outperform RNNs and LSTMs in most cases, with the exception of sequential image recognition tasks where LSTMs perform better. The study also highlights the importance of network depth and architecture in achieving high accuracy in image recognition tasks.

**Similar Documents**: This paper offers an exhaustive comparison of deep learning models for image recognition tasks. It meticulously examines the efficiency of diverse deep learning frameworks, including Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and Long Short-Term Memory (LSTM) networks. The assessment is performed using established benchmark datasets, such as ImageNet, CIFAR-10, and MNIST. The findings indicate that CNNs generally surpass RNNs and LSTMs in terms of performance, except in cases involving sequential image recognition, where LSTMs exhibit superior capabilities. The analysis underscores the critical role of network depth and design in attaining high accuracy in image recognition.

Although people's handshakes are thought to reflect their personality and influence our first impressions of them, these relations have seldom been formally investigated. One hundred twelve participants had their hand shaken twice by 4 trained coders (2 men and 2 women) and completed 4 personality measures. The participants' handshakes were stable and consistent across time and coders. There were also gender differences on most of the handshaking characteristics. A firm handshake was related positively to extraversion and emotional expressiveness and negatively to shyness and neuroticism; it was also positively related to openness to experience, but only for women. Finally, handshake characteristics were related to the impressions of the participants formed by the coders. These results demonstrate that personality traits, assessed through self-report, can predict specific behaviors assessed by trained observers. The pattern of relations among openness, gender, handshaking, and first impressions suggests that a firm handshake may be an effective form of self-promotion for women.

This paper deals with the problem of author identification. The common N-grams (CNG) method [6] is a language-independent profile-based approach with good results in many author identification experiments so far. A variation of this approach is presented based on new distance measures that are quite stable for large profile length values. Special emphasis is given to the degree upon which the effectiveness of the method is affected by the available training text samples per author. Experiments based on text samples on the same topic from the Reuters Corpus Volume 1 are presented using both balanced and imbalanced training corpora. The results show that CNG with the proposed distance measures is more accurate when only limited training text samples are available, at least for some of the candidate authors, a realistic condition in author identification problems.

Biometrics is rapidly gaining acceptance as the technology that can meet the ever increasing need for security in critical applications. Biometric systems automatically recognize individuals based on their physiological and behavioral characteristics. Hence, the fundamental requirement of any biometric recognition system is a human trait having several desirable properties like universality, distinctiveness, permanence, collectability, acceptability, and resistance to circumvention. However, a human characteristic that possesses all these properties has not yet been identified. As a result, none of the existing biometric systems provide perfect recognition and there is a scope for improving the performance of these systems. Although characteristics like gender, ethnicity, age, height, weight and eye color are not unique and reliable, they provide some information about the user. We refer to these characteristics as “soft” biometric traits and argue that these traits can complement the identity information provided by the primary biometric identifiers like fingerprint and face. This paper presents the motivation for utilizing soft biometric information and analyzes how the soft biometric traits can be automatically extracted and incorporated in the decision making process of the primary biometric system. Preliminary experiments were conducted on a fingerprint database of 160 users by synthetically generating soft biometric traits like gender, ethnicity, and height based on known statistics. The results show that the use of additional soft biometric user information significantly improves ( ≈ 6%) the recognition performance of the fingerprint biometric system.

Granting banking facility is one of the most important parts of the financial supplies for each bank. So this activity becomes more valuable economically and always has a degree of risk. These days several various developed Artificial Intelligent systems like Neural Network, Decision Tree, Logistic Regression Analysis, Linear Discriminant Analysis and etc, are used in the field of granting facilities that each of this system owns its advantages and disadvantages. But still studying and working are needed to improve the accuracy and performance of them. In this article among other AI methods, fuzzy expert system is selected. This system is based on data and also extracts rules by using data. Therefore the dependency to experts is omitted and interpretability of rules is obtained. Validity of these rules could be confirmed or rejected by banking affair experts. For investigating the performance of proposed system, this system and some other methods were performed on various datasets. Results show that the proposed algorithm obtained better performance among the others.

This article summarizes the major developments in the history of efforts to use fingerprint patterns to identify individuals, from the earliest fingerprint classification systems of Vucetich and Henry in the 1890s through the advent of automated fingerprint identification. By chronicling the history of “manual” systems for recording storing, matching, and retrieving fingerprints, the article puts advances in automatic fingerprint recognition in historical context and highlights their historical and social significance.

Fingerprint verification is one of the most reliable personal identification methods. However, manual fingerprint verification is so tedious, time-consuming, and expensive that it is incapable of meeting today’s increasing performance requirements. An automatic fingerprint identification system (AFIS) is widely needed. It plays a very important role in forensic and civilian applications such as criminal identification, access control, and ATM card verification. This paper describes the design and implementation of an on-line fingerprint verification system which operates in two stages: minutia extraction and minutia matching. An improved version of the minutia extraction algorithm proposed by Ratha et al., which is much faster and more reliable, is implemented for extracting features from an input fingerprint image captured with an on-line inkless scanner. For minutia matching, an alignment-based elastic matching algorithm has been developed. This algorithm is capable of finding the correspondences between minutiae in the input image and the stored template without resorting to exhaustive search and has the ability of adaptively compensating for the nonlinear deformations and inexact pose transformations between fingerprints. The system has been tested on two sets of fingerprint images captured with inkless scanners. The verification accuracy is found to be acceptable. Typically, a complete fingerprint verification procedure takes, on an average, about eight seconds on a SPARC 20 workstation. These experimental results show that our system meets the response time requirements of on-line verification with high accuracy. Index Terms —Biometrics, fingerprints, matching, verification, minutia, orientation field, ridge extraction. —————————— ✦ ——————————

The task of gesture recognition is highly challenging due to complex background, presence of nongesture hand motions, and different illumination environments. In this paper, a new technique is proposed which begins by detecting the hand and determining its canter, tracking the hands trajectory and analyzing the variations in the hand locations, and finally recognizing the gesture. The proposed technique overcomes background complexity and distance from camera which is up to 2.5 meters. Experimental results show that the proposed technique can recognize 12 gestures with rate above 94%. Keywords— Hand gesture recognition, skin detection, Hand tracking.

The ability to recognize emotions is an important part of building intelligent computers. Emotionally-aware systems would have a rich context from which to make appropriate decisions about how to interact with the user or adapt their system response. There are two main problems with current system approaches for identifying emotions that limit their applicability: they can be invasive and can require costly equipment. Our solution is to determine user emotion by analyzing the rhythm of their typing patterns on a standard keyboard. We conducted a field study where we collected participants' keystrokes and their emotional states via self-reports. From this data, we extracted keystroke features, and created classifiers for 15 emotional states. Our top results include 2-level classifiers for confidence, hesitance, nervousness, relaxation, sadness, and tiredness with accuracies ranging from 77 to 88%. In addition, we show promise for anger and excitement, with accuracies of 84%.

In this paper, we propose a fuzzy local search (FLS) method for intrusion detection. The FLS system is a fuzzy classifier, whose knowledge base is modeled as a fuzzy rule such as "if-then" and improved by a local search metaheuristic. The proposed method is implemented and tested on the benchmark KDD'99 intrusion dataset. The results are encouraging and demonstrate the benefits of our approach.

A new classification approach for human body postures based on a neural fuzzy network is proposed in this paper, and the approach is applied to detect emergencies that are caused by accidental falls. Four main body postures are used for posture classification, including standing, bending, sitting, and lying. After the human body is segmented from the background, the classification features are extracted from the silhouette. The body silhouette is projected onto horizontal and vertical axes, and then, a discrete Fourier transform is applied to each projected histogram. Magnitudes of significant Fourier transform coefficients together with the silhouette length-width ratio are used as features. The classifier is designed by a neural fuzzy network. The four postures can be classified with high accuracy according to experimental results. Classification results are also applicable to home care emergency detection of a person who suddenly falls and remains in the lying posture for a period of time due to experiments that were performed.