Question: Phase-functioned neural networks for animating character control.

Introducing Human3.6M: A Comprehensive Dataset for 3D Human Sensing and Pose Estimation in Real-World Settings
This paper presents Human3.6M, a groundbreaking dataset of over 3.6 million precise 3D human poses, specifically designed for the development and evaluation of advanced human sensing systems and next-generation pose estimation algorithms. The dataset is significantly larger than any existing resource, featuring performances by 11 subjects (5 females and 6 males) recorded from four distinct angles. Its scope is broadened by capturing a wide array of motions and postures associated with everyday activities, such as taking photos, conversing on the phone, posing, greeting, and eating. The dataset is further enriched with synchronized video, motion capture, and time-of-flight (depth) data, as well as detailed 3D body scans of all participating individuals. The authors also offer controlled mixed reality scenarios, where 3D human models animated with motion capture data are seamlessly integrated into complex real environments, accounting for camera movement and occlusion. Accompanying the dataset are large-scale statistical models and evaluation baselines that showcase the dataset's versatility and potential for enhancing research outcomes. Experiments demonstrate that the best large-scale model, when trained on the full dataset, achieves a 20% performance gain over models trained on smaller datasets. The substantial room for improvement with more sophisticated models suggests a fertile ground for future investigation. Access to the dataset, including the learning models, features, visualization tools, and an evaluation server, is provided online at http://vision.imar.ro/human3.6m, facilitating collaborative and innovative research in the field of 3D human sensing.

A three-layered neural network is described for transforming two-dimensional discrete signals into generalized nonorthogonal 2-D “Gabor” representations for image analysis, segmentation, and compression. These transforms are conjoint spat iahpectral representations [lo], [15], which provide a complete image description in terms of locally windowed 2-D spectral coordinates embedded within global 2-D spatial coordinates. Because intrinsic redundancies within images a re extracted, the resulting image codes can be very compact. However, these conjoint transforms are inherently difficult to compute because t e elementary expansion functions a re not orthogonal. One o r t h o g o n k i n g approach developed for 1-D signals by Bastiaans [SI, based on biorthonormal expansions, is restricted by constraints on the conjoint sampling rates and invariance of the windowing function, as well as by the fact that the auxiliary orthogonalizing functions are nonlocal infinite series. In the present “neural network” approach, based upon interlaminar interactions involving two layers with fixed weights and one layer with adjustable weights, the network finds coefficients for complete conjoint 2-D Gabor transforms without these restrictive conditions. For arbitrary noncomplete transforms, in which the coefficients might be interpreted simply as signifying the presence of certain features in the image, the network finds optimal coefficients in the sense of minimal mean-squared-error in representing the image. I n one algebraically complete scheme permitting exact reconstruction, the network finds expansion coefficients that reduce entropy from 7.57 in the pixel representation to 2.55 in the complete 2-D Gabor transform. In “wavelet” expansions based on a biologically inspired log-polar ensemble of dilations, rotations, and translations of a single underlying 2-D Gabor wavelet template, image compression is illustrated with ratios up to 20: 1. Also demonstrated is image segmentation based on the clustering of coefficients in the complete 2-D Gabor transform. This coefficient-finding network for implementing useful nonorthogonal image transforms may also have neuroscientific relevance, because the network layers with fixed weights use empirical 2-D receptive field profiles obtained from orientation-selective neurons in cat visual cortex as the weighting functions, and the resulting transform mimics the biological visual strategy of embedding angular and spectral analysis within global spatial coordinates.

ÐA new method for the representation, recognition, and interpretation of parameterized gesture is presented. By parameterized gesture we mean gestures that exhibit a systematic spatial variation; one example is a point gesture where the relevant parameter is the two-dimensional direction. Our approach is to extend the standard hidden Markov model method of gesture recognition by including a global parametric variation in the output probabilities of the HMM states. Using a linear model of dependence, we formulate an expectation-maximization (EM) method for training the parametric HMM. During testing, a similar EM algorithm simultaneously maximizes the output likelihood of the PHMM for the given sequence and estimates the quantifying parameters. Using visually derived and directly measured three-dimensional hand position measurements as input, we present results that demonstrate the recognition superiority of the PHMM over standard HMM techniques, as well as greater robustness in parameter estimation with respect to noise in the input features. Last, we extend the PHMM to handle arbitrary smooth (nonlinear) dependencies. The nonlinear formulation requires the use of a generalized expectation-maximization (GEM) algorithm for both training and the simultaneous recognition of the gesture and estimation of the value of the parameter. We present results on a pointing gesture, where the nonlinear approach permits the natural spherical coordinate parameterization of pointing direction. Index TermsÐGesture recognition, hidden Markov models, expectation-maximization algorithm, time-series modeling, computer vision.

In this paper, we outline the approach we have developed to construct an emotion-recognising system. It is based on guidance from psychological studies of emotion, as well as from the nature of emotion in its interaction with attention. A neural network architecture is constructed to be able to handle the fusion of different modalities (facial features, prosody and lexical content in speech). Results from the network are given and their implications discussed, as are implications for future direction for the research.

We introduce an educational toy, called curlybot, as the basis for a new class of toys aimed at children in their early stages of development — ages four and up. curlybot is an autonomous two-wheeled vehicle with embedded electronics that can record how it has been moved on any flat surface and then play back that motion accurately and repeatedly. Children can use curlybot to develop intuitions for advanced mathematical and computational concepts, like differential geometry, through play away from a traditional computer.
In our preliminary studies, we found that children learn to use curlybot quickly. They readily establish an affective and body syntonic connection with curlybot, because of its ability to remember all of the intricacies of their original gesture; every pause, acceleration, and even the shaking in their hand is recorded. Programming by example in this context makes the educational ideas implicit in the design of curlybot accessible to young children.

The facial performance of an individual is inherently rich in subtle deformation and timing details. Although these subtleties make the performance realistic and compelling, they often elude both motion capture and hand animation. We present a technique for adding fine-scale details and expressiveness to low-resolution art-directed facial performances, such as those created manually using a rig, via marker-based capture, by fitting a morphable model to a video, or through Kinect reconstruction using recent faceshift technology. We employ a high-resolution facial performance capture system to acquire a representative performance of an individual in which he or she explores the full range of facial expressiveness. From the captured data, our system extracts an expressiveness model that encodes subtle spatial and temporal deformation details specific to that particular individual. Once this model has been built, these details can be transferred to low-resolution art-directed performances. We demonstrate results on various forms of input; after our enhancement, the resulting animations exhibit the same nuances and fine spatial details as the captured performance, with optional temporal enhancement to match the dynamics of the actor. Finally, we show that our technique outperforms the current state-of-the-art in example-based facial animation.

Feedforward neural networks trained by error backpropagation are examples of nonparametric regression estimators. We present a tutorial on nonparametric inference and its relation to neural networks, and we use the statistical viewpoint to highlight strengths and weaknesses of neural models. We illustrate the main points with some recognition experiments involving artificial data as well as handwritten numerals. In way of conclusion, we suggest that current-generation feedforward neural networks are largely inadequate for difficult problems in machine perception and machine learning, regardless of parallel-versus-serial hardware or other implementation issues. Furthermore, we suggest that the fundamental challenges in neural modeling are about representation rather than learning per se. This last point is supported by additional experiments with handwritten numerals.

The recognition of a character begins with analyzing its form and extracting the features that will be exploited for the identification. Primitives can be described as a tool to distinguish an object of one class from another object of another class. It is necessary to define the significant primitives. The size of vector primitives can be large if a large number of primitives are extracted including redundant and irrelevant features. As a result, the performance of the recognition system becomes poor, and as the number of features increases, so does the computing time. Feature selection, therefore, is required to ensure the selection of a subset of features that gives accurate recognition. In our work we propose a feature selection approach based genetic algorithm to improve the discrimination capacity of the Multilayer Perceptron Neural Networks (MLP).

Current robotics have failed to replicate the great dexterity of the animal’s motions. We believe that it is because most of the works have focused on the controller part rather than on its body structure. Indeed, the musculoskeletal system is essential to produce the animal’s motions and several researchers have attempted to exploit it [1][2][3]. However, most of them have used simple mono-articular muscles and torque control which cannot replicate the nonlinearity of the living tissues, muscles and nerves [4]. In contrast, bipedal robots with monoand bi-articular muscles for performing adaptive motions [5][6][7] have been produced. To further investigate the role of the musculoskeletal system for animal motions, we desgned a quadruped robot with monoand bi-articular muscles. We used artificial pneumatic muscles as actuators due to their high mass/power ratio and their high compliance. Although these actuators replicate several characteristics of biological muscles in terms of damping and elasticity, the length-tension relation differs from them [8][9]. In the result, the angle-torque relation of the robot with pneumatic muscles differed substantially from the one of animals. Thus, we tried to bring the angle-torque relation as close as the one of living tissues by devising the angle-moment arm relation.

We introduce AutoHair, the first fully automatic method for 3D hair modeling from a single portrait image, with no user interaction or parameter tuning. Our method efficiently generates complete and high-quality hair geometries, which are comparable to those generated by the state-of-the-art methods, where user interaction is required. The core components of our method are: a novel hierarchical deep neural network for automatic hair segmentation and hair growth direction estimation, trained over an annotated hair image database; and an efficient and automatic data-driven hair matching and modeling algorithm, based on a large set of 3D hair exemplars. We demonstrate the efficacy and robustness of our method on Internet photos, resulting in a database of around 50K 3D hair models and a corresponding hairstyle space that covers a wide variety of real-world hairstyles. We also show novel applications enabled by our method, including 3D hairstyle space navigation and hair-aware image retrieval.

Computer animated agents and robots bring a social dimension to human computer interaction and force us to think in new ways about how computers could be used in daily life. Face to face communication is a real-time process operating at a a time scale in the order of 40 milliseconds. The level of uncertainty at this time scale is considerable, making it necessary for humans and machines to rely on sensory rich perceptual primitives rather than slow symbolic inference processes. In this paper we present progress on one such perceptual primitive. The system automatically detects frontal faces in the video stream and codes them with respect to 7 dimensions in real time: neutral, anger, disgust, fear, joy, sadness, surprise. The face finder employs a cascade of feature detectors trained with boosting techniques [15, 2]. The expression recognizer receives image patches located by the face detector. A Gabor representation of the patch is formed and then processed by a bank of SVM classifiers. A novel combination of Adaboost and SVM's enhances performance. The system was tested on the Cohn-Kanade dataset of posed facial expressions [6]. The generalization performance to new subjects for a 7- way forced choice correct. Most interestingly the outputs of the classifier change smoothly as a function of time, providing a potentially valuable representation to code facial expression dynamics in a fully automatic and unobtrusive manner. The system has been deployed on a wide variety of platforms including Sony's Aibo pet robot, ATR's RoboVie, and CU animator, and is currently being evaluated for applications including automatic reading tutors, assessment of human-robot interaction.


Question: Phase-functioned neural networks for the control of character animation.

Inspired by the hierarchical organization of the cerebral cortex, recent algorithms have aimed to learn deep, hierarchical structures from unlabeled data. While comparisons to computations in visual area V1 and the auditory system have been made, less attention has been paid to assessing these algorithms' accuracy in replicating functions at higher levels of the cortical hierarchy. This paper introduces an unsupervised learning model that accurately reflects properties of visual area V2. Specifically, we develop a sparse variant of deep belief networks, a model introduced by Hinton et al. (2006). Our model learns two layers of nodes, with the first layer producing localized, oriented edge filters, akin to Gabor functions, which are known to model V1 cell receptive fields. Building upon this, the second layer encodes correlations between the first layer's responses to the data, capturing both colinear ("contour") features and more complex shapes like corners and junctions. Notably, in a quantitative analysis, our model's encoding of these intricate "corner" features closely aligns with the findings from Ito and Komatsu's study on biological V2 responses. This indicates that our sparse deep belief network variant shows potential for modeling higher-order visual features effectively.

Question: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

Documents: An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production

A one-layer recurrent neural network with a discontinuous activation function is proposed for linear programming. The number of neurons in the neural network is equal to that of decision variables in the linear programming problem. It is proven that the neural network with a sufficiently high gain is globally convergent to the optimal solution. Its application to linear assignment is discussed to demonstrate the utility of the neural network. Several simulation examples are given to show the effectiveness and characteristics of the neural network.

Dynamic effects such as waves, splashes, fire, smoke, and explosions are an integral part of stylized animations. However, such dynamics are challenging to produce, as manually sketching key-frames requires significant effort and artistic expertise while physical simulation tools lack sufficient expressiveness and user control. We present an interactive interface for designing these elemental dynamics for animated illustrations. Users draw with coarse-scale energy brushes which serve as control gestures to drive detailed flow particles which represent local velocity fields. These fields can convey both realistic and artistic effects based on user specification. This painting metaphor for creating elemental dynamics simplifies the process, providing artistic control, and preserves the fluidity of sketching. Our system is fast, stable, and intuitive. An initial user evaluation shows that even novice users with no prior animation experience can create intriguing dynamics using our system.

We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from just a single RGB video. We propose a novel two-stage analysis-by-synthesis optimization whose formulation and implementation are designed for high performance. In the first stage, a skinned templatemodel is jointly fitted to background subtracted input video, 2D and 3D skeleton joint positions found using a deep neural network, and a set of sparse facial landmark detections. In the second stage, dense non-rigid 3D deformations of skin and even loose apparel are captured based on a novel real-time capable algorithm for non-rigid tracking using dense photometric and silhouette constraints. Our novel energy formulation leverages automatically identified material regions on the template to model the differing non-rigid deformation behavior of skin and apparel. The two resulting nonlinear optimization problems per-frame are solved with specially-tailored data-parallel Gauss-Newton solvers. In order to achieve real-time performance of over 25Hz, we design a pipelined parallel architecture using the CPU and two commodity GPUs. Our method is the first real-time monocular approach for full-body performance capture. Our method yields comparable accuracy with off-line performance capture techniques, while being orders of magnitude faster.

o mimic the quality of everyday human communication, future computer interfaces must combine the benefits of high visual fidelity with conversational intelligence and—above all—the ability to modulate the emotions of their users. From a computer graphics or visual perspective, achieving this goal requires creating synthetic digital humans that have photorealistic faces capable of expressing the finest shades of emotion. From a perceptual point of view, the same system must also read users' emotional reactions and adapt the behavior of the digital human accordingly. Our research focuses on implementing this concept to create a face-to-face system called the Virtual Human Interface (VHI). Photorealistic virtual humans— in contrast to nonrealistic virtual characters—make a more effective communication interface between computers and humans because they can stimulate declarative and procedural memory in the human brain. Declarative memory is our memory of information and events. Procedural memory, on the other hand, is the memory of how to do something, such as riding a bike. Most computer interfaces primarily access users' declarative memory; the person learns information about a particular topic. The VHI system, however, engages the user through entertaining emotional responses that help engrave knowledge in the user's mind and even modify the user's behavior through procedural memory. During the past several decades, researchers have conducted countless studies on agents and human animation. Early work mainly addressed low-polygon virtual humans that users could animate in real time. 1 Many of these animation systems initially addressed purposes other than the needs of human–computer interaction. However, the requirements of facial ani-mation—and especially speech synthesis—demand a different underlying architecture that can effectively model how real faces move and change. 2-4 As a result, research began to focus on creating autonomous agents that could exhibit rich personalities and interact in virtual worlds inhabited by other characters. To provide the illusion of a lifelike character, researchers have developed detailed emotional and personality models that can control the animation channels as a function of the virtual human's personality, mood, and emotions. 7-9 The real-time interaction with these virtual characters poses an extra set of technical challenges in terms of the speed, computational power, and visual quality required to make the user believe that he or she is interacting with a living creature. To achieve this goal, researchers eventually replaced precrafted animated actions with intelligent behavior modules that could control speech, locomotion, gaze, blinks, gestures (including various postures), and interaction with the …

This paper introduces particle systems--a method for modeling fuzzy objects such as fire, clouds, and water. Particle systems model an object as a cloud of primitive particles that define its volume. Over a period of time, particles are generated into the system, move and change form within the system, and die from the system. The resulting model is able to represent motion, changes of form, and dynamics that are not possible with classical surface-based representations. The particles can easily be motion blurred, and therefore do not exhibit temporal aliasing or strobing. Stochastic processes are used to generate and control the many particles within a particle system. The application of particle systems to the wall of fire element from the Genesis Demo sequence of the film Star Trek II: The Wrath of Khan [10] is presented.

One of the key steps in creating quality interactive drama is the ability to create quality interactive characters (or believable agents). Two important aspects of such characters will be that they appear emotional and that they can engage in social interactions. My basic approach to these problems has been to use a broad agent architecture and minimal amounts of modeling of other agent in the environment. This approach is based on an understanding of the artistic nature of the problem. To enable agent-builders (artists) to create emotional agents, I provide a general framework for building emotional agents, default emotion-processing rules, and discussion about how to create quality, emotional characters. My framework gets a lot of its power from being part of a broad agent architecture. The concept is simple: the agent will be emotionally richer if there are more things to have emotions about and more ways to express them. This reliance on breadth has also meant that I have been able to create simple emotion models that rely on perception and motivation instead of deep modeling of other agents and complex cognitive processing. To enable agent builders to create social behaviors for believable agents, I have designed a methodology that provides heuristics for incorporating personality into social behaviors and suggests how to model other agents in the environment. I propose an approach to modeling other agents that calls for limiting the amount of modeling of other agents to that which is sufficient to create the desired behavior. Using this technique, I have been able to build robust social behaviors that use surprisingly little representation. I have used this methodology to build a number of social behaviors, like negotiation and making friends. I have built three simulations containing seven agents to drive and test this work. I have also conducted user studies to demonstrate that these agents appear to be emotional and can engage in non-trivial social interactions while also being good characters with distinct personalities.

The problem of visual presentation of multidimensional data is discussed. The projection methods for dimension reduction are reviewed. The chapter deals with the artificial neural networks that may be used for reducing dimension and data visualization, too. The stress is put on combining the selforganizing map (SOM) and Sammon mapping and on the neural network for Sammon’s mapping SAMANN. Large scale applications are discussed: environmental data analysis, statistical analysis of curricula, comparison of schools, analysis of the economic and social conditions of countries, analysis of data on the fundus of eyes and analysis of physiological data on men’s health.

Werner Reichardt Centre for Integrative Neuroscience and Institute of Theoretical Physics, University of Tübingen, Germany Bernstein Center for Computational Neuroscience, Tübingen, Germany Graduate School for Neural Information Processing, Tübingen, Germany Max Planck Institute for Biological Cybernetics, Tübingen, Germany Department of Neuroscience, Baylor College of Medicine, Houston, TX, USA ∗To whom correspondence should be addressed; E-mail: leon.gatys@bethgelab.org

We propose a non-linear generative model for human motion data that uses an undirected model with binary latent variables and real-valued “visible” variables that represent joint angles. The latent and visible variables at each time step receive directed connections from the visible variables at the last few time-steps. Such an architecture makes on-line inference efficient and allows us to use a simple approximate learning procedure. After training, the model finds a single set of parameters that simultaneously capture several different kinds of motion. We demonstrate the power of our approach by synthesizing various motion sequences and by performing on-line filling in of data lost during motion capture. Website: http://www.cs.toronto.edu/ ∼gwtaylor/publications/nips2006mhmublv/

Procedural representations provide powerful means for generating complex geometric structures. They are also notoriously difficult to control. In this article, we present an algorithm for controlling grammar-based procedural models. Given a grammar and a high-level specification of the desired production, the algorithm computes a production from the grammar that conforms to the specification. This production is generated by optimizing over the space of possible productions from the grammar. The algorithm supports specifications of many forms, including geometric shapes and analytical objectives. We demonstrate the algorithm on procedural models of trees, cities, buildings, and Mondrian paintings.


Question: Phase-functioned neural networks for the control of character animation.

This paper introduces a novel generative model for capturing and synthesizing human motion data, utilizing binary latent variables in conjunction with real-valued "visible" variables that represent joint angles. The proposed architecture features undirected connections between the binary latent variables and the real-valued variables, with each time step's variables receiving directed connections from the previous few time steps' visible variables. This design facilitates efficient online inference and enables the use of a straightforward approximate learning procedure. Post-training, the model identifies a unified set of parameters that can encapsulate various types of motion. The efficacy of this approach is illustrated through the synthesis of diverse motion sequences and the online interpolation of missing data points often encountered in motion capture scenarios. For further details and demonstrations, please visit: http://www.cs.toronto.edu/ ∼gwtaylor/publications/nips2006mhmublv/.

The model's capability to learn a generalized representation of motion from a diverse dataset allows it to generate coherent and realistic motion sequences, even when presented with incomplete or corrupted input data. This is achieved by leveraging the learned parameters to infer missing information based on the context provided by the visible variables from preceding time steps. The use of binary latent variables adds a layer of abstraction that aids in the model's ability to generalize across different motion types, making it a versatile tool for motion synthesis and analysis in various applications, including animation, robotics, and biomechanics research.

We investigate neural circuits in the exacting setting that (i) the acquisition of a piece of knowledge can occur from a single interaction, (ii) the result of each such interaction is a rapidly evaluatable subcircuit, (iii) hundreds of thousands of such subcircuits can be acquired in sequence without substantially degrading the earlier ones, and (iv) recall can be in the form of a rapid evaluation of a composition of subcircuits that have been so acquired at arbitrary different earlier times.We develop a complexity theory, in terms of asymptotically matching upper and lower bounds, on the capacity of a neural network for executing, in this setting, the following action, which we call {\it association}: Each action sets up a subcircuit so that the excitation of a chosen set of neurons A will in future cause the excitation of another chosen set B.% As model of computation we consider the neuroidal model, a fully distributed model in which the quantitative resources n, the neuron numbers, d, the number of other neurons each neuron is connected to, and k, the inverse of the maximum synaptic strength, are all accounted for.A succession of experiences, possibly over a lifetime, results in the realization of a complex set of subcircuits. The composability requirement constrains the model to ensure that, for each association as realized by a subcircuit, the excitation in the triggering set of neurons A is quantitatively similar to that in the triggered set B, and also that the unintended excitation in the rest of the system is negligible. These requirements ensure that chains of associations can be triggeredWe first analyze what we call the Basic Mechanism, which uses only direct connections between neurons in the triggering set A and the target set B. We consider random networks of n neurons with expected number d of connections to and from each. We show that in the composable context capacity growth is limited by d^2, a severe limitation if the network is sparse, as it is in cortex. We go on to study the Expansive Mechanism, that additionally uses intermediate relay neurons which have high synaptic weights. For this mechanism we show that the capacity can grow as dn, to within logarithmic factors. From these two results it follows that in the composable regime, for the realistic cortical estimate of d=n^{\frac{1}{2}, superlinear capacity of order n^{\frac{3}{2}} in terms of the neuron numbers can be realized by the Expansive Mechanism, instead of the linear order n to which the Basic Mechanism is limited. More generally, for both mechanisms, we establish matching upper and lower bounds on capacity in terms of the parameters n, d, and the inverse maximum synaptic strength k.The results as stated above assume that in a set of associations, a target B can be triggered by at most one set A. It can be shown that the capacities are similar if the number m of As that can trigger a B is greater than one but small, but become severely constrained if m exceeds a certain threshold.

In this work we propose a model that enables controlled manipulation of visual attributes of real “target” images (e.g. lighting, expression or pose) using only implicit supervision with synthetic “source” exemplars. Specifically, our model learns a shared low-dimensional representation of input images from both domains in which a property of interest is isolated from other content features of the input. By using triplets of synthetic images that demonstrate modification of the visual property that we would like to control (for example mouth opening) we are able to perform disentanglement of image representations with respect to this property without using explicit attribute labels in either domain. Since our technique relies on triplets instead of explicit labels, it can be applied to shape, texture, lighting, or other properties that are difficult to measure or represent as explicit conditioners. We quantitatively analyze the degree to which trained models learn to isolate the property of interest from other content features with a proof-of-concept digit dataset and demonstrate results in a far more difficult setting, learning to manipulate real faces using a synthetic 3D faces dataset. We also explore limitations of our model with respect to differences in distributions of properties observed in two domains.

The aggregate motion of a flock of birds, a herd of land animals, or a school of fish is a beautiful and familiar part of the n a t u r a l world. But this type of complex motion is rarely seen in computer animation. This paper explores an approach based on simulation as an alternative to scripting the paths of each bird individually. The simulated flock is an elaboration of a particle system, with the simulated birds being the particles. The aggregate motion of the simulated flock is created by a distributed behavioral model much like that at work in a natural flock; the birds choose their own course. Each simulated bird is implemented as an independent actor that navigates according to its local perception of the dynamic environment, the laws of simulated physics that rule its motion, and a set of behaviors programmed into it by the "animator." The aggregate motion of the simulated flock is the result of the dense interaction of the relatively simple behaviors of the individual simulated birds.

We present a method for smoothly blending between existing liquid animations. We introduce a semi-automatic method for matching two existing liquid animations, which we use to create new fluid motion that plausibly interpolates the input. Our contributions include a new space-time non-rigid iterative closest point algorithm that incorporates user guidance, a subsampling technique for efficient registration of meshes with millions of vertices, and a fast surface extraction algorithm that produces 3D triangle meshes from a 4D space-time surface. Our technique can be used to instantly create hundreds of new simulations, or to interactively explore complex parameter spaces. Our method is guaranteed to produce output that does not deviate from the input animations, and it generalizes to multiple dimensions. Because our method runs at interactive rates after the initial precomputation step, it has potential applications in games and training simulations.

We propose an efficient method for hair animation. The motion of hairs is modeled by a simplified physical simulation. Collision between hair and a human body or other objects is used to make a realistic hair animation. Fast collision detection is required because the number of hairs is very large. Therefore, a cylindrical representation of human body or other objects is created for efficient collision detection. Because of the cylindrical representation, collision is detected with table look-up and interpolation, and computation complexity is independent of the complexity of the body. Reaction constraint method is applied for the collision response to simulate inelastic contact. This method is successfully applied to generate hair animation.

Generative 3D face models are a powerful tool in computer vision. They provide pose and illumination invariance by modeling the space of 3D faces and the imaging process. The power of these models comes at the cost of an expensive and tedious construction process, which has led the community to focus on more easily constructed but less powerful models. With this paper we publish a generative 3D shape and texture model, the Basel Face Model (BFM), and demonstrate its application to several face recognition task. We improve on previous models by offering higher shape and texture accuracy due to a better scanning device and less correspondence artifacts due to an improved registration algorithm. The same 3D face model can be fit to 2D or 3D images acquired under different situations and with different sensors using an analysis by synthesis method. The resulting model parameters separate pose, lighting, imaging and identity parameters, which facilitates invariant face recognition across sensors and data sets by comparing only the identity parameters. We hope that the availability of this registered face model will spur research in generative models. Together with the model we publish a set of detailed recognition and reconstruction results on standard databases to allow complete algorithm comparisons.

It is shown how a system consisting of two neuronlike adaptive elements can solve a difficult learning control problem. The task is to balance a pole that is hinged to a movable cart by applying forces to the cart's base. It is argued that the learning problems faced by adaptive elements that are components of adaptive networks are at least as difficult as this version of the pole-balancing problem. The learning system consists of a single associative search element (ASE) and a single adaptive critic element (ACE). In the course of learning to balance the pole, the ASE constructs associations between input and output by searching under the influence of reinforcement feedback, and the ACE constructs a more informative evaluation function than reinforcement feedback alone can provide. The differences between this approach and other attempts to solve problems using neurolike elements are discussed, as is the relation of this work to classical and instrumental conditioning in animal learning studies and its possible implications for research in the neurosciences.

The line of action is a conceptual tool often used by cartoonists and illustrators to help make their figures more consistent and more dramatic. We often see the expression of characters---may it be the dynamism of a super hero, or the elegance of a fashion model---well captured and amplified by a single aesthetic line. Usually this line is laid down in early stages of the drawing and used to describe the body's principal shape. By focusing on this simple abstraction, the person drawing can quickly adjust and refine the overall pose of his or her character from a given viewpoint. In this paper, we propose a mathematical definition of the line of action (LOA), which allows us to automatically align a 3D virtual character to a user-specified LOA by solving an optimization problem. We generalize this framework to other types of lines found in the drawing literature, such as secondary lines used to place arms. Finally, we show a wide range of poses and animations that were rapidly created using our system.

A neural network model for a mechanism of visual pattern recognition is proposed in this paper. The network is self-organized by “learning without a teacher”, and acquires an ability to recognize stimulus patterns based on the geometrical similarity (Gestalt) of their shapes without affected by their positions. This network is given a nickname “neocognitron”. After completion of self-organization, the network has a structure similar to the hierarchy model of the visual nervous system proposed by Hubel and Wiesel. The network consits of an input layer (photoreceptor array) followed by a cascade connection of a number of modular structures, each of which is composed of two layers of cells connected in a cascade. The first layer of each module consists of “S-cells”, which show characteristics similar to simple cells or lower order hypercomplex cells, and the second layer consists of “C-cells” similar to complex cells or higher order hypercomplex cells. The afferent synapses to each S-cell have plasticity and are modifiable. The network has an ability of unsupervised learning: We do not need any “teacher” during the process of self-organization, and it is only needed to present a set of stimulus patterns repeatedly to the input layer of the network. The network has been simulated on a digital computer. After repetitive presentation of a set of stimulus patterns, each stimulus pattern has become to elicit an output only from one of the C-cell of the last layer, and conversely, this C-cell has become selectively responsive only to that stimulus pattern. That is, none of the C-cells of the last layer responds to more than one stimulus pattern. The response of the C-cells of the last layer is not affected by the pattern's position at all. Neither is it affected by a small change in shape nor in size of the stimulus pattern.

The authors show that a multiple-input, single-output, single-hidden-layer feedforward network with (known) hardwired connections from input to hidden layer, monotone squashing at the hidden layer and no squashing at the output embeds as a special case a so-called Fourier network, which yields a Fourier series approximation properties of Fourier series representations. In particular, approximation to any desired accuracy of any square integrable function can be achieved by such a network, using sufficiently many hidden units. In this sense, such networks do not make avoidable mistakes.<<ETX>>


Question: ArSLAT: A Tool for Translating Arabic Sign Language Alphabets

This paper provides an overview of the latest advancements in vision-based gesture recognition, a field driven by the desire for more intuitive human-computer interaction. The recognition of gestures through visual means is a complex area that intersects various disciplines. Here, we focus on summarizing the current strategies employed for static hand posture and dynamic gesture recognition. Additionally, we highlight several practical systems that have been developed using gesture recognition technology. The paper concludes with a speculative look at the future trends and potential research avenues in this evolving field.

**Question**: The Role of Social Media in Political Campaigns

**Documents**: The Influence of Social Media on the 2016 U.S. Presidential Election
The 2016 U.S. Presidential Election was heavily influenced by social media, particularly Facebook and Twitter. Candidates used these platforms to communicate directly with voters, bypassing traditional media channels. Social media also allowed for the rapid spread of information, both accurate and false, influencing voter opinions and behaviors. This paper examines the role of social media in the 2016 election and its impact on the political landscape.

**Similar Documents**: This paper explores the pivotal role that social media platforms, notably Facebook and Twitter, played in shaping the 2016 U.S. Presidential Election. These digital channels provided candidates with a direct line to the electorate, enabling them to circumvent conventional media outlets. The instantaneous dissemination of information, including misinformation, significantly swayed public perceptions and voting patterns. A detailed analysis of the 2016 election reveals how social media has transformed the dynamics of political campaigning and discourse.

Spoken language recognition refers to the automatic process through which we determine or verify the identity of the language spoken in a speech sample. We study a computational framework that allows such a decision to be made in a quantitative manner. In recent decades, we have made tremendous progress in spoken language recognition, which benefited from technological breakthroughs in related areas, such as signal processing, pattern recognition, cognitive science, and machine learning. In this paper, we attempt to provide an introductory tutorial on the fundamentals of the theory and the state-of-the-art solutions, from both phonological and computational aspects. We also give a comprehensive review of current trends and future research directions using the language recognition evaluation (LRE) formulated by the National Institute of Standards and Technology (NIST) as the case studies.

The process to take a Braille document image and convert its content into its equivalent natural language characters, is called Optical Braille Recognition (OBR). It involves two main consecutive steps: Braille cell recognition and Braille cell transcription. Braille cell recognition contains few steps including: Image acquisition, Image De-skewing, Image pre-processing, dot recognition, cell recognition and segmentation. Image transcription aims to convert the segmented Braille cell, into its equivalent natural language characters. In this survey we aim to study the earlier works done by other researchers on both Braille cell recognition and transcription.

Nowadays, the convergence of devices, electronic computing, and massive media produces huge volumes of information, which demands the need for faster and more efficient interaction between users and information. How to make information access manageable, efficient, and easy becomes the major challenge for Human-Computer Interaction (HCI) researchers. The different types of computing devices, such as PDAs (personal digital assistants), tablet PCs, desktops, game consoles, and the next generation phones, provide many different modalities for information access. This makes it possible to dynamically adapt application user interfaces to the changing context. However, as applications go more and more pervasive, these devices show theirs limited input/output capacity caused by small visual displays, use of hands to operate buttons and the lack of an alphanumeric keyboard and mouse (Gu & Gilbert, 2004). Voice User Interface (VUI) systems are capable of, besides recognizing the voice of their users, to understand voice commands, and to provide responses to them, usually, in real time. The state-of-the-art in speech technology already allows the development of automatic systems designed to work in real conditions. VUI is perhaps the most critical factor in the success of any automated speech recognition (ASR) system, determining whether the user experience will be satisfying or frustrating, or even whether the customer will remain one. This chapter describes a practical methodology for creating an effective VUI design. The methodology is scientifically based on principles in linguistics, psychology, and language technology (Cohen et al. 2004; San-Segundo et al., 2005). Given the limited input/output capabilities of mobile devices, speech presents an excellent way to enter and retrieve information either alone or in combination with other modalities. Furthermore, people with disabilities should be provided with a wide range of alternative interaction modalities other than the traditional screen-mouse based desktop computing devices. Whether the disability is temporary or permanent, people with reading difficulty, visual impairment, and/or any difficulty using a keyboard, or mouse can rely on speech as an alternate approach for information access. Source: User Interfaces, Book edited by: Rita Mátrai, ISBN 978-953-307-084-1, pp. 270, May 2010, INTECH, Croatia, downloaded from SCIYO.COM

This paper presents integration and testing scheme for managing a large Multilingual OCR Project. The project is an attempt to implement an integrated platform for OCR of different Indian languages. Software engineering, workflow management and testing processes have been discussed in this paper. The OCR has now been experimentally deployed for some specific applications and currently is being enhanced for handling the space and time constraints, achieving higher recognition accuracies and adding new functionalities.

Large-scale digitisation has led to a number of new possibilities with regard to adaptive and learning based methods in the field of Document Image Analysis and OCR. For ground truth production of large corpora, however, there is still a gap in terms of productivity. Ground truth is not only crucial for training and evaluation at the development stage of tools but also for quality assurance in the scope of production workflows for digital libraries. This paper describes Aletheia, an advanced system for accurate and yet cost-effective ground truthing of large amounts of documents. It aids the user with a number of automated and semi-automated tools which were partly developed and improved based on feedback from major libraries across Europe and from their digitisation service providers which are using the tool in a production environment. Novel features are, among others, the support of top-down ground truthing with sophisticated split and shrink tools as well as bottom-up ground truthing supporting the aggregation of lower-level elements to more complex structures. Special features have been developed to support working with the complexities of historical documents. The integrated rules and guidelines validator, in combination with powerful correction tools, enable efficient production of highly accurate ground truth.

In the era of mobile computing, developers often need to migrate code written for one platform in a programming language to another language for a different platform, e.g., from Java for Android to C# for Windows Phone. The migration process is often performed manually or semi-automatically, in which developers are required to manually define translation rules and API mappings. This paper presents semSMT, an automatic tool to migrate code written in Java to C#. semSMT utilizes statistical machine translation to automatically infer translation rules from existing migrated code, thus, requires no manual defining of rules. The video demonstration on semSMT can be found on YouTube at http://www.youtube.com/watch?v=aRSnl5-7vNo.

Recognized as great works of world literature, Shakespeare’s poems and plays have been translated into dozens of languages for over 300 years. Also, there are many re-translations into the same language, for example, there are more than 60 translations of Othello into German. Every translation is a different interpretation of the play. These large quantities of translations reflect changing culture and express individual thought by the authors. They demonstrate wide connections between different world regions today, and reveal a retrospective view of their cultural, intercultural, and linguistic histories. Researchers from Arts and Humanities at Swansea University are collecting a large number of translations of William Shakespeare’s Othello. In this paper, we have developed an interactive visualization system to present, analyze and explore the variations among these different translations. Our system is composed of two parts: the structure-aware Treemap for document selection and meta data analysis, and Focus + Context parallel coordinates for in-depth document comparison and exploration. In particular, we want to learn more about which content varies highly with each translation, and which content remains stable. We also want to form hypotheses as to the implications behind these variations. Our visualization is evaluated by the domain experts from Arts and Humanities.

With the continued demand for Islamic knowledge, which is mainly based on the Quran as a source of knowledge and wisdom, systems that facilitate an easy search of the content of the Quran remain a considerable challenge. Although in recent years there have been tools for Quran search, most of these tools are based on keyword search, meaning that the user needs to know the correct keywords before being able to retrieve the content of al-Quran. In this paper, we propose a system that supports the end user in querying and exploring the Quran ontology. The system comprises user query reformulation against the Quran ontology stored and annotated in the knowledge base. The Quran ontology consists of noun concepts identified in al-Quran, and the relationship that exists between these concepts. The user writes a query in the natural language and the proposed system reformulates the query to match the content found in the knowledge base in order to retrieve the relevant answer. The answer is represented by the Quranic verse related to the user query.

Interacting with computers is a ubiquitous activity for millions of people. Repetitive or specialized tasks often require creation of small, often one-off, programs. End-users struggle with learning and using the myriad of domain-specific languages (DSLs) to effectively accomplish these tasks.
 We present a general framework for constructing program synthesizers that take natural language (NL) inputs and produce expressions in a target DSL. The framework takes as input a DSL definition and training data consisting of NL/DSL pairs. From these it constructs a synthesizer by learning optimal weights and classifiers (using NLP features) that rank the outputs of a keyword-programming based translation. We applied our framework to three domains: repetitive text editing, an intelligent tutoring system, and flight information queries. On 1200+ English descriptions, the respective synthesizers rank the desired program as the top-1 and top-3 for 80% and 90% descriptions respectively.

There is a great need to provide educational environments for blind and handicapped people. There are many Islamic websites and applications dedicated to the educational services for the Holy Quran and Its Sciences (Quran Recitations, the interpretations, etc.) on the Internet. Unfortunately, blind and handicapped people could not use these services. These people cannot use the keyboard and the mouse. In addition, the ability to read and write is essential to benefit from these services. In this paper, we present an educational environment that allows these people to take full advantage of the scientific materials. This is done through the interaction with the system using voice commands by speaking directly without the need to write or to use the mouse. Google Speech API is used for the universal speech recognition after a preprocessing and post processing phases to improve the accuracy. For blind people, responses of these commands will be played back through the audio device instead of displaying the text to the screen. The text will be displayed on the screen to help other people make use of the system.


Question: ArSLAT: A tool for translating Arabic Sign Language alphabets.

This article explores the strategies for engineering Multiple Classifier Systems (MCS) within the domain of pattern recognition, focusing on the amalgamation of outputs from a diverse ensemble of classifiers to construct high-performance classification systems. The discussion centers on the intricacies of MCS design, highlighting six methodologies rooted in the "overproduce and choose" philosophy. These methods, while showcasing intriguing attributes, fall short of ensuring the creation of the most optimal MCS for the specific classification task. Comparative analyses through experimentation reveal the strengths and limitations of each design approach. Consequently, the paper asserts that the quest for an optimal MCS design strategy remains an unresolved challenge in the field of pattern recognition. Despite the advancements and innovations detailed, the search for a definitive solution to the optimal MCS design continues to engage researchers and practitioners alike.

This paper proposes a two level approach to solve the problem of real-time vision-based hand gesture classification. The lower level of the approach implements the posture recognition with Haar-like features and the AdaBoost learning algorithm. With this algorithm, real-time performance and high recognition accuracy can be obtained. The higher level implements the linguistic hand gesture recognition using a context-free grammar-based syntactic analysis. Given an input gesture, based on the extracted postures, the composite gestures can be parsed and recognized with a set of primitives and production rules.

Paste the appropriate copyright statement here. ACM now supports three different copyright statements: • ACM copyright: ACM holds the copyright on the work. This is the historical approach. • License: The author(s) retain copyright, but ACM receives an exclusive publication license. • Open Access: The author(s) wish to pay for the work to be open access. The additional fee must be paid to ACM. This text field is large enough to hold the appropriate release statement assuming it is single spaced in a sans-serif 7 point font. Every submission will be assigned their own unique DOI string to be included here. Abstract Twitch.tv, a streaming platform known for video game content, has grown tremendously since its inception in 2011. We examine communication practices in Twitch chats for the popular game Hearthstone, comparing massive chats with at least 10,000 concurrent viewers and small chats with fewer than 2,000 concurrent viewers. Due to the large scale and fast pace of massive chats, communication patterns no longer follow models developed in previous studies of computer-mediated communication. Rather than what other studies have described as communication breakdowns and information overload, participants in massive chats communicate in what we call “crowdspeak.”

The use of the gesture system in our daily life as a natural human-human interaction has inspired the researchers to simulate and utilize this gift in human-machine interaction which is appealing and can take place the bore interaction ones that existed such as television, radio, and various home appliances as well as virtual reality will worth and deserve its name. This kind of interaction ensures promising and satisfying outcomes if applied in systematic approach, and supports unadorned human hand when transferring the message to these devices which is easiest, comfort and desired rather than the communication that requires frills to deliver the message to such devices. With the rapid emergence of 3d applications and virtual environments in computer system the need for a new type of interaction device arises. This is because the traditional devices such as mouse, keyboard and joystick become inefficient and cumbersome within this virtual environments.in other words evolution of user interfaces shapes the change in the HumanComputer Interaction (HCI).Intuitive and naturalness characteristics of “Hand Gestures” in the HCI have been the driving force and motivation to develop an interaction device which can replace current unwieldy tools. A survey on the methods of analysing, modelling and recognizing hand gestures in the context of the HCI is provided in this paper.

In this notebook, we describe our approach to cope with the Author Profiling task on PAN17 which consists of both gender and language identification for Twitter’s users. We used our MicroTC (μTC) framework as the primary tool to create our classifiers. μTC follows a simple approach to text classification; it converts the problem of text classification to a model selection problem using several simple text transformations, a combination of tokenizers, a term-weighting scheme, and finally, it classifies using a Support Vector Machine. Our approach reaches accuracies of 0.7838, 0.8054, 0.7957, and 0.8538, for gender identification; and for language variety, it achieves 0.8275, 0.9004, 0.9554, and 0.9850. All these, for Arabic, English, Spanish, and Portuguese languages, respectively.

From our three year experience of developing a large-scale corpus of annotated Arabic text, our paper will address the following: (a) review pertinent Arabic language issues as they relate to methodology choices, (b) explain our choice to use the Penn English Treebank style of guidelines, (requiring the Arabic-speaking annotators to deal with a new grammatical system) rather than doing the annotation in a more traditional Arabic grammar style (requiring NLP researchers to deal with a new system); (c) show several ways in which human annotation is important and automatic analysis difficult, including the handling of orthographic ambiguity by both the morphological analyzer and human annotators; (d) give an illustrative example of the Arabic Treebank methodology, focusing on a particular construction in both morphological analysis and tagging and syntactic analysis and following it in detail through the entire annotation process, and finally, (e) conclude with what has been achieved so far and what remains to be done.

Off-line recognition requires transferring the text under consideration into an image file. This represents the only available solution to bring the printed materials to the electronic media. However, the transferring process causes the system to lose the temporal information of that text. Other complexities that an off-line recognition system has to deal with are the lower resolution of the document and the poor binarisation, which can contribute to readability when essential features of the characters are deleted or obscured. Recognising Arabic script presents two additional challenges: orthography is cursive and letter shape is context sensitive. Certain character combinations form new ligature shapes, which are often font-dependent. Some ligatures involve vertical stacking of characters. Since not all letters connect, word boundary location becomes an interesting problem, as spacing may separate not only words, but also certain characters within a word. Various techniques have been implemented to achieve high recognition rates. These techniques have tackled different aspects of the recognition system. This review is organised into five major sections, covering a general overview, Arabic writing characteristics, Arabic text recognition system, Arabic OCR software and conclusions.

We present a wearable interface that allows a user to silently converse with a computing device without any voice or any discernible movements - thereby enabling the user to communicate with devices, AI assistants, applications or other people in a silent, concealed and seamless manner. A user's intention to speak and internal speech is characterized by neuromuscular signals in internal speech articulators that are captured by the AlterEgo system to reconstruct this speech. We use this to facilitate a natural language user interface, where users can silently communicate in natural language and receive aural output (e.g - bone conduction headphones), thereby enabling a discreet, bi-directional interface with a computing device, and providing a seamless form of intelligence augmentation. The paper describes the architecture, design, implementation and operation of the entire system. We demonstrate robustness of the system through user studies and report 92% median word accuracy levels.

This paper presents an overview of Statistical Machine Translation (SMT), which is currently the dominant approach in Machine Translation (MT) research. In Way and Hearne (2010), we describe how central linguists and translators are to the MT process, so that SMT developers and researchers may better understand how to include these groups in continuing to advance the stateof-the-art. If these constituencies are to make an impact in the field of MT, they need to know how their input is used by SMT systems. Accordingly, our objective in this paper is to present the basic principles underpinning SMT in a way that linguists and translators will find accessible

In this paper we describe a fuzzy logic-based language processing method, which is applied to speech recognition. Our purpose is to create a system that can learn from a linguistic corpus the fuzzy semantic relations between the concepts represented by words and use such relations to process the word sequences generated by speech recognition systems. In particular, the system will be able to predict the words failed to be recognized by a speech recognition system. This will help to increase the accuracy of a speech recognition system. This will also serve as the first stage of deep semantic processing of speech recognition results by providing “semantic relatedness” between the recognized words. We report the fuzzy inference rule learning system, which we have developed and also report the experimental results based on the system. Key-words: Fuzzy logic, Natural Language Analysis, Speech Recognition, Corpus Linguistics.

The process to take a Braille document image and convert its content into its equivalent natural language characters, is called Optical Braille Recognition (OBR). It involves two main consecutive steps: Braille cell recognition and Braille cell transcription. Braille cell recognition contains few steps including: Image acquisition, Image De-skewing, Image pre-processing, dot recognition, cell recognition and segmentation. Image transcription aims to convert the segmented Braille cell, into its equivalent natural language characters. In this survey we aim to study the earlier works done by other researchers on both Braille cell recognition and transcription.


Question: ArSLAT: A Tool for Translating Arabic Sign Language Alphabets

This paper explores the concept of Bagging Predictors, a technique designed to enhance the accuracy of predictions by generating multiple versions of a predictor and aggregating their results. When forecasting numerical outcomes, the aggregation is achieved through averaging, while for class predictions, a plurality vote is utilized. The diversity in predictor versions is created by generating bootstrap replicates of the learning dataset, which are then employed as fresh learning sets. Empirical evaluations on both real-world and simulated datasets, employing classification and regression trees alongside subset selection in linear regression, have demonstrated that bagging can significantly boost predictive accuracy. A critical factor in the effectiveness of bagging is the instability of the prediction method; if small perturbations in the learning set can lead to substantial alterations in the constructed predictor, then bagging is particularly effective in enhancing accuracy.

**Question**: A Comparative Study of Different Techniques for Arabic Sign Language Recognition

**Documents**: A Review of Automatic Speech Recognition: Challenges, Trends, and an Open-Source Recipe for Large Vocabulary Systems
Automatic speech recognition (ASR) has seen significant advancements in recent years, largely due to the advent of deep learning techniques. This paper provides a comprehensive review of ASR technologies, detailing the evolution from traditional hidden Markov models to the current dominance of deep neural networks. We discuss the challenges faced by ASR systems, including acoustic variability, language model limitations, and data scarcity. The review also highlights recent trends in the field, such as end-to-end models and the use of transfer learning. Furthermore, we share an open-source recipe for building large vocabulary ASR systems, encompassing data preparation, model training, and evaluation. This review is intended to serve as a reference for researchers and practitioners interested in the state-of-the-art in ASR.

**Similar Documents**: This paper offers an in-depth analysis of the advancements and challenges in the field of Automatic Speech Recognition (ASR), with a particular focus on the transformative impact of deep learning methodologies. It chronicles the transition from conventional hidden Markov models to the contemporary predominance of deep neural networks in ASR. We explore the hurdles encountered in ASR, which encompass acoustic variability, constraints in language modeling, and the scarcity of training data. The paper also sheds light on emerging trends, such as the integration of end-to-end models and the utilization of transfer learning techniques. Additionally, it provides a detailed open-source guide for the construction of large vocabulary ASR systems, encompassing all stages from data preparation to model training and evaluation. This comprehensive review is aimed at serving as a resource for both

The Internet is frequently used as a medium for exchange of information and opinions, as well as propaganda dissemination. In this study the use of sentiment analysis methodologies is proposed for classification of Web forum opinions in multiple languages. The utility of stylistic and syntactic features is evaluated for sentiment classification of English and Arabic content. Specific feature extraction components are integrated to account for the linguistic characteristics of Arabic. The entropy weighted genetic algorithm (EWGA) is also developed, which is a hybridized genetic algorithm that incorporates the information-gain heuristic for feature selection. EWGA is designed to improve performance and get a better assessment of key features. The proposed features and techniques are evaluated on a benchmark movie review dataset and U.S. and Middle Eastern Web forum postings. The experimental results using EWGA with SVM indicate high performance levels, with accuracies of over 91% on the benchmark dataset as well as the U.S. and Middle Eastern forums. Stylistic features significantly enhanced performance across all testbeds while EWGA also outperformed other feature selection methods, indicating the utility of these features and techniques for document-level classification of sentiments.

In recent years, there has been a global growing demand for Islamic knowledge by both Muslims and non-Muslims. This has brought about a number of automated applications that ease the retrieval of knowledge from the Holy Book, being the major source of Knowledge in Islam. However, the current retrieval methods in the Quranic domain lack adequate semantic search capabilities; they are mostly based on the keywords matching approach. There is a lack of adequate linked data to provide a better description of concepts found in the Holy Quran. In this study we propose an Ontology assisted semantic search system in the Qur’an domain. The system makes use of Quran ontology and various relationships and restrictions. This will enable the user to semantically search for verses related to their query in Al-Quran. The system has improved the search capability of the Holy Quran knowledge to 95 percent accuracy level.

The social media network phenomenon creates massive amounts of valuable data that is available online and easy to access. Many users share images, videos, comments, reviews, news and opinions on different social networks sites, with Twitter being one of the most popular ones. Data collected from Twitter is highly unstructured, and extracting useful information from tweets is a challenging task. Twitter has a huge number of Arabic users who mostly post and write their tweets using the Arabic language. While there has been a lot of research on sentiment analysis in English, the amount of researches and datasets in Arabic language is limited. This paper introduces an Arabic language dataset, which is about opinions on health services and has been collected from Twitter. The paper will first detail the process of collecting the data from Twitter and also the process of filtering, pre-processing and annotating the Arabic text in order to build a big sentiment analysis dataset in Arabic. Several Machine Learning algorithms (Naïve Bayes, Support Vector Machine and Logistic Regression) alongside Deep and Convolutional Neural Networks were utilized in our experiments of sentiment analysis on our health dataset.

Review of Semantic-Free Utterances in Social Human–Robot Interaction Selma Yilmazyildiz, Robin Read, Tony Belpeame & Werner Verhelst To cite this article: Selma Yilmazyildiz, Robin Read, Tony Belpeame & Werner Verhelst (2016) Review of Semantic-Free Utterances in Social Human–Robot Interaction, International Journal of Human–Computer Interaction, 32:1, 63-85, DOI: 10.1080/10447318.2015.1093856 To link to this article: http://dx.doi.org/10.1080/10447318.2015.1093856

This paper extends research on automated image captioning in the dimension of language, studying how to generate Chinese sentence descriptions for unlabeled images. To evaluate image captioning in this novel context, we present Flickr8k-CN, a bilingual extension of the popular Flickr8k set. The new multimedia dataset can be used to quantitatively assess the performance of Chinese captioning and English-Chinese machine translation. The possibility of re-using existing English data and models via machine translation is investigated. Our study reveals to some extent that a computer can master two distinct languages, English and Chinese, at a similar level for describing the visual world. Data is publicly available at http://tinyurl.com/flickr8kcn

We describe here a Web inventory named WIT3 that offers access to a collection of transcribed and translated talks. The core of WIT3 is the TED Talks corpus, that basically redistributes the original content published by the TED Conference website (http://www.ted.com). Since 2007, the TED Conference, based in California, has been posting all video recordings of its talks together with subtitles in English and their translations in more than 80 languages. Aside from its cultural and social relevance, this content, which is published under the Creative Commons BYNC-ND license, also represents a precious language resource for the machine translation research community, thanks to its size, variety of topics, and covered languages. This effort repurposes the original content in a way which is more convenient for machine translation researchers.

Automatic key phrase extraction has many important applications including but not limited to summarization, cataloging/indexing, feature extraction for clustering and classification, and data mining. This paper presents the KP-Miner system, and demonstrates through experimentation and comparison with existing systems that the it is effective in extracting keyphrases from both English and Arabic documents of varied length. Unlike other existing keyphrase extraction systems, the KP-Miner system has the advantage of being configurable as the rules and heuristics adopted by the system are related to the general nature of documents and keyphrase. This implies that the users of this system can use their understanding of the document(s) being input into the system, to fine tune it to their particular needs.

The goal of the research described in this paper is to develop an application-independent presentation tool that automatically designs effective graphical presentations (such as bar charts, scatter plots, and connected graphs) of relational information. Two problems are raised by this goal: The codification of graphic design criteria in a form that can be used by the presentation tool, and the generation of a wide variety of designs so that the presentation tool can accommodate a wide variety of information. The approach described in this paper is based on the view that graphical presentations are sentences of graphical languages. The graphic design issues are codified as expressiveness and effectiveness criteria for graphical languages. Expressiveness criteria determine whether a graphical language can express the desired information. Effectiveness criteria determine whether a graphical language exploits the capabilities of the output medium and the human visual system. A wide variety of designs can be systematically generated by using a composition algebra that composes a small set of primitive graphical languages. Artificial intelligence techniques are used to implement a prototype presentation tool called APT (A Presentation Tool), which is based on the composition algebra and the graphic design criteria.

We have developed Preksha: a Hindi text visualizer. Preksha generates a 3D scene by understanding a given Hindi text. We propose the Earley's parsing algorithm using Tree adjoining grammar (TAG) for Hindi language understanding. A brief overview of the TAG formalism and the Earley's parsing algorithm is presented. Preksha's Hindi language parsing is explainedwith the Lisp notation. The maincontribution of this paper is to apply TAG parsing for morphologically-richfree-order language Hindi for automatic text visualization (ATV).

Computational Morphology is an urgent problem for Arabic Natural Language Processing, because Arabic is a highly inflected language. We have found, however, that a full solution to this problem is not required for effective information retrieval. Light stemming allows remarkably good information retrieval without providing correct morphological analyses. We developed several light stemmers for Arabic, and assessed their effectiveness for information retrieval using standard TREC data. We have also compared light stemming with several stemmers based on morphological analysis.. The light stemmer, light10, outperformed the other approaches. It has been included in the Lemur toolkit, and is becoming widely used Arabic information retrieval.


Question: ArSLAT: A Tool for Translating Arabic Sign Language Alphabets

This research introduces a real-time, continuous gesture recognition framework for sign language interpretation, specifically tailored for Taiwanese Sign Language (TWL). Addressing the significant challenge of accurately pinpointing gesture end-points in a continuous stream of sign language input, this paper proposes a solution and subsequently conducts a statistical analysis based on four critical gesture attributes: posture, position, orientation, and motion. The developed prototype system boasts a comprehensive lexicon of 250 TWL vocabularies. To achieve real-time recognition, the system employs hidden Markov models (HMMs) for 51 core postures, 6 orientations, and 8 motion primitives. Through a signer-dependent approach, this framework enables the seamless recognition of continuous gesture sentences, achieving an impressive average recognition rate of 80.4%. The system's design and performance highlight its potential for enhancing communication accessibility for the deaf community, particularly in real-world scenarios.

Abstractive single document summarization is considered as a challenging problem in the field of artificial intelligence and natural language processing. Meanwhile and specifically in the last two years, several deep learning summarization approaches were proposed that once again attracted the attention of researchers to this field.
 It is a well-known issue that deep learning approaches do not work well with small amounts of data. With some exceptions, this is, unfortunately, the case for most of the datasets available for the summarization task. Besides this problem, it should be considered that phonetic, morphological, semantic and syntactic features of the language are constantly changing over the time and unfortunately most of the summarization corpora are constructed from old resources. Another problem is the language of the corpora. Not only in the summarization field, but also in other fields of natural language processing, most of the corpora are only available in English. In addition to the above problems, license terms, and fees of the corpora are obstacles that prevent many academics and specifically non-academics from accessing these data.
 This work describes an open source framework to create an extendable multilingual corpus for abstractive single document summarization that addresses the above-mentioned problems. We describe a tool consisted of a scalable crawler and a centralized key-value store database to construct a corpus of an arbitrary size using a news aggregator service.

We describe a syntax-based algorithm that automatically builds Finite State Automata (word lattices) from semantically equivalent translation sets. These FSAs are good representations of paraphrases. They can be used to extract lexical and syntactic paraphrase pairs and to generate new, unseen sentences that express the same meaning as the sentences in the input sets. Our FSAs can also predict the correctness of alternative semantic renderings, which may be used to evaluate the quality of translations.

Engineering automatic speech recognition (ASR) for speech to speech (S2S) translation systems, especially targeting languages and domains that do not have readily available spoken language resources, is immensely challenging due to a number of reasons. In addition to contending with the conventional data-hungry speech acoustic and language modeling needs, these designs have to accommodate varying requirements imposed by the domain needs and characteristics, target device and usage modality (such as phrase-based, or spontaneous free form interactions, with or without visual feedback) and huge spoken language variability arising due to socio-linguistic and cultural differences of the users. This paper, using case studies of creating speech translation systems between English and languages such as Pashto and Farsi, describes some of the practical issues and the solutions that were developed for multilingual ASR development. These include novel acoustic and language modeling strategies such as language adaptive recognition, active-learning based language modeling, class-based language models that can better exploit resource poor language data, efficient search strategies, including N-best and confidence generation to aid multiple hypotheses translation, use of dialog information and clever interface choices to facilitate ASR, and audio interface design for meeting both usability and robustness requirements

We describe a software tool named “Speechalyzer” which is optimized to process large speech data sets with respect to transcription, labeling and annotation. It is implemented as a client server based framework in Java and interfaces software for speech recognition, synthesis, speech classification and quality evaluation. The application is mainly the processing of training data for speech recognition and classification models and performing benchmarking tests on speech to text, text to speech and speech categorization software systems.

This paper presents an overview of Statistical Machine Translation (SMT), which is currently the dominant approach in Machine Translation (MT) research. In Way and Hearne (2010), we describe how central linguists and translators are to the MT process, so that SMT developers and researchers may better understand how to include these groups in continuing to advance the stateof-the-art. If these constituencies are to make an impact in the field of MT, they need to know how their input is used by SMT systems. Accordingly, our objective in this paper is to present the basic principles underpinning SMT in a way that linguists and translators will find accessible

This paper proposes a two level approach to solve the problem of real-time vision-based hand gesture classification. The lower level of the approach implements the posture recognition with Haar-like features and the AdaBoost learning algorithm. With this algorithm, real-time performance and high recognition accuracy can be obtained. The higher level implements the linguistic hand gesture recognition using a context-free grammar-based syntactic analysis. Given an input gesture, based on the extracted postures, the composite gestures can be parsed and recognized with a set of primitives and production rules.

We present a wearable interface that allows a user to silently converse with a computing device without any voice or any discernible movements - thereby enabling the user to communicate with devices, AI assistants, applications or other people in a silent, concealed and seamless manner. A user's intention to speak and internal speech is characterized by neuromuscular signals in internal speech articulators that are captured by the AlterEgo system to reconstruct this speech. We use this to facilitate a natural language user interface, where users can silently communicate in natural language and receive aural output (e.g - bone conduction headphones), thereby enabling a discreet, bi-directional interface with a computing device, and providing a seamless form of intelligence augmentation. The paper describes the architecture, design, implementation and operation of the entire system. We demonstrate robustness of the system through user studies and report 92% median word accuracy levels.

Opinion mining is an interested area of research, which epitomize the customer reviews of a product or service and express whether the opinions are positive or negative. Various methods have been proposed as classifiers for opinion mining such as Naïve Bayesian, and Support vector machine, these methods classify opinion without giving us the reasons about why the instance opinion is classified to certain class. Therefore, in our work, we investigate opinion mining of Arabic text at the document level, by applying decision trees classification classifier to have clear, understandable rule, also we apply parallel decision trees classifiers to have efficient results. We applied parallel decision trees on two Arabic corpus of text documents by using parallel implementation of RapidMiner tools. In case of applying parallel decision tree family on OCA we get the best results of accuracy (93.83%), f-measure (93.22) and consumed time 42 Sec at thread 4, one of the resulted rule is Urdu language lines. In case of applying parallel decision tree family on BHA we get the best results of accuracy (90.63%), f-measure (82.29) and consumed time 219 Sec at thread 4, one of the resulted rule is Urdu language lines.

This work presents the design and development of a web-based system that supports cross-language similarity analysis and plagiarism detection. A suspicious document dq in a language Lq is to be submitted to the system via a PHP web-based interface. The system will accept the text through either uploading or pasting it directly to a text-area. In order to lighten large texts and provide an ideal set of queries, we introduce the idea of query document reduction via summarisation. Our proposed system utilised a fuzzy swarm-based summarisation tool originally built in Java. Then, the summary is used as a query to find similar web resources in languages Lx other than Lq via a dictionary-based translation. Thereafter, a detailed similarity analysis across the languages Lq and Lx is performed and friendly report of results is produced. Such report has global similarity score on the whole document, which assures high flexibility of utilisation.

Yunglung Chen, Yuping Wang, Kinshuk, Nian-Shing Chen Department of Information Management, National Sun Yat-sen University, No. 70, Lienhai Rd., Kaohsiung 80424, Taiwan School of Languages and Linguistics, Griffith University, Nathan, 4111, Australia School of Computing and Information System, Athabasca University, Athabasca, Canada henryylchen@gmail.com, y.wang@griffith.edu.au, kinshuk@athabascau.ca, nschen@mis.nsysu.edu.tw


Question: ArSLAT: A tool for translating Arabic Sign Language alphabets.

This research delves into the strategies for integrating the classification capabilities of multiple classifiers, a critical issue in the broad spectrum of pattern recognition applications. A thorough exploration of this topic has been undertaken, with potential solutions categorized into three distinct classes based on the information level accessible from each classifier. Four methodologies are introduced, tailored to address this amalgamation challenge. One technique is specifically designed for integrating classifiers such as Bayesian, k-NN, and various distance-based classifiers. The remaining three methods are versatile, applicable to any type of individual classifier. When these strategies are applied to the recognition of unconstrained handwritten numerals by combining several classifiers, the outcomes reveal a substantial enhancement in individual classifier performance. For instance, when tested on the U.S. zipcode database, the results showcase a recognition rate of 98.9%, with a 0.90% substitution rate and a 0.2% rejection rate. Moreover, a high level of reliability is achieved with 95% recognition, 0% substitution, and a 5% rejection rate. These findings are comparable to or surpass those reported in research efforts across Europe, Asia, and North America.

Problem statement: The identification of collocations is very importa n part in natural language processing applications that require some degree of semantic interpretation such as, machine translation, information retrieval and text summari z tion. Because of the complexities of Arabic, the collocations undergo some variations such as, morph ological, graphical, syntactic variation that constitutes the difficulties of identifying the col location. Approach: We used the hybrid method for extracting the collocations from Arabic corpus that is based on linguistic information and association measures. Results: This method extracted the bi-gram candidates of Ar abic collocation from corpus and evaluated the association measures by using the n-best evaluation method. We reported the precision values for each association measure in ea ch n-best list. Conclusion: The experimental results showed that the log-likelihood ratio is the best as ociation measure that achieved highest precision.

This paper presents integration and testing scheme for managing a large Multilingual OCR Project. The project is an attempt to implement an integrated platform for OCR of different Indian languages. Software engineering, workflow management and testing processes have been discussed in this paper. The OCR has now been experimentally deployed for some specific applications and currently is being enhanced for handling the space and time constraints, achieving higher recognition accuracies and adding new functionalities.

This paper addresses challenges of Natural Language Processing (NLP) on non-canonical multilingual data in which two or more languages are mixed. It refers to code-switching which has become more popular in our daily life and therefore obtains an increasing amount of attention from the research community. We report our experience that covers not only core NLP tasks such as normalisation, language identification, language modelling, part-of-speech tagging and dependency parsing but also more downstream ones such as machine translation and automatic speech recognition. We highlight and discuss the key problems for each of the tasks with supporting examples from different language pairs and relevant previous work.

In this paper we focus on the problem of question ranking in community question answering (cQA) forums in Arabic. We address the task with machine learning algorithms using advanced Arabic text representations. The latter are obtained by applying tree kernels to constituency parse trees combined with textual similarities, including word embeddings. Our two main contributions are: (i) an Arabic language processing pipeline based on UIMA —from segmentation to constituency parsing— built on top of Farasa, a state-of-the-art Arabic language processing toolkit; and (ii) the application of long short-term memory neural networks to identify the best text fragments in questions to be used in our tree-kernel-based ranker. Our thorough experimentation on a recently released cQA dataset shows that the Arabic linguistic processing provided by Farasa produces strong results and that neural networks combined with tree kernels further boost the performance in terms of both efficiency and accuracy. Our approach also enables an implicit comparison between different processing pipelines as our tests on Farasa and Stanford parsers demonstrate.

Approach • Social information independent of physical analog s – Text (e.g., emote) – abstract graphical representations (e.g., chat cir cles) • Interested in Abstract Approach a. Creates and deploys working systems b. Lack of attention Babble Prototype

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L’archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d’enseignement et de recherche français ou étrangers, des laboratoires publics ou privés. Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation Alexandre Bérard, Olivier Pietquin, Laurent Besacier, Christophe Servan

In this notebook, we describe our approach to cope with the Author Profiling task on PAN17 which consists of both gender and language identification for Twitter’s users. We used our MicroTC (μTC) framework as the primary tool to create our classifiers. μTC follows a simple approach to text classification; it converts the problem of text classification to a model selection problem using several simple text transformations, a combination of tokenizers, a term-weighting scheme, and finally, it classifies using a Support Vector Machine. Our approach reaches accuracies of 0.7838, 0.8054, 0.7957, and 0.8538, for gender identification; and for language variety, it achieves 0.8275, 0.9004, 0.9554, and 0.9850. All these, for Arabic, English, Spanish, and Portuguese languages, respectively.

In the era of mobile computing, developers often need to migrate code written for one platform in a programming language to another language for a different platform, e.g., from Java for Android to C# for Windows Phone. The migration process is often performed manually or semi-automatically, in which developers are required to manually define translation rules and API mappings. This paper presents semSMT, an automatic tool to migrate code written in Java to C#. semSMT utilizes statistical machine translation to automatically infer translation rules from existing migrated code, thus, requires no manual defining of rules. The video demonstration on semSMT can be found on YouTube at http://www.youtube.com/watch?v=aRSnl5-7vNo.

Interacting with computers is a ubiquitous activity for millions of people. Repetitive or specialized tasks often require creation of small, often one-off, programs. End-users struggle with learning and using the myriad of domain-specific languages (DSLs) to effectively accomplish these tasks.
 We present a general framework for constructing program synthesizers that take natural language (NL) inputs and produce expressions in a target DSL. The framework takes as input a DSL definition and training data consisting of NL/DSL pairs. From these it constructs a synthesizer by learning optimal weights and classifiers (using NLP features) that rank the outputs of a keyword-programming based translation. We applied our framework to three domains: repetitive text editing, an intelligent tutoring system, and flight information queries. On 1200+ English descriptions, the respective synthesizers rank the desired program as the top-1 and top-3 for 80% and 90% descriptions respectively.

1 Qatar Computing Research Institute (QCRI), Qatar Foundation, Doha, Qatar; E-Mail: ikhalil@uaeu.ac.ae 2 Department of Electrical and Computer Engineering, Newark College of Engineering, New Jersey Institute of Technology, University Heights, Newark, NJ 07102, USA; E-Mail: abdallah@njit.edu 3 College of Information Technology, United Arab Emirates University, PO Box 15551, Al Ain, United Arab Emirates; E-Mail: azeemsamma@hotmail.com