Question: Real-time implementation of semi-global matching on the central processing unit.

This paper tackles the challenge of visual correspondence without the common assumption of similar intensities across different views, a scenario often encountered in non-lambertian scenes or due to variations between cameras. To address this issue, we employ the maximization of mutual information, a robust technique for image registration that does not require any preconceived model of the relationship between scene intensities in various perspectives. Although mutual information has proven challenging for dense visual correspondence due to the limitations of fixed-size window comparisons—namely, poor performance at discontinuities and in low-texture areas—we propose a solution to these issues. By integrating mutual information into the standard energy minimization framework used in early vision, through a straightforward approximation, we can efficiently compute visual correspondence. The energy is minimized using graph cuts, a method that effectively preserves discontinuities and manages low-texture regions. The outcome is an algorithm that combines the precision of disparity maps generated by graph cuts with the resilience to intensity changes offered by mutual information, resulting in accurate and robust visual correspondence under varying lighting and camera conditions.

DTAM is a system for real-time camera tracking and reconstruction which relies not on feature extraction but dense, every pixel methods. As a single hand-held RGB camera flies over a static scene, we estimate detailed textured depth maps at selected keyframes to produce a surface patchwork with millions of vertices. We use the hundreds of images available in a video stream to improve the quality of a simple photometric data term, and minimise a global spatially regularised energy functional in a novel non-convex optimisation framework. Interleaved, we track the camera's 6DOF motion precisely by frame-rate whole image alignment against the entire dense model. Our algorithms are highly parallelisable throughout and DTAM achieves real-time performance using current commodity GPU hardware. We demonstrate that a dense model permits superior tracking performance under rapid motion compared to a state of the art method using features; and also show the additional usefulness of the dense model for real-time scene interaction in a physics-enhanced augmented reality application.

Modern graphics hardware architectures excel at compute-intensive tasks such as ray-triangle intersection, making them attractive target platforms for raytracing. To date, most GPU-based raytracers have relied upon uniform grid acceleration structures. In contrast, the kd-tree has gained widespread use in CPU-based raytracers and is regarded as the best general-purpose acceleration structure. We demonstrate two kd-tree traversal algorithms suitable for GPU implementation and integrate them into a streaming raytracer. We show that for scenes with many objects at different scales, our kd-tree algorithms are up to 8 times faster than a uniform grid. In addition, we identify load balancing and input data recirculation as two fundamental sources of inefficiency when raytracing on current graphics hardware.

Recently dense trajectories were shown to be an efficient video representation for action recognition and achieved state-of-the-art results on a variety of datasets. This paper improves their performance by taking into account camera motion to correct them. To estimate camera motion, we match feature points between frames using SURF descriptors and dense optical flow, which are shown to be complementary. These matches are, then, used to robustly estimate a homography with RANSAC. Human motion is in general different from camera motion and generates inconsistent matches. To improve the estimation, a human detector is employed to remove these matches. Given the estimated camera motion, we remove trajectories consistent with it. We also use this estimation to cancel out camera motion from the optical flow. This significantly improves motion-based descriptors, such as HOF and MBH. Experimental results on four challenging action datasets (i.e., Hollywood2, HMDB51, Olympic Sports and UCF50) significantly outperform the current state of the art.

We present an efficient method for volume rendering by raycasting on the CPU. We employ coherent packet traversal of an implicit bounding volume hierarchy, heuristically pruned using preintegrated transfer functions, to exploit empty or homogeneous space. We also detail SIMD optimizations for volumetric integration, trilinear interpolation, and gradient lighting. The resulting system performs well on low-end and laptop hardware, and can outperform out-of-core GPU methods by orders of magnitude when rendering large volumes without level-of-detail (LOD) on a workstation. We show that, while slower than GPU methods for low-resolution volumes, an optimized CPU renderer does not require LOD to achieve interactive performance on large data sets.

The paper considers two problems associated with the detection and classification of motion in image sequences obtained from a static camera. Motion is detected by differencing a reference and the "current" image frame, and therefore requires a suitable reference image and the selection of an appropriate detection threshold. Several threshold selection methods are investigated, and an algorithm based on hysteresis thresholding is shown to give acceptably good results over a number of test image sets. The second part of the paper examines the problem of detecting shadow regions within the image which are associated with the object motion. This is based on the notion of a shadow as a semi-transparent region in the image which retains a (reduced contrast) representation of the underlying surface pattern, texture or grey value. The method uses a region growing algorithm which uses a growing criterion based on a fixed attenuation of the photometric gain over the shadow region, in comparison to the reference image.

We consider brightness/contrast-invariant and rotation-discriminating template matching that searches an image to analyze A for a query image Q. We propose to use the complex coefficients of the discrete Fourier transform of the radial projections to compute new rotation-invariant local features. These coefficients can be efficiently obtained via FFT. We classify templates in “stable” and “unstable” ones and argue that any local feature-based template matching may fail to find unstable templates. We extract several stable sub-templates of Q and find them in A by comparing the features. The matchings of the sub-templates are combined using the Hough transform. As the features of A are computed only once, the algorithm can find quickly many different sub-templates in A, and it is suitable for: finding many query images in A; multi-scale searching and partial occlusion-robust template matching.

Modern CPUs have instructions that allow basic operations to be performed on several data elements in parallel. These instructions are called SIMD instructions, since they apply a single instruction to multiple data elements. SIMD technology was initially built into commodity processors in order to accelerate the performance of multimedia applications. SIMD instructions provide new opportunities for database engine design and implementation. We study various kinds of operations in a database context, and show how the inner loop of the operations can be accelerated using SIMD instructions. The use of SIMD instructions has two immediate performance benefits: It allows a degree of parallelism, so that many operands can be processed at once. It also often leads to the elimination of conditional branch instructions, reducing branch mispredictions.We consider the most important database operations, including sequential scans, aggregation, index operations, and joins. We present techniques for implementing these using SIMD instructions. We show that there are significant benefits in redesigning traditional query processing algorithms so that they can make better use of SIMD technology. Our study shows that using a SIMD parallelism of four, the CPU time for the new algorithms is from 10% to more than four times less than for the traditional algorithms. Superlinear speedups are obtained as a result of the elimination of branch misprediction effects.

Most pedestrian detection approaches that achieve high accuracy and precision rate and that can be used for realtime applications are based on histograms of gradient orientations. Usually multiscale detection is attained by resizing the image several times and by recomputing the image features or using multiple classifiers for different scales. In this paper we present a pedestrian detection approach that uses the same classifier for all pedestrian scales based on image features computed for a single scale. We go beyond the low level pixel-wise gradient orientation bins and use higher level visual words organized into Word Channels. Boosting is used to learn classification features from the integral Word Channels. The proposed approach is evaluated on multiple datasets and achieves outstanding results on the INRIA and Caltech-USA benchmarks. By using a GPU implementation we achieve a classification rate of over 10 million bounding boxes per second and a 16 FPS rate for multiscale detection in a 640×480 image.

We describe new techniques to detect and analyze periodic mo t on as seen from both a static and moving camera. By tracking objects of interest, we compute a n object’s self-similarity as it evolves in time. For periodic motion, the self-similarity measure is a l o periodic, and we apply Time-Frequency analysis to detect and characterize the periodic motion. Th e periodicity is also analyzed robustly using the 2-D lattice structures inherent in similarity matrices . A real-time system has been implemented to track and classify objects using periodicity. Examples of o bject classification (people, running dogs, vehicles), person counting, and non-stationary periodici ty are provided.

Pedestrian detection will play a major role in future driver assistance and autonomous driving. One powerful algorithm in this field uses HOG features to describe the specific properties of pedestrians in images. To determine their locations, features are extracted and classified window-wise from different scales of an input image. The results of the classification are finally merged to remove overlapping detections. The real-time execution of this method requires specific FPGA- or ASIC-architectures. Recent work focused on accelerating the feature extraction and classification. Although merging is an important step in the algorithm, it is only rarely considered in hardware implementations. A reason for that could be its complexity and irregularity that is not trivial to implement in hardware. In this paper, we present a new bottom-up FPGA architecture that maps the full HOG-based algorithm for pedestrian detection including feature extraction, SVM classification, and multi-scale processing in combination with merging. For that purpose, we also propose a new hardware-optimized merging method. The resulting architecture is highly efficient. Additionally, we present an FPGA-based full real-time and multi-scale pedestrian detection demonstration system.


Question: Real-time implementation of semi-global matching on the central processing unit.

This paper introduces a comprehensive analysis and classification of dense, two-frame stereo correspondence algorithms, a critical area within computer vision. Despite the proliferation of stereo matching algorithms, there has been a notable scarcity of research dedicated to systematically evaluating their performance. To address this gap, we propose a taxonomy designed to dissect and compare the distinct elements and design choices that underpin individual stereo algorithms. This taxonomy facilitates a detailed comparison of existing methods and enables the assessment of numerous variants through a series of rigorous experiments. To foster a unified software framework and a standardized set of data for evaluation, we have developed a standalone, adaptable C++ implementation. This platform supports the evaluation of individual components and is easily expandable to accommodate new algorithms. Additionally, we have created several multi-frame stereo data sets with ground truth labels, which are now publicly accessible. Accompanying our taxonomy is a thorough comparative evaluation of a wide selection of state-of-the-art stereo algorithms, showcasing their relative strengths and weaknesses.

Local space-time features have recently become a popular video representation for action recognition. Several methods for feature localization and description have been proposed in the literature, and promising recognition results were demonstrated for different action datasets. The comparison of those methods, however, is limited given the different experimental settings and various recognition methods used. The purpose of this paper is first to define a common evaluation setup to compare local space-time detectors and descriptors. All experiments are reported for the same bag-of-features SVM recognition framework. Second, we provide a systematic evaluation of different spatio-temporal features. We evaluate the performance of several space-time interest point detectors and descriptors along with their combinations on datasets with varying degree of difficulty. We also include a comparison with dense features obtained by regular sampling of local space-time patches. Feature detectors. In our experimental evaluation, we consider the following feature detectors. (1) The Harris3D detector [3] extends the Harris detector for images to image sequences. At each video point, a spatio-temporal second-moment matrix μ is computed using a separable Gaussian smoothing function and space-time gradients. Interest points are located at local maxima of H = det(μ)− k trace3(μ). (2) The Cuboid detector [1] is based on temporal Gabor filters. The response function has the form: R = (I ∗ g ∗ hev) + (I ∗ g ∗ hod), where g(x,y;σ) is the 2D Gaussian smoothing kernel, and hev and hod are 1D Gabor filters. Interest points are detected at local maxima of R. (3) The Hessian detector [6] is a spatio-temporal extension of the Hessian saliency measure. The determinant of the 3D Hessian matrix is used to measure saliency. The determinant of the Hessian is computed over several spatial and temporal scales. A non-maximum suppression algorithm selects extrema as interest points. (4) Dense sampling extracts multi-scale video blocks at regular positions in space and time and for varying scales. In our experiments, we sample cuboids with 50% spatial and temporal overlap. Feature descriptors. The following feature descriptors are investigated. (1) For the Cuboid descriptor [1], gradients computed for each pixel in a cuboid region are concatenated into a single vector. PCA projects vectors to a lower dimensional space. (2) The HOG/HOF descriptors [4] divide a cuboid region into a grid of cells. For each cell, 4-bin histograms of gradient orientations (HOG) and 5-bin histograms of optic flow (HOF) are computed. Normalized histograms are concatenated into HOG, HOF as well as HOG/HOF descriptor vectors. (3) The HOG3D descriptor [2] is based on histograms of 3D gradient orientations. Gradients are computed via an integral video representations. Regular polyhedrons are used to uniformly quantize the orientation of spatio-temporal gradients. A given 3D volume is divided into a grid of cells. The corresponding descriptor concatenates gradient histograms of all cells. (4) The extended SURF (ESURF) descriptor [6] extends the image SURF descriptor to videos. Again 3D cuboids are divided into a grid of cells. Each cell is represented by a weighted sum of uniformly sampled responses of Haar-wavelets aligned with the three axes. Experimental Setup. We represent video sequences as a bag of local spatio-temporal features [5]. Spatio-temporal features are first quantized HOG3D HOG/HOF HOG HOF Cuboids ESURF Harris3D 89.0% 91.8% 80.9% 92.1% – – Cuboids 90.0% 88.7% 82.3% 88.2% 89.1% – Hessian 84.6% 88.7% 7767% 88.6% – 81.4% Dense 85.3% 86.1% 79.0% 88.0% – – Table 1: Average accuracy on the KTH actions dataset.

This paper presents an approach to textureless object detection and tracking of the 3D pose. Our detection and tracking schemes are coherently integrated in a particle filtering framework on the special Euclidean group, SE(3), in which the visual tracking problem is tackled by maintaining multiple hypotheses of the object pose. For textureless object detection, an efficient chamfer matching is employed so that a set of coarse pose hypotheses is estimated from the matching between 2D edge templates of an object and a query image. Particles are then initialized from the coarse pose hypotheses by randomly drawing based on costs of the matching. To ensure the initialized particles are at or close to the global optimum, an annealing process is performed after the initialization. While a standard edge-based tracking is employed after the annealed initialization, we employ a refinement process to establish improved correspondences between projected edge points from the object model and edge points from an input image. Comparative results for several image sequences with clutter are shown to validate the effectiveness of our approach.

We present an efficient method for volume rendering by raycasting on the CPU. We employ coherent packet traversal of an implicit bounding volume hierarchy, heuristically pruned using preintegrated transfer functions, to exploit empty or homogeneous space. We also detail SIMD optimizations for volumetric integration, trilinear interpolation, and gradient lighting. The resulting system performs well on low-end and laptop hardware, and can outperform out-of-core GPU methods by orders of magnitude when rendering large volumes without level-of-detail (LOD) on a workstation. We show that, while slower than GPU methods for low-resolution volumes, an optimized CPU renderer does not require LOD to achieve interactive performance on large data sets.

We present a new method for describing images for the purposes of matching and registration. We take the point of view that large, coherent regions in the image provide a concise and stable basis for image description. We develop a new algorithm for feature detection that operates on several projections (feature spaces) of the image using kernel-based optimization techniques to locate local extrema of a continuous scale-space of image regions. Descriptors of these image regions and their relative geometry then form the basis of an image description. The emphasis of the work is on features that summarize image content and are highly robust to viewpoint changes and occlusion yet remain discriminative for matching and registration. We present experimental results of these methods applied to the problem of image retrieval. We find that our method performs comparably to two published techniques: Blobworld and SIFT features. However, compared to these techniques two significant advantages of our method are its (1) stability under large changes in the images and (2) its representational efficiency. 2008 Elsevier Inc. All rights reserved.

A real time static isolated gesture recognition application using a Hidden Markov Model approach with features extracted from gestures' silhouettes is presented. Nine different hand poses with various degrees of rotation are considered. The system, both simple and effective, uses color images of the hands to be recognized directly from the camera and is capable of processing 23 frames per second on a Quad Core Intel Processor.

This paper proposes an accurate and dense wide-baseline stereo matching method using Scaled Window Phase-Only Correlation (SW-POC). The wide-baseline setting of the stereo camera can improve the accuracy of the 3D reconstruction compared with the short-baseline setting. However, it is difficult to find accurate and dense correspondence from wide-baseline stereo images due to its large perspective distortion. Addressing this problem, we employ the SW-POC, which is a correspondence matching method using 1D POC with the concept of Scale Window Matching (SWM). The use of SW-POC makes it possible to find the accurate and dense correspondence from a wide-baseline stereo image pair with low computational cost. We also apply the proposed method to 3D reconstruction using a moving and uncalibrated consumer digital camera.

We propose a multi-resolution framework inspired by human visual search for general object detection. Different resolutions are represented using a coarse-to-fine feature hierarchy. During detection, the lower resolution features are initially used to reject the majority of negative windows at relatively low cost, leaving a relatively small number of windows to be processed in higher resolutions. This enables the use of computationally more expensive higher resolution features to achieve high detection accuracy. We applied this framework on Histograms of Oriented Gradient (HOG) features for object detection. Our multi-resolution detector produced better performance for pedestrian detection than state-of-the-art methods (Dalal and Triggs, 2005), and was faster during both training and testing. Testing our method on motorbikes and cars from the VOC database revealed similar improvements in both speed and accuracy, suggesting that our approach is suitable for realtime general object detection applications.

We present an energy-based approach to visual odometry from RGB-D images of a Microsoft Kinect camera. To this end we propose an energy function which aims at finding the best rigid body motion to map one RGB-D image into another one, assuming a static scene filmed by a moving camera. We then propose a linearization of the energy function which leads to a 6×6 normal equation for the twist coordinates representing the rigid body motion. To allow for larger motions, we solve this equation in a coarse-to-fine scheme. Extensive quantitative analysis on recently proposed benchmark datasets shows that the proposed solution is faster than a state-of-the-art implementation of the iterative closest point (ICP) algorithm by two orders of magnitude. While ICP is more robust to large camera motion, the proposed method gives better results in the regime of small displacements which are often the case in camera tracking applications.

In today’s advancing market, the growing performance and decreasing price of embedded processors are opening many doors for developers to design highly sophisticated solutions for different end applications. The complexities of these systems can create bottlenecks for developers in the form of longer development times, more complicated development environments and issues with application stability and quality. Developers can address these problems using sophisticated software packages such as OpenCV, but migrating this software to embedded platforms poses its own set of challenges. This paper will review how to mitigate some of these issues, including C++ implementation, memory constraints, floating-point support and opportunities to maximize performance using vendor-optimized libraries and integrated accelerators or co-processors. Finally, we will introduce a new effort by Texas Instruments (TI) to optimize vision systems by running OpenCV on the C6000TM digital signal processor (DSP) architecture. Benchmarks will show the advantage of using the DSP by comparing the performance of a DSP+ARM® system-on-chip (SoC) processor against an ARM-only device. OpenCV on TI’s DSP+ARM platforms: Mitigating the challenges of porting OpenCV to embedded platforms Introduction OpenCV is a free and open-source computer vision library that offers a broad range of functionality under the permissive Berkeley Software Distribution (BSD) license. The library itself is written in C++ and is also usable through C or Python language applications. Thousands of developers use OpenCV to power their own specialized applications, making it the most widely-used library of its kind. The OpenCV project is under active development, with regular updates to eliminate bugs and add new functionality. The mainline development effort targets the x86 architecture and supports acceleration via Intel’s proprietary Integrated Performance Primitives (IPP) library. A recent release also added support for graphics processing unit (GPU) acceleration using NVIDIA’s Compute Unifi ed Device Architecture (CUDA) standard. OpenCV’s greatest asset is the sheer breadth of algorithms included in its standard distribution. Figure 1 on page 2 shows an incomplete list of some of the key function categories included in OpenCV. These range from low-level image fi ltering and transformation to sophisticated feature analysis and machine learning functionality. A complete listing of every function and use case is beyond the scope of this article, but we will consider the unique requirements of developers in the embedded vision space. For these developers, OpenCV represents an attractively comprehensive toolbox of useful, well-tested algorithms that can serve as building blocks for their own specialized applications. The question then becomes whether or not OpenCV can be used directly in their embedded systems. Despite its original development focus for use with PC workstations, OpenCV can also be a useful tool for embedded development. There are vendor-specifi c libraries that offer OpenCV-like capabilities on various embedded systems, but few can match OpenCV’s ubiquity in the computer vision fi eld or the sheer breadth of its included algorithms. It should come as no surprise that OpenCV has already been ported to the ARM® architecture, a popular CPU choice for embedded processors. It’s certainly possible to cross-compile the OpenCV source code as-is and use the result with embedded devices, but memory constraints and other architectural considerations may pose a problem. This white paper will examine some of the specifi c obstacles that must be overcome for OpenCV to achieve acceptable performance on Joseph Coombs, Applications engineering, Texas Instruments Rahul Prabhu, Applications engineering, Texas Instruments W H I T E P A P E R

This paper describes the design and implementation of IRIS Performer, a toolkit for visual simulation, virtual reality, and other real-time 3D graphics applications. The principal design goal is to allow application developers to more easily obtain maximal performance from 3D graphics workstations which feature multiple CPUs and support an immediate-mode rendering library. To this end, the toolkit combines a low-level library for high-performance rendering with a high-level library that implements pipelined, parallel traversals of a hierarchical scene graph. While discussing the toolkit architecture, the paper illuminates and addresses performance issues fundamental to immediate-mode graphics and coarse-grained, pipelined multiprocessing. Graphics optimizations focus on efficient data transfer to the graphics subsystem, reduction of mode settings, and restricting state inheritance. The toolkit's multiprocessing features solve the problems of how to partition work among multiple processes, how to synchronize these processes, and how to manage data in a pipelined, multiprocessing environment. The paper also discusses support for intersection detection, fixed-frame rates, run-time profiling and special effects such as geometric morphing.


Question: Regarding the significance of lexical and world knowledge in the third Recognizing Textual Entailment competition.

The Berkeley FrameNet Initiative: A Corpus-Based Approach to Computational Lexicography

The Berkeley FrameNet project represents a significant undertaking in the realm of computational lexicography, funded by the National Science Foundation (NSF) and currently in its second year (Grant: NSF IRI-9618838, "Tools for Lexicon Building"). The project's methodology is distinguished by two key aspects: reliance on corpus evidence to inform semantic and syntactic generalizations, and the utilization of frame semantics to represent the valences of its focal words, predominantly nouns, adjectives, and verbs. The resulting database will encompass detailed descriptions of the semantic frames underlying the meanings of these lexical items, accompanied by valence representations that integrate both semantic and syntactic aspects. Each lexical item will be associated with a collection of annotated corpus attestations, illustrating the observed connections between "frame elements" and their syntactic manifestations, such as grammatical function, phrase type, and other syntactic attributes.

This report aims to outline the project's objectives, workflow, and the computational tools that have been either adapted or developed in-house to support this endeavor. The Berkeley FrameNet project is housed at the International Computer Science Institute (1947 Center Street, Berkeley, CA). The main corpus utilized in this project is the British National Corpus, accessed courtesy of Oxford University Press, and the POS-tagged and lemmatized version is courtesy of the Institut für Maschinelle Sprachverarbeitung of the University of Stuttgart.

The project's core activity involves the human encoding of semantic knowledge in a machine-readable format, guided and constrained by corpus-based research facilitated by sophisticated software tools. The semantic domains targeted include HEALTH CARE, CHANCE, PERCEPTION, COMMUNICATION, TRANSACTION, TIME, SPACE, BODY (including body parts and functions), MOTION, LIFE STAGES, SOCIAL CONTEXT, EMOTION, and COGNITION.

The project's deliverables comprise (a) a lexical resource, the FrameNet database, and (b) accompanying software tools. The FrameNet database is structured around three principal components: a lexicon with entries that include conventional dictionary information for human readers, formulas that encapsulate the morphosyntactic realizations of frame elements within phrases or sentences constructed around the word, and links to semantically annotated examples. The database is envisioned to contain a minimum of 5,000 lexical entries, complemented by a parallel annotated corpus, all formatted for seamless integration into applications that utilize other lexical resources such as WordNet and COM

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while ‘multi-sense’ methods have been proposed and tested on artificial wordsimilarity tasks, we don’t know if they improve real natural language understanding tasks. In this paper we introduce a multisense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language un-

Since their inception, distributional models of semantics have been criticized as inadequate cognitive theories of human semantic learning and representation. A principal challenge is that the representations derived by distributional models are purely symbolic and are not grounded in perception and action; this challenge has led many to favor feature-based models of semantic representation. We argue that the amount of perceptual and other semantic information that can be learned from purely distributional statistics has been underappreciated. We compare the representations of three feature-based and nine distributional models using a semantic clustering task. Several distributional models demonstrated semantic clustering comparable with clustering-based on feature-based representations. Furthermore, when trained on child-directed speech, the same distributional models perform as well as sensorimotor-based feature representations of children's lexical semantic knowledge. These results suggest that, to a large extent, information relevant for extracting semantic categories is redundantly coded in perceptual and linguistic experience. Detailed analyses of the semantic clusters of the feature-based and distributional models also reveal that the models make use of complementary cues to semantic organization from the two data streams. Rather than conceptualizing feature-based and distributional models as competing theories, we argue that future focus should be on understanding the cognitive mechanisms humans use to integrate the two sources.

The aim of this paper is to introduce LexIt, a computational framework for the automatic acquisition and exploration of distributional information about Italian verbs, nouns and adjectives, freely available through a web interface at the address http://sesia.humnet.unipi.it/lexit. LexIt is the first large-scale resource for Italian in which subcategorization and semantic selection properties are characterized fully on distributional ground: in the paper we describe both the process of data extraction and the evaluation of the subcategorization frames extracted with LexIt.

Identifying discourse relations that are not overtly marked with discourse connectives remains a challenging problem. The absence of explicit clues indicates a need for the combination of world knowledge and weak contextual clues, which can hardly be learned from a small amount of manually annotated data. In this paper, we address this problem by augmenting the input text with external knowledge and context and by adopting a neural network model that can effectively handle the augmented text. Experiments show that external knowledge did improve the classification accuracy. On the other hand, contextual information provided no significant gain for implicit discourse relations while it worked for explicit ones.

In this paper, we describe our system submitted for the semantic textual similarity (STS) task at SemEval 2012. We implemented two approaches to calculate the degree of similarity between two sentences. First approach combines corpus-based semantic relatedness measure over the whole sentence with the knowledge-based semantic similarity scores obtained for the words falling under the same syntactic roles in both the sentences. We fed all these scores as features to machine learning models to obtain a single score giving the degree of similarity of the sentences. Linear Regression and Bagging models were used for this purpose. We used Explicit Semantic Analysis (ESA) as the corpus-based semantic relatedness measure. For the knowledgebased semantic similarity between words, a modified WordNet based Lin measure was used. Second approach uses a bipartite based method over the WordNet based Lin measure, without any modification. This paper shows a significant improvement in calculating the semantic similarity between sentences by the fusion of the knowledge-based similarity measure and the corpus-based relatedness measure against corpus based measure taken alone.

The reading comprehension task, that asks questions about a given evidence document, is a central problem in natural language understanding. Recent formulations of this task have typically focused on answer selection from a set of candidates pre-defined manually or through the use of an external NLP pipeline. However, Rajpurkar et al. (2016) recently released the SQUAD dataset in which the answers can be arbitrary strings from the supplied text. In this paper, we focus on this answer extraction task, presenting a novel model architecture that efficiently builds fixed length representations of all spans in the evidence document with a recurrent network. We show that scoring explicit span representations significantly improves performance over other approaches that factor the prediction into separate predictions about words or start and end markers. Our approach improves upon the best published results of Wang & Jiang (2016) by 5% and decreases the error of Rajpurkar et al.’s baseline by > 50%.

Type-level word embeddings use the same set of parameters to represent all instances of a word regardless of its context, ignoring the inherent lexical ambiguity in language. Instead, we embed semantic concepts (or synsets) as defined in WordNet and represent a word token in a particular context by estimating a distribution over relevant semantic concepts. We use the new, context-sensitive embeddings in a model for predicting prepositional phrase (PP) attachments and jointly learn the concept embeddings and model parameters. We show that using context-sensitive embeddings improves the accuracy of the PP attachment model by 5.4% absolute points, which amounts to a 34.4% relative reduction in errors.

Deep neural networks for machine comprehension typically utilizes only word or character embeddings without explicitly taking advantage of structured linguistic information such as constituency trees and dependency trees. In this paper, we propose structural embedding of syntactic trees (SEST), an algorithm framework to utilize structured information and encode them into vector representations that can boost the performance of algorithms for the machine comprehension. We evaluate our approach using a state-of-the-art neural attention model on the SQuAD dataset. Experimental results demonstrate that our model can accurately identify the syntactic boundaries of the sentences and extract answers that are syntactically coherent over the baseline methods.

We present our winning solution for the WSDM Cup 2017 triple scoring task. We devise an ensemble of four base scorers, so as to leverage the power of both text and knowledge bases for that task. Then we further refine the outputs of the ensemble by trigger word detection, achieving even better predictive accuracy. The code is available at https://github.com/wsdm-cup-2017/bokchoy.

Compounding is a highly productive word-formation process in some languages that is often problematic for natural language processing applications. In this paper, we investigate whether distributional semantics in the form of word embeddings can enable a deeper, i.e., more knowledge-rich, processing of compounds than the standard string-based methods. We present an unsupervised approach that exploits regularities in the semantic vector space (based on analogies such as “bookshop is to shop as bookshelf is to shelf”) to produce compound analyses of high quality. A subsequent compound splitting algorithm based on these analyses is highly effective, particularly for ambiguous compounds. German to English machine translation experiments show that this semantic analogy-based compound splitter leads to better translations than a commonly used frequency-based method.


Question: Regarding the significance of lexical and world knowledge in the RTE3 task.

The met* Approach: A Computational Technique for Differentiating Metonymy and Metaphor
The met* approach is a computational technique designed to distinguish metonymy from metaphor, as well as from literalness and anomaly, within short English sentences. In this method, literalness is identified by its compliance with contextual constraints, which are typically violated by nonliteral expressions. The met* approach uniquely differentiates metonymy from metaphor and anomaly in a manner that aligns with Lakoff and Johnson's (1980) perspective, where metonymy involves one entity representing another, whereas metaphor involves viewing one entity as another. This approach also accommodates chains of metonymies, as suggested by Reddy (1979), and allows for the co-occurrence of metonymies with literalness, metaphor, or anomaly. Moreover, it distinguishes metaphor from anomaly by the presence of a relevant analogy in the former, which is absent in the latter. The met* approach is an integral part of Collative Semantics, a semantic framework for natural language processing, and has been operationalized in a program named meta5. The paper provides examples of meta5's analysis of metaphor and metonymy, and contrasts the met* approach with methodologies from artificial intelligence, linguistics, philosophy, and psychology.

We present a novel approach to recognizing Textual Entailment. Structural features are constructed from abstract tree descriptions, which are automatically extracted from syntactic dependency trees. These features are then applied in a subsequence-kernel-based classifier to learn whether an entailment relation holds between two texts. Our method makes use of machine learning techniques using a limited data set, no external knowledge bases (e.g. WordNet), and no handcrafted inference rules. We achieve an accuracy of 74.5% for text pairs in the Information Extraction and Question Answering task, 63.6% for the RTE-2 test data, and 66.9% for the RET-3 test data.

a r t i c l e i n f o a b s t r a c t We present an automatic approach to the construction of BabelNet, a very large, wide-coverage multilingual semantic network. Key to our approach is the integration of lexicographic and encyclopedic knowledge from WordNet and Wikipedia. In addition, Machine Translation is applied to enrich the resource with lexical information for all languages. We first conduct in vitro experiments on new and existing gold-standard datasets to show the high quality and coverage of BabelNet. We then show that our lexical resource can be used successfully to perform both monolingual and cross-lingual Word Sense Disambiguation: thanks to its wide lexical coverage and novel semantic relations, we are able to achieve state-of the-art results on three different SemEval evaluation tasks.

A major opportunity for NLP to have a realworld impact is in helping educators score student writing, particularly content-based writing (i.e., the task of automated short answer scoring). A major challenge in this enterprise is that scored responses to a particular question (i.e., labeled data) are valuable for modeling but limited in quantity. Additional information from the scoring guidelines for humans, such as exemplars for each score level and descriptions of key concepts, can also be used. Here, we explore methods for integrating scoring guidelines and labeled responses, and we find that stacked generalization (Wolpert, 1992) improves performance, especially for small training sets.

This paper presents a computational study of part of the lexical-acquisition task faced by children, namely the acquisition of word-to-meaning mappings. It first approximates this task as a formal mathematical problem. It then presents an implemented algorithm for solving this problem, illustrating its operation on a small example. This algorithm offers one precise interpretation of the intuitive notions of cross-situational learning and the principle of contrast applied between words in an utterance. It robustly learns a homonymous lexicon despite noisy multi-word input, in the presence of referential uncertainty, with no prior knowledge that is specific to the language being learned. Computational simulations demonstrate the robustness of this algorithm and illustrate how algorithms based on cross-situational learning and the principle of contrast might be able to solve lexical-acquisition problems of the size faced by children, under weak, worst-case assumptions about the type and quantity of data available.

This paper presents an analysis of existing methods for the intrinsic evaluation of word embeddings. We show that the main methodological premise of such evaluations is “interpretability” of word embeddings: a “good” embedding produces results that make sense in terms of traditional linguistic categories. This approach is not only of limited practical use, but also fails to do justice to the strengths of distributional meaning representations. We argue for a shift from abstract ratings of word embedding “quality” to exploration of their strengths and weaknesses.

Vector-space word representations have been very successful in recent years at improving performance across a variety of NLP tasks. However, common to most existing work, words are regarded as independent entities without any explicit relationship among morphologically related words being modeled. As a result, rare and complex words are often poorly estimated, and all unknown words are represented in a rather crude way using only one or a few vectors. This paper addresses this shortcoming by proposing a novel model that is capable of building representations for morphologically complex words from their morphemes. We combine recursive neural networks (RNNs), where each morpheme is a basic unit, with neural language models (NLMs) to consider contextual information in learning morphologicallyaware word representations. Our learned models outperform existing word representations by a good margin on word similarity tasks across many datasets, including a new dataset we introduce focused on rare words to complement existing ones in an interesting way.

In this paper we introduce the TTCS system, so named after Terms To Conceptual Spaces, that exploits a resource-driven approach relying on BabelNet, NASARI and ConceptNet. TTCS takes in input a term and its context of usage and produces as output a specific type of vector-based semantic representation, where conceptual information is encoded through the Conceptual Spaces (a geometric framework for common-sense knowledge representation and reasoning). The system has been evaluated in a twofold experimentation. In the first case we assessed the quality of the extracted common-sense conceptual information with respect to human judgments with an online questionnaire. In the second one we compared the performances of a conceptual categorization system that was run twice, once fed with extracted annotations and once with hand-crafted annotations. In both cases the results are encouraging and provide precious insights to make substantial improvements.

Distributional representations of words have been recently used in supervised settings for recognizing lexical inference relations between word pairs, such as hypernymy and entailment. We investigate a collection of these state-of-the-art methods, and show that they do not actually learn a relation between two words. Instead, they learn an independent property of a single word in the pair: whether that word is a “prototypical hypernym”.

QA systems have been making steady advances in the challenging elementary science exam domain. In this work, we develop an explanation-based analysis of knowledge and inference requirements, which supports a fine-grained characterization of the challenges. In particular, we model the requirements based on appropriate sources of evidence to be used for the QA task. We create requirements by first identifying suitable sentences in a knowledge base that support the correct answer, then use these to build explanations, filling in any necessary missing information. These explanations are used to create a fine-grained categorization of the requirements. Using these requirements, we compare a retrieval and an inference solver on 212 questions. The analysis validates the gains of the inference solver, demonstrating that it answers more questions requiring complex inference, while also providing insights into the relative strengths of the solvers and knowledge sources. We release the annotated questions and explanations as a resource with broad utility for science exam QA, including determining knowledge base construction targets, as well as supporting information aggregation in automated inference.

We describe a neural network model that jointly learns distributed representations of texts and knowledge base (KB) entities. Given a text in the KB, we train our proposed model to predict entities that are relevant to the text. Our model is designed to be generic with the ability to address various NLP tasks with ease. We train the model using a large corpus of texts and their entity annotations extracted from Wikipedia. We evaluated the model on three important NLP tasks (i.e., sentence textual similarity, entity linking, and factoid question answering) involving both unsupervised and supervised settings. As a result, we achieved state-of-the-art results on all three of these tasks. Our code and trained models are publicly available for further academic research.


Question: Regarding the significance of lexical and world knowledge in the RTE3 task.

This paper explores an innovative method for identifying inference rules to bridge the gap between question formulations and textual information in question-answering systems. A significant hurdle in this field is the discrepancy between the language used in questions and that found in text, a challenge humans navigate using intuitive inference rules such as "X writes Y" leading to "X is the author of Y." However, these rules are typically absent from automated question-answering systems due to the complexity of their creation. Herein, we introduce an unsupervised algorithm designed to uncover such inference rules from textual data. Our methodology is grounded in an augmented version of Harris's Distributional Hypothesis, which posits that words sharing similar contexts are likely to have similar meanings. Rather than applying this hypothesis to individual words, we extend its application to paths within the dependency trees derived from a parsed corpus. Our premise is that paths linking the same set of words are indicative of analogous meanings. Through illustrative examples, we demonstrate the algorithm's efficacy in discovering a multitude of inference rules that might otherwise escape human notice.

Discourse relations bind smaller linguistic units into coherent texts. Automatically identifying discourse relations is difficult, because it requires understanding the semantics of the linked arguments. A more subtle challenge is that it is not enough to represent the meaning of each argument of a discourse relation, because the relation may depend on links between lowerlevel components, such as entity mentions. Our solution computes distributed meaning representations for each discourse argument by composition up the syntactic parse tree. We also perform a downward compositional pass to capture the meaning of coreferent entity mentions. Implicit discourse relations are then predicted from these two representations, obtaining substantial improvements on the Penn Discourse Treebank.

Distributional representations of words have been recently used in supervised settings for recognizing lexical inference relations between word pairs, such as hypernymy and entailment. We investigate a collection of these state-of-the-art methods, and show that they do not actually learn a relation between two words. Instead, they learn an independent property of a single word in the pair: whether that word is a “prototypical hypernym”.

With the rise of social media, learning from informal text has become increasingly important. We present a novel semantic lexicon induction approach that is able to learn new vocabulary from social media. Our method is robust to the idiosyncrasies of informal and open-domain text corpora. Unlike previous work, it does not impose restrictions on the lexical features of candidate terms – e.g. by restricting entries to nouns or noun phrases – while still being able to accurately learn multiword phrases of variable length. Starting with a few seed terms for a semantic category, our method first explores the context around seed terms in a corpus, and identifies context patterns that are relevant to the category. These patterns are used to extract candidate terms – i.e. multiword segments that are further analyzed to ensure meaningful term boundary segmentation. We show that our approach is able to learn high quality semantic lexicons from informally written social media text of Twitter, and can achieve accuracy as high as 92% in the top 100 learned category members.

The mathematical representation of semantics is a key issue for Natural Language Processing (NLP). A lot of research has been devoted to finding ways of representing the semantics of individual words in vector spaces. Distributional approaches—meaning distributed representations that exploit co-occurrence statistics of large corpora—have proved popular and successful across a number of tasks. However, natural language usually comes in structures beyond the word level, with meaning arising not only from the individual words but also the structure they are contained in at the phrasal or sentential level. Modelling the compositional process by which the meaning of an utterance arises from the meaning of its parts is an equally fundamental task of NLP. This dissertation explores methods for learning distributed semantic representations and models for composing these into representations for larger linguistic units. Our underlying hypothesis is that neural models are a suitable vehicle for learning semantically rich representations and that such representations in turn are suitable vehicles for solving important tasks in natural language processing. The contribution of this thesis is a thorough evaluation of our hypothesis, as part of which we introduce several new approaches to representation learning and compositional semantics, as well as multiple state-of-the-art models which apply distributed semantic representations to various tasks in NLP. Part I focuses on distributed representations and their application. In particular, in Chapter 3 we explore the semantic usefulness of distributed representations by evaluating their use in the task of semantic frame identification. Part II describes the transition from semantic representations for words to compositional semantics. Chapter 4 covers the relevant literature in this field. Following this, Chapter 5 investigates the role of syntax in semantic composition. For this, we discuss a series of neural network-based models and learning mechanisms, and demonstrate how syntactic information can be incorporated into semantic composition. This study allows us to establish the effectiveness of syntactic information as a guiding parameter for semantic composition, and answer questions about the link between syntax and semantics. Following these discoveries regarding the role of syntax, Chapter 6 investigates whether it is possible to further reduce the impact of monolingual surface forms and syntax when attempting to capture semantics. Asking how machines can best approximate human signals of semantics, we propose multilingual information as one method for grounding semantics, and develop an extension to the distributional hypothesis for multilingual representations. Finally, Part III summarizes our findings and discusses future work.

This study compared the nature of text comprehension as measured by multiple-choice format and open-ended format questions. Participants read a short text while explaining preselected sentences. After reading the text, participants answered open-ended and multiple-choice versions of the same questions based on their memory of the text content. The results indicated that performance on open-ended questions was correlated with the quality of self-explanations, but performance on multiple-choice questions was correlated with the level of prior knowledge related to the text. These results suggest that open-ended and multiple-choice format questions measure different aspects of comprehension processes. The results are discussed in terms of dual process theories of text comprehension.

Several recent approaches use distributional similarity for making symbolic reasoning more flexible. While an important step in the right direction, the use of similarity has a number of inherent limitations. We argue that similarity-based reasoning should be complemented with commonsense reasoning patterns such as interpolation and a fortiori inference. We show how the required background knowledge for these inference patterns can be obtained from distributional models.

We introduce an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge (WSC), a common sense reasoning task that requires diverse, complex forms of inference and knowledge. Our method uses a knowledge hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine, then extracts and classifies knowledge from the returned results and weighs them to make a resolution. Our approach improves F1 performance on the full WSC by 0.21 over the previous best and represents the first system to exceed 0.5 F1. We further demonstrate that the approach is competitive on the Choice of Plausible Alternatives (COPA) task, which suggests that it is generally applicable.

Processing language requires the retrieval of concepts from memory in response to an ongoing stream of information. This retrieval is facilitated if one can infer the gist of a sentence, conversation, or document and use that gist to predict related concepts and disambiguate words. This article analyzes the abstract computational problem underlying the extraction and use of gist, formulating this problem as a rational statistical inference. This leads to a novel approach to semantic representation in which word meanings are represented in terms of a set of probabilistic topics. The topic model performs well in predicting word association and the effects of semantic association and ambiguity on a variety of language-processing and memory tasks. It also provides a foundation for developing more richly structured statistical models of language, as the generative process assumed in the topic model can easily be extended to incorporate other kinds of semantic and syntactic structure.

In recent years, there has been renewed interest in the NLP community in genuine language understanding and dialogue. Thus the long-standing issue of how the semantic content of language should be represented is reentering the communal discussion. This paper provides a brief “opinionated survey” of broadcoverage semantic representation (SR). It suggests multiple desiderata for such representations, and then outlines more than a dozen approaches to SR–some longstanding, and some more recent, providing quick characterizations, pros, cons, and some comments on imple-

Most Web-based Q/A systems work by finding pages that contain an explicit answer to a question. These systems are helpless if the answer has to be inferred from multiple sentences, possibly on different pages. To solve this problem, we introduce the HOLMES system, which utilizes textual inference (TI) over tuples extracted from text. Whereas previous work on TI (e.g., the literature on textual entailment) has been applied to paragraph-sized texts, HOLMES utilizes knowledge-based model construction to scale TI to a corpus of 117 million Web pages. Given only a few minutes, HOLMES doubles recall for example queries in three disparate domains (geography, business, and nutrition). Importantly, HOLMES’s runtime is linear in the size of its input corpus due to a surprising property of many textual relations in the Web corpus—they are “approximately” functional in a well-defined sense.


Question: Regarding the significance of lexical and world knowledge in the RTE3 task.

This paper delves into the significance and extensive application of WordNet, an electronic lexical database that has become a cornerstone for natural language processing systems. The book, "WordNet: An Electronic Lexical Database," edited by Christiane Fellbaum, explores the theoretical and historical foundations of WordNet's design, offers a current overview of the lexical database, and illustrates its practical uses across various domains. The book comprises a foreword by George Miller, an introduction penned by Fellbaum, seven chapters authored by researchers from Princeton University's Cognitive Sciences Laboratory, where WordNet was developed, and nine additional chapters contributed by scholars from other institutions. Miller's foreword provides an engaging narrative of WordNet's origins, detailing the foundational principles guiding the creation of the database, the determination of top-level noun categories, and the origins of the words incorporated into WordNet. It also chronicles WordNet's transformation from a rudimentary dictionary interface to a comprehensive lexicon, highlighting the contributions of key individuals throughout its decade-long evolution. This account is a captivating read for both casual and dedicated users of WordNet, as well as anyone appreciative of the resource's existence. Structured in three sections, the book's first part centers on WordNet itself, encompassing four chapters: "Nouns in WordNet" by George Miller, "Modifiers in WordNet" by Katherine Miller, "A semantic network of English verbs" by Christiane Fellbaum, and "Design and implementation of the WordNet lexical database and search software" by Randee Tengi. These chapters essentially represent revised versions of four articles from Miller (1990), with a greater emphasis on the underlying principles and rationales informing the design choices. While the information about WordNet's contents is not as exhaustive as in Miller's earlier work, this book introduces new elements, such as an explanation of sense grouping in George Miller's chapter, a segment on adverbs in Katherine Miller's chapter, observations on autohyponymy (a word's sense being a hyponym of another of its senses) and autoantonymy (a word's sense being an antonym of another of its senses) in Fellbaum's chapter, and Tengi's description of the Grinder, a software tool that transforms the lexicographers' files into searchable databases. The three papers in Part II are designated as "extensions, enhancements, and

We propose a framework for discriminative Information Retrieval (IR) atop linguistic features, trained to improve the recall of tasks such as answer candidate passage retrieval, the initial step in text-based Question Answering (QA). We formalize this as an instance of linear feature-based IR (Metzler and Croft, 2007), illustrating how a variety of knowledge discovery tasks are captured under this approach, leading to a 44% improvement in recall for candidate

We describe an inventory of semantic relations that are expressed by prepositions. We define these relations by building on the word sense disambiguation task for prepositions and propose a mapping from preposition senses to the relation labels by collapsing semantically related senses across prepositions.

Since the experiments of Saffran et al. [Saffran, J., Aslin, R., & Newport, E. (1996). Statistical learning in 8-month-old infants. Science, 274, 1926-1928], there has been a great deal of interest in the question of how statistical regularities in the speech stream might be used by infants to begin to identify individual words. In this work, we use computational modeling to explore the effects of different assumptions the learner might make regarding the nature of words--in particular, how these assumptions affect the kinds of words that are segmented from a corpus of transcribed child-directed speech. We develop several models within a Bayesian ideal observer framework, and use them to examine the consequences of assuming either that words are independent units, or units that help to predict other units. We show through empirical and theoretical results that the assumption of independence causes the learner to undersegment the corpus, with many two- and three-word sequences (e.g. what's that, do you, in the house) misidentified as individual words. In contrast, when the learner assumes that words are predictive, the resulting segmentation is far more accurate. These results indicate that taking context into account is important for a statistical word segmentation strategy to be successful, and raise the possibility that even young infants may be able to exploit more subtle statistical patterns than have usually been considered.

Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation models. We show that translation-based embeddings outperform those learned by cutting-edge monolingual models at single-language tasks requiring knowledge of conceptual similarity and/or syntactic role. The findings suggest that, while monolingual models learn information about how concepts are related, neural-translation models better capture their true ontological status. It is well known that word representations can be learned from the distributional patterns in corpora. Originally, such representations were constructed by counting word co-occurrences, so that the features in one word’s representation corresponded to other words [11, 17]. Neural language models, an alternative means to learn word representations, use language data to optimise (latent) features with respect to a language modelling objective. The objective can be to predict either the next word given the initial words of a sentence [4, 14, 8], or simply a nearby word given a single cue word [13, 15]. The representations learned by neural models (sometimes called embeddings) generally outperform those acquired by co-occurrence counting models when applied to NLP tasks [3]. Despite these clear results, it is not well understood how the architecture of neural models affects the information encoded in their embeddings. Here, we explore this question by considering the embeddings learned by architectures with a very different objective function to monolingual language models: neural machine translation models. We show that translation-based embeddings outperform monolingual embeddings on two types of task: those that require knowledge of conceptual similarity (rather than simply association or relatedness), and those that require knowledge of syntactic role. We discuss what the findings indicate about the information content of different embeddings, and suggest how this content might emerge as a consequence of the translation objective. 1 Learning embeddings from language data Both neural language models and translation models learn real-valued embeddings (of specified dimension) for words in some pre-specified vocabulary, V , covering many or all words in their training corpus. At each training step, a ‘score’ for the current training example (or batch) is computed based on the embeddings in their current state. This score is compared to the model’s objective function, and the error is backpropagated to update both the model weights (affecting how the score is computed from the embeddings) and the embedding features. At the end of this process, the embeddings should encode information that enables the model to optimally satisfy its objective. 1.1 Monolingual models In the original neural language model [4] and subsequent variants [8], each training example consists of n subsequent words, of which the model is trained to predict the n-th word given the first n − 1 ar X iv :1 41 0. 07 18 v2 [ cs .C L ] 1 3 N ov 2 01 4 1 words. The model first represents the input as an ordered sequence of embeddings, which it transforms into a single fixed length ‘hidden’ representation by, e.g., concatenation and non-linear projection. Based on this representation, a probability distribution is computed over the vocabulary, from which the model can sample a guess at the next word. The model weights and embeddings are updated to maximise the probability of correct guesses for all sentences in the training corpus. More recent work has shown that high quality word embeddings can be learned via models with no nonlinear hidden layer [13, 15]. Given a single word in the corpus, these models simply predict which other words will occur nearby. For each word w in V , a list of training cases (w, c) : c ∈ V is extracted from the training corpus. For instance, in the skipgram approach [13], for each ‘cue word’ w the ‘context words’ c are sampled from windows either side of tokens of w in the corpus (with c more likely to be sampled if it occurs closer to w).1 For each w in V , the model initialises both a cue-embedding, representing the w when it occurs as a cue-word, and a context-embedding, used when w occurs as a context-word. For a cue word w, the model can use the corresponding cueembedding and all context-embeddings to compute a probability distribution over V that reflects the probability of a word occurring in the context of w. When a training example (w, c) is observed, the model updates both the cue-word embedding of w and the context-word embeddings in order to increase the conditional probability of c. 1.2 Translation-based embeddings Neural translation models generate an appropriate sentence in their target language St given a sentence Ss in their source language [see, e.g., 16, 6]. In doing so, they learn distinct sets of embeddings for the vocabularies Vs and Vt in the source and target languages respectively. Observing a training case (Ss, St), such a model represents Ss as an ordered sequence of embeddings of words from Vs. The sequence for Ss is then encoded into a single representation RS .2 Finally, by referencing the embeddings in Vt, RS and a representation of what has been generated thus far, the model decodes a sentence in the target language word by word. If at any stage the decoded word does not match the corresponding word in the training target St, the error is recorded. The weights and embeddings in the model, which together parameterise the encoding and decoding process, are updated based on the accumulated error once the sentence decoding is complete. Although neural translation models can differ in low-level architecture [7, 2], the translation objective exerts similar pressure on the embeddings in all cases. The source language embeddings must be such that the model can combine them to form single representations for ordered sequences of multiple words (which in turn must enable the decoding process). The target language embeddings must facilitate the process of decoding these representations into correct target-language sentences. 2 Comparing Mono-lingual and Translation-based Embeddings To learn translation-based embeddings, we trained both the RNN encoder-decoder [RNNenc, 7] and the RNN Search architectures [2] on a 300m word corpus of English-French sentence pairs. We conducted all experiments with the resulting (English) source embeddings from these models. For comparison, we trained a monolingual skipgram model [13] and its Glove variant [15] for the same number of epochs on the English half of the bilingual corpus. We also extracted embeddings from a full-sentence language model [CW, 8] trained for several months on a larger 1bn word corpus. As in previous studies [1, 5, 3], we evaluate embeddings by calculating pairwise (cosine) distances and correlating these distances with (gold-standard) human judgements. Table 1 shows the correlations of different model embeddings with three such gold-standard resources, WordSim-353 [1], MEN [5] and SimLex-999 [10]. Interestingly, translation embeddings perform best on SimLex-999, while the two sets of monolingual embeddings perform better on modelling the MEN and WordSim353. To interpret these results, it should be noted that SimLex-999 evaluation quantifies conceptual similarity (dog wolf ), whereas MEN and WordSim-353 (despite its name) quantify more general relatedness (dog collar) [10]. The results seem to indicate that translation-based embeddings better capture similarity, while monolingual embeddings better capture relatedness. 1 Subsequent variants use different algorithms for selecting the (w, c) from the training corpus [9, 12] Alternatively, subsequences (phrases) of Ss may be encoded at this stage in place of the whole sentence [2].

Current research in natural language generation is situated in a computational linguistics tradition that was founded several decades ago. We critically analyse some of the architectural assumptions underlying existing systems and point out some problems in the domains of text planning and lexicalization. Guided by the identification of major generation challenges viewed from the angles of knowledge-based systems and cognitive psychology, we sketch some new directions for future research.

We present the components of the ColumbiaGWU contribution to the 2016 TAC KBP BeSt Evaluation.

Experimentol corroboration was obtained for the hypothesis that the proportion of words common to the contexts of word A and to the contexts of word B is a function of the degree to which A and B are similar in meaning. The tests were carried out for variously defined contexts. The shapes of the functions, however, indicate that similarity of context is reliable as criterion only for detecting pairs of words that are very similar in meaning.

Humans learn language by grounding word meaning in the physical world. Recent efforts in natural language processing have attempted to model such multimodal learning by incorporating visual information into word and sentence representations. Here, we explore the task of grounding lexical color descriptions in their visual referents. We propose an RNN-based autoencoder model to learn vector representations of sentences that reflect their associated color values. Our model effectively learns a joint visual-lexical space that demonstrates compositionality and generalizes to unseen color names. As a demonstration of such a space learned, we show that our model can predict captions from color representations and color representations from captions. In addition to successfully modeling color language, this work provides a novel framework for grounded language learning.

We introduce an extended naive Bayes model for word sense induction (WSI) and apply it to a WSI task. The extended model incorporates the idea the words closer to the target word are more relevant in predicting its sense. The proposed model is very simple yet effective when evaluated on SemEval-2010 WSI data.

Many Knowledge Bases (KBs) are now readily available and encompass colossal quantities of information thanks to either a long-term funding effort (e.g. WordNet, OpenCyc) or a collaborative process (e.g. Freebase, DBpedia). However, each of them is based on a different rigid symbolic framework which makes it hard to use their data in other systems. It is unfortunate because such rich structured knowledge might lead to a huge leap forward in many other areas of AI like natural language processing (word-sense disambiguation, natural language understanding, ...), vision (scene classification, image semantic annotation, ...) or collaborative filtering. In this paper, we present a learning process based on an innovative neural network architecture designed to embed any of these symbolic representations into a more flexible continuous vector space in which the original knowledge is kept and enhanced. These learnt embeddings would allow data from any KB to be easily used in recent machine learning methods for prediction and information retrieval. We illustrate our method on WordNet and Freebase and also present a way to adapt it to knowledge extraction from raw text.


Question: Regarding the significance of lexical and world knowledge in the RTE3 task.

This paper introduces an innovative method for efficiently clustering large, high-dimensional datasets, a challenge often encountered in various domains including reference matching. Traditionally, clustering techniques are effective when datasets are limited in terms of the number of clusters, feature dimensionality, or the volume of data points. However, clustering becomes significantly more complex when datasets are vast in all three aspects, such as those with millions of data points, thousands of dimensions, and numerous clusters. Our proposed approach tackles this issue by employing an inexpensive, approximate distance measure to divide the data into overlapping subsets, termed "canopies." This division facilitates clustering by only requiring exact distance calculations between points within the same canopy. By utilizing canopies, previously intractable clustering problems become feasible, with no compromise in accuracy under reasonable assumptions about the approximate distance metric. This method is versatile, applicable across different domains, and compatible with various clustering algorithms such as Greedy Agglomerative Clustering, K-means, and Expectation-Maximization. We validate the effectiveness of the canopy approach through experimental results on the clustering of bibliographic citations from research papers' reference sections. Compared to traditional clustering methods, the canopy approach achieves a reduction in computational time by over an order of magnitude and a 25% decrease in error rate relative to previously used algorithms.

Humans learn language by grounding word meaning in the physical world. Recent efforts in natural language processing have attempted to model such multimodal learning by incorporating visual information into word and sentence representations. Here, we explore the task of grounding lexical color descriptions in their visual referents. We propose an RNN-based autoencoder model to learn vector representations of sentences that reflect their associated color values. Our model effectively learns a joint visual-lexical space that demonstrates compositionality and generalizes to unseen color names. As a demonstration of such a space learned, we show that our model can predict captions from color representations and color representations from captions. In addition to successfully modeling color language, this work provides a novel framework for grounded language learning.

In this work we present a novel approach for the utilization of observed relations between entity pairs in the task of triple argument prediction. The approach is based on representing observations in a shared, continuous vector space of structured relations and text. Results on a recent benchmark dataset demonstrate that the new model is superior to existing sparse feature models. In combination with state-of-the-art models, we achieve substantial improvements when observed relations are available.

We introduce an extended naive Bayes model for word sense induction (WSI) and apply it to a WSI task. The extended model incorporates the idea the words closer to the target word are more relevant in predicting its sense. The proposed model is very simple yet effective when evaluated on SemEval-2010 WSI data.

A number of studies have shown that second language learners acquire vocabulary through reading, but only relatively small amounts. However, most of these studies used only short texts, measured only the acquisition of meaning, and did not credit partial learning of words. This case study of a learner of French explores whether an extensive reading program can enhance lexical knowledge. The study assessed a relatively large number of words (133), and examined whether one month of extensive reading enhanced knowledge of these target words' spelling, meaning, and grammatical characteristics. The measurement procedure was a one-on-one interview that allowed a very good indication of whether learning occurred. The study also explores how vocabulary acquisition varies according to how often words are encountered in the texts. The results showed that knowledge of 65% of the target words was enhanced in some way, for a pickup rate of about 1 of every 1.5 words tested. Spelling was strongly enhanced, even from a small number of exposures. Meaning and grammatical knowledge were also enhanced, but not to the same extent. Overall, the study indicates that more vocabulary acquisition is possible from extensive reading than previous studies have suggested.

We present a novel approach to recognizing Textual Entailment. Structural features are constructed from abstract tree descriptions, which are automatically extracted from syntactic dependency trees. These features are then applied in a subsequence-kernel-based classifier to learn whether an entailment relation holds between two texts. Our method makes use of machine learning techniques using a limited data set, no external knowledge bases (e.g. WordNet), and no handcrafted inference rules. We achieve an accuracy of 74.5% for text pairs in the Information Extraction and Question Answering task, 63.6% for the RTE-2 test data, and 66.9% for the RET-3 test data.

Bootstrapping semantics from text is one of the greatest challenges in natural language learning. We first define a word similarity measure based on the distributional pattern of words. The similarity measure allows us to construct a thesaurus using a parsed corpus. We then present a new evaluation methodology for the automatically constructed thesaurus. The evaluation results show that the thesaurus is significantly closer to WordNet than Roget Thesaurus is.

We develop latent Dirichlet allocation with WORDNET (LDAWN), an unsupervised probabilistic topic model that includes word sense as a hidden variable. We develop a probabilistic posterior inference algorithm for simultaneously disambiguating a corpus and learning the domains in which to consider each word. Using the WORDNET hierarchy, we embed the construction of Abney and Light (1999) in the topic model and show that automatically learned domains improve WSD accuracy compared to alternative contexts.

The goal of lexical pragmatics is to explain how linguistically specified (‘literal’) word meanings are modified in use. While lexical-pragmatic processes such as narrowing, broadening and metaphorical extension are generally studied in isolation from each other, relevance theorists (Carston 2002, Wilson & Sperber 2002) have been arguing for a unified approach. I will continue this work by underlining some of the problems with more standard treatments, and show how a variety of lexicalpragmatic processes may be analysed as special cases of a general pragmatic adjustment process which applies spontaneously, automatically and unconsciously to fine-tune the interpretation of virtually every word.

Filling the gap between natural language expressions and ontology concepts or properties is the new trend in Semantic Web. Ontology lexicalization introduces a new layer of lexical information for ontology properties and concepts. We propose a method based on unsupervised learning for the extraction of the potential lexical expressions of DBpedia propertiesfrom Wikipedia text corpus. It is a resource-driven approach that comprises three main steps. The first step consists of the extraction of DBpedia triples for the aimed property followed by the extraction of Wikipedia articles describing the resources from these triples. In the second step, sentences mostly related to the property are extracted from the articles and they are analyzed with a Semantic Role Labeler resulting in a set of SRL annotated trees. In the last step, clusters of expressions are built using spectral clustering based on the distances between the SRL trees. The clusters with the least variance are considered to be relevant for the lexical expressions of the property.

We introduce an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge (WSC), a common sense reasoning task that requires diverse, complex forms of inference and knowledge. Our method uses a knowledge hunting module to gather text from the web, which serves as evidence for candidate problem resolutions. Given an input problem, our system generates relevant queries to send to a search engine, then extracts and classifies knowledge from the returned results and weighs them to make a resolution. Our approach improves F1 performance on the full WSC by 0.21 over the previous best and represents the first system to exceed 0.5 F1. We further demonstrate that the approach is competitive on the Choice of Plausible Alternatives (COPA) task, which suggests that it is generally applicable.


Question: A single-stage, single-switch, soft-switching power-factor-correction LED driver design.

This paper introduces a single-stage (S2) parallel AC-to-DC converter design, centered around a single-switch two-output boost-flyback converter. The converter architecture is composed of two semi-stages: the boost-flyback semi-stage, which efficiently transfers a portion of the input power directly to the load through a single power flow path. This semi-stage exhibits a notable self-power factor correction feature when operating in discontinuous conduction mode, even when the boost output is nearly equivalent to the peak line voltage. The second semi-stage is a flyback DC-to-DC (DC/DC) converter, which provides output regulation via a parallel power flow path. This innovative design results in enhanced power conversion efficiency and reduced current stress on the control switch. The paper also delves into the power distribution calculation, bulk capacitor voltage, and provides design equations and a step-by-step procedure for determining key parameters. To validate the design, an 80 W prototype converter was constructed and tested. The experimental outcomes confirm that the line harmonic current at the most challenging condition adheres to the IEC61000-3-2 class D limits, with a maximum bulk capacitor voltage of approximately 415.4 V and a peak efficiency of about 85.8%. These results underscore the suitability of the proposed S2 converter for universal input applications.

This work demonstrates the design of an adaptive reconfigurable rectifier to address the issue of early breakdown voltage in a conventional rectifier and extends the rectifier's operation for wide dynamic input power range. A depletion-mode field-effect transistor has been introduced to operate as a switch and compensate at low and high input power levels for rectifier. This design accomplishes 40% of RF-DC power conversion efficiency over a wide dynamic input power range from -10 dBm to 27 dBm, while exhibiting 78% of peak power efficiency at 22 dBm. The power harvester is designed to operate in the 900 MHz ISM band and suitable for Wireless Power Transfer applications.

This paper describes the design of a compact and highly repetitive pulsed power modulator based on semiconductor switches. Output specifications of the proposed modulator are as follows: variable output pulse voltage, 1-10 kV; width, 1-10 μs; pulse repetition rate (PRR), 1-50,000 pps; and average output power, 10 kW. The proposed solid-state pulsed power modulator has two main parts: an IGBT stack that hasa gate drive circuit for the pulse output with a variable pulse width and repetition rate, and a capacitor charger that has a controllable output voltage. To connect all the storage capacitors in a series for high voltage pulses, a simple and reliable IGBT stack structure was proposed based on the Marx generator. In addition, the gate driver circuit, which supplies power and signals to all the IGBTs simultaneously, was introduced. This providesa superior protection function against the arc and the short. To charge the storage capacitors, a novel 10 kW (10 kV, 1 A) high voltage capacitor charger with the combined advantage of a series-loaded resonant converter and a ZVS (zero-voltage-switching) full-bridge pulse width modulation (PWM) converter was proposed and designed especially for the solid-state pulsed power modulator. Theexperiment results verified that the proposed scheme and structure can be used effectively for a high voltage pulsed power modulator that requires variable voltage, repetition rate, and pulse width, depending on the process of the applications.

This paper presents a new topology of higher order power filter for grid-tied voltage-source inverters, named the LLCL filter, which inserts a small inductor in the branch loop of the capacitor in the traditional LCL filter to compose a series resonant circuit at the switching frequency. Particularly, it can attenuate the switching-frequency current ripple components much better than an LCL filter, leading to a decrease in the total inductance and volume. Furthermore, by decreasing the inductance of a grid-side inductor, it raises the characteristic resonance frequency, which is beneficial to the inverter system control. The parameter design criteria of the proposed LLCL filter is also introduced. The comparative analysis and discussions regarding the traditional LCL filter and the proposed LLCL filter have been presented and evaluated through experiment on a 1.8-kW-single-phase grid-tied inverter prototype.

A new multiple-loop design technique for high-performance low-dropout (LDO) regulator designs has been proposed and successfully implemented in many commercial products for portable smart phone and tablet PC applications. The proposed LDO is composed of five loops that allows designers to obtain a good tradeoff between quiescent current and other performances, such as undershoot, overshoot, and so on. A total of one bandgap reference and 38 LDOs (including n-type and p-type LDOs, which will be named NLDO and PLDO, respectively) were integrated in one power-management IC chip for powering an application processor inside mobile devices. The proposed LDO has been fabricated based on 0.13-<inline-formula> <tex-math notation="LaTeX">$\mu {m}$ </tex-math></inline-formula> CMOS process and supplies various current capacities from 50 to 600 mA; 38 LDOs have been designed and supply different output voltage levels from 0.7 to 3.0 V. One of the proposed NLDOs consumes <inline-formula> <tex-math notation="LaTeX">$14~\mu {A}$ </tex-math></inline-formula> of the quiescent current and features under 56/24 mV of undershoot/overshoot at VOUT =1<italic>V</italic> as the load current steps up from 0 to 300 mA with 300 mA/1 <inline-formula> <tex-math notation="LaTeX">$\mu {s}$ </tex-math></inline-formula> on a 1-<inline-formula> <tex-math notation="LaTeX">$\mu {F}$ </tex-math></inline-formula> output capacitor. The measured output load and line regulations are 1.8 and 0.4 mV, respectively. The measured integrated output noise from 10 Hz to 100 kHz at <inline-formula> <tex-math notation="LaTeX">$\text{I}_{\mathrm{ LOAD}}=10\%$ </tex-math></inline-formula> of maximum current shows 80 <inline-formula> <tex-math notation="LaTeX">$\mu {V}$ </tex-math></inline-formula> rms. The package chip size is <inline-formula> <tex-math notation="LaTeX">$6.25 \times 6.25$ </tex-math></inline-formula> mm2 with 169 balls.

A new isolated current-fed pulsewidth modulation dc-dc converter-current-fed dual-bridge dc-dc converter-with small inductance and no deadtime operation is presented and analyzed. The new topology has more than 3times smaller inductance than that of current-fed full-bridge converter, thus having faster transient response speed. Other characteristics include simple self-driven synchronous rectification, simple housekeeping power supply, and smaller output filter capacitance. Detailed analysis shows the proposed converter can have either lower voltage stress on all primary side power switches or soft switching properties when different driving schemes are applied. A 48-V/125-W prototype dc-dc converter with dual output has been tested for the verification of the principles. Both simulations and experiments verify the feasibility and advantages of the new topology

Conventional cascade buck-boost PFC (CBB-PFC) converter suffers from the high conduction loss in the input rectifier bridge. To resolve the above problem, a novel bridgeless buck-boost PFC topology is proposed in this paper. The proposed PFC converter which removes the input rectifier bridge has three conduction semiconductors at every moment. Comparing with CBB-PFC topology, the proposed topology reduces the conduction semiconductors, reduces conduction losses effectively, improves the efficiency of converter and is suitable for use in the wide input voltage range. In this paper, the average current mode control was implemented with UC3854, the theoretical analysis and design of detection circuits was presented. The experimental prototype with 400 V/600 W output and line input voltage range from 220 VAC to 380 VAC was built. Experimental results show that the proposed converter can improve 0.8% efficiency comparing CBB-PFC converter.

Synchronous rectification offers an effective solution to improve efficiency of low-output-voltage flyback converters. Conventional synchronous flyback topologies using discrete synchronous rectifier face challenges in precise timing control on the secondary side. In this paper, MHz-frequency operation of a flyback DC-DC converter using a new monolithic self-synchronized rectifier (SSR) is investigated and demonstrated. The SSR IC, acting solely on its own drain-source voltage instead of external control signals, considerably simplifies converter design, improves system efficiency, and enables MHz operating frequency. Analysis, modeling, and experimental results are presented.

Induction heating is a well-known technique to produce very high temperature for applications. A large number of topologies have been developed in this area such as voltage and current source inverter. This project presents the analysis and design of a new ac–ac resonant converter applied to domestic induction heating. The proposed topology, based on the halfbridge series resonant inverter, uses only two diodes to rectify the mains voltage.The proposed converter can operate with zerovoltage switching during both switches-on and switch-off transitions. This topology doubles the output voltage, and therefore, the current in the load is reduced for the same output power. As a consequence, the converter efficiency is significantly improved.The output power is controlled using the PWM technique. This results in an increase of the net efficiency of the induction heating system. The circuit is simulated using MATLAB. The circuit is implemented using PIC controller. Both simulation and hardware results are compared.

Multi-phase converters have become a topic of great interest due to the high output power capacity and output ripple cancellation effect. They are even more beneficial to nowadays high-frequency fully integrated converters with output capacitor integrated on-chip. As one of the dominant chip area consumers, reducing the size of the on-chip decoupling capacitors directly leads to cost down. It is reported that a 5× capacitor area reduction can be achieved with a four-phase converter compared to a single-phase one [1]. However, the penalty is obvious. Every extra phase comes with an inductor, which is also counted as cost and becomes more dominant with increase in the number of phases.

Analysis methods are developed that fully determine a switched-capacitor (SC) dc-dc converter's steady-state performance through evaluation of its output impedance. This analysis method has been verified through simulation and experimentation. The simple formulation developed permits optimization of the capacitor sizes to meet a constraint such as a total capacitance or total energy storage limit, and also permits optimization of the switch sizes subject to constraints on total switch conductances or total switch volt-ampere (V-A) products. These optimizations then permit comparison among several switched-capacitor topologies, and comparisons of SC converters with conventional magnetic-based dc-dc converter circuits, in the context of various application settings. Significantly, the performance (based on conduction loss) of a ladder-type converter is found to be superior to that of a conventional magnetic-based converter for medium to high conversion ratios.