Question: Consensus recommendations for using calcium hydroxylapatite in jawline rejuvenation procedures.

This paper presents a radiographic method for assessing the aging effects on the midfacial fat compartments and compares the anatomical differences between younger and older human cadavers. The primary objective of facial rejuvenation is to restore a youthful volume distribution. To achieve this, the study utilized computed tomography (CT) scans from 12 nonfixed cadaver heads, categorizing them into two age groups: one ranging from 54 to 75 years (group 1, n = 6), and the other from 75 to 104 years (group 2, n = 6). The analysis focused on measuring the sagittal diameter of the upper, middle, and lower thirds of each fat compartment to evaluate volume distribution, as well as the distance between the cephalad border of the fat compartments and the infraorbital rim to assess compartment "sagging."

The CT scans provided a clear, reproducible depiction of the facial fat compartments, revealing distinct aging patterns. In group 2 (older cadavers), the distance between the fat compartments and the infraorbital rim was notably increased compared to group 1 (younger cadavers). Additionally, the lower thirds of the compartments showed an increased sagittal diameter, whereas the upper thirds displayed a reduced diameter. The study also identified the buccal extension of the buccal fat pad as a separate, independent compartment.

The findings indicate that aging leads to a downward migration of the midfacial fat compartments and a volume shift within these compartments. Furthermore, specific compartmental changes, such as volume loss in the deep medial cheek fat and the distinct buccal fat pad extension, contribute to the characteristic appearance of an aged face. These insights are crucial for developing targeted facial rejuvenation strategies that address the underlying anatomical changes associated with aging.

BACKGROUND
The purposes of this study were to determine the morphological features and conceptualize the anatomical definition of the angular artery (AA) as an aid to practical operations in the clinical field.

MATERIALS AND METHODS
Thirty-one hemifaces from 17 Korean cadavers and 26 hemifaces from 13 Thai cadavers were dissected.

RESULTS
The topography of the AA was classified into 4 types according to its course: Type I (persistent pattern), in which the AA traverses the lateral side of the nose (11%); Type II (detouring pattern), in which the AA traverses the cheek and tear trough area (18%); Type III (alternative pattern), in which the AA traverses the medial canthal area through a branch of the ophthalmic artery (22.8%); and Type IV (latent pattern), in which the AA is absent (26.3%).

CONCLUSION
The findings of this study will contribute toward improved outcomes for cosmetic surgery involving the injection of facial filler by enhancing the understanding of AA anatomy.

The esthetic application of botulinum toxin type A (Botox) is a safe treatment modality; nevertheless complications can occur as a result of patient-and physician-related factors. Fortunately, adverse effects and undesirable sequelae after Botox injections are temporary. Complications may be more serious in patients who have more severe rhytids (which require more Botox), previous facial plastic surgery (altered anatomy), and those who have pre-existing neuromuscular disease. The physician can reduce complications by using proper injection techniques, appropriate regional Botox dosing, and by being conservative in the overall approach to Botox-mediated facial rejuvenation.


Question: Consensus recommendations for using calcium hydroxylapatite in jawline rejuvenation procedures.

A standardized evaluation system for aging features in the lower face region has been developed and validated in this study. The lower face is particularly susceptible to aging, characterized by the appearance of lines, wrinkles, downturned mouth corners, and alterations in lip volume and shape, accompanied by the sagging of the jawline skin. To address these changes, there is a need for precise, user-friendly, and validated scales for clinical research and practice.

The primary aim of this research was to establish the reliability of eight specific scales designed to evaluate nasolabial folds, marionette lines, upper and lower lip fullness, lip wrinkles (both at rest and dynamic), the oral commissure and jawline, aesthetic regions, and the lower face as a whole.

The methodology involved the creation of four 5-point rating scales to objectively measure upper and lower lip wrinkles, oral commissures, and the jawline. A panel of twelve experts independently assessed 50 identical lower face photographs using these eight 5-point scales across two separate rating sessions. The consistency of the ratings, both between and within raters, was then analyzed.

The findings revealed that inter-rater reliability was substantial to near-perfect for all lower face scales, aesthetic areas, and the lower face unit as a whole. Similarly, high levels of intra-rater reliability were observed for all scales, regions, and the lower face unit.

In conclusion, these rating scales have proven to be reliable instruments for accurately and consistently assessing the aging process in the lower face areas. This standardized assessment system offers a valuable tool for both clinical research and patient care in evaluating the severity of aging features in the lower face.

The esthetic application of botulinum toxin type A (Botox) is a safe treatment modality; nevertheless complications can occur as a result of patient-and physician-related factors. Fortunately, adverse effects and undesirable sequelae after Botox injections are temporary. Complications may be more serious in patients who have more severe rhytids (which require more Botox), previous facial plastic surgery (altered anatomy), and those who have pre-existing neuromuscular disease. The physician can reduce complications by using proper injection techniques, appropriate regional Botox dosing, and by being conservative in the overall approach to Botox-mediated facial rejuvenation.

OBJECTIVES
The objectives of this study were to describe and compare two techniques used to correct nasal septum deviations located in the dorsal and/or caudal septum.

STUDY DESIGN
The authors conducted a retrospective clinical chart review.

METHODS
The authors conducted a comparison of functional and technical results between surgery in the L-strut of the septum in 114 patients with septal battens or by septal replacement by subjective self-evaluation and by examination of the position of the septum during follow up.

RESULTS
There was subjective improvement in nasal breathing in 86% of the septal batten group and in 94% of the septal replacement group. This difference was not statistically significant. The technical result was judged by examining the position of the septum during follow up as midline, slightly deviated, or severely deviated. The septum was significantly more often located in the midline during follow up in the septal replacement group than in the septal batten group.

CONCLUSION
Treatment of deformities located in the structurally important L-strut of the septum may be technically challenging and many functional, structural, and esthetic considerations must be taken into account. On the basis of this series, both septal battens and septal replacement techniques may be considered for correction of deviations in this area. The functional improvement rates were not significantly different between the techniques, although during follow up, the septum appeared to be significantly more often located in the midline in the septal replacement group. The techniques are described and their respective advantages and potential drawbacks are discussed.

Of all the craniofacial abnormalities, facial clefts are the most disfiguring. Facial clefts are classified according to the affected anatomical area as described by Tessier. Through this classification, the location and extent of the cleft can be designated numerically.A 2-month-old male infant was referred to authors' craniofacial unit, from a hospital in a rural province of South Africa, with a problem of a supranasal encephalocoele. Bilateral raised eyebrows were noted as was a right-sided upper lid central third coloboma. Computed tomography and magnetic resonance imaging scans confirmed the presence of a supranasal encephalocoele with a large frontal bone defect and splayed nasal bones. Bilateral enlarged orbits were noted with tented orbital roofs that we classified as Tessier number 10 facial clefts. The child was booked for an encephalocoele excision and calvarial reconstruction at 4 months of age.As a result of the encephalocoele, the supraorbital bar with its adjacent nasal bones was cleaved in 2, resulting in a significant frontal bone defect. Osteotomies were performed to remove the supraorbital bar and nasal bones from the calvarium. The supraorbital bar segment was remodeled and plated with absorbable poly-L-lactic acid plates. Osteotomies of the nasal bones allowed them to be united centrally, also with absorbable plates. This entire construct was transferred and secured to the calvarium, but in a more caudal position thereby obliterating the frontal bone and Tessier number 10 facial cleft defects with a naturally contoured construct.

The banana fold, or the infragluteal fold, is a fat deposit on the posterior thigh close to the gluteal crease and parallel to it. A banana fold may form for different reasons, among which an iatrogenic cause is recurrent. Although banana fold is a common problem, unrelished by most women, few procedures are targeted specifically to fight it. This report presents a severe case of iatrogenic banana fold corrected by a modification of the dermotuberal anchorage buttock-lifting technique, as reported by the author for gluteal ptosis. The operation is performed by tucking in part of the banana fold tissue caudal to the gluteal crease, sliding that tissue, after depithelization, under the buttock, and pulling it up toward the ischial tuberosity until the redundant skin on the posterior thigh is tight and the banana fold is reduced. Assessment of the results 1 year after surgery showed that the technique provided a good scar kept within the subgluteal crease, and that it satisfactorily corrected the patient’s major complaint: the banana-shaped fold.

Cryolipolysis has become a popular non-invasive treatment for unwanted fatty collections. The benefits are minimal down time, short recovery time, and decreased pain, although pain is the most frequent complaint. The author recently treated a patient who presented with an enlarging adipose collection on his lower abdomen following cryolipolysis with CoolSculpting(®) (ZELTIQ Aesthetics, Inc., Pleasanton, CA).

BACKGROUND
Embolia cutis medicamentosa (ECM) is a rare phenomenon attributed to intra-arterial drug injection. Glabellar filler injections can result in potentially devastating visual loss from inadvertent retrograde arteriolar embolization due to the extensive vasculature within the upper face. The minimum amount of filler necessary to potentiate this complication has not yet been reported.

OBJECTIVES
We aim to determine the volume of filler necessary to occupy the supratrochlear artery from the glabella to the bifurcation of the ophthalmic and central retinal arteries. We specifically examine the volume of the supratrochlear artery from the glabella to orbital apex.

METHODS
The study was approved by Duke University Institutional Review Board and involved surgical dissection of six fresh tissue cadaver heads (12 hemifaces). The arterial system in each cadaver head was injected with latex for visualization. The supratrochlear arteries were isolated anteriorly from the glabella to the orbital apex posteriorly. Intra-orbital vessel radius and length were measured. The vessel volume was calculated by water displacement of the intra-arterial latex.

RESULTS
The vessel volumes ranged from 0.04 to 0.12 mL. The average vessel volume was calculated as 0.085 mL, the average length as 51.75 mm, and the average radius as 0.72 mm.

CONCLUSIONS
Vascular occlusion from filler injections can lead to devastating visual consequences due to inadvertent retrograde intra-arterial embolization. Our findings indicate that the average entire volume of the supratrochlear artery from the glabella to the orbital apex is 0.085 mL. Injectors should be aware that a bolus of this critical volume may lead to a significant adverse outcome.

BACKGROUND
The purpose of this study was to identify the environmental factors that contribute to facial aging in identical twins.

METHODS
During the Twins Day Festival in Twinsburg, Ohio, 186 pairs of identical twins completed a comprehensive questionnaire, and digital images were obtained. A panel reviewed the images independently and recorded the differences in the perceived twins' ages and their facial features. The perceived age differences were then correlated with multiple factors.

RESULTS
Four-point higher body mass index was associated with an older appearance in twins younger than age 40 but resulted in a younger appearance after age 40 (p = 0.0001). Eight-point higher body mass index was associated with an older appearance in twins younger than age 55 but was associated with a younger appearance after age 55 (p = 0.0001). The longer the twins smoked, the older they appeared (p < 0.0001). Increased sun exposure was associated with an older appearance and accelerated with age (p = 0.015), as was a history of outdoor activities and lack of sunscreen use. Twins who used hormone replacement had a younger appearance (p = 0.002). Facial rhytids were more evident in twins with a history of skin cancer (p = 0.05) and in those who smoked (p = 0.005). Dark and patchy skin discoloration was less prevalent in twins with a higher body mass index (p = 0.01) and more common in twins with a history of smoking (p = 0.005) and those with sun exposure (p = 0.005). Hair quantity was better with a higher body mass index (p = 0.01) although worse with a history of skin cancer (p = 0.005) and better with the use of hormones (p = 0.05).

CONCLUSION
This study offers strong statistical evidence to support the role of some of the known factors that govern facial aging.


Question: Generating adversarial texts utilizing gradient techniques.

This paper presents a novel global log-bilinear regression model designed for word representation, named Global Vectors (GloVe). Unlike previous models that have struggled to elucidate the underlying mechanisms for capturing semantic and syntactic regularities through vector arithmetic, GloVe clarifies the model properties necessary for these patterns to manifest in word vectors. By synthesizing the strengths of the two predominant model categories—global matrix factorization and local context window methods—GloVe offers a fresh perspective on word representation learning. The key innovation lies in its efficient utilization of statistical data by focusing training solely on the non-zero elements within a word-word co-occurrence matrix. This targeted approach sidesteps the need for processing the entire sparse matrix or individual context windows across a vast corpus. The resultant vector space exhibits meaningful substructure, as evidenced by its impressive performance of 75% on a recent word analogy task. Moreover, GloVe surpasses competing models in tasks related to word similarity and named entity recognition, showcasing its efficacy and versatility in natural language processing applications.

Automated scoring of written and spoken responses is an NLP application that can significantly impact lives especially when deployed as part of high-stakes tests such as the GRE® and the TOEFL®. Ethical considerations require that automated scoring algorithms treat all testtakers fairly. The educational measurement community has done significant research on fairness in assessments and automated scoring systems must incorporate their recommendations. The best way to do that is by making available automated, non-proprietary tools to NLP researchers that directly incorporate these recommendations and generate the analyses needed to help identify and resolve biases in their scoring systems. In this paper, we attempt to provide such a solution.

Automatic image description systems typically produce generic sentences that only make use of a small subset of the vocabulary available to them. In this paper, we consider the production of generic descriptions as a lack of diversity in the output, which we quantify using established metrics and two new metrics that frame image description as a word recall task. This framing allows us to evaluate system performance on the head of the vocabulary, as well as on the long tail, where system performance degrades. We use these metrics to examine the diversity of the sentences generated by nine state-of-the-art systems on the MS COCO data set. We find that the systems trained with maximum likelihood objectives produce less diverse output than those trained with additional adversarial objectives. However, the adversarially-trained models only produce more types from the head of the vocabulary and not the tail. Besides vocabulary-based methods, we also look at the compositional capacity of the systems, specifically their ability to create compound nouns and prepositional phrases of different lengths. We conclude that there is still much room for improvement, and offer a toolkit to measure progress towards the goal of generating more diverse image descriptions.

Previous neural machine translation models used some heuristic search algorithms (e.g., beam search) in order to avoid solving the maximum a posteriori problem over translation sentences at test phase. In this paper, we propose the GumbelGreedy Decoding which trains a generative network to predict translation under a trained model. We solve such a problem using the Gumbel-Softmax reparameterization, which makes our generative network differentiable and trainable through standard stochastic gradient methods. We empirically demonstrate that our proposed model is effective for generating sequences of discrete words.

We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.

Reiter and Dale’s book is about natural language generation (NLG), and describes methods for building systems of this kind. It offers a comprehensive overview of the field of NLG, from a perspective that has rarely been described elsewhere. It is very valuable for people with little or even no knowledge about NLG particulars who intend to investigate applications where NLG technology might play a certain role, as well as for people who want to obtain an application-oriented overview of the field. In this sense, all communities intended to be served by the book—students, academics in related areas, and software developers—are well addressed. In addition, some sections provide material that is probably new even to researchers with considerable experience in the field, such as the discussions on corpus analysis and domain modeling. In the introduction, the field of NLG is briefly characterized from researchand application-oriented perspectives and is illustrated by screen shots produced by several systems. Then conditions for effective uses of this technology are elaborated and contrasted with conditions where other techniques are more appropriate, and methods for determining the intended functionality of a system to be built are discussed. The main sections of the book are devoted to the prototypical architecture of applicationoriented NLG systems and their major processing phases: document planning, microplanning, and surface realization. Each of these three phases is illustrated by a number of detailed examples, demonstrating the successive refinements of utterance specifications in the course of processing. In the final section, the embedding of natural language processing technology is discussed, featuring typography, combined uses with graphics and hypertext, and integration with speech. The methods are illustrated by a large number of examples—the book contains more than 120 figures on its 248 pages. At the end of each section, a number of useful references for further reading are related to the section topics. In the appendix, a table summarizing the 35 systems referred to in the book is given. In general, the presentations in the book are beneficial for their high degree of detail in documenting representations. Running examples are illustrated by intermediate representations at various stages of processing. They demonstrate very well the increasing degrees of precision in which utterances are specified and the associated commitments about possible text portions that express these specifications. Particularly valuable are the comparisons of alternative representations, ranging from skeletal propositions over lexicalized case frames to canned text, including a comparison of the consequences for the distribution of work over the system components involved and discussions of the associated pros and cons. These chapters provide a number of valuable hints, given the currently low degree of standardization in the field. Further important aspects addressed, which have been widely ignored in the literature so far,

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is then extended to handwriting synthesis by allowing the network to condition its predictions on a text sequence. The resulting system is able to generate highly realistic cursive handwriting in a wide variety of styles.

Synthesizing high-quality images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing textto- image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose Stacked Generative Adversarial Networks (StackGAN) to generate 256.256 photo-realistic images conditioned on text descriptions. We decompose the hard problem into more manageable sub-problems through a sketch-refinement process. The Stage-I GAN sketches the primitive shape and colors of the object based on the given text description, yielding Stage-I low-resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high-resolution images with photo-realistic details. It is able to rectify defects in Stage-I results and add compelling details with the refinement process. To improve the diversity of the synthesized images and stabilize the training of the conditional-GAN, we introduce a novel Conditioning Augmentation technique that encourages smoothness in the latent conditioning manifold. Extensive experiments and comparisons with state-of-the-arts on benchmark datasets demonstrate that the proposed method achieves significant improvements on generating photo-realistic images conditioned on text descriptions.

Prediction without justification has limited applicability. As a remedy, we learn to extract pieces of input text as justifications – rationales – that are tailored to be short and coherent, yet sufficient for making the same prediction. Our approach combines two modular components, generator and encoder, which are trained to operate well together. The generator specifies a distribution over text fragments as candidate rationales and these are passed through the encoder for prediction. Rationales are never given during training. Instead, the model is regularized by desiderata for rationales. We evaluate the approach on multi-aspect sentiment analysis against manually annotated test cases. Our approach outperforms attention-based baseline by a significant margin. We also successfully illustrate the method on the question retrieval task.1

Automatically describing an image is an important capability for virtual assistants. Significant progress has been achieved in recent years on this task of image captioning. However, classical prediction techniques based on maximum likelihood trained LSTM nets don’t embrace the inherent ambiguity of image captioning. To address this concern, recent variational auto-encoder and generative adversarial network based methods produce a set of captions by sampling from an abstract latent space. But, this latent space has limited interpretability and therefore, a control mechanism for captioning remains an open problem. This paper proposes a captioning technique conditioned on part-of-speech. Our method provides human interpretable control in form of part-of-speech. Importantly, part-of-speech is a language prior, and conditioning on it provides: (i) more diversity as evaluated by counting n-grams and the novel sentences generated, (ii) achieves high accuracy for the diverse captions on standard captioning metrics.

This paper describes an ensemble system submitted as part of the LSDSem Shared Task 2017 the Story Cloze Test. The main conclusion from our results is that an approach based on semantic similarity alone may not be enough for this task. We test various approaches and compare them with two ensemble systems. One is based on voting and the other on logistic regression based classifier. Our final system is able to outperform the previous state of the art for the Story Cloze test. Another very interesting observation is the performance of sentiment based approach which works almost as well on its own as our final en-


Question: Creating adversarial texts utilizing gradient methods.

This paper explores the linguistic regularities found in continuous space word representations, focusing on the neural network language model and the distributed representation of words as vectors. These vector representations are designed to capture both syntactic and semantic regularities in language, demonstrating superior performance compared to state-of-the-art methods. By utilizing a neural network approach, the model is able to encode words in a high-dimensional vector space, where the proximity of vectors reflects the similarity between words in terms of meaning and usage. This method not only outperforms traditional approaches but also reveals underlying patterns and relationships in language that are essential for natural language processing tasks. Through extensive experimentation and analysis, the paper establishes the efficacy of these continuous space word representations in capturing linguistic regularities, thereby enhancing the performance of various language-related applications.

While the costs of human violence have attracted a great deal of attention from the research community, the effects of the network-on-network (NoN) violence popularised by Generative Adversarial Networks have yet to be addressed. In this work, we quantify the financial, social, spiritual, cultural, grammatical and dermatological impact of this aggression and address the issue by proposing a more peaceful approach which we term Generative Unadversarial Networks (GUNs). Under this framework, we simultaneously train two models: a generator G that does its best to capture whichever data distribution it feels it can manage, and a motivator M that helps G to achieve its dream. Fighting is strictly verboten and both models evolve by learning to respect their differences. The framework is both theoretically and electrically grounded in game theory, and can be viewed as a winner-shares-all two-player game in which both players work as a team to achieve the best score. Experiments show that by working in harmony, the proposed model is able to claim both the moral and log-likelihood high ground. Our work builds on a rich history of carefully argued position-papers, published as anonymous YouTube comments, which prove that the optimal solution to NoN violence is more GUNs. Takes skill to be real, time to heal each other Tupac Shakur, Changes, 1998

We posit that visually descriptive language offers computer vision researchers both information about the world, and information about how people describe the world. The potential benefit from this source is made more significant due to the enormous amount of language data easily available today. We present a system to automatically generate natural language descriptions from images that exploits both statistics gleaned from parsing large quantities of text data and recognition algorithms from computer vision. The system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work.

Neural vector representations are ubiquitous throughout all subfields of NLP. While word vectors have been studied in much detail, thus far only little light has been shed on the properties of sentence embeddings. In this paper, we assess to what extent prominent sentence embedding methods exhibit select semantic properties. We propose a framework that generate triplets of sentences to explore how changes in the syntactic structure or semantics of a given sentence affect the similarities obtained between their sentence embeddings.

Previous neural machine translation models used some heuristic search algorithms (e.g., beam search) in order to avoid solving the maximum a posteriori problem over translation sentences at test phase. In this paper, we propose the GumbelGreedy Decoding which trains a generative network to predict translation under a trained model. We solve such a problem using the Gumbel-Softmax reparameterization, which makes our generative network differentiable and trainable through standard stochastic gradient methods. We empirically demonstrate that our proposed model is effective for generating sequences of discrete words.

Many learning tasks such as spam filtering and credit card fraud detection face an active adversary that tries to avoid detection. For learning problems that deal with an active adversary, it is important to model the adversary's attack strategy and develop robust learning models to mitigate the attack. These are the two objectives of this paper. We consider two attack models: a free-range attack model that permits arbitrary data corruption and a restrained attack model that anticipates more realistic attacks that a reasonable adversary would devise under penalties. We then develop optimal SVM learning strategies against the two attack models. The learning algorithms minimize the hinge loss while assuming the adversary is modifying data to maximize the loss. Experiments are performed on both artificial and real data sets. We demonstrate that optimal solutions may be overly pessimistic when the actual attacks are much weaker than expected. More important, we demonstrate that it is possible to develop a much more resilient SVM learning model while making loose assumptions on the data corruption models. When derived under the restrained attack model, our optimal SVM learning strategy provides more robust overall performance under a wide range of attack parameters.

We present a system to automatically generate natural language descriptions from images. This system consists of two parts. The first part, content planning, smooths the output of computer vision-based detection and recognition algorithms with statistics mined from large pools of visually descriptive text to determine the best content words to use to describe an image. The second step, surface realization, chooses words to construct natural language sentences based on the predicted content and general statistics from natural language. We present multiple approaches for the surface realization step and evaluate each using automatic measures of similarity to human generated reference descriptions. We also collect forced choice human evaluations between descriptions from the proposed generation system and descriptions from competing approaches. The proposed system is very effective at producing relevant sentences for images. It also generates descriptions that are notably more true to the specific image content than previous work.

We present an effective end-to-end memory network model that jointly (i) predicts whether a given document can be considered as relevant evidence for a given claim, and (ii) extracts snippets of evidence that can be used to reason about the factuality of the target claim. Our model combines the advantages of convolutional and recurrent neural networks as part of a memory network. We further introduce a similarity matrix at the inference level of the memory network in order to extract snippets of evidence for input claims more accurately. Our experiments on a public benchmark dataset, FakeNewsChallenge, demonstrate the effectiveness of our approach.

Identity theft is one of the most profitable crimes committed by felons. In the cyber space, this is commonly achieved using phishing. We propose here robust server side methodology to detect phishing attacks, called phishGILLNET, which incorporates the power of natural language processing and machine learning techniques. phishGILLNET is a multi-layered approach to detect phishing attacks. The first layer (phishGILLNET1) employs Probabilistic Latent Semantic Analysis (PLSA) to build a topic model. The topic model handles synonym (multiple words with similar meaning), polysemy (words with multiple meanings), and other linguistic variations found in phishing. Intentional misspelled words found in phishing are handled using Levenshtein editing and Google APIs for correction. Based on term document frequency matrix as input PLSA finds phishing and non-phishing topics using tempered expectation maximization. The performance of phishGILLNET1 is evaluated using PLSA fold in technique and the classification is achieved using Fisher similarity. The second layer of phishGILLNET (phishGILLNET2) employs AdaBoost to build a robust classifier. Using probability distributions of the best PLSA topics as features the classifier is built using AdaBoost. The third layer (phishGILLNET3) further expands phishGILLNET2 by building a classifier from labeled and unlabeled examples by employing Co-Training. Experiments were conducted using one of the largest public corpus of email data containing 400,000 emails. Results show that phishGILLNET3 outperforms state of the art phishing detection methods and achieves F-measure of 100%. Moreover, phishGILLNET3 requires only a small percentage (10%) of data be annotated thus saving significant time, labor, and avoiding errors incurred in human annotation.

In this paper, we proposed a sentence encoding-based model for recognizing text entailment. In our approach, the encoding of sentence is a two-stage process. Firstly, average pooling was used over word-level bidirectional LSTM (biLSTM) to generate a firststage sentence representation. Secondly, attention mechanism was employed to replace average pooling on the same sentence for better representations. Instead of using target sentence to attend words in source sentence, we utilized the sentence’s first-stage representation to attend words appeared in itself, which is called ”Inner-Attention” in our paper . Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus has proved the effectiveness of ”Inner-Attention” mechanism. With less number of parameters, our model outperformed the existing best sentence encoding-based approach by a large margin.

In practice, there are often explicit constraints on what representations or decisions are acceptable in an application of machine learning. For example it may be a legal requirement that a decision must not favour a particular group. Alternatively it can be that that representation of data must not have identifying information. We address these two related issues by learning flexible representations that minimize the capability of an adversarial critic. This adversary is trying to predict the relevant sensitive variable from the representation, and so minimizing the performance of the adversary ensures there is little or no information in the representation about the sensitive variable. We demonstrate this adversarial approach on two problems: making decisions free from discrimination and removing private information from images. We formulate the adversarial model as a minimax problem, and optimize that minimax objective using a stochastic gradient alternate min-max optimizer. We demonstrate the ability to provide discriminant free representations for standard test problems, and compare with previous state of the art methods for fairness, showing statistically significant improvement across most cases. The flexibility of this method is shown via a novel problem: removing annotations from images, from separate training examples of annotated and unannotated images, and with no a priori knowledge of the form of annotation provided to the model.


Question: Generating adversarial texts utilizing gradient techniques.

This paper uncovers a significant vulnerability in state-of-the-art deep neural network classifiers: the existence of a universal, minuscule perturbation vector that, when applied to natural images, leads to a high likelihood of misclassification. We present a methodical approach to calculate these universal perturbations and demonstrate that even the most advanced neural networks are highly susceptible to them, despite their subtlety to human perception. Moreover, we conduct extensive empirical analysis on these universal perturbations, revealing their remarkable consistency across different neural network architectures. The discovery of these universal perturbations not only highlights unexpected geometric correlations within the high-dimensional decision boundaries of classifiers but also points to potential security loopholes. These loopholes stem from the existence of singular directions in the input space that adversaries could exploit to undermine a classifier's performance on most natural images, raising critical concerns regarding the robustness and security of deep learning systems.

We present the first domain adaptation model for authorship attribution to leverage unlabeled data. The model includes extensions to structural correspondence learning needed to make it appropriate for the task. For example, we propose a median-based classification instead of the standard binary classification used in previous work. Our results show that punctuation-based character n-grams form excellent pivot features. We also show how singular value decomposition plays a critical role in achieving domain adaptation, and that replacing (instead of concatenating) non-pivot features with correspondence features yields better performance.

A recent line of research proposed (either implicitly or explicitly) gradient-masking preprocessing techniques to improve adversarial robustness. However, as shown by Athaley-Carlini-Wagner, essentially all these defenses can be circumvented if an attacker leverages approximate gradient information with respect to the preprocessing. This thus raises a natural question of whether there is a useful preprocessing technique in the context of white-box attacks, even just for only mildly complex datasets such as MNIST. In this paper we provide an affirmative answer to this question. Our key observation is that for several popular datasets, one can approximately encode entire dataset using a small set of separable codewords derived from the training set, while retaining high accuracy on natural images. The separability of the codewords in turn prevents small perturbations as in `∞ attacks from changing feature encoding, leading to adversarial robustness. For example, for MNIST our code consists of only two codewords, 0 and 1, and the encoding of any pixel is simply 1[x > 0.5] (i.e., whether a pixel x is at least 0.5). Applying this code to a naturally trained model already gives high adversarial robustness even under strong white-box attacks based on Backward Pass Differentiable Approximation (BPDA) method of Athaley-Carlini-Wagner that takes the codes into account. We give density-estimation based algorithms to construct such codes, and provide theoretical analysis and certificates of when our method can be effective. Systematic evaluation demonstrates that our method is effective in improving adversarial robustness on MNIST, CIFAR-10, and ImageNet, for either naturally or adversarially trained models.

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a discriminative model D that estimates the probability that a sample came from the training data rather than G. The training procedure for G is to maximize the probability of D making a mistake. This framework corresponds to a minimax two-player game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering the training data distribution and D equal to 1 2 everywhere. In the case where G and D are defined by multilayer perceptrons, the entire system can be trained with backpropagation. There is no need for any Markov chains or unrolled approximate inference networks during either training or generation of samples. Experiments demonstrate the potential of the framework through qualitative and quantitative evaluation of the generated samples.

We present a method for generating English sentences from Abstract Meaning Representation (AMR) graphs, exploiting a parallel corpus of AMRs and English sentences. We treat AMR-to-English generation as phrase-based machine translation (PBMT). We introduce a method that learns to linearize tokens of AMR graphs into an English-like order. Our linearization reduces the amount of distortion in PBMT and increases generation quality. We report a Bleu score of 26.8 on the standard AMR/English test set.

Deep neural network-based classifiers are known to be vulnerable to adversarial examples that can fool them into misclassifying their input through the addition of small-magnitude perturbations. However, recent studies have demonstrated that such adversarial examples are not very effective in the physical world—they either completely fail to cause misclassification or only work in restricted cases where a relatively complex image is perturbed and printed on paper. In this paper we propose a new attack algorithm—Robust Physical Perturbations (RP2)— that generates perturbations by taking images under different conditions into account. Our algorithm can create spatiallyconstrained perturbations that mimic vandalism or art to reduce the likelihood of detection by a casual observer. We show that adversarial examples generated by RP2 achieve high success rates under various conditions for real road sign recognition by using an evaluation methodology that captures physical world conditions. We physically realized and evaluated two attacks, one that causes a Stop sign to be misclassified as a Speed Limit sign in 100% of the testing conditions, and one that causes a Right Turn sign to be misclassified as either a Stop or Added Lane sign in 100% of the testing conditions.

We propose an interactive multimodal framework for language learning. Instead of being passively exposed to large amounts of natural text, our learners (implemented as feed-forward neural networks) engage in cooperative referential games starting from a tabula rasa setup, and thus develop their own language from the need to communicate in order to succeed at the game. Preliminary experiments provide promising results, but also suggest that it is important to ensure that agents trained in this way do not develop an adhoc communication code only effective for the game they are playing.

Many language generation tasks require the production of text conditioned on both structured and unstructured inputs. We present a novel neural network architecture which generates an output sequence conditioned on an arbitrary number of input functions. Crucially, our approach allows both the choice of conditioning context and the granularity of generation, for example characters or tokens, to be marginalised, thus permitting scalable and effective training. Using this framework, we address the problem of generating programming code from a mixed natural language and structured specification. We create two new data sets for this paradigm derived from the collectible trading card games Magic the Gathering and Hearthstone. On these, and a third preexisting corpus, we demonstrate that marginalising multiple predictors allows our model to outperform strong benchmarks.

Semantic error detection and correction is an important task for applications such as fact checking, speech-to-text or grammatical error correction. Current approaches generally focus on relatively shallow semantics and do not account for numeric quantities. Our approach uses language models grounded in numbers within the text. Such groundings are easily achieved for recurrent neural language model architectures, which can be further conditioned on incomplete background knowledge bases. Our evaluation on clinical reports shows that numerical grounding improves perplexity by 33% and F1 for semantic error correction by 5 points when compared to ungrounded approaches. Conditioning on a knowledge base yields further improvements.

One of the biggest challenges of endto-end language generation from meaning representations in dialogue systems is making the outputs more natural and varied. Here we take a large corpus of 50K crowd-sourced utterances in the restaurant domain and develop text analysis methods that systematically characterize types of sentences in the training data. We then automatically label the training data to allow us to conduct two kinds of experiments with a neural generator. First, we test the effect of training the system with different stylistic partitions and quantify the effect of smaller, but more stylistically controlled training data. Second, we propose a method of labeling the style variants during training, and show that we can modify the style of the generated utterances using our stylistic labels. We contrast and compare these methods that can be used with any existing large corpus, showing how they vary in terms of semantic quality and

The use of dialogue systems as a medium for human-machine interaction is an increasingly prevalent paradigm. A growing number of dialogue systems use conversation strategies that are learned from large datasets. There are well documented instances where interactions with these system have resulted in biased or even offensive conversations due to the data-driven training process. Here, we highlight potential ethical issues that arise in dialogue systems research, including: implicit biases in data-driven systems, the rise of adversarial examples, potential sources of privacy violations, safety concerns, special considerations for reinforcement learning systems, and reproducibility concerns. We also suggest areas stemming from these issues that deserve further investigation. Through this initial survey, we hope to spur research leading to robust, safe, and ethically sound dialogue systems.


Question: Gradient methods for generating adversarial texts.

This document explores the efficacy of the Earth Mover's Distance (EMD) as a metric for content-based image retrieval, highlighting its potential to enhance the accuracy of perceptual similarity assessments. Initially proposed by Peleg, Werman, and Rom for specific vision problems, the EMD quantifies the minimal cost required to morph one distribution into another. By integrating this concept with a vector quantization-based representation scheme for distributions, we establish an image comparison framework that outperforms conventional methods in capturing perceptual likeness. The EMD leverages solutions from linear optimization's transportation problem, ensuring computational efficiency and facilitating partial matching. It surpasses histogram-based techniques by accommodating variable-length representations, thereby circumventing quantization issues and other binning-related problems. When applied to distributions of equal mass, the EMD functions as a genuine metric. Our study emphasizes applications in color and texture analysis, comparing the EMD's retrieval effectiveness against alternative distance measures.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

**Similar Documents**: This paper presents a novel methodology, combining elements of genetic algorithm (GA) and particle swarm optimization (PSO), to address the intricate task of recurrent network design. The proposed Hybrid Genetic Algorithm-Particle Swarm Optimization (HGAPSO) framework innovates by not only utilizing GA's crossover and mutation operations but also integrating PSO's unique strategies. The elite strategy, a core component of HGAPSO, involves enhancing the most proficient individuals within a population. These top performers, or elites, are then subjected to PSO's optimization techniques, simulating natural maturation processes. The enhanced elites and their offspring, generated through GA's mechanisms, constitute the new generation. This dual approach ensures a dynamic and adaptive evolutionary process, tailored to recurrent neural/fuzzy network design. HGAPSO's application is demonstrated through the design of a fully connected recurrent neural network for temporal sequence production and a Takagi-Sugeno-Kang-type recurrent fuzzy network for dynamic plant control. Comparative analyses reveal HGAPSO's superior performance compared to standalone GA and PSO in tackling recurrent network design challenges.

We present the first domain adaptation model for authorship attribution to leverage unlabeled data. The model includes extensions to structural correspondence learning needed to make it appropriate for the task. For example, we propose a median-based classification instead of the standard binary classification used in previous work. Our results show that punctuation-based character n-grams form excellent pivot features. We also show how singular value decomposition plays a critical role in achieving domain adaptation, and that replacing (instead of concatenating) non-pivot features with correspondence features yields better performance.

Adversarial training has been shown to produce state of the art results for generative image modeling. In this paper we propose an adversarial training approach to train semantic segmentation models. We train a convolutional semantic segmentation network along with an adversarial network that discriminates segmentation maps coming either from the ground truth or from the segmentation network. The motivation for our approach is that it can detect and correct higher-order inconsistencies between ground truth segmentation maps and the ones produced by the segmentation net. Our experiments show that our adversarial training approach leads to improved accuracy on the Stanford Background and PASCAL VOC 2012 datasets.

In this paper, we provide several alternatives to the classical Bag-Of-Words model for automatic authorship attribution. To this end, we consider linguistic and writing style information such as grammatical structures to construct different document representations. Furthermore we describe two techniques to combine the obtained representations: combination vectors and ensemble based meta classification. Our experiments show the viability of our approach.

The advent of social media has brought Internet memes, a unique social phenomenon, to the front stage of the Web. Embodied in the form of images with text descriptions, little do we know about the “language of memes”. In this paper, we statistically study the correlations among popular memes and their wordings, and generate meme descriptions from raw images. To do this, we take a multimodal approach—we propose a robust nonparanormal model to learn the stochastic dependencies among the image, the candidate descriptions, and the popular votes. In experiments, we show that combining text and vision helps identifying popular meme descriptions; that our nonparanormal model is able to learn dense and continuous vision features jointly with sparse and discrete text features in a principled manner, outperforming various competitive baselines; that our system can generate meme descriptions using a simple pipeline.

Detection and rejection of adversarial examples in security sensitive and safety-critical systems using deep CNNs is essential. In this paper, we propose an approach to augment CNNs with out-distribution learning in order to reduce misclassification rate by rejecting adversarial examples. We empirically show that our augmented CNNs can either reject or classify correctly most adversarial examples generated using well-known methods (> 95% for MNIST and > 75% for CIFAR-10 on average). Furthermore, we achieve this without requiring to train using any specific type of adversarial examples and without sacrificing the accuracy of models on clean samples significantly (< 4%).

Several standard text-categorization techniques were applied to the problem of automated essay grading. Bayesian independence classifiers and knearest-neighbor classifiers were trained to assign scores to manually-graded essays. These scores were combined with several other summary text measures using linear regression. The classifiers and regression equations were then applied to a new set of essays. The classifiers worked very well. The agreement between the automated grader and the final manual grade was as good as the agreement between human graders.

Traditional text similarity measures consider each term similar only to itself and do not model semantic relatedness of terms. We propose a novel discriminative training method that projects the raw term vectors into a common, low-dimensional vector space. Our approach operates by finding the optimal matrix to minimize the loss of the pre-selected similarity function (e.g., cosine) of the projected vectors, and is able to efficiently handle a large number of training examples in the highdimensional space. Evaluated on two very different tasks, cross-lingual document retrieval and ad relevance measure, our method not only outperforms existing state-of-the-art approaches, but also achieves high accuracy at low dimensions and is thus more efficient.

In this paper we explore the bi-directional mapping between images and their sentence-based descriptions. Critical to our approach is a recurrent neural network that attempts to dynamically build a visual representation of the scene as a caption is being generated or read. The representation automatically learns to remember long-term visual concepts. Our model is capable of both generating novel captions given an image, and reconstructing visual features given an image description. We evaluate our approach on several tasks. These include sentence generation, sentence retrieval and image retrieval. State-of-the-art results are shown for the task of generating novel image descriptions. When compared to human generated captions, our automatically generated captions are equal to or preferred by humans 21.0% of the time. Results are better than or comparable to state-of-the-art results on the image and sentence retrieval tasks for methods using similar visual features.

Most text classification approaches model text at the lexical and syntactic level only, lacking domain robustness and explainability. In tasks like sentiment analysis, such approaches can result in limited effectiveness if the texts to be classified consist of a series of arguments. In this paper, we claim that even a shallow model of the argumentation of a text allows for an effective and more robust classification, while providing intuitive explanations of the classification results. Here, we apply this idea to the supervised prediction of sentiment scores for reviews. We combine existing approaches from sentiment analysis with novel features that compare the overall argumentation structure of the given review text to a learned set of common sentiment flow patterns. Our evaluation in two domains demonstrates the benefit of modeling argumentation for text classification in terms of effectiveness and robustness.

Automated grammatical error correction for language learners has attracted a lot of attention in recent years, especially after a number of shared tasks that have encouraged research in the area. Treating the problem as a translation task from 'incorrect' into 'correct' English using statistical machine translation has emerged as a state-of-the-art approach but it requires vast amounts of corrected parallel data to produce useful results. Because manual annotation of incorrect text is laborious and expensive, we can generate artificial error-annotated data by injecting errors deliberately into correct text and thus produce larger amounts of parallel data with much less effort. In this work, we review previous work on artificial error generation and investigate new approaches using random and probabilistic methods for constrained and general error correction. Our methods use error statistics from a reference corpus of learner writing to generate errors in native text that look realistic and plausible in context. We investigate a number of aspects that can play a part in the error generation process, such as the origin of the native texts, the amount of context used to find suitable insertion points, the type of information encoded by the error patterns and the output error distribution. In addition, we explore the use of linguistic information for characterising errors and train systems using different combinations of real and artificial data. Results of our experiments show that the use of artificial errors can improve system performance when they are used in combination with real learner errors, in line with previous research. These improvements are observed for both constrained and general correction, for which probabilistic methods produce the best results. We also demonstrate that systems trained on a combination of real and artificial errors can beat other highly-engineered systems and be more robust, showing that performance can be improved by focusing on the data rather than tuning system parameters. Part of our work is also devoted to the proposal of the I-measure, a new evaluation scheme that scores corrections in terms of improvement on the original text and solves known issues with existing evaluation measures. To my family and L.A. In memory of Kika.


Question: Texts generated through gradient methods for adversarial purposes.

In the realm of Natural Language Processing, handling large corpora presents significant challenges, particularly in the application of the Vector Space Model (VSM), where memory constraints often become the primary bottleneck. This paper addresses a critical shortcoming in current software implementations of popular algorithms by focusing on scalability and user-friendliness. We introduce a novel software framework designed to efficiently process large text corpora through document streaming, enabling sequential document processing without memory limitations. This framework supports a variety of topical inference algorithms, notably including Latent Semantic Analysis and Latent Dirichlet Allocation, optimized for memory-independent operation, thereby removing the dependency on corpus size. A key design principle is the framework's simplicity and intuitiveness, ensuring that both modifications to the algorithms and their practical application by users are seamless and straightforward. The efficacy of our approach is illustrated through a real-world application in a digital library, DML-CZ, where document similarities are computed with notable efficiency.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design
An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and

In this work we explore deep generative models of text in which the latent representation of a document is itself drawn from a discrete language model distribution. We formulate a variational auto-encoder for inference in this model and apply it to the task of compressing sentences. In this application the generative model first draws a latent summary sentence from a background language model, and then subsequently draws the observed sentence conditioned on this latent summary. In our empirical evaluation we show that generative formulations of both abstractive and extractive compression yield state-of-the-art results when trained on a large amount of supervised data. Further, we explore semi-supervised compression scenarios where we show that it is possible to achieve performance competitive with previously proposed supervised models while training on a fraction of the supervised data.

Natural image generation is currently one of the most actively explored fields in Deep Learning. Many approaches, e.g. for state-of-the-art artistic style transfer or natural texture synthesis, rely on the statistics of hierarchical representations in supervisedly trained deep neural networks. It is, however, unclear what aspects of this feature representation are crucial for natural image generation: is it the depth, the pooling or the training of the features on natural images? We here address this question for the task of natural texture synthesis and show that none of the above aspects are indispensable. Instead, we demonstrate that natural textures of high perceptual quality can be generated from networks with only a single layer, no pooling and random filters.

Nowadays, convolutional neural network (CNN) based steganalysis methods achieved great performance. While those methods are also facing security problems. In this paper, we proposed an attack scheme aiming at CNN based steganalyzer including two different attack methods 1) the LSB-Jstego Gradient Based Attack; 2) LSB-Jstego Evolutionary Algorithms Based Attack. The experiment results show that the attack strategies could achieve 96.02% and 90.25% success ratio separately on the target CNN. The proposed attack scheme is an effective way to fool the CNN based steganalyzer and in addition demonstrates the vulnerability of the neural networks in steganalysis.

Procedural texture generation enables the creation of more rich and detailed virtual environments without the help of an artist. However, finding a flexible generative model of real world textures remains an open problem. We present a novel Convolutional Neural Network based texture model consisting of two summary statistics (the Gramian and Translation Gramian matrices), as well as spectral constraints. We investigate the Fourier Transform or Window Fourier Transform in applying spectral constraints, and find that the Window Fourier Transform improved the quality of the generated textures. We demonstrate the efficacy of our system by comparing generated output with that of related state of the art systems.

Low-dimensional vector embeddings, computed using LSTMs or simpler techniques, are a popular approach for capturing the “meaning” of text and a form of unsupervised learning useful for downstream tasks. However, their power is not theoretically understood. The current paper derives formal understanding by looking at the subcase of linear embedding schemes. Using the theory of compressed sensing we show that representations combining the constituent word vectors are essentially information-preserving linear measurements of Bag-of-n-Grams (BonG) representations of text. This leads to a new theoretical result about LSTMs: low-dimensional embeddings derived from a low-memory LSTM are provably at least as powerful on classification tasks, up to small error, as a linear classifier over BonG vectors, a result that extensive empirical work has thus far been unable to show. Our experiments support these theoretical findings and establish strong, simple, and unsupervised baselines on standard benchmarks that in some cases are state of the art among word-level methods. We also show a surprising new property of embeddings such as GloVe and word2vec: they form a good sensing matrix for text that is more efficient than random matrices, the standard sparse recovery tool, which may explain why they lead to better representations in practice.

Although Generative Adversarial Networks achieve state-of-the-art results on a variety of generative tasks, they are regarded as highly unstable and prone to miss modes. We argue that these bad behaviors of GANs are due to the very particular functional shape of the trained discriminators in high dimensional spaces, which can easily make training stuck or push probability mass in the wrong direction, towards that of higher concentration than that of the data generating distribution. We introduce several ways of regularizing the objective, which can dramatically stabilize the training of GAN models. We also show that our regularizers can help the fair distribution of probability mass across the modes of the data generating distribution, during the early phases of training and thus providing a unified solution to the missing modes problem.

An obstacle to the development of many natural language processing products is the vast amount of training examples necessary to get satisfactory results. The generation of these examples is often a tedious and timeconsuming task. This paper this paper proposes a method to transform the sentiment of sentences in order to limit the work necessary to generate more training data. This means that one sentence can be transformed to an opposite sentiment sentence and should reduce by half the work required in the generation of text. The proposed pipeline consists of a sentiment classifier with an attention mechanism to highlight the short phrases that determine the sentiment of a sentence. Then, these phrases are changed to phrases of the opposite sentiment using a baseline model and an autoencoder approach. Experiments are run on both the separate parts of the pipeline as well as on the end-to-end model. The sentiment classifier is tested on its accuracy and is found to perform adequately. The autoencoder is tested on how well it is able to change the sentiment of an encoded phrase and it was found that such a task is possible. We use human evaluation to judge the performance of the full (end-to-end) pipeline and that reveals that a model using word vectors outperforms the encoder model. Numerical evaluation shows that a success rate of 54.7% is achieved on the sentiment change.

NLP algorithms are increasingly used in computational social science to take linguistic observations and predict outcomes like human preferences or actions. Making these social models transparent and interpretable often requires identifying features in the input that predict outcomes while also controlling for potential confounds. We formalize this need as a new task: inducing a lexicon that is predictive of a set of target variables yet uncorrelated to a set of confounding variables. We introduce two deep learning algorithms for the task. The first uses a bifurcated architecture to separate the explanatory power of the text and confounds. The second uses an adversarial discriminator to force confound-invariant text encodings. Both elicit lexicons from learned weights and attentional scores. We use them to induce lexicons that are predictive of timely responses to consumer complaints (controlling for product), enrollment from course descriptions (controlling for subject), and sales from product descriptions (controlling for seller). In each domain our algorithms pick words that are associated with narrative persuasion; more predictive and less confound-related than those of standard feature weighting and lexicon induction techniques like regression and log odds.

Generating an article automatically with computer program is a challenging task in artificial intelligence and natural language processing. In this paper, we target at essay generation, which takes as input a topic word in mind and generates an organized article under the theme of the topic. We follow the idea of text planning (Reiter and Dale, 1997) and develop an essay generation framework. The framework consists of three components, including topic understanding, sentence extraction and sentence reordering. For each component, we studied several statistical algorithms and empirically compared between them in terms of qualitative or quantitative analysis. Although we run experiments on Chinese corpus, the method is language independent and can be easily adapted to other language. We lay out the remaining challenges and suggest avenues for future research.

How much does “free shipping!” help an advertisement’s ability to persuade? This paper presents two methods for performance attribution: finding the degree to which an outcome can be attributed to parts of a text while controlling for potential confounders1. Both algorithms are based on interpreting the behaviors and parameters of trained neural networks. One method uses a CNN to encode the text, an adversarial objective function to control for confounders, and projects its weights onto its activations to interpret the importance of each phrase towards each output class. The other method leverages residualization to control for confounds and performs interpretation by aggregating over learned word vectors. We demonstrate these algorithms’ efficacy on 118,000 internet search advertisements and outcomes, finding language indicative of high and low click through rate (CTR) regardless of who the ad is by or what it is for. Our results suggest the proposed algorithms are high performance and data efficient, able to glean actionable insights from fewer than 10,000 data points. We find that quick, easy, and authoritative language is associated with success, while lackluster embellishment is related to failure. These findings agree with the advertising industry’s emperical wisdom, automatically revealing insights which previously required manual A/B testing to discover.


Question: Tracking pose using natural features on mobile devices.

This work introduces a novel approach for estimating the camera pose within an uncharted environment, particularly tailored for Augmented Reality (AR) applications in confined spaces. Unlike previous efforts that have attempted to repurpose Simultaneous Localization and Mapping (SLAM) algorithms originally designed for robotic navigation, our system is meticulously crafted to handle the erratic motion of a hand-held camera in a small AR workspace. We achieve this by decoupling the processes of tracking and mapping, assigning them to distinct threads running in parallel on a dual-core processor. One thread is dedicated to the robust tracking of the camera's unpredictable movements, while the other concurrently generates a three-dimensional map from previously captured video frames. This parallel processing enables the utilization of batch optimization algorithms, typically too computationally intensive for real-time applications. The outcome is a sophisticated system capable of generating comprehensive maps featuring thousands of distinct landmarks, all while maintaining frame-rate tracking. This system boasts accuracy and reliability on par with the most advanced model-based methods, positioning it as a competitive solution for AR tracking applications.

Human 3D pose estimation from a single image is a challenging task with numerous applications. Convolutional Neural Networks (CNNs) have recently achieved superior performance on the task of 2D pose estimation from a single image, by training on images with 2D annotations collected by crowd sourcing. This suggests that similar success could be achieved for direct estimation of 3D poses. However, 3D poses are much harder to annotate, and the lack of suitable annotated training images hinders attempts towards end-to-end solutions. To address this issue, we opt to automatically synthesize training images with ground truth pose annotations. Our work is a systematic study along this road. We find that pose space coverage and texture diversity are the key ingredients for the effectiveness of synthetic training data. We present a fully automatic, scalable approach that samples the human pose space for guiding the synthesis procedure and extracts clothing textures from real images. Furthermore, we explore domain adaptation for bridging the gap between our synthetic training images and real testing photos. We demonstrate that CNNs trained with our synthetic images out-perform those trained with real photos on 3D pose estimation tasks.

Due to its low price, webcam has become one of the most promising sensors with the rapid development of computer vision. However, the accuracies of eye tracking and eye movement analysis are largely limited by the quality of the webcam videos. To solve this issue, a novel eye movement analysis model is proposed based on five eye feature points rather than a single point (such as the iris center). First, a single convolutional neural network (CNN) is trained for eye feature point detection, and five eye feature points are detected for obtaining more useful eye movement information. Subsequently, six types of original time-varying eye movement signals can be constructed by feature points of each frame, which can reduce the dependency of the iris center in low quality videos. Finally, behaviors-CNN can be trained by the time-varying eye movement signals for recognizing different eye movement patterns, which is capable of avoiding the influence of errors from the basic eye movement type detection and artificial eye movement feature construction. To validate the performance, a webcam-based visual activity data set was constructed, which contained almost 0.5 million frames collected from 38 subjects. The experimental results on this database have demonstrated that the proposed model can obtain promising results for natural and convenient eye movement-based applications.

This paper presents a novel method for eye tracking and blink detection in the video frames obtained from low resolution consumer grade web cameras. It uses a method involving Haar based cascade classifier for eye tracking and a combination of HOG features with SVM classifier for eye blink detection. The presented method is non intrusive and hence provides a comfortable user interaction. The eye tracking method has an accuracy of 92.3% and the blink detection method has an accuracy of 92.5% when tested using standard databases and a combined accuracy of 86% when tested under real world conditions of a normal room.

We present a real-time performance-driven facial animation system based on 3D shape regression. In this system, the 3D positions of facial landmark points are inferred by a regressor from 2D video frames of an ordinary web camera. From these 3D points, the pose and expressions of the face are recovered by fitting a user-specific blendshape model to them. The main technical contribution of this work is the 3D regression algorithm that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions. Experiments show that our system can accurately recover 3D face shapes even for fast motions, non-frontal faces, and exaggerated expressions. In addition, some capacity to handle partial occlusions and changing lighting conditions is demonstrated.

We study the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case. After reviewing existing edge and gradient based descriptors, we show experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform existing feature sets for human detection. We study the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results. The new approach gives near-perfect separation on the original MIT pedestrian database, so we introduce a more challenging dataset containing over 1800 annotated human images with a large range of pose variations and backgrounds.

While feature point recognition is a key component of modern approaches to object detection, existing approaches require computationally expensive patch preprocessing to handle perspective distortion. In this paper, we show that formulating the problem in a Naive Bayesian classification framework makes such preprocessing unnecessary and produces an algorithm that is simple, efficient, and robust. Furthermore, it scales well to handle large number of classes. To recognize the patches surrounding keypoints, our classifier uses hundreds of simple binary features and models class posterior probabilities. We make the problem computationally tractable by assuming independence between arbitrary sets of features. Even though this is not strictly true, we demonstrate that our classifier nevertheless performs remarkably well on image datasets containing very significant perspective changes.

Camera calibration patterns and fiducial markers are basic technique for 3D measurement to create Computer Graphics (CG) and Augmented Reality (AR). A checkerboard is widely used as a calibration pattern and a circle or ring grid provides higher precision [Datta et al. 2009], and matrix codes are common as AR markers.

Locating hands in sign language video is challenging due to a number of factors. Hand appearance varies widely across signers due to anthropometric variations and varying levels of signer proficiency. Video can be captured under varying illumination, camera resolutions, and levels of scene clutter, e.g., high-res video captured in a studio vs. low-res video gathered by a Web cam in a user's home. Moreover, the signers' clothing varies, e.g., skin-toned clothing vs. contrasting clothing, short-sleeved vs. long-sleeved shirts, etc. In this work, the hand detection problem is addressed in an appearance matching framework. The histogram of oriented gradient (HOG) based matching score function is reformulated to allow non-rigid alignment between pairs of images to account for hand shape variation. The resulting alignment score is used within a support vector machine hand/not-hand classifier for hand detection. The new matching score function yields improved performance (in ROC area and hand detection rate) over the vocabulary guided pyramid match kernel (VGPMK) and the traditional, rigid HOG distance on American Sign Language video gestured by expert signers. The proposed match score function is computationally less expensive (for training and testing), has fewer parameters and is less sensitive to parameter settings than VGPMK. The proposed detector works well on test sequences from an inexpert signer in a non-studio setting with cluttered background.

This paper presents a fully automated algorithm for reconstructing a textured 3D model of a face from a single photograph or a raw video stream. The algorithm is based on a combination of Support Vector Machines (SVMs) and a Morphable Model of 3D faces. After SVM face detection, individual facial features are detected using a novel regression- and classification-based approach, and probabilistically plausible configurations of features are selected to produce a list of candidates for several facial feature positions. In the next step, the configurations of feature points are evaluated using a novel criterion that is based on a Morphable Model and a combination of linear projections. To make the algorithm robust with respect to head orientation, this process is iterated while the estimate of pose is refined. Finally, the feature points initialize a model-fitting procedure of the Morphable Model. The result is a high resolution 3D surface model.

We present an approach for real-time, robust and accurate hand pose estimation from moving egocentric RGB-D cameras in cluttered real environments. Existing methods typically fail for hand-object interactions in cluttered scenes imaged from egocentric viewpoints—common for virtual or augmented reality applications. Our approach uses two subsequently applied Convolutional Neural Networks (CNNs) to localize the hand and regress 3D joint locations. Hand localization is achieved by using a CNN to estimate the 2D position of the hand center in the input, even in the presence of clutter and occlusions. The localized hand position, together with the corresponding input depth value, is used to generate a normalized cropped image that is fed into a second CNN to regress relative 3D hand joint locations in real time. For added accuracy, robustness and temporal stability, we refine the pose estimates using a kinematic pose tracking energy. To train the CNNs, we introduce a new photorealistic dataset that uses a merged reality approach to capture and synthesize large amounts of annotated data of natural hand interaction in cluttered scenes. Through quantitative and qualitative evaluation, we show that our method is robust to self-occlusion and occlusions by objects, particularly in moving egocentric perspectives.


Question: Tracking poses using natural features on mobile devices.

This paper presents an efficient and effective method for fast keypoint recognition, a critical aspect of modern object detection techniques. Unlike conventional approaches that often demand substantial computational resources for patch preprocessing to manage perspective distortions, our proposed solution circumvents this need by employing a Naive Bayesian classification framework. This results in an algorithm that is not only simple and efficient but also notably robust. Additionally, it excels in handling a vast number of classes. Our method utilizes a classifier that leverages a multitude of basic binary features to discern the patches around keypoints, estimating class posterior probabilities. To ensure computational feasibility, we adopt an assumption of independence among various feature sets, even though this condition may not always hold true. Despite this simplification, our classifier exhibits exceptional performance on image datasets that feature pronounced perspective variations, validating its practical utility and effectiveness.

A new method for r eal-time tracking of non-rigid objects seen from a moving camera is proposed. The central computational module is based on the mean shift iterations and nds the most probable tar get p osition in the current frame. The dissimilarity between the target model (its c olor distribution) and the target candidates is expr essed by a metric derive d from the Bhattacharyya coe cient. The theoretical analysis of the approach shows that it relates to the Bayesian framework while providing a practical, fast and e cient solution. The capability of the tracker to handle in real-time partial occlusions, signi cant clutter, and target scale variations, is demonstrated for several image sequences.

We accurately estimate the 3D geometry and appearance of the human body from a monocular RGB-D sequence of a user moving freely in front of the sensor. Range data in each frame is first brought into alignment with a multi-resolution 3D body model in a coarse-to-fine process. The method then uses geometry and image texture over time to obtain accurate shape, pose, and appearance information despite unconstrained motion, partial views, varying resolution, occlusion, and soft tissue deformation. Our novel body model has variable shape detail, allowing it to capture faces with a high-resolution deformable head model and body shape with lower-resolution. Finally we combine range data from an entire sequence to estimate a high-resolution displacement map that captures fine shape details. We compare our recovered models with high-resolution scans from a professional system and with avatars created by a commercial product. We extract accurate 3D avatars from challenging motion sequences and even capture soft tissue dynamics.

Human analysis in images and video is a hard problem due to the large variation in human pose, clothing, camera view-points, lighting and other factors. While the explicit modeling of this variability is difficult, the huge amount of available person images motivates for the implicit, data-driven approach to human analysis. In this work we aim to explore this approach using the large amount of images spanning a subspace of human appearance. We model this subspace by connecting images into a graph and propagating information through such a graph using a discriminatively-trained graphical model. We particularly address the problems of human pose estimation and action recognition and demonstrate how image graphs help solving these problems jointly. We report results on still images with human actions from the KTH dataset.

While intelligent environments are often cited as a reason for doing work on visual person-tracking, really making an intelligent environment exposes many realworld problems in visual tracking that must be solved to make the technology practical. In the context of our EasyLiving project in intelligent environments, we created a practical person-tracking system that solves most of the real-world problems. It uses two sets of color stereo cameras for tracking multiple people during live demonstrations in a living room. The stereo images are used for locating people, and the color images are used for maintaining their identities. The system runs quickly enough to make the room feel responsive, and it tracks multiple people standing, walking, sitting, occluding, and entering and leaving the

This paper describes an easy-to-use system to estimate the shape of a human body and his/her clothes. The system uses a Kinect to capture the human's RGB and depth information from different views. Using the depth data, a non-rigid deformation method is devised to compensate motions between different views, thus to align and complete the dressed shape. Given the reconstructed dressed shape, the skin regions are recognized by a skin classifier from the RGB images, and these skin regions are taken as a tight constraints for the body estimation. Subsequently, the body shape is estimated from the skin regions of the dressed shape by leveraging a statistical model of human body. After the body estimation, the body shape is non-rigidly deformed to fit the dressed shape, so as to extract the cloth field of the dressed shape. We demonstrate our system and the therein algorithms by several experiments. The results show the effectiveness of the proposed method. & 2014 Elsevier B.V. All rights reserved.

This paper presents the design and analysis of a computer vision-aided motion sensing algorithm for wheeled mobile robot's indoor navigation. The algorithm is realized using two vision cameras attached on a wheeled mobile robot. The first camera is positioned at front-looking direction while the second camera is positioned at downward-looking direction. An algorithm is developed to process the images acquired from the cameras to yield the mobile robot's positions and orientations. The proposed algorithm is implemented on a wheeled mobile robot for real-world effectiveness testing. Results are compared and shown the accuracy of the proposed algorithm. At the end of the paper, an artificial landmark approach is introduced to improve the navigation efficiency. Future work involved implementing the proposed artificial landmark for indoor navigation applications with minimized accumulated errors.

Current 2D face recognition systems encounter difficulties in recognizing faces with large pose variations. Utilizing the pose-invariant features of 3D face data has the potential to handle multiview face matching. A feature extractor based on the directional maximum is proposed to estimate the nose tip location and the pose angle simultaneously. A nose profile model represented by subspaces is used to select the best candidates for the nose tip. Assisted by a statistical feature location model, a multimodal scheme is presented to extract eye and mouth corners. Using the automatic feature extractor, a fully automatic 3D face recognition system is developed. The system is evaluated on two databases, the MSU database (300 multiview test scans from 100 subjects) and the UND database (953 near frontal scans from 277 subjects). The automatic system provides recognition accuracy that is comparable to the accuracy of a system with manually labeled feature points

Wee present an accurate and robust method for six degree of freedom image localization. There are two key-points of our method, 1). automatic immense photo synthesis and labeling from point cloud model and, 2). pose estimation with deep convolutional neural networks (ConvNets) regression. Our model can directly regresses 6-DOF camera poses from images, accurately describing where and how it was captured. We achieved an accuracy within 1 meters and 1 degree on our outdoor dataset, which covers about 20, 000m on our school campus. Unlike previous point cloud registration solutions, our model supports low resolution images (i.e. 224×224 in our settings), and is tiny in size when finished training. Moreover, in pose estimation, our model uses O(1) time & space complexity as trainset grows. We will show the importance to localization using hundreds of thousands of generated and self-labeled ”photos” came from a short video. We will show our model’s robustness despite of illumination and seasonal variances, which usually fails methods that leverage image feature descriptors like SIFT. Furthermore, we will show the ability of transfer our model trained on one scene to another, and the gains in accuracy and efficiency.

This paper concerns the problem of facial landmark detection. We provide a unique new analysis of the features produced at intermediate layers of a convolutional neural network (CNN) trained to regress facial landmark coordinates. This analysis shows that while being processed by the CNN, face images can be partitioned in an unsupervised manner into subsets containing faces in similar poses (i.e., 3D views) and facial properties (e.g., presence or absence of eye-wear). Based on this finding, we describe a novel CNN architecture, specialized to regress the facial landmark coordinates of faces in specific poses and appearances. To address the shortage of training data, particularly in extreme profile poses, we additionally present data augmentation techniques designed to provide sufficient training examples for each of these specialized sub-networks. The proposed Tweaked CNN (TCNN) architecture is shown to outperform existing landmark detection methods in an extensive battery of tests on the AFW, ALFW, and 300W benchmarks. Finally, to promote reproducibility of our results, we make code and trained models publicly available through our project webpage.

This paper addresses the problem of recovering 3D human pose from a single monocular image, using a discriminative bag-of-words approach. In previous work, the visual words are learned by unsupervised clustering algorithms. They capture the most common patterns and are good features for coarse-grain recognition tasks like object classification. But for those tasks which deal with subtle differences such as pose estimation, such representation may lack the needed discriminative power. In this paper, we propose to jointly learn the visual words and the pose regressors in a supervised manner. More specifically, we learn an individual distance metric for each visual word to optimize the pose estimation performance. The learned metrics rescale the visual words to suppress unimportant dimensions such as those corresponding to background. Another contribution is that we design an appearance and position context (APC) local descriptor that achieves both selectivity and invariance while requiring no background subtraction. We test our approach on both a quasi-synthetic dataset and a real dataset (HumanEva) to verify its effectiveness. Our approach also achieves fast computational speed thanks to the integral histograms used in APC descriptor extraction and fast inference of pose regressors.