Question: A method for authenticating cyber-physical devices in the smart grid electric vehicle ecosystem.

Facing the Security and Privacy Dilemmas in the Smart Grid Revolution
As the world stands on the brink of a monumental shift in power distribution technology, the traditional electrical grid is undergoing a radical transformation into what is now known as the smart grid. This cutting-edge digital network represents a significant leap forward from the outdated systems that have long been responsible for delivering electricity to our homes and workplaces. The smart grid's sophisticated digital infrastructure is designed to significantly enhance the capabilities of both customers and utilities in terms of energy monitoring, management, and forecasting.

The smart grid's integration of advanced communication technologies, data analytics, and automation promises to revolutionize the way we consume and distribute energy. However, this transformation also introduces a complex array of security and privacy challenges. The interconnected nature of the smart grid, which connects a vast network of devices, from smart meters in homes to large-scale power generation and distribution systems, creates a larger attack surface for potential cyber threats. Ensuring the integrity, confidentiality, and availability of data in this environment becomes a critical concern.

Moreover, the smart grid's ability to collect detailed data about energy consumption patterns at the household level raises significant privacy issues. This data can potentially reveal intimate details about a person's lifestyle, habits, and even their physical presence at home. Safeguarding this information from unauthorized access and misuse is essential to maintaining public trust in the smart grid's adoption and operation.

To address these challenges, robust security protocols and privacy-preserving mechanisms must be integrated into the smart grid's design and operation. This includes implementing strong encryption for data transmission, employing multi-factor authentication for device access, and adopting privacy-by-design principles to minimize data collection and retention. Additionally, continuous monitoring and proactive threat detection systems are necessary to identify and mitigate potential security breaches promptly.

In summary, while the smart grid heralds a new era of efficiency and sustainability in energy management, it also necessitates a comprehensive approach to security and privacy to ensure the protection of critical infrastructure and personal data. Ongoing research and development in these areas are crucial to maintaining the smart grid's resilience against cyber threats and upholding the privacy rights of its users.

Counterfeiting of goods in general and of electronic goods in particular is a growing concern with a huge impact on the global economy, the society, and the security of its critical infrastructure. Various examples are known where companies suffer from economic and brand damage due to competition with counterfeit goods. In some cases the use of counterfeit components has even led to tragic accidents in which lives were lost. It has also recently become clear that counterfeit products can penetrate the critical and security infrastructure of our modern societies and hence cause a threat to national security. One of the difficulties to deal with this problem stems from the fact that counterfeit goods can originate from sources that are able to make copies that are very hard to distinguish from their legitimate counterpart. A first well-known aspect of counterfeiting is product cloning. A second much less known but increasingly dangerous aspect consists of overproduction of goods. A special, but modern, case of counterfeiting is theft of Intellectual Property such as software and designs. The attractive part from the attackers’ point of view is that it is relatively easy to steal and has a high value without having to do huge investments in research and development. From a high-level point of view one can state that the attack can be thwarted by using encryption and authentication techniques. Device configuration data or embedded software can, for example, be encrypted such that it will only run on the device possessing the correct cryptographic key. Since encrypted data is still easy to copy, it now becomes essential that the secret key is well protected against copying or cloning. In order to deal with these two aspects of counterfeiting, a secret unclonable identifier is required together with strong cryptographic protocols. In this chapter we focus on a new way to address these problems: Hardware Intrinsic Security. It is based on the implementation and generation of secret physically unclonable identifiers used in conjunction with cryptographic techniques such as encryption and

A major challenge in computer security is about establishing the trustworthiness of remote platforms. Remote attestation is the most common approach to this challenge. It allows a remote platform to measure and report its system state in a secure way to a third party. Unfortunately, existing attestation solutions either provide low security, as they rely on unrealistic assumptions, or are not applicable to commodity low-cost and resource-constrained devices, as they require custom secure hardware extensions that are difficult to adopt across IoT vendors. In this work, we propose a novel remote attestation scheme, named Boot Attestation, that is particularly optimized for low-cost and resource-constrained embedded devices. In Boot Attestation, software integrity measurements are immediately committed to during boot, thus relaxing the traditional requirement for secure storage and reporting. Our scheme is very light on cryptographic requirements and storage, allowing efficient implementations, even on the most low-end IoT platforms available today. We also describe extensions for more flexible management of ownership and third party (public-key) attestation that may be desired in fully Internet-enabled devices. Our scheme is supported by many existing off-the-shelf devices. To this end, we review the hardware protection capabilities for a number of popular device types and present implementation results for two such commercially available platforms.

Cyber-physical Production Systems (CPPS) are one of the technical driving forces behind the transformation of industrial production towards "digital factory of the future" in the context of Industry 4.0. Security is a major concern for such systems as they become more intelligent, interconnected, and coupled with physical devices. For various security activities from security analysis to designing security controls and architecture, a systematic and structured view and presentation of security-related information is required. Based on the draft standard of Reference Architecture Model for Industry 4.0 (RAMI 4.0), we propose a practical approach to establish a security viewpoint in the CPPS reference architecture model. We investigate the feasibility of using an architecture modeling tool to implement the concept and leverage existing work on models of layered architecture. We demonstrate the applicability for security analysis in two example case studies.

We show how to realize two-factor authentication for a Bitcoin wallet. To do so, we explain how to employ an ECDSA adaption of the two-party signature protocol by MacKenzie and Reiter (Int J Inf Secur 2(3–4):218–239, 2004. doi: 10.1007/s10207-004-0041-0 ) in the context of Bitcoin and present a prototypic implementation of a Bitcoin wallet that offers both: two-factor authentication and verification over a separate channel. Since we use a smart phone as the second authentication factor, our solution can be used with hardware already available to most users and the user experience is quite similar to the existing online banking authentication methods.

The quest for sustainable energy models is the main factor driving research on smart grid technology. SGs represent the bridging paradigm to enable highly efficient energy production, transport, and consumption along the whole chain, from the source to the user. Although this concept promises to be very fruitful, the research on how to deploy it in the real world has just begun. A discussion on the enabling technologies for SGs and a possible roadmap for the profitable evolution thereof is the focus of this article. After introducing the recent trends that are pushing the SG paradigm, we will discuss various key scenarios for the SG, and briefly introduce some of its key requirements. We will then provide an analysis of how current and future standard solutions in the areas of communications and networking can be engineered into a system that fulfills the needs of the SG vision. We advocate the use of small, cheap, and resource-constrained devices with pervasive computing capabilities as the key component to deploy a ubiquitous energy control system. To this end, the recent efforts carried out by Internet standardization bodies such as the IETF and W3C toward the vision of the Internet of Things (IoT) are especially relevant. The various components of the proposed solution have been successfully showcased in real-world implementations, and relevant actors such as ETSI, ZigBee, and IPSO are already evaluating their potential for future IoT applications, making the Internet-based smart grid vision considered in this article practically achievable in the not too distant future.

Due to the widespread popularity in both academia and industry, vehicular ad hoc networks (VANETs) have been used in a wide range of applications starting from intelligent transportation to e-health and itinerary planning. This paper proposes a new decentralized lightweight authentication and key agreement scheme for VANETs. In the proposed scheme, there are three types of mutual authentications: 1) between vehicles; 2) between vehicles and their respective cluster heads; and 3) between cluster heads and their respective roadside units. Apart from these authentications, the proposed scheme also maintains secret keys between roadside units for their secure communications. The rigorous formal and informal security analysis shows that the proposed scheme is capable to defend various malicious attacks. Moreover, the ns-2 simulation demonstrates the practicability of the proposed scheme in VANET environment.

Dear editor, As data are one of the most important factors of today’s intelligent systems but are relatively scarce, people try exchanging data with other organizations or public marketplaces. However, data exchange is now lack of a secure execution mechanism to automatically and fairly protect the rights which are seriously concerned by data owners. For instance, data owners cannot limit the identity of a data transferee who is banned to take the data by any way. Moreover, data transferees are also afraid that their applications for exchange may be unfairly treated. For instance, data owners could dynamically change data price when multiple transferees submit their applications at the same time. A data marketplace can take over the role of a trusted third party during data exchange, but single point failure is always a serious problem in a large-scale distributed system. Furthermore, these marketplaces usually use contracts based on natural languages which may be maliciously misinterpreted and cannot be automatically executed. Numerous researches have been presented to provide digital rights management (DRM) [1]. For instance, Zhu et al. [2] presented a privacypreserving video subscription scheme with the limitation of expire date. However, rights of data owners are more complex than rights of digital content owners. Decentralized and trustworthy enforcement for access control policies is a long-term issue. Han et al. [3] proposed an optimized mechanism which is a trusted and decentralized access control framework for the client/server architecture. These studies still need third party authorities which are used to execute or supervise the rights control, leading to a few trust and security issues. Existing studies [4] on attribute-based access control have been applied to many fields, such as e-commerce and Internet of Things. This article introduces attribute-based access control and proposes a secure framework for data exchange based on smart contracts (DESC) to enable automatic and fair access control. Featuring with decentralized, fair, transparent, immutable and traceable advantages, blockchain 2.0 [5] provides a generalized framework for implementing decentralized compute resources named smart contracts which define rights and policies in mathematical and programming forms. Once the predefined smart contract is triggered by a transaction, it can automatically execute the specific contractual clauses. To find out which data rights are seriously concerned by data owners, we investigate the DRM mechanisms and several data exchange platforms: GBDEX, national engineering lab for big data distribution and exchange technologies. Then,

Smart Grid is designed to integrate advanced communication/networking technologies into electrical power grids to make them ‘‘smarter’’. Current situation is that most of the blackouts and voltage sags could be prevented if we have better and faster communication devices and technologies for the electrical grid. In order to make the current electrical power grid a Smart Grid, the design and implementation of a new communication infrastructure for the grid are two important fields of research. However, Smart Grid projects have only been proposed in recent years and only a few proposals for forwardlooking requirements and initial research work have been offered in this field. No any systematic reviews of communication/networking in Smart Grids have been conducted yet. Therefore, we conduct a systematic review of communication/networking technologies in Smart Grid in this paper, including communication/networking architecture, different communication technologies thatwould be employed into this architecture, quality of service (QoS), optimizing utilization of assets, control and management, etc. © 2011 Elsevier B.V. All rights reserved.

IoT is going to be an established part of life by extending the communication and networking anytime, anywhere. Security requirements for IoT will certainly underline the importance of properly formulated, implemented, and enforced security policies throughout their life-cycle. This paper gives a detailed survey and analysis of embedded security, especially in the area of IoT. Together with the conventional security solutions, the paper highlights the need to provide in-built security in the device itself to provide a flexible infrastructure for dynamic prevention, detection, diagnosis, isolation, and countermeasures against successful breaches. Based on this survey and analysis, the paper defines the security needs taking into account computational time, energy consumption and memory requirements of the devices. Finally, this paper proposes an embedded security framework as a feature of software/hardware co-design methodology.

In many Internet of Thing (IoT) application domains security is a critical requirement, because malicious parties can undermine the effectiveness of IoT-based systems by compromising single components and/or communication channels. Thus, a security infrastructure is needed to ensure the proper functioning of such systems even under attack. However, it is also critical that security be at a reasonable resource and energy cost. In this article, we focus on the problem of efficiently and effectively securing IoT networks by carefully allocating security resources in the network area. In particular, given a set of security resources R and a set of attacks to be faced A, our method chooses the subset of R that best addresses the attacks in A, and the set of locations where to place them, that ensure the security coverage of all IoT devices at minimum cost and energy consumption. We model our problem according to game theory and provide a Pareto-optimal solution in which the cost of the security infrastructure, its energy consumption, and the probability of a successful attack are minimized. Our experimental evaluation shows that our technique improves the system robustness in terms of packet delivery rate for different network topologies. Furthermore, we also provide a method for handling the computation of the resource allocation plan for large-scale networks scenarios, where the optimization problem may require an unreasonable amount of time to be solved. We show how our proposed method drastically reduces the computing time, while providing a reasonable approximation of the optimal solution.


Question: Multi-scale, multi-band DenseNets for the task of audio source separation.

This paper explores the challenge of recognizing and deciphering text within natural images, a complex yet critical area in computer vision with growing significance in various applications. Despite substantial advancements in document character recognition, which have been extensively researched and are effectively resolved for tasks such as reading handwritten digits, recognizing characters within the more intricate context of photographs remains daunting. Current methodologies fall significantly short of human-level performance in this domain. To address this gap, we apply unsupervised feature learning techniques to the specific task of identifying house numbers from street-level images, a real-world scenario. To facilitate research in this area, we introduce a novel benchmark dataset, comprising over 600,000 annotated digits extracted from Street View imagery. We highlight the limitations of traditional, hand-designed feature approaches in tackling this problem. Subsequently, we implement and evaluate two recent unsupervised feature learning methodologies, demonstrating their significant superiority over conventional techniques on our benchmark dataset. This work underscores the potential of unsupervised learning in advancing computer vision capabilities, particularly in recognizing textual elements within complex natural scenes.

OBJECTIVE
Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible.

APPROACH
In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR).

MAIN RESULTS
We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features.

SIGNIFICANCE
Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

Making no claim of being exhaustive, a review of the most popular MFCC (Mel Frequency Cepstral Coefficients) implementations is made. These differ mainly in the particular approximation of the nonlinear pitch perception of human, the filter bank design, and the compression of the filter bank output. Then, a comparative evaluation of the presented implementations is performed on the task of text-independent speaker verification, by means of the well-known 2001 NIST SRE (speaker recognition evaluation) one-speaker detection database.

In this paper, we review recent results on audiovisual (AV) fusion. We also discuss some of the challenges and report on approaches to address them. One important issue in AV fusion is how the modalities interact and influence each other. This review will address this question in the context of AV speech processing, and especially speech recognition, where one of the issues is that the modalities both interact but also sometimes appear to desynchronize from each other. An additional issue that sometimes arises is that one of the modalities may be missing at test time, although it is available at training time; for example, it may be possible to collect AV training data while only having access to audio at test time. We will review approaches to address this issue from the area of multiview learning, where the goal is to learn a model or representation for each of the modalities separately while taking advantage of the rich multimodal training data. In addition to multiview learning, we also discuss the recent application of deep learning (DL) toward AV fusion. We finally draw conclusions and offer our assessment of the future in the area of AV fusion.

In this paper, we present a transfer learning approach for music classification and regression tasks. We propose to use a pre-trained convnet feature, a concatenated feature vector using the activations of feature maps of multiple layers in a trained convolutional network. We show how this convnet feature can serve as general-purpose music representation. In the experiments, a convnet is trained for music tagging and then transferred to other music-related classification and regression tasks. The convnet feature outperforms the baseline MFCC feature in all the considered tasks and several previous approaches that are aggregating MFCCs as well as lowand high-level music features.

Convolutional neural networks (CNNs) have demonstrated superior performance in super-resolution (SR). However, most CNN-based SR methods neglect the different importance among feature channels or fail to take full advantage of the hierarchical features. To address these issues, this paper presents a novel recursive unit. Firstly, at the beginning of each unit, we adopt a compact channel attention mechanism to adaptively recalibrate the channel importance of input features. Then, the multi-level features, rather than only deep-level features, are extracted and fused. Additionally, we find that it will force our model to learn more details by using the learnable upsampling method (i.e., transposed convolution) only on residual branch (instead of using it both on residual branch and identity branch) while using the bicubic interpolation on the other branch. Analytic experiments show that our method achieves competitive results compared with the state-of-the-art methods and maintains faster speed as well.

This work describes a system for acoustic scene classification using large-scale audio feature extraction. It is our contribution to the Scene Classification track of the IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (D-CASE). The system classifies 30 second long recordings of 10 different acoustic scenes. From the highly variable recordings, a large number of spectral, cepstral, energy and voicing-related audio features are extracted. Using a sliding window approach, classification is performed on short windows. SVM are used to classify these short segments, and a majority voting scheme is employed to get a decision for longer recordings. On the official development set of the challenge, an accuracy of 73 % is achieved. SVM are compared with a nearest neighbour classifier and an approach called Latent Perceptual Indexing, whereby SVM achieve the best results. A feature analysis using the t-statistic shows that mainly Mel spectra are the most relevant features.

We propose a multi-task end-to-end optimized deep neural network (MEON) for blind image quality assessment (BIQA). MEON consists of two sub-networks—a distortion identification network and a quality prediction network—sharing the early layers. Unlike traditional methods used for training multi-task networks, our training process is performed in two steps. In the first step, we train a distortion type identification sub-network, for which large-scale training samples are readily available. In the second step, starting from the pre-trained early layers and the outputs of the first sub-network, we train a quality prediction sub-network using a variant of the stochastic gradient descent method. Different from most deep neural networks, we choose biologically inspired generalized divisive normalization (GDN) instead of rectified linear unit as the activation function. We empirically demonstrate that GDN is effective at reducing model parameters/layers while achieving similar quality prediction performance. With modest model complexity, the proposed MEON index achieves state-of-the-art performance on four publicly available benchmarks. Moreover, we demonstrate the strong competitiveness of MEON against state-of-the-art BIQA models using the group maximum differentiation competition methodology.

Acoustic source localization and sound recognition are common acoustic scene analysis tasks that are usually considered separately. In this paper, a new source localization technique is proposed that works jointly with an acoustic event detection system. Given the identities and the end-points of simultaneous sounds, the proposed technique uses the statistical models of those sounds to compute a likelihood score for each model and for each signal at the output of a set of null-steering beamformers per microphone array. Those scores are subsequently combined to find the MAP-optimal event source positions in the room. Experimental work is reported for a scenario consisting of meeting-room acoustic events, either isolated or overlapped with speech. From the localization results, which are compared with those from the SRP-PHAT technique, it seems that the proposed model-based approach can be an alternative to current techniques for event-based localization.

This paper presents a novel audio fingerprinting method that is highly robust to a variety of audio distortions. It is based on an unconventional audio fingerprint generation scheme. The robustness is achieved by generating different versions of the spectrogram matrix of the audio signal by using a threshold based on the average of the spectral values to prune this matrix. We transform each version of this pruned spectrogram matrix into a 2-D binary image. Multiple versions of these 2-D images suppress noise to a varying degree. This varying degree of noise suppression improves likelihood of one of the images matching a reference image. To speed up matching, we convert each image into an n-dimensional vector, and perform a nearest neighbor search based on this n-dimensional vector. We give results with two different feature parameters and their combination. We test this method on TRECVID 2010 content-based copy detection evaluation dataset, and we validate the performance on TRECVID 2009 dataset also. Experimental results show the effectiveness of these features even when the audio is distorted. We compare the proposed method to two state-of-the-art audio copy detection systems, namely NN-based and Shazam systems. Our method by far outperforms Shazam system for all audio transformations (or distortions) in terms of detection performance, number of missed queries and localization accuracy. Compared to NN-based system, our approach reduces minimal Normalized Detection Cost Rate (min NDCR) by 23 % and improves localization accuracy by 24 %.

Numerous deep learning applications benefit from multi-task learning with multiple regression and classification objectives. In this paper we make the observation that the performance of such systems is strongly dependent on the relative weighting between each task’s loss. Tuning these weights by hand is a difficult and expensive process, making multi-task learning prohibitive in practice. We propose a principled approach to multi-task deep learning which weighs multiple loss functions by considering the homoscedastic uncertainty of each task. This allows us to simultaneously learn various quantities with different units or scales in both classification and regression settings. We demonstrate our model learning per-pixel depth regression, semantic and instance segmentation from a monocular input image. Perhaps surprisingly, we show our model can learn multi-task weightings and outperform separate models trained individually on each task.


Question: Multi-scale, multi-band DenseNets for the task of audio source separation.

This paper explores the optimal feature sets for robust visual object recognition, with a focus on human detection using linear SVM-based methods. Through an extensive experimental analysis, we demonstrate that a grid of Histograms of Oriented Gradients (HOG) descriptors outperforms conventional feature sets in human detection tasks. We delve into the impact of each computational stage on performance, highlighting the significance of fine-scale gradients, precise orientation binning, coarser spatial binning, and high-quality local contrast normalization within overlapping descriptor blocks. The adoption of HOG descriptors achieves nearly perfect separation on the original MIT pedestrian database. To further challenge the robustness of our approach, we introduce a more demanding dataset comprising over 1800 annotated human images, featuring a wide range of poses and diverse backgrounds, thereby pushing the boundaries of the method's effectiveness.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: A hybrid of genetic algorithm and particle swarm optimization for recurrent network design

**Similar Documents**: This paper presents a novel approach, the Hybrid Genetic Algorithm-Particle Swarm Optimization (HGAPSO), tailored for the design of recurrent neural networks. HGAPSO combines the strengths of genetic algorithms and particle swarm optimization to create an innovative evolutionary learning algorithm. Unlike traditional design methods, HGAPSO employs both crossover and mutation operations, characteristic of genetic algorithms, and the swarm intelligence principles of particle swarm optimization. This dual approach enhances the search for optimal solutions in the design space of recurrent neural networks. The paper also emphasizes the importance of elite strategy, where top-performing individuals, or elites, are not merely replicated but further optimized through particle swarm optimization. This process mimics the maturation phenomenon observed in natural systems, leading to the refinement of these elites. The enhanced elites then form half of the population in the subsequent generation, while the other half is produced by applying crossover and mutation operations on the optimized elites. HGAPSO is applied to the design of both recurrent neural networks and recurrent fuzzy networks, with the latter utilizing a Takagi-Sugeno-Kang-type architecture. The effectiveness of HGAPSO is validated through its superior performance compared to standard genetic algorithms and particle swarm optimization techniques in recurrent network design problems.

Deeper neural networks are more difficult to train. We present a residual learning framework to ease the training of networks that are substantially deeper than those used previously. We explicitly reformulate the layers as learning residual functions with reference to the layer inputs, instead of learning unreferenced functions. We provide comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth. On the ImageNet dataset we evaluate residual nets with a depth of up to 152 layers - 8× deeper than VGG nets [40] but still having lower complexity. An ensemble of these residual nets achieves 3.57% error on the ImageNet test set. This result won the 1st place on the ILSVRC 2015 classification task. We also present analysis on CIFAR-10 with 100 and 1000 layers. The depth of representations is of central importance for many visual recognition tasks. Solely due to our extremely deep representations, we obtain a 28% relative improvement on the COCO object detection dataset. Deep residual nets are foundations of our submissions to ILSVRC & COCO 2015 competitions1, where we also won the 1st places on the tasks of ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation.

We present recurrent transformer networks (RTNs) for obtaining dense correspondences between semantically similar images. Our networks accomplish this through an iterative process of estimating spatial transformations between the input images and using these transformations to generate aligned convolutional activations. By directly estimating the transformations between an image pair, rather than employing spatial transformer networks to independently normalize each individual image, we show that greater accuracy can be achieved. This process is conducted in a recursive manner to refine both the transformation estimates and the feature representations. In addition, a technique is presented for weakly-supervised training of RTNs that is based on a proposed classification loss. With RTNs, state-of-the-art performance is attained on several benchmarks for semantic correspondence.

In this study, a multi-task deep neural network is proposed for skin lesion analysis. The proposed multi-task learning model solves different tasks (e.g., lesion segmentation and two independent binary lesion classifications) at the same time by exploiting commonalities and differences across tasks. This results in improved learning efficiency and potential prediction accuracy for the task-specific models, when compared to training the individual models separately. The proposed multi-task deep learning model is trained and evaluated on the dermoscopic image sets from the International Skin Imaging Collaboration (ISIC) 2017 Challenge “Skin Lesion Analysis towards Melanoma Detection”, which consists of 2000 training samples and 150 evaluation samples. The experimental results show that the proposed multi-task deep learning model achieves promising performances on skin lesion segmentation and classification. The average value of Jaccard index for lesion segmentation is 0.724, while the average values of area under the receiver operating characteristic curve (AUC) on two individual lesion classifications are 0.880 and 0.972, respectively.

In this paper we investigate the use of deep neural networks (DNNs) for a small footprint text-dependent speaker verification task. At development stage, a DNN is trained to classify speakers at the frame-level. During speaker enrollment, the trained DNN is used to extract speaker specific features from the last hidden layer. The average of these speaker features, or d-vector, is taken as the speaker model. At evaluation stage, a d-vector is extracted for each utterance and compared to the enrolled speaker model to make a verification decision. Experimental results show the DNN based speaker verification system achieves good performance compared to a popular i-vector system on a small footprint text-dependent speaker verification task. In addition, the DNN based system is more robust to additive noise and outperforms the i-vector system at low False Rejection operating points. Finally the combined system outperforms the i-vector system by 14% and 25% relative in equal error rate (EER) for clean and noisy conditions respectively.

The CHiME challenge series aims to advance robust automatic speech recognition (ASR) technology by promoting research at the interface of speech and language processing, signal processing, and machine learning. This paper introduces the 5th CHiME Challenge, which considers the task of distant multimicrophone conversational ASR in real home environments. Speech material was elicited using a dinner party scenario with efforts taken to capture data that is representative of natural conversational speech and recorded by 6 Kinect microphone arrays and 4 binaural microphone pairs. The challenge features a single-array track and a multiple-array track and, for each track, distinct rankings will be produced for systems focusing on robustness with respect to distant-microphone capture vs. systems attempting to address all aspects of the task including conversational language modeling. We discuss the rationale for the challenge and provide a detailed description of the data collection procedure, the task, and the baseline systems for array synchronization, speech enhancement, and conventional and end-to-end ASR.

This is a polyphonic music dataset which can be used for versatile research problems, such as Multi-pitch Estimation and Tracking, Audio-score Alignment, Source Separation, etc. This dataset consists of the audio recordings of each part and the ensemble of ten pieces of four-part J.S. Bach chorales, as well as their MIDI scores, the ground-truth alignment between the audio and the score, the ground-truth pitch values of each part and the ground-truth notes of each piece. The audio recordings of the four parts (Soprano, Alto, Tenor and Bass) of each piece are performed by violin, clarinet, saxophone and bassoon, respectively.

In the paper we present new Alternating Least Squares (ALS) algorithms for Nonnegative Matrix Factorization (NMF) and their extensions to 3D Nonnegative Tensor Factorization (NTF) that are robust in the presence of noise and have many potential applications, including multi-way Blind Source Separation (BSS), multi-sensory or multi-dimensional data analysis, and nonnegative neural sparse coding. We propose to use local cost functions whose simultaneous or sequential (one by one) minimization leads to a very simple ALS algorithm which works under some sparsity constraints both for an under-determined (a system which has less sensors than sources) and over-determined model. The extensive experimental results confirm the validity and high performance of the developed algorithms, especially with usage of the multi-layer hierarchical NMF. Extension of the proposed algorithm to multidimensional Sparse Component Analysis and Smooth Component Analysis is also proposed.

This paper describes digital circuit architectures for automatic speech recognition (ASR) and voice activity detection (VAD) with improved accuracy, programmability, and scalability. Our ASR architecture is designed to minimize off-chip memory bandwidth, which is the main driver of system power consumption. A SIMD processor with 32 parallel execution units efficiently evaluates feed-forward deep neural networks (NNs) for ASR, limiting memory usage with a sparse quantized weight matrix format. We argue that VADs should prioritize accuracy over area and power, and introduce a VAD circuit that uses an NN to classify modulation frequency features with 22.3-<inline-formula> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> power consumption. The 65-nm test chip is shown to perform a variety of ASR tasks in real time, with vocabularies ranging from 11 words to 145 000 words and full-chip power consumption ranging from 172 <inline-formula> <tex-math notation="LaTeX">$\mu \text{W}$ </tex-math></inline-formula> to 7.78 mW.

Monaural speech segregation is an important problem in robust speech processing and has been formulated as a supervised learning problem. In supervised learning methods, the ideal binary mask (IBM) is usually used as the target because of its simplicity and large speech intelligibility gains. Recently, the ideal ratio mask (IRM) has been found to improve the speech quality over the IBM. However, the IRM was originally defined in anechoic conditions and did not consider the effect of reverberation. In this paper, the IRM is extended to reverberant conditions where the direct sound and early reflections of target speech are regarded as the desired signal. Deep neural networks (DNNs) is employed to estimate the extended IRM in the noisy reverberant conditions. The estimated IRM is then applied to the noisy reverberant mixture for speech segregation. Experimental results show that the estimated IRM provides substantial improvements in speech intelligibility and speech quality over the unprocessed mixture signals under various noisy and reverberant conditions.

The driving force behind deep networks is their ability to compactly represent rich classes of functions. The primary notion for formally reasoning about this phenomenon is expressive efficiency, which refers to a situation where one network must grow unfeasibly large in order to realize (or approximate) functions of another. To date, expressive efficiency analyses focused on the architectural feature of depth, showing that deep networks are representationally superior to shallow ones. In this paper we study the expressive efficiency brought forth by connectivity, motivated by the observation that modern networks interconnect their layers in elaborate ways. We focus on dilated convolutional networks, a family of deep models delivering state of the art performance in sequence processing tasks. By introducing and analyzing the concept of mixed tensor decompositions, we prove that interconnecting dilated convolutional networks can lead to expressive efficiency. In particular, we show that even a single connection between intermediate layers can already lead to an almost quadratic gap, which in large-scale settings typically makes the difference between a model that is practical and one that is not. Empirical evaluation demonstrates how the expressive efficiency of connectivity, similarly to that of depth, translates into gains in accuracy. This leads us to believe that expressive efficiency may serve a key role in the development of new tools for deep network design.


Question: Multi-scale, multi-band DenseNets for the task of audio source separation.

This paper introduces a set of empirical tests to directly assess the degree of invariance in features learned by deep networks, a critical aspect for pattern recognition tasks where ideal features should remain constant across various confounding variables like illumination and viewing angles. Recent advancements in deep learning have proposed unsupervised training methods for deep architectures to automatically extract useful features. However, evaluating these features often relies solely on their performance in classification tasks. To address this, we propose several metrics that quantitatively measure the invariance of learned features under different input transformations. Our findings indicate that stacked autoencoders, when trained on natural images, generate features that show a modest increase in invariance with each additional layer. More significantly, convolutional deep belief networks demonstrate a substantial increase in invariance at each layer. These results not only reinforce the rationale for using deep representations over shallower ones but also suggest that mechanisms beyond simple layer stacking in autoencoders are crucial for achieving invariance. Our evaluation methods can serve as a benchmark for future research in deep learning, guiding the development of more sophisticated algorithms.

Question: Multi-Scale multi-band densenets for audio source separation

Documents: Multi-scale Multi-band Densenets for Audio Source Separation
Audio source separation is a challenging task that aims to isolate individual sound sources from a mixed audio signal. Recent advancements in deep learning have led to the development of various neural network architectures designed to tackle this problem. This paper introduces a novel approach, Multi-scale Multi-band Densenets (MSMB-Densenets), which combines the benefits of multi-scale and multi-band processing with the dense connectivity of Densenets. The MSMB-Densenets architecture is specifically tailored to enhance the separation of audio sources by leveraging the dense connections between layers, which facilitate the efficient reuse of features. Additionally, the multi-scale and multi-band processing allows the network to capture a wide range of temporal and spectral characteristics of audio signals. Through extensive experiments on various audio datasets, we demonstrate that our proposed MSMB-Densenets outperform state-of-the-art methods in terms of both objective and subjective evaluation metrics for audio source separation. Furthermore, we provide a detailed analysis of the impact of different design choices on the performance of MSMB-Densenets, offering insights for future research in this domain.

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation. Our key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet, the VGG net, and GoogLeNet) into fully convolutional networks and transfer their learned representations by fine-tuning to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional networks achieve improved segmentation of PASCAL VOC (30% relative improvement to 67.2% mean IU on 2012), NYUDv2, SIFT Flow, and PASCAL-Context, while inference takes one tenth of a second for a typical image.

We present a general approach to reduce the size of feedforward deep neural networks (DNNs). We propose a rankconstrained topology, which factors the weights in the input layer of the DNN in terms of a low-rank representation: unlike previous work, our technique is applied at the level of the filters learned at individual hidden layer nodes, and exploits the natural two-dimensional time-frequency structure in the input. These techniques are applied on a small-footprint DNN-based keyword spotting task, where we find that we can reduce model size by 75% relative to the baseline, without any loss in performance. Furthermore, we find that the proposed approach is more effective at improving model performance compared to other popular dimensionality reduction techniques, when evaluated with a comparable number of parameters.

This letter describes modifications to locally normalized filter banks (LNFB), which substantially improve their performance on the Aurora-4 robust speech recognition task using a Deep Neural Network-Hidden Markov Model (DNN-HMM)-based speech recognition system. The modified coefficients, referred to as LNFB features, are a filter-bank version of locally normalized cepstral coefficients (LNCC), which have been described previously. The ability of the LNFB features is enhanced through the use of newly proposed dynamic versions of them, which are developed using an approach that differs somewhat from the traditional development of delta and delta–delta features. Further enhancements are obtained through the use of mean normalization and mean–variance normalization, which is evaluated both on a per-speaker and a per-utterance basis. The best performing feature combination (typically LNFB combined with LNFB delta and delta–delta features and mean–variance normalization) provides an average relative reduction in word error rate of 11.4% and 9.4%, respectively, compared to comparable features derived from Mel filter banks when clean and multinoise training are used for the Aurora-4 evaluation. The results presented here suggest that the proposed technique is more robust to channel mismatches between training and testing data than MFCC-derived features and is more effective in dealing with channel diversity.

Compressed sensing (CS) samples signals at a much lower rate than the Nyquist rate if they are sparse in some basis. In this paper, the CS methodology is applied to sinusoidally modeled audio signals. As this model is sparse by definition in the frequency domain (being equal to the sum of a small number of sinusoids), we investigate whether CS can be used to encode audio signals at low bitrates. In contrast to encoding the sinusoidal parameters (amplitude, frequency, phase) as current state-of-the-art methods do, we propose encoding few randomly selected samples of the time-domain description of the sinusoidal component (per signal segment). The potential of applying compressed sensing both to single-channel and multi-channel audio coding is examined. The listening test results are encouraging, indicating that the proposed approach can achieve comparable performance to that of state-of-the-art methods. Given that CS can lead to novel coding systems where the sampling and compression operations are combined into one low-complexity step, the proposed methodology can be considered as an important step towards applying the CS framework to audio coding applications.

This paper focuses on the causality problem in the task of Blind Source Separation (BSS) of speech signals in nonminimum-phase mixing channels. We propose a new algorithm for solving this problem using filter decomposition approach. Our proposed algorithm uses an integrated cost function in which independence criterion is defined in frequency-domain. The parameters of demixing system are derived in time-domain, so the algorithm has the benefits of both time and frequency-domain approaches. Compared to the previous work in this framework, our proposed algorithm is the extension of filter decomposition idea in multi-channel blind deconvolution to the problem of blind source separation of speech signals. The proposed method is capable of dealing with both minimum-phase and nonminimum-phase mixing situations. Simulation results show considerable improvement in separating speech signals specially when the mixing system is nonminimum-phase.

This is a polyphonic music dataset which can be used for versatile research problems, such as Multi-pitch Estimation and Tracking, Audio-score Alignment, Source Separation, etc. This dataset consists of the audio recordings of each part and the ensemble of ten pieces of four-part J.S. Bach chorales, as well as their MIDI scores, the ground-truth alignment between the audio and the score, the ground-truth pitch values of each part and the ground-truth notes of each piece. The audio recordings of the four parts (Soprano, Alto, Tenor and Bass) of each piece are performed by violin, clarinet, saxophone and bassoon, respectively.

This paper reports on a project for collection of the sound scene data. The sound scene data is necessary for studies such as sound source localization, sound retrieval, sound recognition and hands-free speech recognition in real acoustical environments. There are many kinds of sound scenes in real environments. The sound scene is denoted by sound sources and room acoustics. The number of combination of the sound sources, source positions and rooms is huge in real acoustical environments. However, the sound in the environments can be simulated by convolution of the isolated sound sources and impulse responses. As an isolated sound source, a hundred kinds of non-speech sounds and speech sounds are collected. The impulse responses are collected in various acoustical environments. In this paper, progress of our sound scene database project and application to environment sound recognition are described.

Image super-resolution is a fundamental pre-processing technique for the machine vision applications of robotics and other mobile platforms. Inevitably, images captured by the mobile camera tend to emerge severe motion blur and this degradation will deteriorate the performance of current state-of-the-art super-resolution methods. In this paper, we propose a deep dual-branch convolution neural network (CNN) to generate a clear high-resolution image from a single natural image with severe blurs. Compared to off-the-shelf methods, our method, called DB-SRN, can remove the complex non-uniform motion blurs and restore useful texture details simultaneously. By sharing the features from modified residual blocks (ResBlocks), the dual-branch design can promote the performances of both tasks other while retaining network simplicity. Extensive experiments demonstrate that our method produces remarkable deblurred and super-resolved images in terms of quality and quantity with high computational efficiency.

We present a novel and practical deep fully convolutional neural network architecture for semantic pixel-wise segmentation termed SegNet. This core trainable segmentation engine consists of an encoder network, a corresponding decoder network followed by a pixel-wise classification layer. The architecture of the encoder network is topologically identical to the 13 convolutional layers in the VGG16 network [1] . The role of the decoder network is to map the low resolution encoder feature maps to full input resolution feature maps for pixel-wise classification. The novelty of SegNet lies is in the manner in which the decoder upsamples its lower resolution input feature map(s). Specifically, the decoder uses pooling indices computed in the max-pooling step of the corresponding encoder to perform non-linear upsampling. This eliminates the need for learning to upsample. The upsampled maps are sparse and are then convolved with trainable filters to produce dense feature maps. We compare our proposed architecture with the widely adopted FCN [2] and also with the well known DeepLab-LargeFOV [3] , DeconvNet [4] architectures. This comparison reveals the memory versus accuracy trade-off involved in achieving good segmentation performance. SegNet was primarily motivated by scene understanding applications. Hence, it is designed to be efficient both in terms of memory and computational time during inference. It is also significantly smaller in the number of trainable parameters than other competing architectures and can be trained end-to-end using stochastic gradient descent. We also performed a controlled benchmark of SegNet and other architectures on both road scenes and SUN RGB-D indoor scene segmentation tasks. These quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures. We also provide a Caffe implementation of SegNet and a web demo at http://mi.eng.cam.ac.uk/projects/segnet/.

We present Deep Voice 3, a fully-convolutional attention-based neural textto-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturalness while training an order of magnitude faster. We scale Deep Voice 3 to dataset sizes unprecedented for TTS, training on more than eight hundred hours of audio from over two thousand speakers. In addition, we identify common error modes of attention-based speech synthesis networks, demonstrate how to mitigate them, and compare several different waveform synthesis methods. We also describe how to scale inference to ten million queries per day on a single GPU server.


Question: Multi-scale, multi-band DenseNets for the task of audio source separation.

This paper presents a vast image dataset, encompassing 79,302,017 images, meticulously gathered from the internet, designed to facilitate nonparametric object and scene recognition. Inspired by psychological studies that reveal the human visual system's resilience to image degradation, the dataset features images at a resolution of 32x32 pixels, ensuring efficient storage and processing. Each image is loosely annotated with one of the 75,062 non-abstract nouns found in the English language, as cataloged in the Wordnet lexical database. This comprehensive labeling approach covers an extensive array of object categories and scenes, providing a rich resource for exploring the visual world. By leveraging the semantic information from Wordnet in conjunction with nearest-neighbor algorithms, the dataset supports object classification across various semantic levels, effectively mitigating labeling inaccuracies. For classes that are abundantly represented in the dataset, such as human figures, the recognition performance is comparable to that of specialized Viola-Jones style detectors, demonstrating the dataset's potential for advanced visual recognition tasks.

We propose a deep convolutional neural network architecture codenamed Inception that achieves the new state of the art for classification and detection in the ImageNet Large-Scale Visual Recognition Challenge 2014 (ILSVRC14). The main hallmark of this architecture is the improved utilization of the computing resources inside the network. By a carefully crafted design, we increased the depth and width of the network while keeping the computational budget constant. To optimize quality, the architectural decisions were based on the Hebbian principle and the intuition of multi-scale processing. One particular incarnation used in our submission for ILSVRC14 is called GoogLeNet, a 22 layers deep network, the quality of which is assessed in the context of classification and detection.

This paper introduces a novel large dataset for example-based single image super-resolution and studies the state-of-the-art as emerged from the NTIRE 2017 challenge. The challenge is the first challenge of its kind, with 6 competitions, hundreds of participants and tens of proposed solutions. Our newly collected DIVerse 2K resolution image dataset (DIV2K) was employed by the challenge. In our study we compare the solutions from the challenge to a set of representative methods from the literature and evaluate them using diverse measures on our proposed DIV2K dataset. Moreover, we conduct a number of experiments and draw conclusions on several topics of interest. We conclude that the NTIRE 2017 challenge pushes the state-of-the-art in single-image super-resolution, reaching the best results to date on the popular Set5, Set14, B100, Urban100 datasets and on our newly proposed DIV2K.

This letter describes modifications to locally normalized filter banks (LNFB), which substantially improve their performance on the Aurora-4 robust speech recognition task using a Deep Neural Network-Hidden Markov Model (DNN-HMM)-based speech recognition system. The modified coefficients, referred to as LNFB features, are a filter-bank version of locally normalized cepstral coefficients (LNCC), which have been described previously. The ability of the LNFB features is enhanced through the use of newly proposed dynamic versions of them, which are developed using an approach that differs somewhat from the traditional development of delta and delta–delta features. Further enhancements are obtained through the use of mean normalization and mean–variance normalization, which is evaluated both on a per-speaker and a per-utterance basis. The best performing feature combination (typically LNFB combined with LNFB delta and delta–delta features and mean–variance normalization) provides an average relative reduction in word error rate of 11.4% and 9.4%, respectively, compared to comparable features derived from Mel filter banks when clean and multinoise training are used for the Aurora-4 evaluation. The results presented here suggest that the proposed technique is more robust to channel mismatches between training and testing data than MFCC-derived features and is more effective in dealing with channel diversity.

OBJECTIVE
Brain-computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional neural networks (CNNs), which have been used in computer vision and speech recognition to perform automatic feature extraction and classification, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible.

APPROACH
In this work we introduce EEGNet, a compact convolutional neural network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet, both for within-subject and cross-subject classification, to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR).

MAIN RESULTS
We show that EEGNet generalizes across paradigms better than, and achieves comparably high performance to, the reference algorithms when only limited training data is available across all tested paradigms. In addition, we demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features.

SIGNIFICANCE
Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks. Our models can be found at: https://github.com/vlawhern/arl-eegmodels.

We present SlowFast networks for video recognition. Our model involves (i) a Slow pathway, operating at low frame rate, to capture spatial semantics, and (ii) a Fast pathway, operating at high frame rate, to capture motion at fine temporal resolution. The Fast pathway can be made very lightweight by reducing its channel capacity, yet can learn useful temporal information for video recognition. Our models achieve strong performance for both action classification and detection in video, and large improvements are pin-pointed as contributions by our SlowFast concept. We report 79.0% accuracy on the Kinetics dataset without using any pre-training, largely surpassing the previous best results of this kind. On AVA action detection we achieve a new stateof-the-art of 28.3 mAP. Code will be made publicly available.

The CHiME challenge series aims to advance robust automatic speech recognition (ASR) technology by promoting research at the interface of speech and language processing, signal processing, and machine learning. This paper introduces the 5th CHiME Challenge, which considers the task of distant multimicrophone conversational ASR in real home environments. Speech material was elicited using a dinner party scenario with efforts taken to capture data that is representative of natural conversational speech and recorded by 6 Kinect microphone arrays and 4 binaural microphone pairs. The challenge features a single-array track and a multiple-array track and, for each track, distinct rankings will be produced for systems focusing on robustness with respect to distant-microphone capture vs. systems attempting to address all aspects of the task including conversational language modeling. We discuss the rationale for the challenge and provide a detailed description of the data collection procedure, the task, and the baseline systems for array synchronization, speech enhancement, and conventional and end-to-end ASR.

In our recent work an effective method for multiple source localization has been proposed under the name of cumulative State Coherence Transform (cSCT). Exploiting the physical meaning of the frequency-domain blind source separation and the sparse time-frequency dominance of the acoustic sources, multiple reliable TDOAs can be estimated with only two microphones, regardless of the permutation problem and of the microphone spacing. In this paper we present a multidimensional generalization of the cSCT which allows one to localize several sources in the P-dimensional space. An important approximation is made in order to perform a disjoint TDOA estimation over each dimension which reduces the localization problem to linear complexity. Furthermore the approach is invariant to the array geometry and to the assumed acoustic propagation model. Experimental results on simulated data show a precise 2-D localization of 7 sources by only using an array of three elements.

This letter presents theoretical, algorithmic, and experimental results about nonnegative matrix factorization (NMF) with the Itakura-Saito (IS) divergence. We describe how IS-NMF is underlaid by a well-defined statistical model of superimposed gaussian components and is equivalent to maximum likelihood estimation of variance parameters. This setting can accommodate regularization constraints on the factors through Bayesian priors. In particular, inverse-gamma and gamma Markov chain priors are considered in this work. Estimation can be carried out using a space-alternating generalized expectation-maximization (SAGE) algorithm; this leads to a novel type of NMF algorithm, whose convergence to a stationary point of the IS cost function is guaranteed. We also discuss the links between the IS divergence and other cost functions used in NMF, in particular, the Euclidean distance and the generalized Kullback-Leibler (KL) divergence. As such, we describe how IS-NMF can also be performed using a gradient multiplicative algorithm (a standard algorithm structure in NMF) whose convergence is observed in practice, though not proven. Finally, we report a furnished experimental comparative study of Euclidean-NMF, KL-NMF, and IS-NMF algorithms applied to the power spectrogram of a short piano sequence recorded in real conditions, with various initializations and model orders. Then we show how IS-NMF can successfully be employed for denoising and upmix (mono to stereo conversion) of an original piece of early jazz music. These experiments indicate that IS-NMF correctly captures the semantics of audio and is better suited to the representation of music signals than NMF with the usual Euclidean and KL costs.

Energy disaggregation estimates appliance-by-appliance electricity consumption from a single meter that measures the whole home's electricity demand. Recently, deep neural networks have driven remarkable improvements in classification performance in neighbouring machine learning fields such as image classification and automatic speech recognition. In this paper, we adapt three deep neural network architectures to energy disaggregation: 1) a form of recurrent neural network called `long short-term memory' (LSTM); 2) denoising autoencoders; and 3) a network which regresses the start time, end time and average power demand of each appliance activation. We use seven metrics to test the performance of these algorithms on real aggregate power data from five appliances. Tests are performed against a house not seen during training and against houses seen during training. We find that all three neural nets achieve better F1 scores (averaged over all five appliances) than either combinatorial optimisation or factorial hidden Markov models and that our neural net algorithms generalise well to an unseen house.

In this paper we address three different computer vision tasks using a single basic architecture: depth prediction, surface normal estimation, and semantic labeling. We use a multiscale convolutional network that is able to adapt easily to each task using only small modifications, regressing from the input image to the output map directly. Our method progressively refines predictions using a sequence of scales, and captures many image details without any superpixels or low-level segmentation. We achieve state-of-the-art performance on benchmarks for all three tasks.


Question: Multi-scale, multi-band DenseNets for the separation of audio sources.

This paper explores a compelling error correction scenario involving real-valued input and output data. The primary objective is to accurately reconstruct an input vector \(f \in \mathbb{R}^n\) from measurements that have been corrupted, \(y = Af + e\), where \(A\) is an \(m \times n\) coding matrix and \(e\) is an arbitrary, unknown error vector. The central question is whether it's feasible to precisely recover \(f\) from \(y\). We demonstrate that, under certain conditions on \(A\), the original vector \(f\) is uniquely recoverable via an \(\ell_1\)-minimization problem (where \(\|x\|_{\ell_1} = \sum_i |x_i|\)) by solving \(\min_{g \in \mathbb{R}^n} \|y - Ag\|_{\ell_1}\), provided the error vector's support does not exceed a threshold, i.e., \(\|e\|_{\ell_0} = |{i : e_i \neq 0}|\) is less than or equal to \(\rho \cdot m\) for a positive \(\rho\). In essence, the exact recovery of \(f\) is achieved through a simple convex optimization problem that can be reformulated as a linear program. Empirical evidence suggests that this recovery method performs exceptionally well, even when a considerable portion of the output is corrupted. This work intersects with the challenge of finding sparse solutions to severely underdetermined linear systems and bears significant relevance to the problem of reconstructing signals from highly incomplete measurements. Notably, the findings herein improve upon our previous research. Moreover, the success of the \(\ell_1\) minimization technique hinges on a critical property we term the Uniform Uncertainty Principle, which we will detail extensively.

The above document has been transformed to maintain its length, complexity, and semantic style while preserving its relationship to the question. It discusses a method for reconstructing input vectors from corrupted measurements, emphasizing the use of \(\ell_1\) minimization and the Uniform Uncertainty Principle. This is in line with the original document's focus on error correction and signal recovery.

This paper reports on a project for collection of the sound scene data. The sound scene data is necessary for studies such as sound source localization, sound retrieval, sound recognition and hands-free speech recognition in real acoustical environments. There are many kinds of sound scenes in real environments. The sound scene is denoted by sound sources and room acoustics. The number of combination of the sound sources, source positions and rooms is huge in real acoustical environments. However, the sound in the environments can be simulated by convolution of the isolated sound sources and impulse responses. As an isolated sound source, a hundred kinds of non-speech sounds and speech sounds are collected. The impulse responses are collected in various acoustical environments. In this paper, progress of our sound scene database project and application to environment sound recognition are described.

Distributed speech recognition DSR splits the processing of data between a mobile device and a network server. In the front-end, features are extracted and compressed to transmit over a wireless channel to a back-end server, where the incoming stream is received and reconstructed for recognition tasks. In this paper, we propose a feature compression algorithm termed suppression by selecting wavelets SSW to achieve the two main goals of DSR: Minimizing memory and device requirements while also maintaining or even improving the recognition performance. The SSW approach first applies the discrete wavelet transform DWT to filter the incoming speech feature sequence into two temporal subsequences at the client terminal. Feature compression is achieved by keeping the low modulation frequency subsequence while discarding the high frequency counterpart. The low-frequency subsequence is then transmitted across the remote network for specific feature statistics normalization. Wavelets are favorable for resolving the temporal properties of the feature sequence, and the down-sampling process in DWT achieves data compression by reducing the amount of data at the terminal prior to transmission across the network. Once the compressed features have arrived at the server, the feature sequence can be enhanced by statistics normalization, reconstructed with inverse DWT, and compensated with a simple post filter to alleviate any over-smoothing effects from the compression stage. Results on a standard robustness task Aurora-4 and on a Mandarin Chinese news corpus showed SSW outperforms conventional noise-robustness techniques while also providing nearly a 50% compression rate during the transmission stage of DSR systems.

Deep learning yields great results across many fields, from speech recognition, image classification, to translation. But for each problem, getting a deep model to work well involves research into the architecture and a long period of tuning. We present a single model that yields good results on a number of problems spanning multiple domains. In particular, this single model is trained concurrently on ImageNet, multiple translation tasks, image captioning (COCO dataset), a speech recognition corpus, and an English parsing task. Our model architecture incorporates building blocks from multiple domains. It contains convolutional layers, an attention mechanism, and sparsely-gated layers. Each of these computational blocks is crucial for a subset of the tasks we train on. Interestingly, even if a block is not crucial for a task, we observe that adding it never hurts performance and in most cases improves it on all tasks. We also show that tasks with less data benefit largely from joint training with other tasks, while performance on large tasks degrades only slightly if at all.

We introduce a technique for augmenting neural text-to-speech (TTS) with lowdimensional trainable speaker embeddings to generate different voices from a single model. As a starting point, we show improvements over the two state-ofthe-art approaches for single-speaker neural TTS: Deep Voice 1 and Tacotron. We introduce Deep Voice 2, which is based on a similar pipeline with Deep Voice 1, but constructed with higher performance building blocks and demonstrates a significant audio quality improvement over Deep Voice 1. We improve Tacotron by introducing a post-processing neural vocoder, and demonstrate a significant audio quality improvement. We then demonstrate our technique for multi-speaker speech synthesis for both Deep Voice 2 and Tacotron on two multi-speaker TTS datasets. We show that a single neural TTS system can learn hundreds of unique voices from less than half an hour of data per speaker, while achieving high audio quality synthesis and preserving the speaker identities almost perfectly.

This work describes a system for acoustic scene classification using large-scale audio feature extraction. It is our contribution to the Scene Classification track of the IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (D-CASE). The system classifies 30 second long recordings of 10 different acoustic scenes. From the highly variable recordings, a large number of spectral, cepstral, energy and voicing-related audio features are extracted. Using a sliding window approach, classification is performed on short windows. SVM are used to classify these short segments, and a majority voting scheme is employed to get a decision for longer recordings. On the official development set of the challenge, an accuracy of 73 % is achieved. SVM are compared with a nearest neighbour classifier and an approach called Latent Perceptual Indexing, whereby SVM achieve the best results. A feature analysis using the t-statistic shows that mainly Mel spectra are the most relevant features.

Convolutional networks are powerful visual models that yield hierarchies of features. We show that convolutional networks by themselves, trained end-to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation. Our key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning. We define and detail the space of fully convolutional networks, explain their application to spatially dense prediction tasks, and draw connections to prior models. We adapt contemporary classification networks (AlexNet, the VGG net, and GoogLeNet) into fully convolutional networks and transfer their learned representations by fine-tuning to the segmentation task. We then define a skip architecture that combines semantic information from a deep, coarse layer with appearance information from a shallow, fine layer to produce accurate and detailed segmentations. Our fully convolutional networks achieve improved segmentation of PASCAL VOC (30% relative improvement to 67.2% mean IU on 2012), NYUDv2, SIFT Flow, and PASCAL-Context, while inference takes one tenth of a second for a typical image.

In this paper, we explore different ways to extend a recurrent neural network (RNN) to a deep RNN. We start by arguing that the concept of depth in an RNN is not as clear as it is in feedforward neural networks. By carefully analyzing and understanding the architecture of an RNN, however, we find three points of an RNN which may be made deeper; (1) input-to-hidden function, (2) hidden-tohidden transition and (3) hidden-to-output function. Based on this observation, we propose two novel architectures of a deep RNN which are orthogonal to an earlier attempt of stacking multiple recurrent layers to build a deep RNN (Schmidhuber, 1992; El Hihi and Bengio, 1996). We provide an alternative interpretation of these deep RNNs using a novel framework based on neural operators. The proposed deep RNNs are empirically evaluated on the tasks of polyphonic music prediction and language modeling. The experimental result supports our claim that the proposed deep RNNs benefit from the depth and outperform the conventional, shallow RNNs.

The author investigates automatic recognition of singers’ gender and age through audio features using deep neural network (DNN). Features of each singing voice, fundamental frequency and Mel-Frequency Cepstrum Coefficients (MFCC) are extracted for neural network training. 10,000 singing voice from Smule’s Sing! Karaoke app is used for training and evaluation, and the DNN-based method achieves an average recall of 91% for gender classification and 36% for age identification.

We constructed an acoustic, gesture-based recognition system called Multiwave, which leverages the Doppler Effect to translate multidimensional movements into user interface commands. Our system only requires the use of two speakers and a microphone to be operational. Since these components are already built in to most end user systems, our design makes gesture-based input more accessible to a wider range of end users. By generating a known high frequency tone from multiple speakers and detecting movement using changes in the sound waves, we are able to calculate a Euclidean representation of hand velocity that is then used for more natural gesture recognition and thus, more meaningful interaction mappings. We present the results of a user study of Multiwave to evaluate recognition rates for different gestures and report accuracy rates comparable to or better than the current state of the art. We also report subjective user feedback and some lessons learned from our system that provide additional insight for future applications of multidimensional gesture recognition.

The CHiME challenge series aims to advance robust automatic speech recognition (ASR) technology by promoting research at the interface of speech and language processing, signal processing, and machine learning. This paper introduces the 5th CHiME Challenge, which considers the task of distant multimicrophone conversational ASR in real home environments. Speech material was elicited using a dinner party scenario with efforts taken to capture data that is representative of natural conversational speech and recorded by 6 Kinect microphone arrays and 4 binaural microphone pairs. The challenge features a single-array track and a multiple-array track and, for each track, distinct rankings will be produced for systems focusing on robustness with respect to distant-microphone capture vs. systems attempting to address all aspects of the task including conversational language modeling. We discuss the rationale for the challenge and provide a detailed description of the data collection procedure, the task, and the baseline systems for array synchronization, speech enhancement, and conventional and end-to-end ASR.


Question: Personalized recommendation in online social networks: integrating personal preferences and location-based community trends.

This research delves into the significant challenge of understanding and modeling influence propagation within social networks, a topic that has garnered considerable attention recently. Traditionally, studies in this field have relied on a predefined social graph, where edges denote the probability of influence between users. However, the critical question of how these probabilities are derived from actual social network data has received scant attention. This paper seeks to address this gap by exploring whether, given a social graph and a record of user activities, one can construct accurate models of influence. We introduce novel models and algorithms for estimating the parameters of these models and evaluating their predictive capabilities. Furthermore, we introduce methodologies for forecasting the time frame within which a user might engage in a specific action. Our findings are substantiated using the Flickr dataset, which encompasses a social graph with over 1.3 million nodes, 40 million edges, and an action log comprising 35 million records pertaining to 300,000 distinct actions. Our research not only confirms the presence of substantial influence dynamics in real-world social networks but also demonstrates the remarkable predictive accuracy of our techniques.

Newly emerged event-based online social services, such as Meetup and Plancast, have experienced increased popularity and rapid growth. From these services, we observed a new type of social network - event-based social network (EBSN). An EBSN does not only contain online social interactions as in other conventional online social networks, but also includes valuable offline social interactions captured in offline activities. By analyzing real data collected from Meetup, we investigated EBSN properties and discovered many unique and interesting characteristics, such as heavy-tailed degree distributions and strong locality of social interactions.
 We subsequently studied the heterogeneous nature (co-existence of both online and offline social interactions) of EBSNs on two challenging problems: community detection and information flow. We found that communities detected in EBSNs are more cohesive than those in other types of social networks (e.g. location-based social networks). In the context of information flow, we studied the event recommendation problem. By experimenting various information diffusion patterns, we found that a community-based diffusion model that takes into account of both online and offline interactions provides the best prediction power.
 This paper is the first research to study EBSNs at scale and paves the way for future studies on this new type of social network. A sample dataset of this study can be downloaded from http://www.largenetwork.org/ebsn.

Incorporating users’ personality traits has shown to be instrumental in many personalized retrieval and recommender systems. Analysis of users’ digital traces has become an important resource for inferring personality traits. To date, the analysis of users’ explicit and latent characteristics is typically restricted to a single social networking site (SNS). In this work, we propose a novel method that integrates text, image, and users’ meta features from two different SNSs: Twitter and Instagram. Our preliminary results indicate that the joint analysis of users’ simultaneous activities in two popular SNSs seems to lead to a consistent decrease of the prediction errors for each personality trait.

Existing recommender algorithms mainly focused on recommending individual items by utilizing user-item interactions. However, little attention has been paid to recommend user generated lists (e.g., playlists and booklists). On one hand, user generated lists contain rich signal about item co-occurrence, as items within a list are usually gathered based on a specific theme. On the other hand, a user's preference over a list also indicate her preference over items within the list. We believe that 1) if the rich relevance signal within user generated lists can be properly leveraged, an enhanced recommendation for individual items can be provided, and 2) if user-item and user-list interactions are properly utilized, and the relationship between a list and its contained items is discovered, the performance of user-item and user-list recommendations can be mutually reinforced.
 Towards this end, we devise embedding factorization models, which extend traditional factorization method by incorporating item-item (item-item-list) co-occurrence with embedding-based algorithms. Specifically, we employ factorization model to capture users' preferences over items and lists, and utilize embedding-based models to discover the co-occurrence information among items and lists. The gap between the two types of models is bridged by sharing items' latent factors. Remarkably, our proposed framework is capable of solving the new-item cold-start problem, where items have never been consumed by users but exist in user generated lists. Overall performance comparisons and micro-level analyses demonstrate the promising performance of our proposed approaches.

Recommender systems have traditionally recommended items to individual users, but there has recently been a proliferation of recommenders that address their recommendations to groups of users. The shift of focus from an individual to a group makes more of a difference than one might at first expect. This chapter discusses the most important new issues that arise, organizing them in terms of four subtasks that can or must be dealt with by a group recommender: 1. acquiring information about the user’s preferences; 2. generating recommendations; 3. explaining recommendations; and 4. helping users to settle on a final decision. For each issue, we discuss how it has been dealt with in existing group recommender systems and what open questions call for further research.

This paper introduces a new dataset and compares several methods for the recommendation of non-fiction audio visual material, namely lectures from the TED website. The TED dataset contains 1,149 talks and 69,023 profiles of users, who have made more than 100,000 ratings and 200,000 comments. The corresponding metadata, which we make available, can be used for training and testing generic or personalized recommender systems. We define content-based, collaborative, and combined recommendation methods for TED lectures and use cross-validation to select the best parameters of keyword-based (TFIDF) and semantic vector space-based methods (LSI, LDA, RP, and ESA). We compare these methods on a personalized recommendation task in two settings, a cold-start and a non-cold-start one. In the cold-start setting, semantic vector spaces perform better than keywords. In the non-cold-start setting, where collaborative information can be exploited, content-based methods are outperformed by collaborative filtering ones, but the proposed combined method shows acceptable performances, and can be used in both settings. For the generic recommendation task, LSI and RP again outperform TF-IDF.

Our team intends to develop a recommendation system for job seekers based on the information of current employees in big companies. Several models are implemented to achieve over 60% success rate in classifying employees, and we use these models to help job seekers identify their best fitting company.

Purpose –Academic groups are designed specifically for researchers. A group recommendation procedure is essential to support scholars’ research-based social activities. However, group recommendation methods are rarely applied in online libraries and they often suffer from scalability problem in big data context. The purpose of this paper is to facilitate academic group activities in big data-based library systems by recommending satisfying articles for academic groups. Design/methodology/approach – The authors propose a collaborative matrix factorization (CoMF) mechanism and implement paralleled CoMF under Hadoop framework. Its rationale is collaboratively decomposing researcher-article interaction matrix and group-article interaction matrix. Furthermore, three extended models of CoMF are proposed. Findings – Empirical studies on CiteULike data set demonstrate that CoMF and three variants outperform baseline algorithms in terms of accuracy and robustness. The scalability evaluation of paralleled CoMF shows its potential value in scholarly big data environment. Research limitations/implications – The proposed methods fill the gap of group-article recommendation in online libraries domain. The proposed methods have enriched the group recommendation methods by considering the interaction effects between groups and members. The proposed methods are the first attempt to implement group recommendation methods in big data contexts. Practical implications – The proposed methods can improve group activity effectiveness and information shareability in academic groups, which are beneficial to membership retention and enhance the service quality of online library systems. Furthermore, the proposed methods are applicable to big data contexts and make library system services more efficient. Social implications – The proposed methods have potential value to improve scientific collaboration and research innovation. Originality/value – The proposed CoMF method is a novel group recommendation method based on the collaboratively decomposition of researcher-article matrix and group-article matrix. The process indirectly reflects the interaction between groups and members, which accords with actual library environments and provides an interpretable recommendation result.

Recommendation systems are widely used to recommend products to the end users that are most appropriate. Online book selling websites now-a-days are competing with each other by many means. Recommendation system is one of the stronger tools to increase profit and retaining buyer. The book recommendation system must recommend books that are of buyer's interest. This paper presents book recommendation system based on combined features of content filtering, collaborative filtering and association rule mining.

With the rapid development of the mobile games, mobile game recommendation has become a core technique to mobile game marketplaces. This paper proposes a bipartite graph based recommendation algorithm PKBBR (Prior Knowledge Based for Bipartite Graph Rank). We model the user's interest in mobile game based on bipartite graph structure and use the users' mobile game behavior to describe the edge weights of the graph, then incorporate users' prior knowledge into the projection procedure of the bipartite graph to enrich the information among the nodes. Because the popular games have a great influence on mobile game marketplace, we design a hybrid recommendation algorithm to incorporate popularity recommendation based on users' behaviors. The experiment results show that this hybrid method could achieve a better performance than other approaches.

In mobile application, the push notification is an important function to make users consume an item (eg. movie, novel). If a mobile application sends too many notifications to users without considering their preferences, it can cause them to turn off the notification or even deleting the mobile application. Thus, we develop a framework that selects a set of users who are likely to react to the push notification. To the best of our knowledge, we first apply recommendation techniques (i.e., collaborative filtering and content-based models) to the problem of discovering the target of push notification.