Question: Deep learning methodology for computation offloading in mobile edge computing environments.

This paper presents Adam, an efficient algorithm designed for optimizing stochastic objective functions through first-order gradient-based methods. Adam is characterized by its simplicity in implementation, computational efficiency, minimal memory consumption, and its robustness to diagonal rescaling of gradients. It is particularly advantageous for large-scale problems involving vast amounts of data or parameters, as well as for non-stationary objectives and scenarios with highly noisy or sparse gradients. The algorithm's hyper-parameters are intuitively interpretable and generally require minimal tuning. Adam is inspired by and shares connections with other related optimization algorithms. A theoretical analysis of Adam's convergence properties is provided, alongside a regret bound on the convergence rate that is comparable to the state-of-the-art results in the context of online convex optimization. Empirical evaluations confirm Adam's practical effectiveness and its competitive performance against other stochastic optimization techniques. Additionally, the paper introduces AdaMax, a modification of Adam that utilizes the infinity norm for optimization purposes.

Multi-character recognition in arbitrary photographs on mobile platform is difficult, in terms of both accuracy and real-time performance. In this paper, we focus on the domain of hand-written multi-digit recognition. Convolutional neural network (CNN) is the state-of-the-art solution for object recognition, and presents a workload that is both compute and data intensive. To reduce the workload, we train a shallow CNN offline, achieving 99.07% top-1 accuracy. And we utilize preprocessing and segmentation to reduce input image size fed into CNN. For CNN implementation on the mobile platform, we adopt and modify DeepBeliefSDK to support batching fully-connected layers. On NVIDIA SHIELD tablet, the application processes a frame and extracts 32 digits in approximately 60ms, and batching the fully-connected layers reduces the CNN runtime by another 12%.

Fog Computing is a recent computing paradigm that is extending cloud computing towards the edge of network. Due to its proximity to end-users, dense geographical distribution, open platform and support for high mobility, Fog Computing platforms can provide services with reduced latency and improved QoS. Thus it is becoming an important enabler for consumer centric Internet of Things based applications and services that require real time operations e.g. connected vehicles, smart road intersection management and smart grid. The paper discusses one such architecture for connected vehicles with Road Side Units (RSUs) and M2M gateways including the Fog Computing Platform. M2M data processing with semantics, discovery and management of connected vehicles are briefly discussed as consumer centric IoT services enabled by the distinct characteristics of Fog Computing.

In this paper, a deep learning inspired solution is proposed and developed to enable timeliness and practicality of the pre-existing ground stereo vision guidance system for flxed-wing UAVs' safe landing. Since the ground guidance prototype was restricted within applications due to its untimeliness, eventually the vision-based detection less than 15 fps (frame per second). Under such circumstances, we employ a regression based deep learning algorithm into automatic detection on the flying aircraft in the landing sequential images. The system architecture is upgraded so as to be compatible with the novel deep learning requests, and furthermore, annotated datasets are conducted to support training and testing of the regression-based learning detection algorithm. Experimental results validate that the detection attaches 100 fps or more while the localization accuracy is kept in the same level.

It is widely acknowledged that the development of traditional terrestrial communication technologies cannot provide all users with fair and high quality services due to the scarce network resource and limited coverage areas. To complement the terrestrial connection, especially for users in rural, disasterstricken, or other difficult-to-serve areas, satellites, unmanned aerial vehicles (UAVs), and balloons have been utilized to relay the communication signals. On the basis, Space-Air-Ground Integrated Networks (SAGINs) have been proposed to improve the users’ Quality of Experience (QoE). However, compared with existing networks such as ad hoc networks and cellular networks, the SAGINs are much more complex due to the various characteristics of three network segments. To improve the performance of SAGINs, researchers are facing many unprecedented challenges. In this paper, we propose the Artificial Intelligence (AI) technique to optimize the SAGINs, as the AI technique has shown its predominant advantages in many applications. We first analyze several main challenges of SAGINs and explain how these problems can be solved by AI. Then, we consider the satellite traffic balance as an example and propose a deep learning based method to improve the traffic control performance. Simulation results evaluate that the deep learning technique can be an efficient tool to improve the performance of SAGINs.

For the efficient execution of deep convolutional neural networks (CNN) on edge devices, various approaches have been presented which reduce the bit width of the network parameters down to 1 bit. Binarization of the first layer was always excluded, as it leads to a significant error increase. Here, we present the novel concept of binary input layer (BIL), which allows the usage of binary input data by learning bit specific binary weights. The concept is evaluated on three datasets (PAMAP2, SVHN, CIFAR-10). Our results show that this approach is in particular beneficial for multimodal datasets (PAMAP2) where it outperforms networks using full precision weights in the first layer by 1.92 percentage points (pp) while consuming only 2% of the chip area.

While deep neural networks have been shown in recent years to outperform other machine learning methods in a wide range of applications, one of the biggest challenges with enabling deep neural networks for widespread deployment on edge devices such as mobile and other consumer devices is high computational and memory requirements. Recently, there has been greater exploration into small deep neural network architectures that are more suitable for edge devices, with one of the most popular architectures being SqueezeNet, with an incredibly small model size of 4.8MB. Taking further advantage of the notion that many applications of machine learning on edge devices are often characterized by a low number of target classes, this study explores the utility of combining architectural modifications and an evolutionary synthesis strategy for synthesizing even smaller deep neural architectures based on the more recent SqueezeNet v1.1 macroarchitecture (considered state-of-the-art in efficient architectures) for applications with fewer target classes. In particular, architectural modifications are first made to SqueezeNet v1.1 to accommodate for a 10-class ImageNet-10 dataset, and then an evolutionary synthesis strategy is leveraged to synthesize more efficient deep neural networks based on this modified macroarchitecture. The resulting SquishedNets possess model sizes ranging from 2.4MB to 0.95MB (∼5.17X smaller than SqueezeNet v1.1, or 253X smaller than AlexNet). Furthermore, the SquishedNets are still able to achieve accuracies ranging from 81.2% to 77%, and able to process at speeds of 156 images/sec to as much as 256 images/sec on a Nvidia Jetson TX1 embedded chip. These preliminary results show that a combination of architectural modifications and an evolutionary synthesis strategy can be a useful tool for producing very small deep neural network architectures that are well-suited for edge device scenarios without the need for compression or quantization.

Future 5G wireless networks will rely on agile and automated network management, where the usage of diverse resources must be jointly optimized with surgical accuracy. A number of key wireless network functionalities (e.g., traffic steering, energy savings) give rise to hard optimization problems. What is more, high spatio-temporal traffic variability coupled with the need to satisfy strict per slice/service SLAs in modern networks, suggest that these problems must be constantly (re-)solved, to maintain close-to-optimal performance. To this end, in this paper we propose the framework of Online Network Optimization (ONO), which seeks to maintain both agile and efficient control over time, using an arsenal of data-driven, adaptive, and AI-based techniques. Since the mathematical tools and the studied regimes vary widely among these methodologies, a theoretical comparison is often out of reach. Therefore, the important question “what is the right ONO technique?” remains open to date. In this paper, we discuss the pros and cons of each technique and further attempt a direct quantitative comparison for a specific use case, using real data. Our results suggest that carefully combining the insights of problem modeling with state-of-the-art AI techniques provides significant advantages at reasonable complexity.

Intelligent agents, such as robots, have to serve a multitude of autonomous functions. Examples are, e.g., collision avoidance, navigation and route planning, active sensing of its environment, or the interaction and non-verbal communication with people in the extended reach space. Here, we focus on the recognition of the action of a human agent based on a biologically inspired visual architecture of analyzing articulated movements. The proposed processing architecture builds upon coarsely segregated streams of sensory processing along different pathways which separately process form and motion information (Layher et al., 2014). Action recognition is performed in an event-based scheme by identifying representations of characteristic pose configurations (key poses) in an image sequence. In line with perceptual studies, key poses are selected unsupervised utilizing a feature-driven criterion which combines extrema in the motion energy with the horizontal and the vertical extendedness of a body shape. Per class representations of key pose frames are learned using a deep convolutional neural network consisting of 15 convolutional layers. The network is trained using the energy-efficient deep neuromorphic networks (Eedn) framework (Esser et al., 2016), which realizes the mapping of the trained synaptic weights onto the IBM Neurosynaptic System platform (Merolla et al., 2014). After the mapping, the trained network achieves real-time capabilities for processing input streams and classify input images at about 1,000 frames per second while the computational stages only consume about 70 mW of energy (without spike transduction). Particularly regarding mobile robotic systems, a low energy profile might be crucial in a variety of application scenarios. Cross-validation results are reported for two different datasets and compared to state-of-the-art action recognition approaches. The results demonstrate, that (I) the presented approach is on par with other key pose based methods described in the literature, which select key pose frames by optimizing classification accuracy, (II) compared to the training on the full set of frames, representations trained on key pose frames result in a higher confidence in class assignments, and (III) key pose representations show promising generalization capabilities in a cross-dataset evaluation.

Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performances in applications such as image classification and language modeling. However, these techniques typically ignore device-related objectives such as inference time, memory usage, and power consumption. Optimizing neural architecture for devicerelated objectives is immensely crucial for deploying deep networks on portable devices with limited computing resources. We propose DPPNet: Device-aware Progressive Search for Pareto-optimal Neural Architectures, optimizing for both device-related (e.g., inference time and memory usage) and device-agnostic (e.g., accuracy and model size) objectives. DPP-Net employs a compact search space inspired by current state-of-the-art mobile CNNs, and further improves search efficiency by adopting progressive search (Liu et al. 2017). Experimental results on CIFAR-10 are poised to demonstrate the effectiveness of Pareto-optimal networks found by DPP-Net, for three different devices: (1) a workstation with Titan X GPU, (2) NVIDIA Jetson TX1 embedded system, and (3) mobile phone with ARM Cortex-A53. Compared to CondenseNet and NASNet (Mobile), DPP-Net achieves better performances: higher accuracy & shorter inference time on various devices. Additional experimental results show that models found by DPP-Net also achieve considerablygood performance on ImageNet as well.

Machine learning models benefit from large and diverse datasets. Using such datasets, however, often requires trusting a centralized data aggregator. For sensitive applications like healthcare and finance this is undesirable as it could compromise patient privacy or divulge trade secrets. Recent advances in secure and privacy-preserving computation, including trusted hardware enclaves and differential privacy, offer a way for mutually distrusting parties to efficiently train a machine learning model without revealing the training data. In this work, we introduce Myelin, a deep learning framework which combines these privacy-preservation primitives, and use it to establish a baseline level of performance for fully private machine learning.


Question: Deep learning method for computation offloading in mobile edge computing systems.

This document delves into the topic of multi-label classification, a critical component in contemporary applications like protein function classification, music categorization, and semantic scene interpretation. It provides an overview of the multi-label classification task, consolidating the scattered relevant literature into a coherent framework. The paper also presents comparative empirical outcomes for various multi-label classification techniques, offering insights into their performance. Additionally, it introduces a set of metrics designed to quantify the multi-label characteristics of a dataset, enhancing the understanding of data complexity in multi-label scenarios.

**Question**: Computation offloading for mobile edge computing: A deep learning approach

**Documents**: Efficient computation offloading for mobile edge computing using deep reinforcement learning
In this paper, we propose a novel deep reinforcement learning (DRL)-based computation offloading scheme for mobile edge computing (MEC) systems. The proposed scheme is designed to minimize the total energy consumption of mobile devices by efficiently offloading computation tasks to the MEC server. The DRL algorithm is used to learn the optimal offloading policy, which takes into account the current state of the system, including the available resources at the MEC server, the computation requirements of the mobile devices, and the channel conditions. The proposed scheme is evaluated in a realistic scenario with multiple mobile devices and an MEC server. The results show that the proposed scheme can significantly reduce the total energy consumption of mobile devices compared to existing offloading schemes.

**Similar Documents**: This paper presents an innovative computation offloading strategy for mobile edge computing (MEC) environments, utilizing deep reinforcement learning (DRL) to optimize the process. The goal of this strategy is to minimize the overall energy usage of mobile devices by efficiently distributing computation tasks to the MEC server. The DRL algorithm is employed to learn the optimal offloading policy, factoring in the system's current state, such as the MEC server's resource availability, the computational demands of mobile devices, and the prevailing channel conditions. The efficacy of the proposed strategy is assessed in a simulation featuring multiple mobile devices and an MEC server. The findings indicate that the proposed approach can substantially decrease the total energy consumption of mobile devices relative to conventional offloading methods.

The purpose of this article is to introduce the readers to the emerging technologies enabled by deep learning and to review the research work conducted in this area that is of direct relevance to signal processing. We also point out, in our view, the future research directions that may attract interests of and require efforts from more signal processing researchers and practitioners in this emerging area for advancing signal and information processing technology and applications.

Convolutional neural network (CNN)-based systems are increasingly used in autonomous vehicles for detecting obstacles. CNN-based object detection and per-pixel classification (semantic segmentation) algorithms are trained for detecting and classifying a predefined set of object types. These algorithms have difficulties in detecting distant and heavily occluded objects and are, by definition, not capable of detecting unknown object types or unusual scenarios. The visual characteristics of an agriculture field is homogeneous, and obstacles, like people, animals and other obstacles, occur rarely and are of distinct appearance compared to the field. This paper introduces DeepAnomaly, an algorithm combining deep learning and anomaly detection to exploit the homogenous characteristics of a field to perform anomaly detection. We demonstrate DeepAnomaly as a fast state-of-the-art detector for obstacles that are distant, heavily occluded and unknown. DeepAnomaly is compared to state-of-the-art obstacle detectors including "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks" (RCNN). In a human detector test case, we demonstrate that DeepAnomaly detects humans at longer ranges (45-90 m) than RCNN. RCNN has a similar performance at a short range (0-30 m). However, DeepAnomaly has much fewer model parameters and (182 ms/25 ms =) a 7.28-times faster processing time per image. Unlike most CNN-based methods, the high accuracy, the low computation time and the low memory footprint make it suitable for a real-time system running on a embedded GPU (Graphics Processing Unit).

Big deep neural network (DNN) models trained on large amounts of data have recently achieved the best accuracy on hard tasks, such as image and speech recognition. Training these DNNs using a cluster of commodity machines is a promising approach since training is time consuming and compute-intensive. To enable training of extremely large DNNs, models are partitioned across machines. To expedite training on very large data sets, multiple model replicas are trained in parallel on different subsets of the training examples with a global parameter server maintaining shared weights across these replicas. The correct choice for model and data partitioning and overall system provisioning is highly dependent on the DNN and distributed system hardware characteristics. These decisions currently require significant domain expertise and time consuming empirical state space exploration.
 This paper develops performance models that quantify the impact of these partitioning and provisioning decisions on overall distributed system performance and scalability. Also, we use these performance models to build a scalability optimizer that efficiently determines the optimal system configuration that minimizes DNN training time. We evaluate our performance models and scalability optimizer using a state-of-the-art distributed DNN training framework on two benchmark applications. The results show our performance models estimate DNN training time with high estimation accuracy and our scalability optimizer correctly chooses the best configurations, minimizing the training time of distributed DNNs.

Cloud computing is a model of sharing computing resources over any communication network by using virtualization. Virtualization allows a server to be sliced in virtual machines. Each virtual machine has its own operating system/applications that rapidly adjust resource allocation. Cloud computing offers many benefits, one of them is elastic resource allocation. To fulfill the requirements of clients, cloud environment should be flexible in nature and can be achieve by efficient resource allocation. Resource allocation is the process of assigning available resources to clients over the internet and plays vital role in Infrastructure-as-aService (IaaS) model of cloud computing. Elastic resource allocation is required to optimize the allocation of resources, minimizing the response time and maximizing the throughput to improve the performance of cloud computing. Sufficient solutions have been proposed for cloud computing to improve the performance but for fog computing still efficient solution have to be found. Fog computing is the virtualized intermediate layer between clients and cloud. It is a highly virtualized technology which is similar to cloud and provide data, computation, storage, and networking services between end users and cloud servers. This paper presents an efficient architecture and algorithm for resources provisioning in fog computing environment by using virtualization technique.

Recently, deep learning has demonstrated great success in computer vision with the capability to learn powerful image features from a large training set. However, most of the published work has been confined to solving 2D problems, with a few limited exceptions that treated the 3D space as a composition of 2D orthogonal planes. The challenge of 3D deep learning is due to a much larger input vector, compared to 2D, which dramatically increases the computation time and the chance of over-fitting, especially when combined with limited training samples (hundreds to thousands), typical for medical imaging applications. To address this challenge, we propose an efficient and robust deep learning algorithm capable of full 3D detection in volumetric data. A two-step approach is exploited for efficient detection. A shallow network (with one hidden layer) is used for the initial testing of all voxels to obtain a small number of promising candidates, followed by more accurate classification with a deep network. In addition, we propose two approaches, i.e., separable filter decomposition and network sparsification, to speed up the evaluation of a network. To mitigate the over-fitting issue, thereby increasing detection robustness, we extract small 3D patches from a multi-resolution image pyramid. The deeply learned image features are further combined with Haar wavelet features to increase the detection accuracy. The proposed method has been quantitatively evaluated for carotid artery bifurcation detection on a head-neck CT dataset from 455 patients. Compared to the state-ofthe-art, the mean error is reduced by more than half, from 5.97 mm to 2.64 mm, with a detection speed of less than 1 s/volume.

Implementing E cient, Portable Computations for Machine Learning

Deep networks are state-of-the-art models used for understanding the content of images, videos, audio and raw input data. Current computing systems are not able to run deep network models in real-time with low power consumption. In this paper we present nn-X: a scalable, low-power coprocessor for enabling real-time execution of deep neural networks. nn-X is implemented on programmable logic devices and comprises an array of configurable processing elements called collections. These collections perform the most common operations in deep networks: convolution, subsampling and non-linear functions. The nn-X system includes 4 high-speed direct memory access interfaces to DDR3 memory and two ARM Cortex-A9 processors. Each port is capable of a sustained throughput of 950 MB/s in full duplex. nn-X is able to achieve a peak performance of 227 G-ops/s, a measured performance in deep learning applications of up to 200 G-ops/s while consuming less than 4 watts of power. This translates to a performance per power improvement of 10 to 100 times that of conventional mobile and desktop processors.

Going deeper and wider in neural architectures improves their accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) practitioners either need to change to less desired network architectures, or nontrivially dissect a network across multiGPUs. These distract DL practitioners from concentrating on their original machine learning tasks. We present SuperNeurons: a dynamic GPU memory scheduling runtime to enable the network training far beyond the GPU DRAM capacity. SuperNeurons features 3 memory optimizations, Liveness Analysis, Unified Tensor Pool, and Cost-Aware Recomputation; together they effectively reduce the network-wide peak memory usage down to the maximal memory usage among layers. We also address the performance issues in these memory-saving techniques. Given the limited GPU DRAM, SuperNeurons not only provisions the necessary memory for the training, but also dynamically allocates the memory for convolution workspaces to achieve the high performance. Evaluations against Caffe, Torch, MXNet and TensorFlow have demonstrated that SuperNeurons trains at least 3.2432 deeper network than current ones with the leading performance. Particularly, SuperNeurons can train ResNet2500 that has 104 basic network layers on a 12GB K40c.

We present a practical approach for processing mobile sensor time series data for continual deep learning predictions. The approach comprises data cleaning, normalization, capping, time-based compression, and finally classification with a recurrent neural network. We demonstrate the effectiveness of the approach in a case study with 279 participants. On the basis of sparse sensor events, the network continually predicts whether the participants would attend to a notification within 10 minutes. Compared to a random baseline, the classifier achieves a 40% performance increase (AUC of 0.702) on a withheld test set. This approach allows to forgo resource-intensive, domain-specific, error-prone feature engineering, which may drastically increase the applicability of machine learning to mobile phone sensor data.

Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network’s behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.


Question: A deep learning method for computation offloading in mobile edge computing systems.

Innovative Offloading Strategies for Mobile Edge Computing with Unreliable Connectivity

The advent of mobile edge computing has revolutionized the way mobile devices utilize nearby resource-rich systems (referred to as mobile edge servers) for offloading tasks, thereby optimizing energy usage and enhancing computational efficiency. Nevertheless, the intermittent nature of connectivity between mobile devices and these edge servers, influenced by mobility and server capacity, poses significant challenges. This intermittency can lead to failed offloading attempts when a device moves out of the server's communication range. Addressing this issue, we present an optimal offloading algorithm designed for mobile devices within an unreliable mobile edge computing environment, taking into account the device's local processing load and the availability of edge servers. Our analysis incorporates the dynamics of user mobility and edge server admission policies, enabling us to calculate the likelihood of successful task offloading. We employ a Markov decision process (MDP) framework to derive an optimal policy for mobile devices, aiming to minimize the combined costs of computation and offloading. We demonstrate that the optimal policy exhibits a threshold structure, facilitating quick decision-making for energy-limited devices. Additionally, we propose a streamlined algorithm to assist these devices in making informed offloading choices. Empirical evaluations confirm the accuracy of our analytical model for successful offloading probability across diverse mobility scenarios. Moreover, our MDP-based offloading strategy for mobile devices consistently outperforms conventional methods in terms of efficiency and reliability.

Mobile computing is emerging as the prime focus of next generation computing .One of the prime issues of mobile computing is to provide infrastructure support in terms of computing devices, seamless mobility, application middleware, data and user security, and user applications/services. Mobile commerce is one of the driving forces that has evinced enormous interest in mobile computing .The thought of conducting commerce on the go is what is driving the huge investments corporations are making in researching this area. This paper discusses the various challenges in providing infrastructure for wireless computing.

Transferring a huge amount of data between different network locations over the network links depends on the network's traffic capacity and data rate. Traditionally, a mobile device may be moved to achieve the operations of vertical handover, considering only one criterion, that is the Received Signal Strength (RSS). The use of a single criterion may cause service interruption, an unbalanced network load and an inefficient vertical handover. In this paper, we propose an enhanced vertical handover decision algorithm based on multiple criteria in the heterogeneous wireless network. The algorithm consists of three technology interfaces: Long-Term Evolution (LTE), Worldwide interoperability for Microwave Access (WiMAX) and Wireless Local Area Network (WLAN). It also employs three types of vertical handover decision algorithms: equal priority, mobile priority and network priority. The simulation results illustrate that the three types of decision algorithms outperform the traditional network decision algorithm in terms of handover number probability and the handover failure probability. In addition, it is noticed that the network priority handover decision algorithm produces better results compared to the equal priority and the mobile priority handover decision algorithm. Finally, the simulation results are validated by the analytical model.

The network on-chip (NoC) is a critical resource shared by various on-chip components. An efficient NoC arbitration policy is crucial in providing global fairness and improving system performance. In this preliminary work, we demonstrate an idea of utilizing deep reinforcement learning to guide the design of more efficient NoC arbitration policies. We relate arbitration to a self-learning decision making process. Results show that the deep reinforcement learning approach can effectively reduce packet latency and has potential for identifying interesting features that could be utilized in more practical hardware designs.

Deep Machine Learning (DML) algorithms have proven to be highly successful at challenging, high-dimensional learning problems, but their widespread deployment is limited by their heavy computational requirements and the associated power consumption. Analog computational circuits offer the potential for large improvements in power efficiency, but noise, mismatch, and other effects cause deviations from ideal computations. In this paper we describe circuits useful for DML algorithms, including a tunable-width bump circuit and a configurable distance calculator. We also discuss the impacts of computational errors on learning performance. Finally we will describe a complete deep learning engine implemented using current-mode analog circuits and compare its performance to digital equivalents.

The demand for rich multimedia services over mobile networks has been soaring at a tremendous pace over recent years. However, due to the centralized architecture of current cellular networks, the wireless link capacity as well as the bandwidth of the radio access networks and the backhaul network cannot practically cope with the explosive growth in mobile traffic. Recently, we have observed the emergence of promising mobile content caching and delivery techniques, by which popular contents are cached in the intermediate servers (or middleboxes, gateways, or routers) so that demands from users for the same content can be accommodated easily without duplicate transmissions from remote servers; hence, redundant traffic can be significantly eliminated. In this article, we first study techniques related to caching in current mobile networks, and discuss potential techniques for caching in 5G mobile networks, including evolved packet core network caching and radio access network caching. A novel edge caching scheme based on the concept of content-centric networking or information-centric networking is proposed. Using trace-driven simulations, we evaluate the performance of the proposed scheme and validate the various advantages of the utilization of caching content in 5G mobile networks. Furthermore, we conclude the article by exploring new relevant opportunities and challenges.

While demands on video traffic over mobile networks have been souring, the wireless link capacity cannot keep up with the traffic demand. The gap between the traffic demand and the link capacity, along with time-varying link conditions, results in poor service quality of video streaming over mobile networks such as long buffering time and intermittent disruptions. Leveraging the cloud computing technology, we propose a new mobile video streaming framework, dubbed AMES-Cloud, which has two main parts: adaptive mobile video streaming (AMoV) and efficient social video sharing (ESoV). AMoV and ESoV construct a private agent to provide video streaming services efficiently for each mobile user. For a given user, AMoV lets her private agent adaptively adjust her streaming flow with a scalable video coding technique based on the feedback of link quality. Likewise, ESoV monitors the social network interactions among mobile users, and their private agents try to prefetch video content in advance. We implement a prototype of the AMES-Cloud framework to demonstrate its performance. It is shown that the private agents in the clouds can effectively provide the adaptive streaming, and perform video sharing (i.e., prefetching) based on the social network analysis.

Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning. One aspect of the field receiving considerable attention is efficiently executing deep models in resource-constrained environments, such as mobile or embedded devices. This paper focuses on this problem, and proposes two new compression methods, which jointly leverage weight quantization and distillation of larger networks, called “teachers,” into compressed “student” networks. The first method we propose is called quantized distillation and leverages distillation during the training process, by incorporating distillation loss, expressed with respect to the teacher network, into the training of a smaller student network whose weights are quantized to a limited set of levels. The second method, differentiable quantization, optimizes the location of quantization points through stochastic gradient descent, to better fit the behavior of the teacher model. We validate both methods through experiments on convolutional and recurrent architectures. We show that quantized shallow students can reach similar accuracy levels to state-of-the-art full-precision teacher models, while providing up to order of magnitude compression, and inference speedup that is almost linear in the depth reduction. In sum, our results enable DNNs for resource-constrained environments to leverage architecture and accuracy advances developed on more powerful devices.

Training of large-scale deep neural networks is often constrained by the available computational resources. We study the effect of limited precision data representation and computation on neural network training. Within the context of low-precision fixed-point computations, we observe the rounding scheme to play a crucial role in determining the network’s behavior during training. Our results show that deep networks can be trained using only 16-bit wide fixed-point number representation when using stochastic rounding, and incur little to no degradation in the classification accuracy. We also demonstrate an energy-efficient hardware accelerator that implements low-precision fixed-point arithmetic with stochastic rounding.

We describe Spectra, a remote execution system for batterypowered clients used in pervasive computing. Spectra enables applications to combine the mobility of small devices with the greater processing power of static compute servers. Spectra is self-tuning: it monitors both application resource usage and the availability of resources in the environment, and dynamically determines how and where to execute application components. In making this determination, Spectra balances the competing goals of performance, energy conservation, and application quality. We have validated Spectra’s approach on the Compaq Itsy v2.2 and IBM ThinkPad 560X using a speech recognizer, a document preparation system, and a natural language translator. Our results confirm that Spectra almost always selects the best execution plan, and that its few suboptimal choices are very close to optimal.

Sub-8-bit representation of DNNs incur some noticeable loss of accuracy despite rigorous (re)training at low-precision. Such loss of accuracy essentially makes them equivalent to a much shallower counterpart, diminishing the power of being deep networks. To address this problem of accuracy drop we introduce the notion of residual networks where we add more low-precision edges to sensitive branches of the sub-8-bit network to compensate for the lost accuracy. Further, we present a perturbation theory to identify such sensitive edges. Aided by such an elegant trade-off between accuracy and model size, the 8-2 architecture (8-bit activations, ternary weights), enhanced by residual ternary edges, turns out to be sophisticated enough to achieve similar accuracy as 8-8 representation (∼ 1% drop from our FP-32 baseline), despite ∼ 1.6× reduction in model size, ∼ 26× reduction in number of multiplications , and potentially ∼ 2× inference speed up comparing to 8-8 representation, on the state-of-the-art deep network ResNet-101 pre-trained on ImageNet dataset. Moreover, depending on the varying accuracy requirements in a dynamic environment, the deployed low-precision model can be upgraded/downgraded on-the-fly by partially enabling/disabling residual connections. For example, disabling the least important residual connections in the above enhanced network, the accuracy drop is ∼ 2% (from our FP-32 baseline), despite ∼ 1.9× reduction in model size, ∼ 32× reduction in number of multiplications, and potentially ∼ 2.3× inference speed up comparing to 8-8 representation. Finally, all the ternary connections are sparse in nature, and the residual ternary conversion can be done in a resource-constraint setting without any low-precision (re)training and without accessing the data.


Question: Deep learning method for computation offloading in mobile edge computing systems.

In the realm of Cloud Computing, a vision akin to computing as a utility, there's a transformative potential to reshape significant portions of the IT sector, enhancing the allure of software as a service and dictating the design and procurement of IT hardware. Innovators with ideas for novel Internet services are liberated from the financial burdens of hardware investments and operational expenditures, eliminating the risk of overprovisioning for services that underperform or underprovisioning for those that exceed expectations, thereby avoiding lost opportunities and revenue. Additionally, companies grappling with large-scale batch tasks can expedite results in proportion to their program's scalability, as the cost of harnessing 1000 servers for an hour is equivalent to utilizing one server for 1000 hours—a financial elasticity unparalleled in IT history.

Cloud Computing encapsulates both the Internet-delivered services and the hardware and systems software in datacenters that facilitate these services. These services, often referred to as Software as a Service (SaaS), are complemented by the datacenter infrastructure, termed the Cloud. When a Cloud is offered to the public on a pay-per-use basis, it is designated a Public Cloud, and the service is known as Utility Computing. Conversely, an organization's internal datacenter, not accessible to the general public, is a Private Cloud. Therefore, Cloud Computing is the amalgamation of SaaS and Utility Computing, excluding Private Clouds. Individuals can assume roles as users or providers of SaaS, or users or providers of Utility Computing. Our focus here is on SaaS Providers (Cloud Users) and Cloud Providers, aspects that have garnered less attention compared to SaaS Users. From a hardware perspective, Cloud Computing introduces three novel dimensions.

**Question**: A Deep Learning Approach for Computation Offloading in Mobile Edge Computing

**Documents**: Deep Learning for Computation Offloading in Mobile Edge Computing: A Survey
The exponential growth in mobile data traffic and the increasing demand for high-performance mobile applications have led to a surge in research on Mobile Edge Computing (MEC). MEC is a promising paradigm that enables computation offloading from mobile devices to nearby edge servers, thereby alleviating the resource constraints of mobile devices and enhancing the quality of service (QoS) for mobile users. In this context, deep learning (DL) has emerged as a powerful tool for optimizing computation offloading decisions in MEC systems. DL algorithms can learn the complex relationships between the various parameters affecting offloading decisions, such as the workload of mobile devices, the capacity of edge servers, and the network conditions. This

Mobile devices come with an assortment of networks: WiFi in two different frequency bands, each of which can run in infrastructure-mode, WiFi-Direct mode, or ad hoc mode; cellular radios, which can run in LTE/4G, 3G, or EDGE modes; and Bluetooth. But how should an app choose which network to use? There is no systematic solution to this problem today: in current practice the choice is almost always left to the user, who usually has no idea what's best. In fact, what's best for a user depends on the app's performance objectives (throughput, delay, object load time, etc.) and the user's constraints on cost and battery life. Besides, what's best for a single user or app must be balanced with what's best for the wireless network as a whole (individual optimality vs. social utility). This paper introduces Delphi, a transport-layer module to resolve these issues. Delphi has three noteworthy components: "local learning", in which a mobile device estimates or infers useful properties of different networks efficiently, "property sharing", in which mobile devices share what they learn with other nearby devices, and "selection", in which each node selects a network using what it has observed locally and/or from its neighbors.

Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performances in applications such as image classification and language modeling. However, these techniques typically ignore device-related objectives such as inference time, memory usage, and power consumption. Optimizing neural architecture for devicerelated objectives is immensely crucial for deploying deep networks on portable devices with limited computing resources. We propose DPPNet: Device-aware Progressive Search for Pareto-optimal Neural Architectures, optimizing for both device-related (e.g., inference time and memory usage) and device-agnostic (e.g., accuracy and model size) objectives. DPP-Net employs a compact search space inspired by current state-of-the-art mobile CNNs, and further improves search efficiency by adopting progressive search (Liu et al. 2017). Experimental results on CIFAR-10 are poised to demonstrate the effectiveness of Pareto-optimal networks found by DPP-Net, for three different devices: (1) a workstation with Titan X GPU, (2) NVIDIA Jetson TX1 embedded system, and (3) mobile phone with ARM Cortex-A53. Compared to CondenseNet and NASNet (Mobile), DPP-Net achieves better performances: higher accuracy & shorter inference time on various devices. Additional experimental results show that models found by DPP-Net also achieve considerablygood performance on ImageNet as well.

We evaluated the effectiveness in classifying food images of a deep-learning approach based on the specifications of Google's image recognition architecture Inception. The architecture is a deep convolutional neural network (DCNN) having a depth of 54 layers. In this study, we fine-tuned this architecture for classifying food images from three well-known food image datasets: ETH Food-101, UEC FOOD 100, and UEC FOOD 256. On these datasets we achieved, respectively, 88.28%, 81.45%, and 76.17% as top-1 accuracy and 96.88%, 97.27%, and 92.58% as top-5 accuracy. To the best of our knowledge, these results significantly improve the best published results obtained on the same datasets, while requiring less computation power, since the number of parameters and the computational complexity are much smaller than the competitors?. Because of this, even if it is still rather large, the deep network based on this architecture appears to be at least closer to the requirements for mobile systems.

In this paper, a deep learning inspired solution is proposed and developed to enable timeliness and practicality of the pre-existing ground stereo vision guidance system for flxed-wing UAVs' safe landing. Since the ground guidance prototype was restricted within applications due to its untimeliness, eventually the vision-based detection less than 15 fps (frame per second). Under such circumstances, we employ a regression based deep learning algorithm into automatic detection on the flying aircraft in the landing sequential images. The system architecture is upgraded so as to be compatible with the novel deep learning requests, and furthermore, annotated datasets are conducted to support training and testing of the regression-based learning detection algorithm. Experimental results validate that the detection attaches 100 fps or more while the localization accuracy is kept in the same level.

With the demand of high data rate and low latency in fifth generation (5G), deep neural network decoder (NND) has become a promising candidate due to its capability of one-shot decoding and parallel computing. In this paper, three types of NND, i.e., multi-layer perceptron (MLP), convolution neural network (CNN) and recurrent neural network (RNN), are proposed with the same parameter magnitude. The performance of these deep neural networks are evaluated through extensive simulation. Numerical results show that RNN has the best decoding performance, yet at the price of the highest computational overhead. Moreover, we find there exists a saturation length for each type of neural network, which is caused by their restricted learning abilities.

In mobile cloud computing, mobile devices can rely on cloud computing and information storage resource to perform computationally intensive operations such as searching, data mining, and multimedia processing. In addition to providing traditional computation services, mobile cloud also enhances the operation of traditional ad hoc network by treating mobile devices as service nodes, e.g., sensing services. The sensed information, such as location coordinates, health related information, should be processed and stored in a secure fashion to protect user's privacy in the cloud. To this end, we present a new mobile cloud data processing framework through trust management and private data isolation. Finally, an implementation pilot for improving teenagers' driving safety, which is called FocusDrive, is presented to demonstrate the solution.

Edge computing systems (Cloudlet, Fog Computing, Multi-access Edge Computing) provide numerous benefits to information technology: reduced latency, improved bandwidth, battery lifetime, etc. Despite all the benefits, edge computing systems have several issues that could significantly reduce the performance of certain applications. Indeed, current edge computing technologies do not assure ultra-low latency for real-time applications and they encounter overloading issues for data processing. To solve the aforementioned issues, we propose Home Edge Computing (HEC): a new three-tier edge computing architecture that provides data storage and processing in close proximity to the users. The term “Home” in Home Edge Computing does not restrain our work to the homes of the users, we take into account other places where the users could connect to the Internet such as: companies, shopping malls, hospitals, etc. Our three-tier architecture is composed of a Home Server, an Edge Server and a Central Cloud which we also find in traditional edge computing architectures. The Home Server is located within the vicinities of the users which allow the achievement of ultra-low latency for applications that could be processed by the said server; this also help reduce the amount of data that could be treated in the Edge Server and the Central Cloud. We demonstrate the validity of our architecture by leveraging the EdgeCloudSim simulation platform. The results of the simulation show that our proposal can, in fact, help achieve ultra-low latency and reduce overloading issues.

In this paper, we apply the concept of a software-defined data plane to defining new services for mobile virtual network operators (MVNOs) to empower them with the capability of tailoring fine-grained subscription plans that can meet users' demands. For this purpose, we have recently proposed the concept of application-and/ or device-specific slicing that classifies application-and/or device-specific traffic into slices and introduced various applications of our proposed system [Optical Fiber Communication Conf. (OFC), 2016, paper Th3I.4]. This paper reports the prototype implementation of the proposal into a real MVNO connecting customized smartphones so we can identify applications/devices from the given traffic with 100% accuracy. We first characterize the traffic patterns of popular applications to build a detailed understanding of how network resources are utilized for real users and applications and what their demands on different network resources are. Then, we classify traffic according to the devices and find that the flow characteristics of LTE RAN slices are very similar to those of 3G. We discover that most of the bandwidth is consumed by those flows with the duration of several hundreds of seconds, and the flow size is larger than 5 Mbytes in both LTE and 3G MVNO networks.

The emerging Internet of Things (IoT) is facing significant scalability and security challenges. On the one hand, IoT devices are “weak” and need external assistance. Edge computing provides a promising direction addressing the deficiency of centralized cloud computing in scaling massive number of devices. On the other hand, IoT devices are also relatively “vulnerable” facing malicious hackers due to resource constraints. The emerging blockchain and smart contracts technologies bring a series of new security features for IoT and edge computing. In this paper, to address the challenges, we design and prototype an edge-IoT framework named “EdgeChain” based on blockchain and smart contracts. The core idea is to integrate a permissioned blockchain and the internal currency or “coin” system to link the edge cloud resource pool with each IoT device’ account and resource usage, and hence behavior of the IoT devices. EdgeChain uses a credit-based resource management system to control how much resource IoT devices can obtain from edge servers, based on pre-defined rules on priority, application types and past behaviors. Smart contracts are used to enforce the rules and policies to regulate the IoT device behavior in a non-deniable and automated manner. All the IoT activities and transactions are recorded into blockchain for secure data logging and auditing. We implement an EdgeChain prototype and conduct extensive experiments to evaluate the ideas. The results show that while gaining the security benefits of blockchain and smart contracts, the cost of integrating them into EdgeChain is within a reasonable and acceptable range.

With the support of mobile cloud computing, wireless body area networks can be significantly enhanced for massive deployment of pervasive healthcare applications. However, several technical issues and challenges are associated with the integration of WBANs and MCC. In this article, we study a cloud-enabled WBAN architecture and its applications in pervasive healthcare systems. We highlight the methodologies for transmitting vital sign data to the cloud by using energy-efficient routing, cloud resource allocation, semantic interactions, and data security mechanisms.


Question: EmoBGM: Assessing sound emotion for generating slideshows with fitting background music.

This paper presents an innovative method for robust image sentiment analysis utilizing progressively trained and domain-transferred deep networks. The analysis of online user-generated content's sentiment is crucial for various social media analytics applications. Traditionally, textual sentiment analysis has been the primary approach for predicting political outcomes, gauging economic indicators, and other tasks. However, the growing trend of using images and videos for expression and sharing experiences necessitates the development of sentiment analysis techniques for visual content. This can significantly enhance the extraction of user sentiments regarding events or topics found in images, such as those in tweets, complementing textual sentiment analysis.

To tackle the challenge of image sentiment analysis using large-scale but noisy training data, we utilize Convolutional Neural Networks (CNNs). Our approach begins with the design of a specialized CNN architecture tailored for image sentiment analysis. We generate a substantial training dataset of approximately half a million images by employing a baseline sentiment algorithm to label Flickr images. To effectively utilize this noisy, machine-labeled data, we implement a progressive strategy for fine-tuning the deep network. Additionally, to enhance performance specifically for Twitter images, we introduce domain transfer by leveraging a small set of manually labeled Twitter images.

Extensive experimentation on manually labeled Twitter images has been conducted, and the outcomes demonstrate that our proposed CNN outperforms competing algorithms in image sentiment analysis, showcasing its robustness and effectiveness in this domain.

Given a short video we create a representation that captures a spectrum of looping videos with varying levels of dynamism, ranging from a static image to a highly animated loop. In such a progressively dynamic video, scene liveliness can be adjusted interactively using a slider control. Applications include background images and slideshows, where the desired level of activity may depend on personal taste or mood. The representation also provides a segmentation of the scene into independently looping regions, enabling interactive local adjustment over dynamism. For a landscape scene, this control might correspond to selective animation and deanimation of grass motion, water ripples, and swaying trees. Converting arbitrary video to looping content is a challenging research problem. Unlike prior work, we explore an optimization in which each pixel automatically determines its own looping period. The resulting nested segmentation of static and dynamic scene regions forms an extremely compact representation.

Nowadays, almost any web site that provides means for sharing user-generated multimedia content, like Flickr, Facebook, YouTube and Vimeo, has tagging functionalities to let users annotate the material that they want to share. The tags are then used to retrieve the uploaded content, and to ease browsing and exploration of these collections, e.g. using tag clouds. However, while tagging a single image is straightforward, and sites like Flickr and Facebook allow also to tag easily portions of the uploaded photos, tagging a video sequence is more cumbersome, so that users just tend to tag the overall content of a video. Moreover, the tagging process is completely manual, and often users tend to spend as few time as possible to annotate the material, resulting in a sparse annotation of the visual content. A semi-automatic process, that helps the users to tag a video sequence would improve the quality of annotations and thus the overall user experience. While research on image tagging has received a considerable attention in the latest years, there are still very few works that address the problem of automatically assigning tags to videos, locating them temporally within the video sequence. In this paper we present a system for video tag suggestion and temporal localization based on collective knowledge and visual similarity of frames. The algorithm suggests new tags that can be associated to a given keyframe exploiting the tags associated to videos and images uploaded to social sites like YouTube and Flickr and visual features.

Despite the enormous interest in emotion classification from speech, the impact of noise on emotion classification is not well understood. This is important because, due to the tremendous advancement of the smartphone technology, it can be a powerful medium for speech emotion recognition in the outside laboratory natural environment, which is likely to incorporate background noise in the speech. We capitalize on the current breakthrough of Recurrent Neural Network (RNN) and seek to investigate its performance for emotion classification from noisy speech. We particularly focus on the recently proposed Gated Recurrent Unit (GRU), which is yet to be explored for emotion recognition from speech. Experiments conducted with speech compounded with eight different types of noises reveal that GRU incurs an 18.16% smaller run-time while performing quite comparably to the Long Short-Term Memory (LSTM), which is the most popular Recurrent Neural Network proposed to date. This result is promising for any embedded platform in general and will initiate further studies to utilize GRU to its full potential for emotion recognition on smartphones.

Images can convey rich emotions to viewers. Recent research on image emotion analysis mainly focused on affective image classification, trying to find features that can classify emotions better. We concentrate on affective image retrieval and investigate the performance of different features on different kinds of images in a multi-graph learning framework. Firstly, we extract commonly used features of different levels for each image. Generic features and features derived from elements-of-art are extracted as low-level features. Attributes and interpretable principles-of-art based features are viewed as mid-level features, while semantic concepts described by adjective noun pairs and facial expressions are extracted as high-level features. Secondly, we construct single graph for each kind of feature to test the retrieval performance. Finally, we combine the multiple graphs together in a regularization framework to learn the optimized weights of each graph to efficiently explore the complementation of different features. Extensive experiments are conducted on five datasets and the results demonstrate the effectiveness of the proposed method.

Emotion Detection is one of the most emerging issues in human computer interaction. A sufficient amount of work has been done by researchers to detect emotions from facial and audio information whereas recognizing emotions from textual data is still a fresh and hot research area. This paper presented a knowledge based survey on emotion detection based on textual data and the methods used for this purpose. At the next step paper also proposed a new architecture for recognizing emotions from text document. Proposed architecture is composed of two main parts, emotion ontology and emotion detector algorithm. Proposed emotion detector system takes a text document and the emotion ontology as inputs and produces one of the six emotion classes (i.e. love, joy, anger, sadness, fear and surprise) as the output.

Visual-textual presentation layout (e.g., digital magazine cover, poster, Power Point slides, and any other rich media), which combines beautiful image and overlaid readable texts, can result in an eye candy touch to attract users’ attention. The designing of visual-textual presentation layout is therefore becoming ubiquitous in both commercially printed publications and online digital magazines. However, handcrafting aesthetically compelling layouts still remains challenging for many small businesses and amateur users. This article presents a system to automatically generate visual-textual presentation layouts by investigating a set of aesthetic design principles, through which an average user can easily create visually appealing layouts. The system is attributed with a set of topic-dependent layout templates and a computational framework integrating high-level aesthetic principles (in a top-down manner) and low-level image features (in a bottom-up manner). The layout templates, designed with prior knowledge from domain experts, define spatial layouts, semantic colors, harmonic color models, and font emotion and size constraints. We formulate the typography as an energy optimization problem by minimizing the cost of text intrusion, the utility of visual space, and the mismatch of information importance in perception and semantics, constrained by the automatically selected template and further preserving color harmonization. We demonstrate that our designs achieve the best reading experience compared with the reimplementation of parts of existing state-of-the-art designs through a series of user studies.

In emotion classification of speech signals, the popular features employed are statistics of fundamental frequency, energy contour, duration of silence and voice quality. However, the performance of systems employing these features degrades substantially when more than two categories of emotion are to be classified. In this paper, a text independent method of emotion classification of speech is proposed. The proposed method makes use of short time log frequency power coefficients (LFPC) to represent the speech signals and a discrete hidden Markov model (HMM) as the classifier. The emotions are classified into six categories. The category labels used are, the archetypal emotions of Anger, Disgust, Fear, Joy, Sadness and Surprise. Adatabase consisting of 60 emotional utterances, each from twelve speakers is constructed and used to train and test the proposed system. Performance of the LFPC feature parameters is compared with that of the linear prediction Cepstral coefficients (LPCC) and mel-frequency Cepstral coefficients (MFCC) feature parameters commonly used in speech recognition systems. Results show that the proposed system yields an average accuracy of 78% and the best accuracy of 96% in the classification of six emotions. This is beyond the 17% chances by a random hit for a sample set of 6 categories. Results also reveal that LFPC is a better choice as feature parameters for emotion classification than the traditional feature parameters. 2003 Elsevier B.V. All rights reserved.

We introduce the openSMILE feature extraction toolkit, which unites feature extraction algorithms from the speech processing and the Music Information Retrieval communities. Audio low-level descriptors such as CHROMA and CENS features, loudness, Mel-frequency cepstral coefficients, perceptual linear predictive cepstral coefficients, linear predictive coefficients, line spectral frequencies, fundamental frequency, and formant frequencies are supported. Delta regression and various statistical functionals can be applied to the low-level descriptors. openSMILE is implemented in C++ with no third-party dependencies for the core functionality. It is fast, runs on Unix and Windows platforms, and has a modular, component based architecture which makes extensions via plug-ins easy. It supports on-line incremental processing for all implemented features as well as off-line and batch processing. Numeric compatibility with future versions is ensured by means of unit tests. openSMILE can be downloaded from http://opensmile.sourceforge.net/.

The importance of automatically recognizing emotions from human speech has grown with the increasing role of spoken language interfaces in human-computer interaction applications. This paper explores the detection of domain-specific emotions using language and discourse information in conjunction with acoustic correlates of emotion in speech signals. The specific focus is on a case study of detecting negative and non-negative emotions using spoken language data obtained from a call center application. Most previous studies in emotion recognition have used only the acoustic information contained in speech. In this paper, a combination of three sources of information-acoustic, lexical, and discourse-is used for emotion recognition. To capture emotion information at the language level, an information-theoretic notion of emotional salience is introduced. Optimization of the acoustic correlates of emotion with respect to classification error was accomplished by investigating different feature sets obtained from feature selection, followed by principal component analysis. Experimental results on our call center data show that the best results are obtained when acoustic and language information are combined. Results show that combining all the information, rather than using only acoustic information, improves emotion classification by 40.7% for males and 36.4% for females (linear discriminant classifier used for acoustic information).

A significant amount of work in automatic music genre recognition has used a dataset whose composition and integrity has never been formally analyzed. For the first time, we provide an analysis of its composition, and create a machine-readable index of artist and song titles. We also catalog numerous problems with its integrity, such as replications, mislabelings, and distortions.


Question: EmoBGM: Assessing sound emotion to generate fitting background music for slideshows.

This study offers fresh insights into the cross-cultural assessment of facial expressions and their emotional content. Participants from 10 distinct cultures were engaged in a more intricate judgment task than those typically employed in previous cross-cultural studies. Unlike past methodologies that confined subjects to choosing a single emotion term per expression, this study allowed participants to indicate the presence of multiple emotions along with the intensity of each. Across cultures, there was a remarkable consensus regarding the most intense emotion portrayed by facial expressions. Moreover, the participating cultures agreed on the second most intense emotion conveyed by an expression and the comparative intensity among expressions of the same emotion. However, the study also revealed cultural disparities in the judgments of the absolute level of emotional intensity, highlighting the nuanced interplay between universal expressions and cultural interpretations.

**Question**: How to use the Google Analytics API to track website traffic

**Documents**: Using the Google Analytics API to track website traffic
To use the Google Analytics API for tracking website traffic, you first need to set up a project in the Google Developers Console and enable the Google Analytics API. Next, you must create credentials for your application. You can use OAuth 2.0 credentials if you need to access data for multiple Google Analytics accounts. For a single account, you can use a Service Account. Once you have your credentials, you can use a library such as the Google Analytics Reporting API client library for Python to make API calls. You will need to authorize your application to access the Google Analytics data, which involves creating a project in the Google Developers Console and downloading a JSON file with your credentials. Then, you can use the library to make API calls to retrieve data from the Google Analytics account.

**Similar Documents**: Leveraging the Google Analytics API for Website Traffic Monitoring
To harness the power of the Google Analytics API for monitoring website traffic, you must initiate the process by establishing a project within the Google Developers Console and activating the Google Analytics API. Following this, you should generate credentials for your application. If your requirement involves accessing data across multiple Google Analytics accounts, OAuth 2.0 credentials are the preferred choice. Alternatively, for a singular account, a Service Account is suitable. After securing your credentials, you can employ a library, such as the Google Analytics Reporting API client library for Python, to facilitate API calls. Authorization of your application to access Google Analytics data is essential, which includes creating a project in the Google Developers Console and downloading a JSON file containing your credentials. Subsequently, you can utilize the library to execute API calls and fetch data from your Google Analytics account, enabling

Timbre distances and similarities are an expression of the phenomenon that some music appears similar while other songs sound very different to us. The notion of genre is often used to categorize music, but songs from a single genre do not necessarily sound similar and vice versa. Instead we aim at a visualization of timbre similarities of sound within a music collection. We analyzed and compared a large amount of different audio features and psychoacoustic variants thereof for the purpose of modelling timbre distance of sound. The sound of polyphonic music is commonly described by extracting audio features on short time windows during which the sound is assumed to be stationary. The resulting down sampled time series are aggregated to form a high level feature vector describing the music. We generated high level features by systematically applying static and temporal statistics for aggregation. Especially the temporal structure of features has previously been largely neglected. A novel supervised feature selection method is applied to the huge set of possible features. Distances between vectors of the selected features correspond to timbre differences in music. The features show few redundancies and have high potential for explaining possible clusters. They outperform seven other previously proposed feature sets on several datasets w.r.t. the separation of the known groups of timbrally different music. Clustering and visualization based on these feature vectors can discover emergent structures in collections of music. Visualization based on Emergent Self-Organizing Maps in particular enables the unsupervised discovery of timbrally consistent clusters that may or may not correspond to musical genres and artists. We demonstrate the visualizations capabilities of the U-Map and related methods based on the new audio features. An intuitive browsing of large music collections is offered based on the paradigm of topographic maps. The user can navigate the sound space and interact with the maps to play music or show the context of a song. Data Bionics Research Group, Philipps-University Marburg, 35032 Marburg, Germany

Predicting when a person might be frustrated can provide an intelligent system with important information about when to initiate interaction. For example, an automated Learning Companion or Intelligent Tutoring System might use this information to intervene, providing support to the learner who is likely to otherwise quit, while leaving engaged learners free to discover things without interruption. This paper presents the first automated method that assesses, using multiple channels of affect-related information, whether a learner is about to click on a button saying ‘‘I’m frustrated.’’ The new method was tested on data gathered from 24 participants using an automated Learning Companion. Their indication of frustration was automatically predicted from the collected data with 79% accuracy ðchance 1⁄4 58%Þ. The new assessment method is based on Gaussian process classification and Bayesian inference. Its performance suggests that non-verbal channels carrying affective cues can help provide important information to a system for formulating a more intelligent response. r 2007 Elsevier Ltd. All rights reserved.

Robots have the potential to facilitate future therapies for children on the autism spectrum. However, existing robots are limited in their ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Their inference challenge is made even harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in children with autism, we used the latest advances in deep learning to formulate a personalized machine learning (ML) framework for automatic perception of the children’s affective states and engagement during robot-assisted autism therapy. Instead of using the traditional one-size-fits-all ML approach, we personalized our framework to each child using their contextual information (demographics and behavioral assessment scores) and individual characteristics. We evaluated this framework on a multimodal (audio, video, and autonomic physiology) data set of 35 children (ages 3 to 13) with autism, from two cultures (Asia and Europe), and achieved an average agreement (intraclass correlation) of ~60% with human experts in the estimation of affect and engagement, also outperforming nonpersonalized ML solutions. These results demonstrate the feasibility of robot perception of affect and engagement in children with autism and have implications for the design of future autism therapies.

Images can convey rich emotions to viewers. Recent research on image emotion analysis mainly focused on affective image classification, trying to find features that can classify emotions better. We concentrate on affective image retrieval and investigate the performance of different features on different kinds of images in a multi-graph learning framework. Firstly, we extract commonly used features of different levels for each image. Generic features and features derived from elements-of-art are extracted as low-level features. Attributes and interpretable principles-of-art based features are viewed as mid-level features, while semantic concepts described by adjective noun pairs and facial expressions are extracted as high-level features. Secondly, we construct single graph for each kind of feature to test the retrieval performance. Finally, we combine the multiple graphs together in a regularization framework to learn the optimized weights of each graph to efficiently explore the complementation of different features. Extensive experiments are conducted on five datasets and the results demonstrate the effectiveness of the proposed method.

Personality-based personalized systems are increasingly gaining interest as personality traits has been shown to be a stable construct within humans. In order to provide a personality-based experience to the user, users' behavior, preferences, and needs in relation to their personality need to be investigated. Although for a technological mediated environment the search for these relationships is often new territory, there are findings from personality research of the real world that can be used in personalized systems. However, for these findings to be implementable, we need to investigate whether they hold in a technologically mediated environment. In this study we assess prior work on personality-based music genre preferences from traditional personality research. We analyzed a dataset consisting of music listening histories and personality scores of 1415 Last.fm users. Our results show agreements with prior work, but also important differences that can help to inform personalized systems.

While many lexica annotated with words polarity are available for sentiment analysis, very few tackle the harder task of emotion analysis and are usually quite limited in coverage. In this paper, we present a novel approach for extracting – in a totally automated way – a highcoverage and high-precision lexicon of roughly 37 thousand terms annotated with emotion scores, called DepecheMood. Our approach exploits in an original way ‘crowd-sourced’ affective annotation implicitly provided by readers of news articles from rappler.com. By providing new state-of-the-art performances in unsupervised settings for regression and classification tasks, even using a naı̈ve approach, our experiments show the beneficial impact of harvesting social media data for affective lexicon building.

Multimodal emotion recognition is the task of detecting emotions present in user-generated multimedia content. Such resources contain complementary information in multiple modalities. A stiff challenge often faced is the complexity associated with feature-level fusion of these heterogeneous modes. In this paper, we propose a new feature-level fusion method based on self-attention mechanism. We also compare it with traditional fusion methods such as concatenation, outer-product, etc. Analyzed using textual and speech (audio) modalities, our results suggest that the proposed fusion method outperforms others in the context of utterance-level emotion recognition in videos.

This paper describes experiments in using audible sound as a means for wireless device communications. The direct application of standard modulation techniques to sound, without further improvements, results in sounds that are immediately perceived as digital communications and that are fairly aggressive and intrusive. We observe that some parameters of the modulation that have an impact in the data rate, the error probability and the computational overhead at the receiver also have a tremendous impact in the quality of the sound as perceived by humans. This paper focuses on how to vary those parameters in standard modulation techniques such as ASK, FSK and Spread-Spectrum to obtain communication systems in which the messages are musical and other familiar sounds, rather than modem sounds. A prototype called Digital Voices demonstrates the feasibility of this music-based communication technology. Our goal is to lay out the basis of sound design for aerial acoustic communications so that the presence of such communications, though noticeable, is not intrusive and can even be considered as part of musical compositions and sound tracks.

The interaction between human beings and computers will be more natural if computers are able to perceive and respond to human non-verbal communication such as emotions. Although several approaches have been proposed to recognize human emotions based on facial expressions or speech, relatively limited work has been done to fuse these two, and other, modalities to improve the accuracy and robustness of the emotion recognition system. This paper analyzes the strengths and the limitations of systems based only on facial expressions or acoustic information. It also discusses two approaches used to fuse these two modalities: decision level and feature level integration. Using a database recorded from an actress, four emotions were classified: sadness, anger, happiness, and neutral state. By the use of markers on her face, detailed facial motions were captured with motion capture, in conjunction with simultaneous speech recordings. The results reveal that the system based on facial expression gave better performance than the system based on just acoustic information for the emotions considered. Results also show the complementarily of the two modalities and that when these two modalities are fused, the performance and the robustness of the emotion recognition system improve measurably.

Multimedia database retrieval is rapidly growing and its popularity in online retrieval systems is consequently increasing. Large datasets are major challenges for searching, retrieving, and organizing the music content. Therefore, a robust automatic music-genre classification method is needed for organizing this music data into different classes according to specific viable information. Two fundamental components are to be considered for genre classification: audio feature extraction and classifier design. In this paper, we propose diverse audio features to precisely characterize the music content. The feature sets belong to four groups: dynamic, rhythmic, spectral, and harmonic. From the features, five statistical parameters are considered as representatives, including the fourth-order central moments of each feature as well as covariance components. Ultimately, insignificant representative parameters are controlled by minimum redundancy and maximum relevance. This algorithm calculates the score level of all feature attributes and orders them. Only high-score audio features are considered for genre classification. Moreover, we can recognize those audio features and distinguish which of the different statistical parameters derived from them are important for genre classification. Among them, mel frequency cepstral coefficient statistical parameters, such as covariance components and variance, are more frequently selected than the feature attributes of other groups. This approach does not transform the original features as do principal component analysis and linear discriminant analysis. In addition, other feature reduction methodologies, such as locality-preserving projection and non-negative matrix factorization are considered. The performance of the proposed system is measured based on the reduced features from the feature pool using different feature reduction techniques. The results indicate that the overall classification is competitive with existing state-of-the-art frame-based methodologies.


Question: EmoBGM: Assessing sound emotion to generate appropriate background music for slideshows.

The Role of Background Music in Shaping Customer Experience in Restaurants

This study explores the impact of background music on the behavior and overall experience of patrons in restaurant settings. Background music, often an overlooked aspect of the dining environment, plays a crucial role in setting the ambiance and influencing customer mood, leading to potential effects on dining behavior, satisfaction, and loyalty. The research delves into how different genres, tempos, and volumes of music can alter the perceived atmosphere, affecting how long patrons stay, the amount they consume, and their overall emotional state. Findings suggest that carefully selected background music can enhance the dining experience, contributing to a more positive perception of the restaurant and potentially increasing customer satisfaction and repeat visits. This highlights the importance of considering music as a strategic tool in the restaurant industry, aiming to create a harmonious and inviting atmosphere that complements the culinary offerings and enhances the overall customer experience.

We develop a method for emotion recognition from facial imagery. This problem is challenging in part because of the subjectivity of ground truth labels and in part because of the relatively small size of existing labeled datasets. We use the FER+ dataset [8], a dataset with multiple emotion labels per image, in order to build an emotion recognition model that encompasses a full range of emotions. Since the amount of data in the FER+ dataset is limited, we explore the use of a much larger face dataset, MS-Celeb-1M [41], in conjunction with the FER+ dataset. Specific layers within an Inception-ResNet-v1 [13, 38] model trained for facial recognition are used for the emotion recognition problem. Thus, we leverage the MS-Celeb-1M dataset in addition to the FER+ dataset and experiment with different architectures to assess the overall performance of neural networks to recognize emotion using facial imagery.

This paper presents an approach to automatic visual emotion recognition from two modalities: face and body. Firstly, individual classifiers are trained from individual modalities. Secondly, we fuse facial expression and affective body gesture information first at a feature-level, in which the data from both modalities are combined before classification, and later at a decision-level, in which we integrate the outputs of the monomodal systems by the use of suitable criteria. We then evaluate these two fusion approaches, in terms of performance over monomodal emotion recognition based on facial expression modality only. In the experiments performed the emotion classification using the two modalities achieved a better recognition accuracy outperforming the classification using the individual facial modality. Moreover, fusion at the feature-level proved better recognition than fusion at the decision-level.

The Face Expression Based Emoticon Identification (FEBEI) system is an open source extension to the Tracker.js framework which converts a human facial expression to the best matching emoticon. The contribution of this project was to build this robust classifier which can identify facial expression in real time without any reliance on an external server or computation node. An entirely client-side JavaScript implementation has clear privacy benefits as well as the avoidance of any lag inherent in uploading and downloading images. We accomplished this by utilizing several computationally efficient methods. Tracking.js provided a Viola Jones based face detector which we used to pass facial images to our own implementation of an eigenemotion detection system which was trained to distinguish between happy and angry faces. We have implemented a similar eigenface classifier in python and have trained a Convoluted Neural Network (CNN) to classify emotions to provide a comparative view of its advantages. We aim to make FEBEI easily extendable so that the developer community will be able to add classifiers for more emoticon.

The advancement in technology indicates that there is an opportunity to enhance Human-Computer Interaction by way of affective state recognition. Affective state recognition is typically based on passive stimuli such as watching video clips, which does not reflect genuine interaction. This paper presents a study on affective state recognition using active stimuli, i.e. facial expressions of users when they attempt computerised tasks, particularly across typical usage of computer systems. A data collection experiment is presented for acquiring data from normal users whilst they interact with software, attempting to complete a set of predefined tasks. In addition, a hierarchical machine learning approach is presented for facial expression-based affective state recognition, which employs an Euclidean distance-based feature representation, conjointly with a customised encoding for users’ self-reported affective states. Consequently, the aim is to find the potential relationship between the facial expressions, as defined by Paul Ekman, and the self-reported emotional states specified by users using Russells Circumplex model, in relation to the actual feelings and affective states. The main findings of this study suggest that facial expressions cannot precisely reveal the actual feelings of users whilst interacting with common computerised tasks. Moreover, during active interaction tasks more variation occurs within the facial expressions of participants than occurs within passive interaction.

In emotion classification of speech signals, the popular features employed are statistics of fundamental frequency, energy contour, duration of silence and voice quality. However, the performance of systems employing these features degrades substantially when more than two categories of emotion are to be classified. In this paper, a text independent method of emotion classification of speech is proposed. The proposed method makes use of short time log frequency power coefficients (LFPC) to represent the speech signals and a discrete hidden Markov model (HMM) as the classifier. The emotions are classified into six categories. The category labels used are, the archetypal emotions of Anger, Disgust, Fear, Joy, Sadness and Surprise. Adatabase consisting of 60 emotional utterances, each from twelve speakers is constructed and used to train and test the proposed system. Performance of the LFPC feature parameters is compared with that of the linear prediction Cepstral coefficients (LPCC) and mel-frequency Cepstral coefficients (MFCC) feature parameters commonly used in speech recognition systems. Results show that the proposed system yields an average accuracy of 78% and the best accuracy of 96% in the classification of six emotions. This is beyond the 17% chances by a random hit for a sample set of 6 categories. Results also reveal that LFPC is a better choice as feature parameters for emotion classification than the traditional feature parameters. 2003 Elsevier B.V. All rights reserved.

Future Work Dataset and Features Music has been an integral part of our culture all throughout human history. In 2012 alone, the U.S. music industry generated $15 billion. Having a fundamental understanding of what makes a song popular has major implications to businesses that thrive on popular music, namely radio stations, record labels, and digital and physical music market places. Many private companies in these industries have solutions for this problem, but details have been kept private for competitive reasons. For our project, we will predict song popularity based on an song’s audio features and metadata. Methods & Results

We present a method to classify fixed-duration windows of speech as expressing anger or not, which does not require speech recognition, utterance segmentation, or separating the utterances of different speakers and can, thus, be easily applied to real-world recordings. We also introduce the task of ranking a set of spoken dialogues by decreasing percentage of anger duration, as a step towards helping call center supervisors and analysts identify conversations requiring further action. Our work is among the very few attempts to detect emotions in spontaneous human-human dialogues recorded in call centers, as opposed to acted studio recordings or human-machine dialogues. We show that despite the non-perfect performance (approx. 70% accuracy) of the window-level classifier, its decisions help produce a ranking of entire conversations by decreasing percentage of anger duration that is clearly better than a random ranking, which represents the case where supervisors and analysts randomly select conversations to inspect.

Robots have the potential to facilitate future therapies for children on the autism spectrum. However, existing robots are limited in their ability to automatically perceive and respond to human affect, which is necessary for establishing and maintaining engaging interactions. Their inference challenge is made even harder by the fact that many individuals with autism have atypical and unusually diverse styles of expressing their affective-cognitive states. To tackle the heterogeneity in children with autism, we used the latest advances in deep learning to formulate a personalized machine learning (ML) framework for automatic perception of the children’s affective states and engagement during robot-assisted autism therapy. Instead of using the traditional one-size-fits-all ML approach, we personalized our framework to each child using their contextual information (demographics and behavioral assessment scores) and individual characteristics. We evaluated this framework on a multimodal (audio, video, and autonomic physiology) data set of 35 children (ages 3 to 13) with autism, from two cultures (Asia and Europe), and achieved an average agreement (intraclass correlation) of ~60% with human experts in the estimation of affect and engagement, also outperforming nonpersonalized ML solutions. These results demonstrate the feasibility of robot perception of affect and engagement in children with autism and have implications for the design of future autism therapies.

This paper is concerned with the discovery and aggregation of events that provoke a particular emotion in the person who experiences them, or emotion-provoking events. We first describe the creation of a small manually-constructed dictionary of events through a survey of 30 subjects. Next, we describe first attempts at automatically acquiring and aggregating these events from web data, with a baseline from previous work and some simple extensions using seed expansion and clustering. Finally, we propose several evaluation measures for evaluating the automatically acquired events, and perform an evaluation of the effectiveness of automatic event extraction.

In this paper, we present our latest progress in Emotion Recognition techniques, which combines acoustic features and facial features in both non-temporal and temporal mode. This paper presents the details of our techniques used in the Audio-Video Emotion Recognition subtask in the 2018 Emotion Recognition in the Wild (EmotiW) Challenge. After the multimodal results fusion, our final accuracy in Acted Facial Expression in Wild (AFEW) test dataset achieves 61.87%, which is 1.53% higher than the best results last year. Such improvements prove the effectiveness of our methods.