Question: Variance minimization techniques to achieve accelerated non-convex optimization.

This paper introduces a novel spectral algorithm designed for the efficient learning of Hidden Markov Models (HMMs), a critical statistical tool widely utilized for modeling discrete time series. Traditionally, HMMs have been learned using search heuristics like the Baum-Welch algorithm or Expectation-Maximization (EM), which can be plagued by the issue of converging to local optima. While learning HMMs from samples drawn from the underlying distribution is generally acknowledged to be challenging, we present the first algorithm that can provably learn HMMs in an efficient manner, both in terms of sample and computational complexity, under a natural separation condition. This condition is akin to the separation requirements considered for the learning of mixture distributions, where models are known to be difficult to learn without such conditions. Notably, our sample complexity analysis avoids explicit dependence on the number of distinct observations, instead basing this dependence on the spectral properties of the underlying HMM. This makes our algorithm especially suitable for scenarios involving a vast number of observations, such as those encountered in natural language processing, where the observation space might encompass the entire vocabulary of a language. Moreover, the simplicity of the algorithm is a significant advantage, as it relies solely on singular value decomposition and matrix multiplications, making it straightforward to implement and apply.

While it has not yet been proven, empirical evidence suggests that model generalization is related to local properties of the optima which can be described via the Hessian. We connect model generalization with the local property of a solution under the PAC-Bayes paradigm. In particular, we prove that model generalization ability is related to the Hessian, the higher-order “smoothness” terms characterized by the Lipschitz constant of the Hessian, and the scales of the parameters. Guided by the proof, we propose a metric to score the generalization capability of the model, as well as an algorithm that optimizes the perturbed model accordingly.

Recent work in unsupervised feature learning and deep learning has shown that being able to train large models can dramatically improve performance. In this paper, we consider the problem of training a deep network with billions of parameters using tens of thousands of CPU cores. We have developed a software framework called DistBelief that can utilize computing clusters with thousands of machines to train large models. Within this framework, we have developed two algorithms for large-scale distributed training: (i) Downpour SGD, an asynchronous stochastic gradient descent procedure supporting a large number of model replicas, and (ii) Sandblaster, a framework that supports a variety of distributed batch optimization procedures, including a distributed implementation of L-BFGS. Downpour SGD and Sandblaster L-BFGS both increase the scale and speed of deep network training. We have successfully used our system to train a deep network 30x larger than previously reported in the literature, and achieves state-of-the-art performance on ImageNet, a visual object recognition task with 16 million images and 21k categories. We show that these same techniques dramatically accelerate the training of a more modestlysized deep network for a commercial speech recognition service. Although we focus on and report performance of these methods as applied to training large neural networks, the underlying algorithms are applicable to any gradient-based machine learning algorithm.

One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth. This paper demystifies this surprising phenomenon for two-layer fully connected ReLU activated neural networks. For anm hidden node shallow neural network with ReLU activation and n training data, we show as long asm is large enough and no two inputs are parallel, randomly initialized gradient descent converges to a globally optimal solution at a linear convergence rate for the quadratic loss function. Our analysis relies on the following observation: over-parameterization and random initialization jointly restrict every weight vector to be close to its initialization for all iterations, which allows us to exploit a strong convexity-like property to show that gradient descent converges at a global linear rate to the global optimum. We believe these insights are also useful in analyzing deep models and other first order methods.

TheSGD-QN algorithm is a stochastic gradient descent algorithm that m akes careful use of secondorder information and splits the parameter update into inde pendently scheduled components. Thanks to this design, SGD-QN iterates nearly as fast as a first-order stochastic gradient escent but requires less iterations to achieve the same accuracy. This algorith m won the “Wild Track” of the first PASCAL Large Scale Learning Challenge (Sonnenburg et al., 2008 ).

In this paper we give a new fast iterative algorithm for support vector machine (SVM) classifier design. The basic problem treated is one that does not allow classification violations. The problem is converted to a problem of computing the nearest point between two convex polytopes. The suitability of two classical nearest point algorithms, due to Gilbert, and Mitchell et al., is studied. Ideas from both these algorithms are combined and modified to derive our fast algorithm. For problems which require classification violations to be allowed, the violations are quadratically penalized and an idea due to Cortes and Vapnik and Friess is used to convert it to a problem in which there are no classification violations. Comparative computational evaluation of our algorithm against powerful SVM methods such as Platt's sequential minimal optimization shows that our algorithm is very competitive.

Distributed training of deep learning models on large-scale training data is typically conducted with asynchronous stochastic optimization to maximize the rate of updates, at the cost of additional noise introduced from asynchrony. In contrast, the synchronous approach is often thought to be impractical due to idle time wasted on waiting for straggling workers. We revisit these conventional beliefs in this paper, and examine the weaknesses of both approaches. We demonstrate that a third approach, synchronous optimization with backup workers, can avoid asynchronous noise while mitigating for the worst stragglers. Our approach is empirically validated and shown to converge faster and to better test accuracies.

Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence. But they are rarely applied to deep learning in practice because of high computational cost and the need for model-dependent algorithmic variations. We introduce a variant of the Hessian-free method that leverages a block-diagonal approximation of the generalized Gauss-Newton matrix. Our method computes the curvature approximation matrix only for pairs of parameters from the same layer or block of the neural network and performs conjugate gradient updates independently for each block. Experiments on deep autoencoders, deep convolutional networks, and multilayer LSTMs demonstrate better convergence and generalization compared to the original Hessian-free approach and the Adam method.

It is widely believed that in order to solve large scale global optimization problems an appropriate mixture of local approximation and global exploration is necessary. Local approximation, if first order information on the objective function is available, is efficiently performed by means of local optimization methods. Unfortunately, global exploration, in absence of some kind of global information on the problem, is a “blind” procedure, aimed at placing observations as evenly as possible in the search domain. Often this procedure reduces to uniform random sampling (like in Multistart algorithms, or in techniques based on clustering). In this paper we propose a new framework for global exploration which tries to guide random exploration towards the region of attraction of low-level local optima. The main idea originated by the use of smoothing techniques (based on gaussian convolutions): the possibility of applying a smoothing transformation not to the objective function but to the result of local searches seems to have never been explored yet. Although an exact smoothing of the results of local searches is impossible to implement, in this paper we propose a computational approximation scheme which has proven to be very efficient and (maybe more important) extremely robust in solving large scale global optimization problems with huge numbers of local optima. ∗Email: b.addis@ing.unifi.it Dip. Sistemi e Informatica Università di Firenze †Email: locatell@di.unito.it Dip. Informatica Università di Torino ‡Email: schoen@ing.unifi.it Dip. Sistemi e Informatica Università di Firenze

We propose a simple yet effective technique to simplify the training and the resulting model of neural networks. In back propagation, only a small subset of the full gradient is computed to update the model parameters. The gradient vectors are sparsified in such a way that only the top-k elements (in terms of magnitude) are kept. As a result, only k rows or columns (depending on the layout) of the weight matrix are modified, leading to a linear reduction in the computational cost. Based on the sparsified gradients, we further simplify the model by eliminating the rows or columns that are seldom updated, which will reduce the computational cost both in the training and decoding, and potentially accelerate decoding in real-world applications. Surprisingly, experimental results demonstrate that most of time we only need to update fewer than 5% of the weights at each back propagation pass. More interestingly, the accuracy of the resulting models is actually improved rather than degraded, and a detailed analysis is given. The model simplification results show that we could adaptively simplify the model which could often be reduced by around 9x, without any loss on accuracy or even with improved accuracy.

Kernel method has been developed as one of the standard approaches for nonlinear learning, which however, does not scale to large data set due to its quadratic complexity in the number of samples. A number of kernel approximation methods have thus been proposed in the recent years, among which the random features method gains much popularity due to its simplicity and direct reduction of nonlinear problem to a linear one. Different random feature functions have since been proposed to approximate a variety of kernel functions. Among them the Random Binning (RB) feature, proposed in the first random-feature paper [21], has drawn much less attention than the Random Fourier (RF) feature proposed also in [21]. In this work, we observe that the RB features, with right choice of optimization solver, could be orders-of-magnitude more efficient than other random features and kernel approximation methods under the same requirement of accuracy. We thus propose the first analysis of RB from the perspective of optimization, which by interpreting RB as a Randomized Block Coordinate Descent in the infinite-dimensional space, gives a faster convergence rate compared to that of other random features. In particular, we show that by drawing R random grids with at least κ number of non-empty bins per grid in expectation, RB method achieves a convergence rate of O(1/κ R)), which not only sharpens its O(1/√R) rate from Monte Carlo analysis, but also shows a κ times speedup over other random features under the same analysis framework. In addition, we demonstrate another advantage of RB in the L1-regularized setting, where unlike other random features, a RB-based Coordinate Descent solver can be parallelized with guaranteed speedup proportional to κ. Our extensive experiments demonstrate the superior performance of the RB features over other random features and kernel approximation methods.


Question: Reduced Variance Techniques in Accelerating Non-Convex Optimization Algorithms

This paper revisits the Stochastic Variance Reduced Gradient (SVRG) method, initially developed for strongly convex objectives, and demonstrates its adaptability and robustness in non-strongly convex and sum-of-non-convex scenarios. When dealing with objectives that are the sum of smooth, convex functions but lack strong convexity, such as in Lasso regression or logistic regression, we propose an enhanced version of SVRG that employs a strategic, increasing epoch length. This modification results in a more efficient SVRG variant for these non-strongly convex settings, offering faster convergence than standard SVRG. In cases where the objective function is a sum of non-convex functions yet still strongly convex overall, we establish that the convergence rate of SVRG is directly influenced by the non-convexity parameter of the component functions. This finding not only outperforms the current state-of-the-art results but also leads to improved run times for Stochastic Principal Component Analysis (PCA), showcasing SVRG's versatility and effectiveness in various optimization landscapes.

Valuable training data is often owned by independent organizations and located in multiple data centers. Most deep learning approaches require to centralize the multi-datacenter data for performance purpose. In practice, however, it is often infeasible to transfer all data to a centralized data center due to not only bandwidth limitation but also the constraints of privacy regulations. Model averaging is a conventional choice for data parallelized training, but its ineffectiveness is claimed by previous studies as deep neural networks are often non-convex. In this paper, we argue that model averaging can be effective in the decentralized environment by using two strategies, namely, the cyclical learning rate and the increased number of epochs for local model training. With the two strategies, we show that model averaging can provide competitive performance in the decentralized mode compared to the data-centralized one. In a practical environment with multiple data centers, we conduct extensive experiments using state-of-the-art deep network architectures on different types of data. Results demonstrate the effectiveness and robustness of the proposed method.

In this paper, we consider the problem of federated (or decentralized) learning using ADMM with multiple agents. We consider a scenario where a certain fraction of agents (referred to as Byzantines) provide falsified data to the system. In this context, we study the convergence behavior of the decentralized ADMM algorithm. We show that ADMM converges linearly to a neighborhood of the solution to the problem under certain conditions. We next provide guidelines for network structure design to achieve faster convergence. Next, we provide necessary conditions on the falsified updates for exact convergence to the true solution. To tackle the data falsification problem, we propose a robust variant of ADMM. We also provide simulation results to validate the analysis and show the resilience of the proposed algorithm to Byzantines.

Many recent deep learning platforms rely on thirdparty libraries (such as cuBLAS) to utilize the computing power of modern hardware accelerators (such as GPUs). However, we observe that they may achieve suboptimal performance because the library functions are not used appropriately. In this paper, we target at optimizing the operations of multiplying a matrix with the transpose of another matrix (referred to as NT operation hereafter), which contribute half of the training time of fully connected deep neural networks. Rather than directly calling the library function, we propose a supervised learning based algorithm selection approach named MTNN, which uses a gradient boosted decision tree to select one from two alternative NT implementations intelligently: (1) calling the cuBLAS library function; (2) calling our proposed algorithm TNN that uses an efficient out-of-place matrix transpose. We evaluate the performance of MTNN on two modern GPUs: NVIDIA GTX 1080 and NVIDIA Titan X Pascal. MTNN can achieve 96% of prediction accuracy with very low computational overhead, which results in an average of 54% performance improvement on a range of NT operations. To further evaluate the impact of MTNN on the training process of deep neural networks, we have integrated MTNN into a popular deep learning platform Caffe. Our experimental results show that the revised Caffe can outperform the original one by an average of 28%. Both MTNN and the revised Caffe are open-source.

We study a distributed computation model for optimizing a sum of convex objective functions corresponding to multiple agents. For solving this (not necessarily smooth) optimization problem, we consider a subgradient method that is distributed among the agents. The method involves every agent minimizing his/her own objective function while exchanging information locally with other agents in the network over a time-varying topology. We provide convergence results and convergence rate estimates for the subgradient method. Our convergence rate results explicitly characterize the tradeoff between a desired accuracy of the generated approximate optimal solutions and the number of iterations needed to achieve the accuracy.

Deep and recurrent neural networks (DNNs and RNNs respectively) are powerful models that were considered to be almost impossible to train using stochastic gradient descent with momentum. In this paper, we show that when stochastic gradient descent with momentum uses a well-designed random initialization and a particular type of slowly increasing schedule for the momentum parameter, it can train both DNNs and RNNs (on datasets with long-term dependencies) to levels of performance that were previously achievable only with Hessian-Free optimization. We find that both the initialization and the momentum are crucial since poorly initialized networks cannot be trained with momentum and well-initialized networks perform markedly worse when the momentum is absent or poorly tuned. Our success training these models suggests that previous attempts to train deep and recurrent neural networks from random initializations have likely failed due to poor initialization schemes. Furthermore, carefully tuned momentum methods su ce for dealing with the curvature issues in deep and recurrent network training objectives without the need for sophisticated second-order methods.

We propose a novel training algorithm for reinforcement learning which combines the strength of deep Q-learning with a constrained optimization approach to tighten optimality and encourage faster reward propagation. Our novel technique makes deep reinforcement learning more practical by drastically reducing the training time. We evaluate the performance of our approach on the 49 games of the challenging Arcade Learning Environment, and report significant improvements in both training time and accuracy.

Kernel approximation via nonlinear random feature maps is w idely used in speeding up kernel machines. There are two main challenges for the conventional kernel ap proximation methods. First, before performing kernel approximation, a good kernel has to be chosen. Pickin g a good kernel is a very challenging problem in itself. Second, high-dimensional maps are often required i n order to achieve good performance. This leads to high computational cost in both generating the nonlinear ma ps, nd in the subsequent learning and prediction process. In this work, we propose to optimize the nonlinear m ps directly with respect to the classification objective in a data-dependent fashion. The proposed approa ch achieves kernel approximation and kernel learning in a joint framework. This leads to much more compact maps wit hout hurting the performance. As a by-product, the same framework can also be used to achieve more compact ke rnel maps to approximate a known kernel. We also introduce Circulant Nonlinear Maps, which uses a circu lant-structured projection matrix to speed up the nonlinear maps for high-dimensional data.

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we theoretically study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this general optimization dynamics, we analyze the behavior of SGD on escaping from minima and its regularization effects. A novel indicator is derived to characterize the efficiency of escaping from minima through measuring the alignment of noise covariance and the curvature of loss function. Based on this indicator, two conditions are established to show which type of noise structure is superior to isotropic noise in term of escaping efficiency. We further show that the anisotropic noise in SGD satisfies the two conditions, and thus helps to escape from sharp and poor minima effectively, towards more stable and flat minima that typically generalize well. We verify our understanding through comparing this anisotropic diffusion with full gradient descent plus isotropic diffusion (i.e. Langevin dynamics) and other types of position-dependent noise.

A new global optimization algorithm for functions of many continuous variables is presented, derived from the basic Simulated annealing method. Our main contribution lies in dealing with high-dimensionality minimization problems, which are often difficult to solve by all known minimization methods with or without gradient. In this article we take a special interest in the variables discretization issue. We also develop and implement several complementary stopping criteria. The original Metropolis iterative random search, which takes place in a Euclidean space Rn, is replaced by another similar exploration, performed within a succession of Euclidean spaces Rp, with p <<n. This Enhanced Simulated Annealing (ESA) algorithm was validated first on classical highly multimodal functions of 2 to 100 variables. We obtained significant reductions in the number of function evaluations compared to six other global optimization algorithms, selected according to previously published computational results for the same set of test functions. In most cases, ESA was able to closely approximate known global optima. The reduced ESA computational cost helped us to refine further the obtained global results, through the use of some local search. We have used this new minimizing procedure to solve complex circuit design problems, for which the objective function evaluation can be exceedingly costly.

In order to extract the best possible performance from asynchronous stochastic gradient descent one must increase the mini-batch size and scale the learning rate accordingly. In order to achieve further speedup we introduce a technique that delays gradient updates effectively increasing the mini-batch size. Unfortunately with the increase of mini-batch size we worsen the stale gradient problem in asynchronous stochastic gradient descent (SGD) which makes the model convergence poor. We introduce local optimizers which mitigate the stale gradient problem and together with fine tuning our momentum we are able to train a shallow machine translation system 27% faster than an optimized baseline with negligible penalty in BLEU.


Question: Variance minimization to achieve accelerated non-convex optimization.

This paper presents a novel variance reduction technique for accelerating the convergence of Stochastic Gradient Descent (SGD) in large-scale optimization problems. Known for its efficiency in handling large datasets, SGD suffers from slow convergence rates due to inherent variance in the gradient estimates. To address this issue, we propose a method called Stochastic Variance Reduced Gradient (SVRG), which explicitly reduces variance in the gradient updates. For optimization problems involving smooth and strongly convex functions, SVRG achieves the same rapid convergence rates as other state-of-the-art methods like Stochastic Dual Coordinate Ascent (SDCA) and Stochastic Average Gradient (SAG), but with a more straightforward and intuitive theoretical analysis. A key advantage of SVRG over SDCA and SAG is that it does not require the storage of gradients, making it particularly suitable for complex optimization tasks such as structured prediction problems and neural network learning. This feature significantly simplifies the implementation and application of SVRG in real-world scenarios, particularly in domains where memory constraints are a concern.

In this paper, we present two new communication-efficient methods for distributed minimization of an average of functions. The first algorithm is an inexact variant of the DANE algorithm [20] that allows any local algorithm to return an approximate solution to a local subproblem. We show that such a strategy does not affect the theoretical guarantees of DANE significantly. In fact, our approach can be viewed as a robustification strategy since the method is substantially better behaved than DANE on data partition arising in practice. It is well known that DANE algorithm does not match the communication complexity lower bounds. To bridge this gap, we propose an accelerated variant of the first method, called AIDE, that not only matches the communication lower bounds but can also be implemented using a purely first-order oracle. Our empirical results show that AIDE is superior to other communication efficient algorithms in settings that naturally arise in machine learning applications.

This paper presents a combinatorial polynomial-time algorithm for minimizing submodular functions, answering an open question posed in 1981 by Grötschel, Lovász, and Schrijver. The algorithm employs a scaling scheme that uses a flow in the complete directed graph on the underlying set with each arc capacity equal to the scaled parameter. The resulting algorithm runs in time bounded by a polynomial in the size of the underlying set and the length of the largest absolute function value. The paper also presents a strongly polynomial version in which the number of steps is bounded by a polynomial in the size of the underlying set, independent of the function values.

We study the numerical performance of a limited memory quasi Newton method for large scale optimization which we call the L BFGS method We compare its performance with that of the method developed by Buckley and LeNir which combines cyles of BFGS steps and conjugate direction steps Our numerical tests indicate that the L BFGS method is faster than the method of Buckley and LeNir and is better able to use additional storage to accelerate convergence We show that the L BFGS method can be greatly accelerated by means of a simple scaling We then compare the L BFGSmethod with the partitioned quasi Newton method of Griewank and Toint a The results show that for some problems the partitioned quasi Newton method is clearly superior to the L BFGS method However we nd that for other problems the L BFGS method is very competitive due to its low iteration cost We also study the convergence properties of the L BFGS method and prove global convergence on uniformly convex problems

In this paper, we show how to transform any optimization problem that arises from fitting a machine learning model into one that (1) detects and removes contaminated data from the training set while (2) simultaneously fitting the trimmed model on the uncontaminated data that remains. To solve the resulting nonconvex optimization problem, we introduce a fast stochastic proximal-gradient algorithm that incorporates prior knowledge through nonsmooth regularization. For datasets of size n, our approach requires O(n/ε) gradient evaluations to reach ε-accuracy and, when a certain error bound holds, the complexity improves to O(κn log(1/ε)). These rates are n times better than those achieved by typical, full gradient methods.

We develop a 2nd-order optimization method based on the “Hessian-free” approach, and apply it to training deep auto-encoders. Without using pre-training, we obtain results superior to those reported by Hinton & Salakhutdinov (2006) on the same tasks they considered. Our method is practical, easy to use, scales nicely to very large datasets, and isn’t limited in applicability to autoencoders, or any specific model class. We also discuss the issue of “pathological curvature” as a possible explanation for the difficulty of deeplearning and how 2nd-order optimization, and our method in particular, effectively deals with it.

Most machine learning researchers perform quantitative experiments to estimate generalization error and compare algorithm performances. In order to draw statistically convincing conclusions, it is important to estimate the uncertainty of such estimates. This paper studies the estimation of uncertainty around the K-fold cross-validation estimator. The main theorem shows that there exists no universal unbiased estimator of the variance of K-fold cross-validation. An analysis based on the eigendecomposition of the covariance matrix of errors helps to better understand the nature of the problem and shows that naive estimators may grossly underestimate variance, as con£rmed by numerical experiments.

This paper presents an algorithm to estimate simultaneously both mean and variance of a non parametric regression problem. The key point is that we are able to estimate variance locally unlike standard Gaussian Process regression or SVMs. This means that our estimator adapts to the local noise. The problem is cast in the setting of maximum a posteriori estimation in exponential families. Unlike previous work, we obtain a convex optimization problem which can be solved via Newton's method.

This paper is the result of contrived efforts to break the barrier between numerical accuracy and run time efficiency in computing the fundamental decomposition of numerical linear algebra – the singular value decomposition (SVD) of a general dense matrix. It is an unfortunate fact that the numerically most accurate one–sided Jacobi SVD algorithm is several times slower than generally less accurate bidiagonalization based methods such as the QR or the divide and conquer algorithm. Despite its sound numerical qualities, the Jacobi SVD is not included in the state of the art matrix computation libraries and it is even considered obsolete by some leading researches. Our quest for a highly accurate and efficient SVD algorithm has led us to a new, superior variant of the Jacobi algorithm. The new algorithm has inherited all good high accuracy properties, and it outperforms not only the best implementations of the one–sided Jacobi algorithm but also the QR algorithm. Moreover, it seems that the potential of the new approach is yet to be fully exploited.

Variational methods have been previously explored as a tractable approximation to Bayesian inference for neural networks. However the approaches proposed so far have only been applicable to a few simple network architectures. This paper introduces an easy-to-implement stochastic variational method (or equivalently, minimum description length loss function) that can be applied to most neural networks. Along the way it revisits several common regularisers from a variational perspective. It also provides a simple pruning heuristic that can both drastically reduce the number of network weights and lead to improved generalisation. Experimental results are provided for a hierarchical multidimensional recurrent neural network applied to the TIMIT speech corpus.

We study a distributed computation model for optimizing a sum of convex objective functions corresponding to multiple agents. For solving this (not necessarily smooth) optimization problem, we consider a subgradient method that is distributed among the agents. The method involves every agent minimizing his/her own objective function while exchanging information locally with other agents in the network over a time-varying topology. We provide convergence results and convergence rate estimates for the subgradient method. Our convergence rate results explicitly characterize the tradeoff between a desired accuracy of the generated approximate optimal solutions and the number of iterations needed to achieve the accuracy.


Question: A method to decrease variance for accelerated non-convex optimization.

This paper explores the application of accelerated proximal gradient (APG) methods to nonconvex and nonsmooth optimization problems, which have gained prominence in various fields including signal processing, statistics, and machine learning. Despite the popularity of APG for convex programming, its efficacy in nonconvex settings has been unexplored until now. By introducing a monitoring mechanism that ensures sufficient descent, we have adapted APG for nonconvex and nonsmooth optimization. This has led to the development of two APG variants: a monotone APG and a nonmonotone APG. The nonmonotone APG, in particular, eliminates the need for the objective function to monotonically decrease, thus reducing computational requirements per iteration. We believe our work is pioneering in demonstrating the feasibility of APG-type algorithms for broad nonconvex and nonsmooth problems, guaranteeing that each accumulation point is a critical point. Moreover, the convergence rates of our algorithms match those of convex problems, remaining at O (1/k^2), where k represents the iteration count. Empirical results validate the superior speed of our proposed algorithms.

Understanding the behavior of stochastic gradient descent (SGD) in the context of deep neural networks has raised lots of concerns recently. Along this line, we theoretically study a general form of gradient based optimization dynamics with unbiased noise, which unifies SGD and standard Langevin dynamics. Through investigating this general optimization dynamics, we analyze the behavior of SGD on escaping from minima and its regularization effects. A novel indicator is derived to characterize the efficiency of escaping from minima through measuring the alignment of noise covariance and the curvature of loss function. Based on this indicator, two conditions are established to show which type of noise structure is superior to isotropic noise in term of escaping efficiency. We further show that the anisotropic noise in SGD satisfies the two conditions, and thus helps to escape from sharp and poor minima effectively, towards more stable and flat minima that typically generalize well. We verify our understanding through comparing this anisotropic diffusion with full gradient descent plus isotropic diffusion (i.e. Langevin dynamics) and other types of position-dependent noise.

In distributed training of deep neural networks, parallel minibatch SGD is widely used to speed up the training process by using multiple workers. It uses multiple workers to sample local stochastic gradient in parallel, aggregates all gradients in a single server to obtain the average, and update each worker’s local model using a SGD update with the averaged gradient. Ideally, parallel mini-batch SGD can achieve a linear speed-up of the training time (with respect to the number of workers) compared with SGD over a single worker. However, such linear scalability in practice is significantly limited by the growing demand for gradient communication as more workers are involved. Model averaging, which periodically averages individual models trained over parallel workers, is another common practice used for distributed training of deep neural networks since (Zinkevich et al. 2010) (McDonald, Hall, and Mann 2010). Compared with parallel mini-batch SGD, the communication overhead of model averaging is significantly reduced. Impressively, tremendous experimental works have verified that model averaging can still achieve a good speed-up of the training time as long as the averaging interval is carefully controlled. However, it remains a mystery in theory why such a simple heuristic works so well. This paper provides a thorough and rigorous theoretical study on why model averaging can work as well as parallel mini-batch SGD with significantly less communication overhead. Introduction Consider the distributed training of deep neural networks over multiple workers (Dean et al. 2012), where all workers can access all or partial training data and aim to find a common model that yields the minimum training loss. Such a scenario can be modeled as the following distributed parallel non-convex optimization min x∈Rm f(x) ∆ = 1 N N ∑

A new global optimization algorithm for functions of many continuous variables is presented, derived from the basic Simulated annealing method. Our main contribution lies in dealing with high-dimensionality minimization problems, which are often difficult to solve by all known minimization methods with or without gradient. In this article we take a special interest in the variables discretization issue. We also develop and implement several complementary stopping criteria. The original Metropolis iterative random search, which takes place in a Euclidean space Rn, is replaced by another similar exploration, performed within a succession of Euclidean spaces Rp, with p <<n. This Enhanced Simulated Annealing (ESA) algorithm was validated first on classical highly multimodal functions of 2 to 100 variables. We obtained significant reductions in the number of function evaluations compared to six other global optimization algorithms, selected according to previously published computational results for the same set of test functions. In most cases, ESA was able to closely approximate known global optima. The reduced ESA computational cost helped us to refine further the obtained global results, through the use of some local search. We have used this new minimizing procedure to solve complex circuit design problems, for which the objective function evaluation can be exceedingly costly.

In this paper, we propose the primal-dual method of multipliers (PDMM) for distributed optimization over a graph. In particular, we optimize a sum of convex functions defined over a graph, where every edge in the graph carries a linear equality constraint. In designing the new algorithm, an augmented primal-dual Lagrangian function is constructed which smoothly captures the graph topology. It is shown that a saddle point of the constructed function provides an optimal solution of the original problem. Further under both the synchronous and asynchronous updating schemes, PDMM has the convergence rate of <inline-formula><tex-math notation="LaTeX">$O(1/K)$</tex-math></inline-formula> (where <inline-formula><tex-math notation="LaTeX">$K$</tex-math></inline-formula> denotes the iteration index) for general closed, proper, and convex functions. Other properties of PDMM such as convergence speeds versus different parameter-settings and resilience to transmission failure are also investigated through the experiments of distributed averaging.

We propose a new class of support vector algorithms for regression and classification. In these algorithms, a parameter lets one effectively control the number of support vectors. While this can be useful in its own right, the parameterization has the additional benefit of enabling us to eliminate one of the other free parameters of the algorithm: the accuracy parameter in the regression case, and the regularization constant C in the classification case. We describe the algorithms, give some theoretical results concerning the meaning and the choice of , and report experimental results.

The goal of this paper is not to introduce a single algorithm or method, but to make theoretical steps towards fully understanding the training dynamics of generative adversarial networks. In order to substantiate our theoretical analysis, we perform targeted experiments to verify our assumptions, illustrate our claims, and quantify the phenomena. This paper is divided into three sections. The first section introduces the problem at hand. The second section is dedicated to studying and proving rigorously the problems including instability and saturation that arize when training generative adversarial networks. The third section examines a practical and theoretically grounded direction towards solving these problems, while introducing new tools to study them.

We study a distributed computation model for optimizing a sum of convex objective functions corresponding to multiple agents. For solving this (not necessarily smooth) optimization problem, we consider a subgradient method that is distributed among the agents. The method involves every agent minimizing his/her own objective function while exchanging information locally with other agents in the network over a time-varying topology. We provide convergence results and convergence rate estimates for the subgradient method. Our convergence rate results explicitly characterize the tradeoff between a desired accuracy of the generated approximate optimal solutions and the number of iterations needed to achieve the accuracy.

We study the resilience to Byzantine failures of distributed implementations of Stochastic Gradient Descent (SGD). So far, distributed machine learning frameworks have largely ignored the possibility of failures, especially arbitrary (i.e., Byzantine) ones. Causes of failures include software bugs, network asynchrony, biases in local datasets, as well as attackers trying to compromise the entire system. Assuming a set of n workers, up to f being Byzantine, we ask how resilient can SGD be, without limiting the dimension, nor the size of the parameter space. We first show that no gradient aggregation rule based on a linear combination of the vectors proposed by the workers (i.e, current approaches) tolerates a single Byzantine failure. We then formulate a resilience property of the aggregation rule capturing the basic requirements to guarantee convergence despite f Byzantine workers. We propose Krum, an aggregation rule that satisfies our resilience property, which we argue is the first provably Byzantine-resilient algorithm for distributed SGD. We also report on experimental evaluations of Krum.

We study two communication-efficient algorithms for distributed statistical optimization on large-scale data. The first algorithm is an averaging method that distributes the N data samples evenly to m machines, performs separate minimization on each subset, and then averages the estimates. We provide a sharp analysis of this average mixture algorithm, showing that under a reasonable set of conditions, the combined parameter achieves mean-squared error that decays as O(N-1 + (N/m)-2). Whenever m ≤ √N, this guarantee matches the best possible rate achievable by a centralized algorithm with access to all N samples. The second algorithm is a novel method, based on an appropriate form of bootstrap. Requiring only a single round of communication, it has mean-squared error that decays as O(N-1 + (N/m)-3), and so is more robust to the amount of parallelization.

This paper introduces a novel parameter automation strategy for the particle swarm algorithm and two further extensions to improve its performance after a predefined number of generations. Initially, to efficiently control the local search and convergence to the global optimum solution, time-varying acceleration coefficients (TVAC) are introduced in addition to the time-varying inertia weight factor in particle swarm optimization (PSO). From the basis of TVAC, two new strategies are discussed to improve the performance of the PSO. First, the concept of "mutation" is introduced to the particle swarm optimization along with TVAC (MPSO-TVAC), by adding a small perturbation to a randomly selected modulus of the velocity vector of a random particle by predefined probability. Second, we introduce a novel particle swarm concept "self-organizing hierarchical particle swarm optimizer with TVAC (HPSO-TVAC)". Under this method, only the "social" part and the "cognitive" part of the particle swarm strategy are considered to estimate the new velocity of each particle and particles are reinitialized whenever they are stagnated in the search space. In addition, to overcome the difficulties of selecting an appropriate mutation step size for different problems, a time-varying mutation step size was introduced. Further, for most of the benchmarks, mutation probability is found to be insensitive to the performance of MPSO-TVAC method. On the other hand, the effect of reinitialization velocity on the performance of HPSO-TVAC method is also observed. Time-varying reinitialization step size is found to be an efficient parameter optimization strategy for HPSO-TVAC method. The HPSO-TVAC strategy outperformed all the methods considered in this investigation for most of the functions. Furthermore, it has also been observed that both the MPSO and HPSO strategies perform poorly when the acceleration coefficients are fixed at two.


Question: The connections between attachment style, personality traits, interpersonal competency, and the use of Facebook.

This study explores the correlations between personality traits and the use of Facebook, a leading social networking platform. Unlike other online social networks, Facebook exhibits a unique offline-to-online transition, where most friendships established on the platform originate from offline encounters. The research focuses on the Five-Factor Model of personality, examining how its components—Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism—interact with Facebook usage patterns. Contrary to initial hypotheses, the study reveals that personality traits do not exert a significant influence on Facebook use as previously theorized. Instead, a strong motivation to communicate emerges as a key factor driving engagement with the platform. The findings suggest that diverse motivations may underpin the choice to use Facebook and other social networking tools, with particular functions of Facebook playing a role in this decision-making process. It is posited that the interplay between personality, motivations, and social network functionalities merits further investigation to fully understand the complexities of online social behavior.

Online Social Networks are so popular nowadays that they are a major component of an individual’s social interaction. They are also emotionally-rich environments where close friends share their emotions, feelings and thoughts. In this paper, a new framework is proposed for characterizing emotional interactions in social networks, and then using these characteristics to distinguish friends from acquaintances. The goal is to extract the emotional content of texts in online social networks. The interest is in whether the text is an expression of the writer’s emotions or not. For this purpose, text mining techniques are performed on comments retrieved from a social network. The framework includes a model for data collection, database schemas, data processing and data mining steps. The informal language of online social networks is a main point to consider before performing any text mining techniques. This is why the framework includes the development of special lexicons. In general, the paper presents a new perspective for studying friendship relations and emotions’ expression in online social networks where it deals with the nature of these sites and the nature of the language used. It considers Lebanese Face book users as a case study. The technique adopted is unsupervised, it mainly uses the k-means clustering algorithm. Experiments show high accuracy for the model in both determining subjectivity of texts and predicting friendship.

A national survey asked 323 U.S. adults about paralinguistic digital affordances (PDAs) and how these forms of lightweight feedback within social media were associated with their perceived social support. People perceived PDAs (e.g., Likes, Favorites, and Upvotes) as socially supportive both quantitatively and qualitatively, even without implicit meaning associated with them. People who are highly sensitive about what others think of them and have high self-esteem are more likely to perceive higher social support from PDAs.

Electronic communication is emotionally gratifying, but how do such technological distractions impact academic learning? The current study observed 263 middle school, high school and university students studying for 15 min in their homes. Observers noted technologies present and computer windows open in the learning environment prior to studying plus a minute-by-minute assessment of on-task behavior, off-task technology use and open computer windows during studying. A questionnaire assessed study strategies, task-switching preference, technology attitudes, media usage, monthly texting and phone calling, social networking use and grade point average (GPA). Participants averaged less than six minutes on task prior to switching most often due to technological distractions including social media, texting and preference for task-switching. Having a positive attitude toward technology did not affect being on-task during studying. However, those who preferred to task-switch had more distracting technologies available and were more likely to be off-task than others. Also, those who accessed Facebook had lower GPAs than those who avoided it. Finally, students with relatively high use of study strategies were more likely to stay on-task than other students. The educational implications include allowing students short ‘‘technology breaks’’ to reduce distractions and teaching students metacognitive strategies regarding when interruptions negatively impact learning. 2012 Elsevier Ltd. All rights reserved.

Personality disorders are presumed to be stable because of underlying stable and maladaptive personality traits, but while previous research has demonstrated a link between personality traits and personality disorders cross-sectionally, personality disorders and personality traits have not been linked longitudinally. This study explores the extent to which relevant personality traits are stable in individuals diagnosed with 4 personality disorders (schizotypal, borderline, avoidant, and obsessive-compulsive personality disorders) and examines the assumption that these personality disorders are stable by virtue of stable personality traits. This assumption was tested via the estimation of a series of latent longitudinal models that evaluated whether changes in relevant personality traits lead to subsequent changes in personality disorders. In addition to offering large consistency estimates for personality traits and personality disorders, the results demonstrate significant cross-lagged relationships between trait change and later disorder change for 3 of the 4 personality disorders studied.

Recent studies have found associations between narcissistic personality features and retrospective accounts of early experiences. The current study sought to extend these previous findings by examining whether adaptive and maladaptive features of narcissism were associated with recollections of early life experiences in a non-clinical sample of undergraduate students (N = 334). Results revealed that the Entitlement/Exploitativeness feature of narcissism was associated with low security, high parental discipline, and high threats of separation. Narcissistic Grandiosity was positively associated with peer affectional support and parental discipline, whereas Narcissistic Vulnerability was not uniquely associated with memories of early life experiences. The results provide partial support for models of narcissism in which parents are recalled as failing to provide a secure base while inducing threats of separation and discipline. 2011 Elsevier Ltd. All rights reserved.

Advancement of technology and the Internet proliferation have visible effects in the world. One of the most important effects is the increased social media usage among the Internet users. For this purpose, factors having impacts on users’ behavior intention to social media usage are investigated within the scope of the study. A research model based on technology acceptance model 2 is proposed and revised by the taking social media platforms into consideration. The effects of perceived ease of use, perceived usefulness, social influence, facilitating conditions, playfulness, and trust are measured. Data collected from 462 respondents are analyzed by structural equation modeling technique. According to results of the measurement and structural model validities, a fit and acceptable model is achieved to measure the related effects of variables. The results of study reveal the both direct and indirect positive impacts of the factors on users’ behavior intention to use social media.

Facebook is a popular social networking site. It, like many other new technologies, has potential for teaching and learning because of its unique built-in functions that offer pedagogical, social and technological affordances. In this study, the Facebook group was used as a learning management system (LMS) in two courses for putting up announcements, sharing resources, organizing weekly tutorials and conducting online discussions at a teacher education institute in Singapore. This study explores using the Facebook group as an LMS and the students’ perceptions of using it in their courses. Results showed that students were basically satisfied with the affordances of Facebook as the fundamental functions of an LMS could be easily implemented in the Facebook group. However, using the Facebook group as an LMS has certain limitations. It did not support other format files to be uploaded directly, and the discussion was not organized in a threaded structure. Also, the students did not feel safe and comfortable as their privacy might be revealed. Constraints of using the Facebook group as an LMS, implications for practice and limitations of this study are discussed. Practitioner notes What is already known about this topic • Facebook has been popularly used by tertiary students, but many students do not want their teachers to be friends on Facebook • Teacher’s self-disclosure on Facebook can promote classroom atmosphere, teacher’s credibility and student–teacher relationship • Commercial learning management systems (LMSs) have limitations What this paper adds • The Facebook group can be used an LMS as it has certain pedagogical, social and technological affordances • Students are satisfied with the way of using the Facebook group as an LMS British Journal of Educational Technology (2011) doi:10.1111/j.1467-8535.2011.01195.x © 2011 The Authors. British Journal of Educational Technology © 2011 Bera. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. • Younger students are more acceptable with the idea of using the Facebook group as an LMS • Using the Facebook group as an LMS has limitations: it does not support other format files; its discussions are not listed in threads; and it is not perceived as a safe environment Implications for practice and/or policy • The Facebook group can be used an LMS substitute or supplement • Third-party applications are needed to extend the capability of the Facebook group as an LMS • Using Facebook seems to be more appropriate for young learners than adults • Teachers do not have to be students’ friends on Facebook. Introduction Social networking sites (SNSs) are virtual spaces where people of similar interest gather to communicate, share photos and discuss ideas with one another (Boyd & Ellison, 2008; Raacke & Bonds-Raacke, 2008). In recent years, Facebook has become one of the most prominent SNSs. Like any new technology, Facebook seems to offer great potentials for teaching and learning as many students are using Facebook daily. One possible way of using Facebook for teaching and learning is to use its group as an LMS. Research shows that using LMSs possesses numerous benefits for teaching and learning. It enables faculty to shift the focus from content-based learning to process-based learning (Vogel & Klassen, 2001) and helps to “facilitate change from passive to active learning” (Herse & Lee, 2005, p. 51). Using LMSs also has the potential to increase student enrollment (Nunes & McPherson, 2003) and to promote interaction between students and faculty members (Lonn & Teasley, 2009; West, Waddoups & Graham, 2007). Using existing commercial LMSs like Blackboard, however, often has practical constraints (Sanchez-Franco, 2010). For example, LMSs tend to be expensive and that not every school can afford to purchase and maintain them over the long run. Trainee teachers cannot access certain features such as creating a course, enrolling students and setting up student groups as these functions are usually open to instructors or administrators only. The resources in the present LMS are often no longer accessible to trainee teachers after their graduation. Also, the LMS used at school in the future may be different from the one being currently used. They have to shift to a new LMS, and research shows learning a new system is often a painful experience (Black, Beck, Dawson, Jinks & DiPietro, 2007). If the Facebook group can be used as an alternative LMS, it would help to overcome some of the abovementioned constraints. For instance, it would enable a teacher to easily create a new course and enroll students in person if the class size is small. As presented in the following literature review section, many research studies have investigated the usage of Facebook, the effect of teacher’s self-disclosure via Facebook on teacher–student relationship improvement and the academic performance of Facebook users. However, few studies have examined if and how Facebook can be effectively used as an LMS. In this exploratory study, the Facebook group was used as an LMS to put up announcements, share resources, organize weekly tutorial sessions and conduct online discussions. The purpose of this paper is to describe how the Facebook group was used as an LMS in the study and to report students’ perceptions on it. Literature review on Facebook

This article explores the relationship between Internet use and the individual-level production of social capital. To do so, the authors adopt a motivational perspective to distinguish among types of Internet use when examining the factors predicting civic engagement, interpersonal trust, and life contentment. The predictive power of new media use is then analyzed relative to key demographic, contextual, and traditional media use variables using the 1999 DDB Life Style Study. Although the size of associations is generally small, the data suggest that informational uses of the Internet are positively related to individual differences in the production of social capital, whereas social-recreational uses are negatively related to these civic indicators. Analyses within subsamples defined by generational age breaks further suggest that social capital production is related to Internet use among Generation X, while it is tied to television use among Baby Boomers and newspaper use among members of the Civic Generation. The possibility of life cycle and cohort effects is discussed.

Online social networks have become extremely popular; numerous sites allow users to interact and share content using social links. Users of these networks often establish hundreds to even thousands of social links with other users. Recently, researchers have suggested examining the activity network - a network that is based on the actual interaction between users, rather than mere friendship - to distinguish between strong and weak links. While initial studies have led to insights on how an activity network is structurally different from the social network itself, a natural and important aspect of the activity network has been disregarded: the fact that over time social links can grow stronger or weaker. In this paper, we study the evolution of activity between users in the Facebook social network to capture this notion. We find that links in the activity network tend to come and go rapidly over time, and the strength of ties exhibits a general decreasing trend of activity as the social network link ages. For example, only 30% of Facebook user pairs interact consistently from one month to the next. Interestingly, we also find that even though the links of the activity network change rapidly over time, many graph-theoretic properties of the activity network remain unchanged.

We report the results of a field experiment where we sent to over 1200 university students an email or a Facebook message with a link to (non-existing) party pictures from a non-existing person, and later asked them about the reasons for their link clicking behavior. We registered a significant difference in clicking rates: 20% of email versus 42.5% of Facebook recipients clicked. The most frequently reported reason for clicking was curiosity (34%), followed by the explanations that the message fit recipient’s expectations (27%). Moreover, 16% thought that they might know the sender. These results show that people’s decisional heuristics are relatively easy to misuse in a targeted attack, making defense especially challenging.


Question: The connections between attachment style, personality characteristics, interpersonal skills, and the utilization of Facebook.

The Nexus Between Personality, Significant Outcomes, and Social Dynamics
Personality traits hold significant sway over life's outcomes, shaping various aspects of human experience. When considering the Big Five personality factors as a framework, a plethora of consequential associations emerge. At the individual level, personality dispositions are linked to subjective well-being, health status—both physical and mental—and the development of personal identity and spiritual beliefs. On an interpersonal scale, personality traits influence the quality of social connections, encompassing friendships, family bonds, and romantic relationships. At the broader social institutional level, personality plays a pivotal role in career decisions, job satisfaction, and performance, as well as engagement in community activities, legal behavior, and political leanings. This comprehensive exploration underscores the profound impact of personality on a wide spectrum of life's facets, from personal happiness to societal involvement.

In this study, the authors used meta-analytical techniques to examine the relationship between personality and entrepreneurial status. Personality variables used in previous studies were categorized according to the five-factor model of personality. Results indicate significant differences between entrepreneurs and managers on 4 personality dimensions such that entrepreneurs scored higher on Conscientiousness and Openness to Experience and lower on Neuroticism and Agreeableness. No difference was found for Extraversion. Effect sizes for each personality dimension were small, although the multivariate relationship for the full set of personality variables was moderate (R = .37). Considerable heterogeneity existed for all of the personality variables except Agreeableness, suggesting that future research should explore possible moderators of the personality-entrepreneurial status relationship.

Despite the fact that longitudinal data have been compiled over the past 30 years among undergraduate students in higher education settings regarding narcissism, the literature is devoid of empirical investigations that explore the relationships between narcissism and learning. Because the data suggest that narcissism scores are increasing each year among this population, an exploration of the relationship between narcissism and learning is timely and warranted. Sampling from university undergraduate students, this study uses the Narcissistic Personality Inventory, the Big Five Inventory, and the Achievement Goal Questionnaire to verify the known relationships between narcissism and the Big Five personality traits of extraversion and agreeableness; to verify the known relationships between the Big Five personality traits of extraversion and agreeableness and goal orientation; and to explore a previously undocumented empirical relationship between narcissism and performance goal orientation. Results of this exploratory study indicate that while narcissism does contribute to a performance goal orientation, it is not a substantial variable in determining achievement goal orientation in general. The study addresses the implications and limitations of this research in addition to areas for additional investigation.

The rapid growth of contemporary social network sites (SNSs) has coincided with an increasing concern over personal privacy. College students and adolescents routinely provide personal information on profiles that can be viewed by large numbers of unknown people and potentially used in harmful ways. SNSs like Facebook and MySpace allow users to control the privacy level of their profile, thus limiting access to this information. In this paper, we take the preference for privacy itself as our unit of analysis, and analyze the factors that are predictive of a student having a private versus public profile. Drawing upon a new social network dataset based on Facebook, we argue that privacy behavior is an upshot of both social influences and personal incentives. Students are more likely to have a private profile if their friends and roommates have them; women are more likely to have private profiles than are men; and having a private profile is associated with a higher level of online activity. Finally, students who have private versus public profiles are characterized by a unique set of cultural preferences—of which the ‘‘taste for privacy’’ may be only a small but integral part.

Relationship researchers have focused on the frequency of conflict in couples' relationships and the manner in which couples engage in and try to resolve conflicts. conflict, under some conditions, may facilitate the development and maintenance of associated with patterns of behavior (e.g., negative affect reciprocity, demand-withdraw) and thought that tend to escalate conflict and make it more difficult to negotiate a facilitates intimacy or exacerbates distress may depend on individual differences in the way in which people interpret and respond to conflict. framework for understanding different responses to conflict. People are thought to differ in their working models of attachment, which include expectations, beliefs, and goals These working models are likely to shape people's thoughts, feelings, and behavior during conflict. For example, a person who expects close others to be generally responsive and available is likely to interpret and respond to conflict very differently from a person who expects close others to be rejecting and unavailable. Attachment theory may be able to inform the literature on 3 conflict in close relationships by suggesting how individuals might differ in how they construe conflict. At the same time, the study of relationship conflict provides a useful context for testing important aspects of attachment theory. Conflict may be particularly likely to reveal attachment processes because (a) it may act as a stressor on the relationship and thereby activate the attachment system (Simpson, Rholes, & Phillips, 1996), (b) it challenges partners' abilities to regulate their emotions and behavior (Kobak & Duemmler, 1994), which are thought to be connected to attachment processes and (c) it may trigger behaviors (e.g., personal disclosures) that typically promote intimacy, thereby providing evidence relevant to different attachment goals such as achieving intimacy or maintaining self-reliance (Pietromonaco & Feldman Barrett, 1997). In this chapter, we first discuss how conflict can be conceptualized within an attachment framework. Specifically, we propose that conflict may pose a threat to the attachment bond, but that it also may provide an opportunity for perceiving or experiencing greater intimacy. Furthermore, the degree to which people perceive conflict as a threat, opportunity, or both will depend on the content (e.g., expectations, beliefs, goals) of their working models of attachment. Next, we identify a set of predictions that follow from this framework, and evaluate extent to which empirical findings support these predictions; in particular, we attempt to integrate divergent findings in the empirical literature. Finally, we outline several critical …

This paper describes a pilot study in using Facebook as an alternative to a learning management system (LMS). The paper reviews the current research on the use of Facebook in academia and analyzes the differences between a Facebook group and a regular LMS. The paper reports on a precedent-setting attempt to use a Facebook group as a course Web site, serving as a platform for delivering content and maintaining interactions among the students and between the students and the instructor. The paper presents findings from the students’ self-assessments and reflections on their experience. The students expressed satisfaction with learning in Facebook and willingness to continue using these groups in future courses.

Seven experts on personality measurement here discuss the viability of public-domain personality measures, focusing on the International Personality Item Pool (IPIP) as a prototype. Since its inception in 1996, the use of items and scales from the IPIP has increased dramatically. Items from the IPIP have been translated from English into more than 25 other languages. Currently over 80 publications using IPIP scales are listed at the IPIP Web site (http://ipip.ori.org), and the rate of IPIPrelated publications has been increasing rapidly. The growing popularity of the IPIP can be attributed to Wve factors: (1) It is cost free; (2) its items can be obtained instantaneously via the Internet; (3) it includes over 2000 items, all easily available for inspection; (4) scoring keys for IPIP scales are This article represents a synthesis of contributions to the presidential symposium, The International Personality Item Pool and the Future of Public-Domain Personality Measures (L.R. Goldberg, Chair) at the sixth annual meeting of the Association for Research in Personality, New Orleans, January 20, 2005. Authorship order is based on the order of participation in the symposium. The IPIP project has been continually supported by Grant MH049227 from the National Institute of Mental Health, U.S. Public Health Service. J.A. Johnson’s research was supported by the DuBois Educational Foundation. The authors thank Paul T. Costa Jr., Samuel D. Gosling, Leonard G. Rorer, Richard Reed, and Krista Trobst for their helpful suggestions. * Corresponding author. Fax: +1 814 375 4784. E-mail address: j5j@psu.edu (J.A. Johnson). 0092-6566/$ see front matter  2005 Elsevier Inc. All rights reserved. doi:10.1016/j.jrp.2005.08.007 L.R. Goldberg et al. / Journal of Research in Personality 40 (2006) 84–96 85 provided; and (5) its items can be presented in any order, interspersed with other items, reworded, translated into other languages, and administered on the World Wide Web without asking permission of anyone. The unrestricted availability of the IPIP raises concerns about possible misuse by unqualiWed persons, and the freedom of researchers to use the IPIP in idiosyncratic ways raises the possibility of fragmentation rather than scientiWc uniWcation in personality research.  2005 Elsevier Inc. All rights reserved.

There is a general consensus that Internet addiction (IA) is mainly related to social aspects of the Web, especially among adolescents. The empirical link between poor social skills and IA is well documented; however, theoretical explanations for this relationship are less developed. One possibility is that people with poor social skills are especially prone to develop a preference for online social interaction (POSI), which, in turn, predicts problematic usage. This hypothesis has been tested for loneliness and social anxiety, but not for self-esteem (SE; one of the main antecedents of IA). Furthermore, the mediating role of the perceived relevance of some Internet features (e.g., anonymity) in the relationship between SE and POSI has never been investigated. A cross-sectional study was conducted with 257 adolescents. Using mediation analyses, we found evidence among females for the mediating role of (a) POSI in the relationship between SE and IA, and (b) the subjective relevance of some Internet features in the association between SE and POSI. No significant effects were found for males.

Recommender systems have emergedrecently in many aspects due to enormous amount of information or items existed in e-commerce or social network services. Users find it convenient if the service can recommend interesting items to them automatically. Past systems adopted users' preference or social connections for such recommendation. In this work, we will try to identify a game player's personality traits and use them to recommend games that may be interested by the player. The personality traits of a player is identified by applying a text mining process on textual contents related to the player. The same process is also applied on the comments of the games to identified the personality traits of games. Recommendation is then performed based on the similarity between the traits of the user and the game. We performed experiments on 63 players and 2050 games and obtained promising results which may demonstrate the plausibility of personality-based recommender systems.

The precise nature and etiopathogenesis of borderline personality disorder (BPD) continues to elude researchers and clinicians. Yet, increasing evidence from various strands of research converges to suggest that affect dysregulation, impulsivity, and unstable relationships constitute the core features of BPD. Over the last two decades, the mentalization-based approach to BPD has attempted to provide a theoretically consistent way of conceptualizing the interrelationship between these core features of BPD, with the aim of providing clinicians with a conceptually sound and empirically supported approach to BPD and its treatment. This paper presents an extended version of this approach to BPD based on recently accumulated data. In particular, we suggest that the core features of BPD reflect impairments in different facets of mentalization, each related to impairments in relatively distinct neural circuits underlying these facets. Hence, we provide a comprehensive account of BPD by showing how its core features are related to each other in theoretically meaningful ways. More specifically, we argue that BPD is primarily associated with a low threshold for the activation of the attachment system and deactivation of controlled mentalization, linked to impairments in the ability to differentiate mental states of self and other, which lead to hypersensitivity and increased susceptibility to contagion by other people's mental states, and poor integration of cognitive and affective aspects of mentalization. The combination of these impairments may explain BPD patients' propensity for vicious interpersonal cycles, and their high levels of affect dysregulation and impulsivity. Finally, the implications of this expanded mentalization-based approach to BPD for mentalization-based treatment and treatment of BPD more generally are discussed.

Online social networks have become extremely popular; numerous sites allow users to interact and share content using social links. Users of these networks often establish hundreds to even thousands of social links with other users. Recently, researchers have suggested examining the activity network - a network that is based on the actual interaction between users, rather than mere friendship - to distinguish between strong and weak links. While initial studies have led to insights on how an activity network is structurally different from the social network itself, a natural and important aspect of the activity network has been disregarded: the fact that over time social links can grow stronger or weaker. In this paper, we study the evolution of activity between users in the Facebook social network to capture this notion. We find that links in the activity network tend to come and go rapidly over time, and the strength of ties exhibits a general decreasing trend of activity as the social network link ages. For example, only 30% of Facebook user pairs interact consistently from one month to the next. Interestingly, we also find that even though the links of the activity network change rapidly over time, many graph-theoretic properties of the activity network remain unchanged.


Question: The connections between attachment style, personality traits, interpersonal competency, and the utilization of Facebook.

The Digital Dilemma: Does the Internet Diminish Social Engagement and Mental Health?
As the Internet has become an integral part of daily life, akin to the transformative impact of the telephone in the early 20th century and television in the mid-20th century, there is a growing debate over its effects on community engagement and interpersonal relationships. This study delves into the social and psychological consequences of Internet usage by tracking 169 individuals across 73 households during their initial 1 to 2 years online. Our analysis, which utilized longitudinal data, aimed to discern the influence of the Internet on social engagement and mental health. Despite the extensive use of the Internet for communication purposes, our findings revealed a paradoxical outcome: increased Internet usage was correlated with reduced face-to-face communication within households, a decrease in the size of social networks, and a rise in feelings of depression and loneliness. These results have significant implications for future research, public policy formulation, and the development of digital technologies, suggesting a need for a more nuanced understanding of how the Internet shapes our social and psychological well-being.

As the use and influence of social networking continues to grow, researchers have begun to explore its consequences for psychological well-being. Some research suggests that Facebook use can have negative consequences for well-being. Instagram, a photo-sharing social network created in 2010, has particular characteristics that may make users susceptible to negative consequences. This study tested a theoretically grounded moderated meditation model of the association between Instagram use and depressive symptoms through the mechanism of negative social comparison, and moderation by amount of strangers one follows. One hundred and seventeen 18-29 year olds completed online questionnaires containing demographics, frequency of Instagram use, amount of strangers followed on Instagram, the Center for Epidemiological Resources Scale for Depression, and the Social Comparison Rating Scale. Instagram use was marginally positively associated with depressive symptoms, and positive social comparison was significantly negatively associated with depressive symptoms. Amount of strangers followed moderated the associations of Instagram use with social comparison (significantly) and depressive symptoms (marginally), and further significantly moderated the indirect association of Instagram use with depressive symptoms through social comparison. Findings generally suggest that more frequent Instagram use has negative associations for people who follow more strangers, but positive associations for people who follow fewer strangers, with social comparison and depressive symptoms. Implications of negative associations of social networking for people who follow strangers and the need for more research on Instagram use given its increasing popularity are explored.

The authors explored the relations among self-esteem, perceived regard, and satisfaction in dating relationships. On the basis of the dependency regulation model (T. DeHart, B. Pelham, & S. Murray, 2004), the authors hypothesized that self-esteem would influence individuals' self-perceptions and views of how their partners perceive them (metaperception). They also hypothesized that perceived regard (self-perception minus metaperception) would predict relationship satisfaction. Regression analyses indicated that for moderate relationship-relevant traits (i.e., caring, loving), high self-esteem was associated with self-enhancement (idealization), whereas low self-esteem was associated with self-deprecation. For low relationship-relevant traits (i.e., quiet, reserved), both low and high self-esteem individuals self-enhanced. Hierarchical regression analyses indicated that self-esteem and perceived regard for moderate relationship-relevant traits predicted satisfaction. The authors discuss the implications of idealization, self-verification, and self-deprecation for the perceivers and their relationships.

Of the offensive yet non-pathological personalities in the literature, three are especially prominent: Machiavellianism, subclinical narcissism, and subclinical psychopathy. We evaluated the recent contention that, in normal samples, this Dark Triad of constructs are one and the same. In a sample of 245 students, we measured the three constructs with standard measures and examined a variety of laboratory and self-report correlates. The measures were moderately inter-correlated, but certainly were not equivalent. Their only common Big Five correlate was disagreeableness. Subclinical psychopaths were distinguished by low neuroticism; Machiavellians, and psychopaths were low in conscientiousness; narcissism showed small positive associations with cognitive ability. Narcissists and, to a lesser extent, psychopaths exhibited self-enhancement on two objectively scored indexes. We conclude that the Dark Triad of personalities, as currently measured, are overlapping but distinct constructs. 2002 Elsevier Science (USA). All rights reserved.

Readers of fiction tend to have better abilities of empathy and theory of mind (Mar et al., 2006). We present a study designed to replicate this finding, rule out one possible explanation, and extend the assessment of social outcomes. In order to rule out the role of personality, we first identified Openness as the most consistent correlate. This trait was then statistically controlled for, along with two other important individual differences: the tendency to be drawn into stories and gender. Even after accounting for these variables, fiction exposure still predicted performance on an empathy task. Extending these results, we also found that exposure to fiction was positively correlated with social support. Exposure to nonfiction, in contrast, was associated with loneliness, and negatively related to social support.

Attachment theory provides a model for understanding how the attachment styles formed in infancy systematically affect subsequent psychological functioning across the life span. Attachment styles provide the cognitive schemas, or working models, through which individuals perceive and relate to their worlds. In turn, these schemas predispose the development of psychopathologies and influence outcomes when people undergo psychotherapy. After reviewing recent empirical findings, the authors conclude that an understanding of attachment theory facilitates the conceptualization of clients’ problems and the selection of appropriate interventions. Accordingly, attachment styles should be assessed as a standard part of treatment planning. Furthermore, the authors propose that attachment styles should be assessed as individual difference variables in psychotherapy outcome research because adult attachment styles dictate how people perceive and respond to their environments and, therefore, how clients respond differentially to various treatments.

Recent years have seen unprecedented growth in the popularity of social network systems, with Facebook being an archetypical example. The access control paradigm behind the privacy preservation mechanism of Facebook is distinctly different from such existing access control paradigms as Discretionary Access Control, Role-Based Access Control, Capability Systems, and Trust Management Systems. This work takes a first step in deepening the understanding of this access control paradigm, by proposing an access control model that formalizes and generalizes the privacy preservation mechanism of Facebook. The model can be instantiated into a family of Facebook-style social network systems, each with a recognizably different access control mechanism, so that Facebook is but one instantiation of the model. We also demonstrate that the model can be instantiated to express policies that are not currently supported by Facebook but possess rich and natural social significance. This work thus delineates the design space of privacy preservation mechanisms for Facebook-style social network systems, and lays out a formal framework for policy analysis in these systems.

Introduction: Online social networking sites are being used all around the world. However, only recently researchers have started to investigate their relationship with mental health. Evidence coming from literature suggests that they have both advantages and disadvantages for individuals. The aim of this study is to critically review the existent research conducted on the relationship between online social networking and social anxiety. Method: According to PRISMA guidelines, comprehensive systematic searches of electronic databases were conducted (PsychInfo, Cochrane, PubMed, Scopus, Web of Science). Terms related to online social networking were combined with social anxiety terms. After identifying relevant papers, the relationship between social media networking and social anxiety as well as limitations of each study were presented. All of the papers included were cross-sectional, relying on self-report assessments. Conclusion: There are several papers published on the relationship between social media and social anxiety; however, more research needs to be done in order to clarify the influence of one variable over the other. Rigorous experimental studies need to be conducted with different time-point assessments, and bidirectionality should be investigated. Important mediators and moderators have to be considered in order to explain the relationship between social media networking and social anxiety.

We conducted a comprehensive analysis of assortative mating (i.e., the similarity between wives and husbands on a given characteristic) in a newlywed sample. These newlyweds showed (a) strong similarity in age, religiousness, and political orientation; (b) moderate similarity in education and verbal intelligence; (c) modest similarity in values; and (d) little similarity in matrix reasoning, self- and spouse-rated personality, emotional experience and expression, and attachment. Further analyses established that similarity was not simply due to background variables such as age and education and reflected initial assortment (i.e., similarity at the time of marriage) rather than convergence (i.e., increasing similarity with time). Finally, marital satisfaction primarily was a function of the rater's own traits and showed little relation to spousal similarity.

Online Social Networks are so popular nowadays that they are a major component of an individual’s social interaction. They are also emotionally-rich environments where close friends share their emotions, feelings and thoughts. In this paper, a new framework is proposed for characterizing emotional interactions in social networks, and then using these characteristics to distinguish friends from acquaintances. The goal is to extract the emotional content of texts in online social networks. The interest is in whether the text is an expression of the writer’s emotions or not. For this purpose, text mining techniques are performed on comments retrieved from a social network. The framework includes a model for data collection, database schemas, data processing and data mining steps. The informal language of online social networks is a main point to consider before performing any text mining techniques. This is why the framework includes the development of special lexicons. In general, the paper presents a new perspective for studying friendship relations and emotions’ expression in online social networks where it deals with the nature of these sites and the nature of the language used. It considers Lebanese Face book users as a case study. The technique adopted is unsupervised, it mainly uses the k-means clustering algorithm. Experiments show high accuracy for the model in both determining subjectivity of texts and predicting friendship.

We show how online social networks such as Facebook can be used in Behavioral Game Theory research. We report the deployment of a Facebook application ‘Project Waterloo’ that allows users to play the Colonel Blotto game against their friends and strangers. Unlike conventional studies performed in the laboratory environment, which rely on monetary incentives to attract human subjects to play games, our framework does not use money and instead relies on reputation and entertainment incentives. We describe the Facebook application we created for conducting this experiment, and perform a preliminary analysis of the data collected in the game. We conclude by discussing the advantages of our approach and list some ideas for future work.