Question: Velocity regulation for DC motor drives powered by buck converters.

This document presents an in-depth comparison between Proportional Integral (PI) and Backstepping control strategies for a Buck converter, specifically when interfaced with a resistive (R) load and a DC motor. Backstepping control is highlighted as a robust and efficient method for addressing both regulation and tracking challenges. This control technique is grounded in a systematic approach that ensures global stability for both regulation and tracking objectives. The primary goal of the proposed control scheme is to maintain the output (be it voltage or motor speed) at a desired level and to ensure that the tracking error diminishes to zero over time. To validate the effectiveness of these control methods, the Buck converter system was meticulously simulated in MATLAB, employing state reconstruction techniques for enhanced accuracy. The simulation outcomes for the Buck converter operating with an R-load and a Permanent Magnet DC (PMDC) motor clearly indicate that the Backstepping controller exhibits a shorter settling time compared to the PI controller, thus demonstrating its superiority in terms of response time and stability.

A quasi-resonant (QR) controller with an adaptive frequency reduction scheme has been developed for a flyback converter. While maintaining the valley switching, the QR controller reduces the switching frequency for lighter load by skipping some valleys to reduce the power loss and thereby achieve better light-load efficiency. If the QR controller cannot detect any valley due to the damped oscillation of switch voltage, then the valley switching is given up and the nonvalley switching is employed to keep reducing the switching frequency for lighter load. The proposed QR controller has been implemented in a 0.35-μm 700-V BCDMOS process and applied to a 40-W flyback converter. The power efficiency of the flyback converter is improved by up to 3.0% when the proposed QR controller is employed compared to the one employing the conventional QR controller.

A new neutral-point-clamped pulsewidth modulation (PWM) inverter composed of main switching devices which operate as switches for PWM and auxiliary switching devices to clamp the output terminal potential to the neutral point potential has been developed. This inverter output contains less harmonic content as compared with that of a conventional type. Two inverters are compared analytically and experimentally. In addition, a new PWM technique suitable for an ac drive system is applied to this inverter. The neutral-point-clamped PWM inverter adopting the new PWM technique shows an excellent drive system efficiency, including motor efficiency, and is appropriate for a wide-range variable-speed drive system.

A sensorless direct rotor field-oriented control (SDRFOC) scheme of three-phase induction motors for low-cost applications is presented in this paper. The SDRFOC algorithm is based on a sensorless closed-loop rotor flux observer whose main advantages are simplicity and robustness to motor parameter detuning. The whole algorithm has been developed and implemented on a low-cost fixed-point digital signal processor controller. Experimental results are presented for a 0.5-kW induction motor drive for a primary vacuum pump used in industry applications.

Well known as an efficient and eco-friendly power source, fuel cell, unfortunately, offers slow dynamics. When attached as primary energy source in a vehicle, fuel cell would not be able to respond to abrupt load variations. Supplementing battery and/or supercapacitor to the system will provide a solution to this shortcoming. On the other hand, a current regulation that is vital for lengthening time span of the energy storage system is needed. This can be accomplished by keeping fuel cell's and batteries' current slope in reference to certain values, as well as attaining a stable dc output voltage. For that purpose, a feedback control system for regulating the hybrid of fuel cell, batteries, and supercapacitor was constructed for this study. Output voltage of the studied hybrid power sources (HPS) was administered by assembling three dc-dc converters comprising two bidirectional converters and one boost converter. Current/voltage output of fuel cell was regulated by boost converter, whereas the bidirectional converters regulated battery and supercapacitor. Reference current for each converter was produced using Model Predictive Control (MPC) and subsequently tracked using hysteresis control. These functions were done on a controller board of a dSPACE DS1104. Subsequently, on a test bench made up from 6 V, 4.5 Ah battery and 7.5 V, 120 F supercapacitor together with a fuel cell of 50 W, 10 A, experiment was conducted. Results show that constructing a control system to restrict fuel cell's and batteries' current slope and maintaining dc bus voltage in accordance with the reference values using MPC was feasible and effectively done.

The main aim of this paper is to improve the performance of high current dual active bridge converters when operated over a wide voltage range. A typical application is for fuel cell vehicles where a bi-directional interface between a 12V battery and a high voltage DC bus is required. The battery side voltage ranges from 11V to 16V while the fuel cell is operated between 220V and 447V and the required power is typically 1kW. Careful analysis shows that the high currents on the battery side cause significant design issues in order to obtain a high efficiency. The standard phase shift modulation method can result in high conduction and switching losses. This paper proposes a combined triangular and trapezoidal modulation method to reduce losses over the wide operating range. Approximately, a 2% improvement in efficiency can be expected. An experimental system is used to verify the improved performance of the dual active bridge using the proposed advanced modulation method.

Modular photovoltaic (PV) power conditioning systems (PCSs) require a high-efficiency dc-dc converter stage capable of regulation over a wide input voltage range for maximum power point tracking. In order to mitigate ground leakage currents and to be able to use a high-efficiency, nonisolated grid-tied inverter, it is also desirable for this microconverter to provide galvanic isolation between the PV module and the inverter. This paper presents a novel, isolated topology that is able to meet the high efficiency over a wide input voltage range requirement. This topology yields high efficiency through low circulating currents, zero-voltage switching (ZVS) and low-current switching of the primary side devices, ZCS of the output diodes, and direct power transfer to the load for the majority of switching cycle. This topology is also able to provide voltage regulation through basic fixed-frequency pulsewidth modulated (PWM) control. These features are able to be achieved with the simple addition of a secondary-side bidirectional ac switch to the isolated series resonant converter. Detailed analysis of the operation of this converter is discussed along with a detailed design procedure. Experimental results of a 300-W prototype are given. The prototype reached a peak power stage efficiency of 98.3% and a California Energy Commission (CEC) weighted power stage efficiency of 98.0% at the nominal input voltage.

Predictive control is a very wide class of controllers that have found rather recent application in the control of power converters. Research on this topic has been increased in the last years due to the possibilities of today's microprocessors used for the control. This paper presents the application of different predictive control methods to power electronics and drives. A simple classification of the most important types of predictive control is introduced, and each one of them is explained including some application examples. Predictive control presents several advantages that make it suitable for the control of power converters and drives. The different control schemes and applications presented in this paper illustrate the effectiveness and flexibility of predictive control.

This paper proposes the design and implementation of a high power full bridge bidirectional isolated DC-DC converter (BIDC) which comprises of two symmetrical voltage source converters and a high frequency transformer. In the proposed BIDC, a well-known PI controller based single phase-shift (SPS) modulation technique is used in order to achieve high power transfer. Besides, different phase shift methods such as extended phase-shift (EPS) and double phase-shift (DPS) are compared with SPS. Both simulation and experimental results are caried out to verify PI controller based simple phase-shift controlled BIDC prototype that is designed for 300-V 2.4-kW and operating at 20 kHz.

In this paper, the main concepts related to the development of a single-stage three-phase high power factor rectifier, with high-frequency isolation and regulated DC bus are described. The structure operation is presented, being based on the DC-DC SEPIC converter operating in the discontinuous conduction mode. This operational mode provides to the rectifier a high power factor feature, with sinusoidal input current, without the use of any current sensors and current loop control. A design example and simulation results are presented in order to validate the theoretical analysis.

This paper presents a compact battery-powered switched-reluctance motor (SRM) drive for an electric vehicle with voltage-boosting and on-board power-factor-corrected-charging capabilities. Although the boost-type front-end DC/DC converter is externally equipped, the on-board charger is formed by the embedded components of SRM windings and converter. In the driving mode, the DC/DC boost converter is controlled to establish boostable well-regulated DC-link voltage for the SRM drive from the battery. During demagnetization of each stroke, the winding stored energy is automatically recovered back to the battery. Moreover, the regenerative braking is achieved by properly setting the SRM in the regenerating mode. The controls of two power stages are properly designed and digitally realized using a common digital signal processor. Good winding current and speed dynamic responses of the SRM drive are obtained. In addition, the static operation characteristics are also improved. In the charging mode, the power devices that were embedded in the SRM converter and the three motor-phase windings are used to form a buck-boost switch-mode rectifier to charge the battery from utility with good power quality. All constituted circuit components of the charger are placed on board, and only the insertion of power cords is needed.


Question: Speed regulation for DC motor drives powered by buck converters.

This document details an innovative control strategy for a buck-boost converter-powered DC motor system, employing an Active Disturbance Rejection (ADR) technique. In response to the challenges posed by unpredictable, time-varying load torques and the non-measurability of the motor's angular velocity, a Generalized Proportional Integral (GPI) observer is integrated into the ADR controller, guided by passivity principles. This observer facilitates real-time estimation of both angular velocity and external disturbance torques, enabling their cancellation through the ADR control mechanism. Consequently, the control system operates sensorlessly, incorporating robustness into the conventional energy shaping and damping injection methods. The continuous feedback control law's discrete switching control implementation is achieved through a standard Pulse Width Modulation (PWM) modulation scheme. To ensure system stability, an Input to State Stability (ISS) property is established for the closed-loop system. The efficacy of the proposed control scheme is substantiated through a combination of experimental and simulation outcomes.

The efficiency of zero voltage switching half-bridge series resonant inverter can be decreased under certain load conditions due to the high switching frequencies required. The proposed variable frequency duty cycle (VFDC) control is intended to improve the efficiency in the medium and low output power levels because of the decreased switching frequencies. The study performed in this letter includes, in a first step, a theoretical analysis of power balance as a function of control parameters. In addition, restrictions due to snubber capacitors and deadtime, and variability of the loads have been considered. Afterward, an efficiency analysis has been carried out to determine the optimum operation point. Switching and conduction losses have been calculated to examine the overall efficiency improvement. VFDC strategy efficiency improvement is achieved by means of a switching-frequency reduction, mainly at low-medium power range, and with low-quality factor loads. Domestic induction heating application is suitable for the use of VFDC strategy due to its special load characteristics. For this reason, the simulation results have been validated using an induction heating inverter with a specially designed load.

Classical direct torque control (DTC) has advantage in absence of coordinate transform and voltage modulator block. However, it may have disadvantage in controlling electromagnetic torque. DTC produces high ripple in electromagnetic torque as it is not directly controlled. High torque ripple causes vibrations to the motor which may lead to component lose, bearing failure or resonance. Thus, Fuzzy Logic controller is applied to reduce electromagnetic torque ripple. This paper presents the simulation analysis of Fuzzy Logic Direct Torque Controller (FLDTC) of induction machine drives. However, only the design of the controller using built in DTC induction motor drive model of Simulink AC4 will be discussed in this paper. Using FLDTC, the resulting electromagnetic torque from produced less ripple than classical DTC.

A power processing system is required to convert electrical energy from one voltage, current, or frequency to another with, ideally, 100-percent efficiency, together with adjustability of the conversion ratio. The allowable elements are switches, capacitors, and magnetic devices. Essential to the design is a knowledge of the transfer functions, and since the switching converter is nonlinear, a suitable modeling approach is needed. This paper discusses the state-space averaging method, which is an almost ideal compromise between accuracy and simplicity. The treatment here emphasizes the motivation and objectives of modeling by equivalent circuits based upon physical interpretation of state-space averaging, and is limited to pulse-width modulated dc-to-dc converters.

We investigate the implementation of field-weakening control for interior permanent magnet synchronous motor (IPMSM), and analyze the algorithm of field-weakening control in d-q axes. To deal with the problem that the dc-link voltage is not fully utilized when the voltage regulation is employed, we propose a new field-weakening scheme based on the space-vector modulation technique. The duty-time of zero-voltage vector is used as the feedback signal to decide the switching of fieldweakening. To avoid the regulation lag in the q-axis component, we apply the lead-angle control during the field-weakening progress. The proposed scheme is validated with Matlab/Simulink tool. Simulation results show that the scheme is feasible. It not only improves the utilization ratio of the space-vector-pulse-width-modulation (SVPWM) inverter, but also achieves a smooth transition between the constant-torque mode below the base-speed and the constant-power mode above the basespeed.

The narrow duty cycle in the buck converter limits its application for high-step-down dc-dc conversion. With a simple structure, the tapped-inductor buck converter shows promise for extending the duty cycle. However, the leakage inductance causes a huge turn-off voltage spike across the top switch. Also, the gate drive for the top switch is not simple due to its floating source connection. This paper solves all these problems by modifying the tapped-inductor structure. A simple lossless clamp circuit can effectively clamp the switch turn-off voltage spike and totally recover the leakage energy. Experimental results for 12V-to-1.5V and 48V-to-6V dc-dc conversions show significant improvements in efficiency.

This paper describes a novel one-axis actively regulated bearingless motor with a spinning top-shaped rotor, which is intended for a use as a cooling fan. This motor requires only one set of three-phase winding and thus one three-phase inverter to realize both rotation and magnetic suspension, and thus it is dubbed a single-drive bearingless motor. An axial position of the rotor is actively regulated and radial and tilting motions are passively stabilized by the passive magnetic bearing. Torque and axial suspension force are regulated by q-axis and d-axis currents, respectively. In this paper, the structure and principle, design, and test results of the proposed bearingless motor are presented.

In this paper, a low-time-consuming and low-cost sensorless-control algorithm for high-dynamic performance permanent-magnet synchronous motors, both surface and internal permanent-magnet mounted for position and speed estimation, is introduced, discussed, and experimentally validated. This control algorithm is based on the estimation of rotor speed and angular position starting from the back electromotive force space-vector determination without voltage sensors by using the reference voltages given by the current controllers instead of the actual ones. This choice obviously introduces some errors that must be vanished by means of a compensating function. The novelties of the proposed estimation algorithm are the position-estimation equation and the process of compensation of the inverter phase lag that also suggests the final mathematical form of the estimation. The mathematical structure of the estimation guarantees a high degree of robustness against parameter variation as shown by the sensitivity analysis reported in this paper. Experimental verifications of the proposed sensorless-control system have been made with the aid of a flexible test bench for brushless motor electrical drives. The test results presented in this paper show the validity of the proposed low-cost sensorless-control algorithm and, above all, underline the high dynamic performances of the sensorless-control system also with a reduced equipment.

Here, A PID controller plus a novel linear-to-nonlinear translator is proposed and applied to the boost converter to improve the operating performance of the boost converter under large transient variations in load all over the quiescent dc input voltage range. The proposed translator is inserted between the PID controller and the main power stage such that the PID controller considers the behavior of the boost converter to be quasi-linear, thereby suppressing the effect of nonlinear behavior of the boost converter on the PID controller. Besides, variations in dc gain of the control-to-output transfer function with the translator are smaller than those without the translator, thus making system control easy to attain. Also, one-comparator counter-based sampling without any A-D converter is presented herein to realize the proposed control strategy. As mentioned above, the proposed control strategy can make the boost converter stably operated under large load transient responses all over the quiescent dc input voltage range.

Continuous conduction mode (CCM) boost converter is difficult to compensate due to the right half plane (RHP) zero. This paper presents a quasi fixed frequency constant on time (COT) controlled boost converter, it has no loop compensation requirement. With the proposed COT control, pulse frequency modulation (PFM) comes free at light load, which improves the light load efficiency compared with traditional PWM control. A compact on-time generator that controls the operation frequency is proposed. And a current sink based feedback circuit is used to improve the quality of feedback signal.The COT controller is designed with 1.5 mum BCD process. Simulation results demonstrate that the COT boost converter has fast load transient response and quasi fixed frequency operation.

Input-series and output-parallel (ISOP) converters are suitable for the high input voltage and low output voltage conversion applications. An ISOP current mode AC/AC converter with high frequency link is proposed. The control strategy and operation principle of the ISOP AC/AC converter are investigated. The control strategy ensures the proper sharing of the input voltage and the proper sharing of the output current among the ISOP AC/AC converter. By simulation, the correctness of the operation principle and the control strategy of the ISOP AC/AC converter are proved.


Question: Velocity regulation for DC motor drives powered by buck converters.

This paper presents a comprehensive closed-loop analysis and design of a cascade control strategy for a nonminimum phase boost converter, specifically tailored for voltage regulation in DC motor drives. The control architecture comprises an inner current control loop utilizing sliding-mode control for rapid response and an outer voltage control loop employing proportional-integral (PI) control for precise voltage regulation. The stability and performance of the system are analyzed based on nonlinear closed-loop error dynamics, leading to the optimal selection of PI controller gains. The analysis reveals that the closed-loop system exhibits nonminimum phase characteristics, which are crucial for understanding the system's transient behavior and its response to reference voltage changes. The impact of current ripple and system sensitivity to external disturbances is also thoroughly investigated. The efficacy of the proposed control strategy is validated through extensive simulations conducted under various conditions, including nonideal circuit parameters, diverse system parameters, and different maximum switching frequencies. Simulation outcomes confirm the controller's ability to maintain accurate voltage tracking under parametric variations, system uncertainties, and external disturbances, while demonstrating fast dynamic response, thereby substantiating the robustness and reliability of the proposed cascade control scheme for buck converter-fed DC motor drives.

This paper introduces a novel LLC resonant controller with better light load efficiency and best-in-class transient performance. A new control algorithm — hybrid hysteretic control, is proposed which combines the benefit of charge control and frequency control. It maintains the good transient performance of charge control, but avoids the related stability issues by adding slope compensation. The slope compensation also helps to sense the resonant capacitor voltage by only using lossless capacitor divider. Burst mode operation is developed to improve the light load efficiency. By turning on the power devices for only a short period during light load, it helps to reduce the equivalent switching frequency and achieve high efficiency. The effectiveness of the proposed controller has been validated in a 12 V/120 W half-bridge LLC converter, as well as a commercial power supply unit.

This paper presents analysis and modeling for a bidirectional push-pull dc-dc converter. In particular, attention is paid to leakage current reduction with PWM plus phase-shift control. The leakage current in different operating modes has been compared to identify the efficient modes for various operation conditions. The state space averaging method is adopted to derive the small signal and large signal models of the most efficient mode. A 30~70V/300V prototype was built. Both simulation results and experimental results prove the validity of the model.

This paper presents a LED driver circuit based on single-stage Flyback converter with power factor correction. Voltage following control and Nonlinear carrier control are applied to constant current mode Flyback converter which is combined with constant current driver technique. By designing the outer loop to stabilize the output voltage and the inner loop to shape input current waveform, that makes the system achieve unity power factor. The circuit doesn't need an expensive multiplier as classical multiplier control. To verify the performance, the proposed circuit was confirmed by simulation and experiment. From the results, the proposed system shows high power factor with 0.96 and constant current output.

This paper presents a mechatronic model of a novel slotless permanent magnet DC-motor with air gap winding. Besides technical advantages of this type of motor like high power density, high torque, very low weight and high efficiency, the motor design allows a very precise and efficient modelling with limited effort. A nonlinear model of magnetic field density can be extracted from a detailed nonlinear FE-model build in ANSYS/Maxwell, approximated by Fourier series and then used to model driving torque and back EMF, representing the coupling between electrical and mechanical subsystems. Analytically founded numerical models for driving torque and back EMF will be given. Real geometry of the phase winding is taken into account to improve model accuracy. The electrical subsystem will be described as coupled three phase system, whose parameters can also be extracted from the nonlinear FE-model with high accuracy. Together with a mechanical model of the rotor a MATLAB/Simulink model is build and extended by models of the hall sensors to detect rotor position and commutation logic to control the HEX-Bridge during operation. Finally, results of a complex simulation model, based on the parameters of the prototype of a wheel-hub motor, implementing the new motor design, are getting shown. Simulation results compare very well to measured data. Simulation time is very short due to the efficient approximation of magnetic flux density.

In this paper, a new voltage source converter for medium voltage applications is presented which can operate over a wide range of voltages (2.4-7.2 kV) without the need for connecting the power semiconductor in series. The operation of the proposed converter is studied and analyzed. In order to control the proposed converter, a space-vector modulation (SVM) strategy with redundant switching states has been proposed. SVM usually has redundant switching state anyways. What is the main point we are trying to get to? These redundant switching states help to control the output voltage and balance voltages of the flying capacitors in the proposed converter. The performance of the converter under different operating conditions is investigated in MATLAB/Simulink environment. The feasibility of the proposed converter is evaluated experimentally on a 5-kVA prototype.

An inverter control design in DQ reference frame takes 3-phase sinusoidal signals and converts them to the DQ-frame to make a 2-phase based DC variable control. The error is calculated by subtracting the feedback from reference and then feedding it to PI controllers. The outputs of PI controllers are of the voltage form and transferred back to the ABC frame and fed to SPWM modules. P+Resonant controllers suggest establishing the control strategy in stationary (alpha-beta) frame. In this frame the axis are fix and three phase variables are mapped in a rotating vector with the same angular speed of the original signals. The P+Resonant controller has zero steady state error at the resonant frequency. This paper explains how to apply DQ-frame and P+Resonant for inverter controller design for grid connected single-phase Voltage Source Inverters (VSI). To evaluate the effectiveness of both control techniques, it is also presented simulated results with application of these controllers.

This paper reviews many design issues and analysis techniques for the brushless permanent-magnet machine. It reviews the basic requirements for the use of both ac and dc machines and issues concerning the selection of pole number, winding layout, rotor topology, drive strategy, field weakening, and cooling. These are key issues in the design of a motor. Leading-edge design techniques are illustrated. This paper is aimed as a tutor for motor designers who may be unfamiliar with this particular type of machine.

Direct Torque Control is a control technique in AC drive systems to obtain high performance torque control. The conventional DTC drive contains a pair of hysteresis comparators. DTC drives utilizing hysteresis comparators suffer from high torque ripple and variable switching frequency. The most common solution to those problems is to use the space vector depends on the reference torque and flux. In this Paper The space vector modulation technique (SVPWM) is applied to 2 level inverter control in the proposed DTC-based induction motor drive system, thereby dramatically reducing the torque ripple. Then the controller based on space vector modulation is designed to be applied in the control of Induction Motor (IM) with a three-level Inverter. This type of Inverter has several advantages over the standard two-level VSI, such as a greater number of levels in the output voltage waveforms, Lower dV/dt, less harmonic distortion in voltage and current waveforms and lower switching frequencies. This paper proposes a general SVPWM algorithm for three-level based on standard two-level SVPWM. The proposed scheme is described clearly and simulation results are reported to demonstrate its effectiveness. The entire control scheme is implemented with Matlab/Simulink. Keywords—Direct torque control, space vector Pulsewidth modulation(SVPWM), neutral point clamped(NPC), two-level inverter.

This paper aims to design a PID compensator for the control of Synchronous Rectifier (SR) Buck Converter to improve its conversion efficiency under different load conditions. Since the diode rectifier is replaced by a high frequency MOSFET switch, the SR control method itself will be sufficient under heavy load condition, to attain better normal mode performance. However, this technique does not hold well in light load condition, due to increased switching losses. A new control technique accompanied with PID compensator is introduced in the paper will enable synchronous buck converter to realize ZVS, while feeding light loads. This is also least cost and highly efficient easy technique without use of extra auxiliary switches and RLC components. This control technique also proved to be efficient under input voltage variations. Simulation is done in MATLAB Simulink for proving stabilization provided by the PID compensator for synchronous rectifier (SR) buck converter. Keywords—Synchronous Rectifier (SR), Continuous Condu-ction Mode (CCM), Discontinuous Conduction Mode (DCM), Zero Voltage Switching (ZVS), Proportional-IntegralDerivative (PID).

This paper proposes a small-signal model for average current mode control based on an equivalent circuit. The model uses a three-terminal equivalent circuit model based on a linearized describing function method to include the feedback effect of the sideband frequency components of the inductor current. The model extends the results obtained in peak current mode control to average current mode control. The proposed small-signal model is accurate up to half switching frequency, predicting the subharmonic instability. The proposed model is verified using SIMPLIS simulation and hardware experiments, which show good agreement with the measurement results. Based on the proposed model, new feedback design guidelines are presented. The proposed design guidelines are compared with several conventional, widely used design criteria. By designing the external ramp following the proposed design guidelines, the quality factor of the double poles at half of the switching frequency in the control-to-output transfer function can be precisely controlled. This helps the feedback loop design to achieve wide control bandwidth and proper damping.


Question: Speed regulation for DC motor drives powered by buck converters.

This document outlines a novel methodology for enhancing the robustness of a Positive Buck-Boost converter in response to fluctuations in input voltage and variations in load conditions. The Positive Buck-Boost converter is a versatile DC-DC converter capable of functioning as either a buck or boost converter, maintaining the same polarity of the input voltage. It possesses four distinct switching states, encompassing all those found in conventional buck and boost converters, plus an additional state that offers a unique degree of freedom not found in other DC-DC converter types. This additional state grants the Positive Buck-Boost converter superior adaptability in controlling its inductor current, surpassing the capabilities of standard buck, boost, and inverting buck-boost converters. The paper proposes two innovative control strategies that capitalize on this extra degree of freedom to improve the converter's resilience against input voltage variations and load alterations. These strategies focus on regulating the inductor current and output voltage, ensuring stability in the face of changing conditions. Theoretical analyses for both dynamic and steady-state scenarios are provided, with simulation outcomes confirming the efficacy of the proposed techniques.

The main aim of this paper is to improve the performance of high current dual active bridge converters when operated over a wide voltage range. A typical application is for fuel cell vehicles where a bi-directional interface between a 12V battery and a high voltage DC bus is required. The battery side voltage ranges from 11V to 16V while the fuel cell is operated between 220V and 447V and the required power is typically 1kW. Careful analysis shows that the high currents on the battery side cause significant design issues in order to obtain a high efficiency. The standard phase shift modulation method can result in high conduction and switching losses. This paper proposes a combined triangular and trapezoidal modulation method to reduce losses over the wide operating range. Approximately, a 2% improvement in efficiency can be expected. An experimental system is used to verify the improved performance of the dual active bridge using the proposed advanced modulation method.

In this paper, the authors investigate the iron loss of interior permanent magnet motors driven by pulsewidth modulation (PWM) inverters from both results of the experiments and the finite-element analysis. In the analysis, the iron loss of the motor is decomposed into several components due to their origins, for instance, the fundamental field, carrier of the PWM inverter, slot ripples, and harmonic magnetomotive forces of the permanent magnet in order to clarify the main loss factors. The Fourier transformation and the finite-element method considering the carrier harmonics are applied to this calculation. The calculated iron loss is compared with the measurement at each driving condition. The measured and the calculated results agree well. It is clarified that the iron loss caused by the carrier of the PWM inverter is the largest component at low-speed condition under the maximum torque control, whereas the loss caused by the harmonic magnetomotive forces of the permanent magnet remarkably increase at high-speed condition under the flux-weakening control

This paper presents a combined phase-shift and frequency modulation scheme of a dual-active-bridge (DAB) ac- dc converter with power factor correction (PFC) to achieve zero voltage switching (ZVS) over the full range of the ac mains voltage. The DAB consists of a half bridge with bidirectional switches on the ac side and a full bridge on the dc side of the isolation transformer to accomplish single-stage power conversion. The modulation scheme is described by means of analytical formulas, which are used in an optimization procedure to determine the optimal control variables for minimum switch commutation currents. Furthermore, an ac current controller suitable for the proposed modulation scheme is described. A loss model and measurements on a 3.3-kW electric vehicle battery charger to connect to the 230 Vrms / 50-Hz mains considering a battery voltage range of 280-420 V validate the theoretical analysis.

In the present, a power decoupling method without additional component is proposed for a dc to single-phase ac converter, which consists of a flying capacitor dc/dc converter (FCC) and the voltage source inverter (VSI). In particular, a small flying capacitor in the FCC is used for both a boost operation and a double-line-frequency power ripple reduction. Thus, the dc-link capacitor value can be minimized in order to avoid the use of a large electrolytic capacitor. In addition, component design, of, e.g., the boost inductor and the flying capacitor, is clarified when the proposed control is applied. Experiments were carried out using a 1.5-kW prototype in order to verify the validity of the proposed control. The experimental results revealed that the use of the proposed control reduced the dc-link voltage ripple by 74.5%, and the total harmonic distortion (THD) of the inverter output current was less than 5%. Moreover, a maximum system efficiency of 95.4% was achieved at a load of 1.1 kW. Finally, the high power density design is evaluated by the Pareto front optimization. The power densities of three power decoupling topologies, such as a boost topology, a buck topology, and the proposed topology are compared. As a result, the proposed topology achieves the highest power density (5.3 kW/dm3) among the topologies considered herein.

This paper presents a multi-MHz buck regulator for portable applications using an auto-selectable peak- and valley-current control (ASPVCC) scheme. The proposed ASPVCC scheme can enable the current-mode buck regulator to reduce the settling-time requirement of the current sensing by two times. In addition, the dynamically biased shunt feedback technique is employed to improve the sensing speed and the sensing accuracy of both the peak and valley current sensors. With both ASPVCC scheme and advanced current sensors, the buck regulator can thus operate at high switching frequencies with a wide range of duty ratios for reducing the required off-chip inductance. The proposed current-mode buck regulator was fabricated in a standard 0.35-μm CMOS process and occupies a small chip area of 0.54 mm2. Measurement results show that the buck regulator can deliver a maximum output current of 500 mA, operate at 5 MHz with a wide duty-ratio range of about 0.6, use a small-value off-chip inductor of 1 μH, and achieve the peak power efficiency of 91%.

This paper presents a recursive digital low-dropout (RLDO) regulator that improves response time, quiescent power, and load regulation dynamic range over prior digital LDO designs by 1–2 orders of magnitude. The proposed RLDO enables a practical digital replacement to analog LDOs by using an SAR-like binary search algorithm in a coarse loop and a sub-LSB pulse width modulation duty control scheme in a fine loop. A proportional-derivative compensation scheme is employed to ensure stable operation independent of load current, the size of the output decoupling capacitor, and clock frequency. Implemented in 0.0023 mm2 in 65 nm CMOS, the 7-bit RLDO achieves, at a 0.5-V input, a response time of 15.1 ns with a figure of merit of 199.4 ps, along with stable operation across a 20 000 $\times $  dynamic load range.

This paper describes the design and the experimental investigation of a gate drive unit with closed-loop control of the collector current slope diC/dt for multichip insulated-gate bipolar transistors (IGBTs). Compared to a pure resistive gate drive, the proposed diC/dt control offers the ability to adjust the collector current slope freely which helps to find an optimized relation between switching losses and secure operation of the freewheeling diode for every type of IGBT. Based on the description of IGBT's switching behavior, the design and the realization of the gate drive are presented. The test setup and the comparison of switching tests with and without the proposed diC/dt control are discussed.

-Constant power loads (CPLs) may yield instability due to the well-known negative impedance characteristic. This study analyzes the factors which cause instability of a DC microgrid with multiple DC-DC converters. Two stabilization methods are presented for two operation modes: constant voltage source mode and droop mode. And sufficient conditions for the stability of the DC microgrid are obtained by identifying the eigenvalues of the Jacobian matrix. The key is to transform the eigenvalue problem to a quadratic eigenvalue problem (QEP). When applying the methods in practical engineering, the salient feature is that the stability parameter domains can be estimated by the available constraints, such as the values of capacities, inductances, maximum load power and distances of the cables. Compared with some classical methods, the proposed methods have wider stability region. The simulation results based on MATLAB /simulink platform verify the feasibility of the methods.

This paper presents a dimmable charge-pump driver to power light-emitting diodes (LEDs) with power factor correction (PFC) and Zero Voltage Switching (ZVS). The proposed LED driver does not utilize electrolytic capacitors, providing a high useful lifetime, and it can stabilize the output current in open loop control without needing current sensors, which reduces the cost. The output power is proportional to the switching frequency, which allows the LEDs dimming. A prototype with 22 W was implemented and experimental results were discussed. The prototype presented a power factor of 0.996 and an efficiency of 89.5 %. The driver output power was reduced by more than 40% through the switching frequency while varying of 53 kHz to 30 kHz and the converter has continues to operate in ZVS.

In this paper, a monolithically integrated gate voltage pull-down circuitry is presented to avoid the unintentional C·dV/dt induced turn-on. The concept of a low threshold voltage MOSFET with this integrated gate voltage pull-down circuitry is introduced as a contributing factor to the next generation high frequency DC-DC converter efficiency improvement. Design considerations on this new device and influences of critical design parameters on device/circuit performance will be fully discussed. In synchronous buck application, this integrated power module achieves more than 2% efficiency improvement over reference solution at high operation frequency (1MHz) under 19V input and 1.3V output condition.


Question: Using multiplicative recurrent neural networks to model compositional structures.

This paper introduces two innovative model architectures designed for the efficient computation of continuous vector representations of words from vast datasets. The efficacy of these representations is assessed through a word similarity task, with the results juxtaposed against those obtained from previously established methods utilizing various neural network configurations. Notably, our models demonstrate a significant enhancement in accuracy, achieved at a considerably reduced computational expense. Specifically, we achieve the learning of high-quality word vectors from a dataset comprising 1.6 billion words in under a day. Additionally, we present evidence that these vector representations yield top-tier performance in our test set, which evaluates both syntactic and semantic word similarities, surpassing existing benchmarks.

**Question**: A Direct Search Method to solve Economic Dispatch Problem with Valve-Point Effect

**Documents**: An efficient algorithm for economic dispatch with valve-point effect
This paper presents an efficient algorithm for solving the economic dispatch problem with valve-point effect. The proposed algorithm is based on a new approach to model the valve-point effect and a modified version of the quadratic programming method. The algorithm is tested on a 10-unit system and the results are compared with those obtained from other methods. The proposed algorithm is found to be more efficient and accurate than the other methods in terms of both solution time and solution quality.

**Similar Documents**: This work introduces an innovative algorithm designed to efficiently address the economic dispatch problem, particularly in scenarios involving valve-point effects. The algorithm's foundation lies in a novel modeling technique for valve-point effects, coupled with an enhanced implementation of the quadratic programming method. To validate its efficacy, the algorithm is applied to a 10-unit system, and its outcomes are contrasted with those derived from alternative methodologies. Notably, the proposed algorithm outperforms competing methods in terms of both computational speed and solution precision, thereby establishing its superiority in the context of economic dispatch problems with valve-point considerations.

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-ofthe-art discriminative constituency parser. The use of attention makes explicit the manner in which information is propagated between different locations in the sentence, which we use to both analyze our model and propose potential improvements. For example, we find that separating positional and content information in the encoder can lead to improved parsing accuracy. Additionally, we evaluate different approaches for lexical representation. Our parser achieves new state-ofthe-art results for single models trained on the Penn Treebank: 93.55 F1 without the use of any external data, and 95.13 F1 when using pre-trained word representations. Our parser also outperforms the previous best-published accuracy figures on 8 of the 9 languages in the SPMRL dataset.

Many data such as social networks, movie preferences or knowledge bases are multi-relational, in that they describe multiple relations between entities. While there is a large body of work focused on modeling these data, modeling these multiple types of relations jointly remains challenging. Further, existing approaches tend to breakdown when the number of these types grows. In this paper, we propose a method for modeling large multi-relational datasets, with possibly thousands of relations. Our model is based on a bilinear structure, which captures various orders of interaction of the data, and also shares sparse latent factors across different relations. We illustrate the performance of our approach on standard tensor-factorization datasets where we attain, or outperform, state-of-the-art results. Finally, a NLP application demonstrates our scalability and the ability of our model to learn efficient and semantically meaningful verb representations.

In a conversation or a dialogue process, attention and intention play intrinsic roles. This paper proposes a neural network based approach that models the attention and intention processes. It essentially consists of three recurrent networks. The encoder network is a word-level model representing source side sentences. The intention network is a recurrent network that models the dynamics of the intention process. The decoder network is a recurrent network that produces responses to the input from the source side. It is a language model that is dependent on the intention and has an attention mechanism to attend to particular source side words, when predicting a symbol in the response. The model is trained end-toend without labeling data. Experiments show that this model generates natural responses to user inputs.

Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SRU’s architecture is simple, un-gated, and contains a comparable number of parameters to LSTMs; yet, SRUs perform favorably to more sophisticated LSTM and GRU alternatives, often outperforming one or both in various tasks. We show the efficacy of SRUs as compared to LSTMs and GRUs in an unbiased manner by optimizing respective architectures’ hyperparameters in a Bayesian optimization scheme for both synthetic and realworld tasks.

We introduce an encoder-decoder recurrent neural network model called Recurrent Neural Aligner (RNA) that can be used for sequence to sequence mapping tasks. Like connectionist temporal classification (CTC) models, RNA defines a probability distribution over target label sequences including blank labels corresponding to each time step in input. The probability of a label sequence is calculated by marginalizing over all possible blank label positions. Unlike CTC, RNA does not make a conditional independence assumption for label predictions; it uses the predicted label at time t−1 as an additional input to the recurrent model when predicting the label at time t. We apply this model to end-to-end speech recognition. RNA is capable of streaming recognition since the decoder does not employ attention mechanism. The model is trained on transcribed acoustic data to predict graphemes and no external language and pronunciation models are used for decoding. We employ an approximate dynamic programming method to optimize negative log likelihood, and a sampling-based sequence discriminative training technique to fine-tune the model to minimize expected word error rate. We show that the model achieves competitive accuracy without using an external language model nor doing beam search decoding.

We present a model of visually-grounded language learning based on stacked gated recurrent neural networks which learns to predict visual features given an image description in the form of a sequence of phonemes. The learning task resembles that faced by human language learners who need to discover both structure and meaning from noisy and ambiguous data across modalities. We show that our model indeed learns to predict features of the visual context given phonetically transcribed image descriptions, and show that it represents linguistic information in a hierarchy of levels: lower layers in the stack are comparatively more sensitive to form, whereas higher layers are more sensitive to meaning.

We describe a simple neural language model that relies only on character-level inputs. Predictions are still made at the word-level. Our model employs a convolutional neural network (CNN) and a highway network over characters, whose output is given to a long shortterm memory (LSTM) recurrent neural network language model (RNN-LM). On the English Penn Treebank the model is on par with the existing state-ofthe-art despite having 60% fewer parameters. On languages with rich morphology (Czech, German, French, Spanish, Russian), the model consistently outperforms a Kneser-Ney baseline and word-level/morpheme-level LSTM baselines, again with far fewer parameters. Our results suggest that on many languages, character inputs are sufficient for language modeling.

In statistical relational learning, the link prediction problem is key to automatically understand the structure of large knowledge bases. As in previous studies, we propose to solve this problem through latent factorization. However, here we make use of complex valued embeddings. The composition of complex embeddings can handle a large variety of binary relations, among them symmetric and antisymmetric relations. Compared to state-of-the-art models such as Neural Tensor Network and Holographic Embeddings, our approach based on complex embeddings is arguably simpler, as it only uses the Hermitian dot product, the complex counterpart of the standard dot product between real vectors. Our approach is scalable to large datasets as it remains linear in both space and time, while consistently outperforming alternative approaches on standard link prediction benchmarks.1

An evolutionary recurrent network which automates the design of recurrent neural/fuzzy networks using a new evolutionary learning algorithm is proposed in this paper. This new evolutionary learning algorithm is based on a hybrid of genetic algorithm (GA) and particle swarm optimization (PSO), and is thus called HGAPSO. In HGAPSO, individuals in a new generation are created, not only by crossover and mutation operation as in GA, but also by PSO. The concept of elite strategy is adopted in HGAPSO, where the upper-half of the best-performing individuals in a population are regarded as elites. However, instead of being reproduced directly to the next generation, these elites are first enhanced. The group constituted by the elites is regarded as a swarm, and each elite corresponds to a particle within it. In this regard, the elites are enhanced by PSO, an operation which mimics the maturing phenomenon in nature. These enhanced elites constitute half of the population in the new generation, whereas the other half is generated by performing crossover and mutation operation on these enhanced elites. HGAPSO is applied to recurrent neural/fuzzy network design as follows. For recurrent neural network, a fully connected recurrent neural network is designed and applied to a temporal sequence production problem. For recurrent fuzzy network design, a Takagi-Sugeno-Kang-type recurrent fuzzy network is designed and applied to dynamic plant control. The performance of HGAPSO is compared to both GA and PSO in these recurrent networks design problems, demonstrating its superiority.

It is a known fact that training recurrent neural networks for tasks that have long term dependencies is challenging. One of the main reasons is the vanishing or exploding gradient problem, which prevents gradient information from propagating to early layers. In this paper we propose a simple recurrent architecture, the Fourier Recurrent Unit (FRU), that stabilizes the gradients that arise in its training while giving us stronger expressive power. Specifically, FRU summarizes the hidden states h along the temporal dimension with Fourier basis functions. This allows gradients to easily reach any layer due to FRU’s residual learning structure and the global support of trigonometric functions. We show that FRU has gradient lower and upper bounds independent of temporal dimension. We also show the strong expressivity of sparse Fourier basis, from which FRU obtains its strong expressive power. Our experimental study also demonstrates that with fewer parameters the proposed architecture outperforms other recurrent architectures on many tasks.


Question: Using multiplicative recurrent neural networks to model compositional structures.

This paper presents a comprehensive convolutional neural network architecture designed to process and predict a multitude of linguistic features from a given sentence. The network's output encompasses a variety of natural language processing tasks, including part-of-speech tagging, chunking, named entity recognition, semantic role labeling, semantic similarity scoring, and language model assessment, which evaluates the sentence's grammatical and semantic coherence. A key feature of this network is its ability to learn from multiple tasks simultaneously through weight-sharing, a form of multitask learning. Notably, all tasks except the language model are trained on labeled data, while the language model leverages unlabeled text, offering a novel approach to semi-supervised learning for the shared tasks. The paper demonstrates that both multitask and semi-supervised learning strategies enhance the generalization capabilities of the shared tasks, leading to state-of-the-art performance across these language processing tasks.

In this paper, we propose TopicRNN, a recurrent neural network (RNN)-based language model designed to directly capture the global semantic meaning relating words in a document via latent topics. Because of their sequential nature, RNNs are good at capturing the local structure of a word sequence – both semantic and syntactic – but might face difficulty remembering long-range dependencies. Intuitively, these long-range dependencies are of semantic nature. In contrast, latent topic models are able to capture the global semantic structure of a document but do not account for word ordering. The proposed TopicRNN model integrates the merits of RNNs and latent topic models: it captures local (syntactic) dependencies using an RNN and global (semantic) dependencies using latent topics. Unlike previous work on contextual RNN language modeling, our model is learned endto-end. Empirical results on word prediction show that TopicRNN outperforms existing contextual RNN baselines. In addition, TopicRNN can be used as an unsupervised feature extractor for documents. We do this for sentiment analysis on the IMDB movie review dataset and report an error rate of 6.28%. This is comparable to the state-of-the-art 5.91% resulting from a semi-supervised approach. Finally, TopicRNN also yields sensible topics, making it a useful alternative to document models such as latent Dirichlet allocation.

We introduce a new neural architecture to learn the conditional probability of an output sequence with elements that are discrete tokens corresponding to positions in an input sequence. Such problems cannot be trivially addressed by existent approaches such as sequence-to-sequence [1] and Neural Turing Machines [2], because the number of target classes in each step of the output depends on the length of the input, which is variable. Problems such as sorting variable sized sequences, and various combinatorial optimization problems belong to this class. Our model solves the problem of variable size output dictionaries using a recently proposed mechanism of neural attention. It differs from the previous attention attempts in that, instead of using attention to blend hidden units of an encoder to a context vector at each decoder step, it uses attention as a pointer to select a member of the input sequence as the output. We call this architecture a Pointer Net (Ptr-Net). We show Ptr-Nets can be used to learn approximate solutions to three challenging geometric problems – finding planar convex hulls, computing Delaunay triangulations, and the planar Travelling Salesman Problem – using training examples alone. Ptr-Nets not only improve over sequence-to-sequence with input attention, but also allow us to generalize to variable size output dictionaries. We show that the learnt models generalize beyond the maximum lengths they were trained on. We hope our results on these tasks will encourage a broader exploration of neural learning for discrete problems.

Neural networks have been shown to be an effective tool for learning algorithms over graph-structured data. However, graph representation techniques—that convert graphs to real-valued vectors for use with neural networks—are still in their infancy. Recent works have proposed several approaches (e.g., graph convolutional networks), but these methods have difficulty scaling and generalizing to graphs with different sizes and shapes. We present GRAPH2SEQ, a new technique that represents vertices of graphs as infinite time-series. By not limiting the representation to a fixed dimension, GRAPH2SEQ scales naturally to graphs of arbitrary sizes and shapes. GRAPH2SEQ is also reversible, allowing full recovery of the graph structure from the sequences. By analyzing a formal computational model for graph representation, we show that an unbounded sequence is necessary for scalability. Our experimental results with GRAPH2SEQ show strong generalization and new state-of-the-art performance on a variety of graph combinatorial optimization problems.

Most tasks in natural language processing can be cast into question answering (QA) problems over language input. We introduce the dynamic memory network (DMN), a unified neural network framework which processes input sequences and questions, forms semantic and episodic memories, and generates relevant answers. Questions trigger an iterative attention process which allows the model to condition its attention on the result of previous iterations. These results are then reasoned over in a hierarchical recurrent sequence model to generate answers. The DMN can be trained end-to-end and obtains state of the art results on several types of tasks and datasets: question answering (Facebook’s bAbI dataset), sequence modeling for part of speech tagging (WSJ-PTB), coreference resolution (Quizbowl dataset) and text classification for sentiment analysis (Stanford Sentiment Treebank). The model relies exclusively on trained word vector representations and requires no string matching or manually engineered features.

Several approaches to implementing symbol-like representations in neurally plausible models have been proposed. These approaches include binding through synchrony (Shastri & Ajjanagadde, ), "mesh" binding (van der Velde & de Kamps, ), and conjunctive binding (Smolensky, ). Recent theoretical work has suggested that most of these methods will not scale well, that is, that they cannot encode structured representations using any of the tens of thousands of terms in the adult lexicon without making implausible resource assumptions. Here, we empirically demonstrate that the biologically plausible structured representations employed in the Semantic Pointer Architecture (SPA) approach to modeling cognition (Eliasmith, ) do scale appropriately. Specifically, we construct a spiking neural network of about 2.5 million neurons that employs semantic pointers to successfully encode and decode the main lexical relations in WordNet, which has over 100,000 terms. In addition, we show that the same representations can be employed to construct recursively structured sentences consisting of arbitrary WordNet concepts, while preserving the original lexical structure. We argue that these results suggest that semantic pointers are uniquely well-suited to providing a biologically plausible account of the structured representations that underwrite human cognition.

We present a supervised neural network model for polyphonic piano music transcription. The architecture of the proposed model is analogous to speech recognition systems and comprises an acoustic model and a music language model. The acoustic model is a neural network used for estimating the probabilities of pitches in a frame of audio. The language model is a recurrent neural network that models the correlations between pitch combinations over time. The proposed model is general and can be used to transcribe polyphonic music without imposing any constraints on the polyphony. The acoustic and language model predictions are combined using a probabilistic graphical model. Inference over the output variables is performed using the beam search algorithm. We perform two sets of experiments. We investigate various neural network architectures for the acoustic models and also investigate the effect of combining acoustic and music language model predictions using the proposed architecture. We compare performance of the neural network-based acoustic models with two popular unsupervised acoustic models. Results show that convolutional neural network acoustic models yield the best performance across all evaluation metrics. We also observe improved performance with the application of the music language models. Finally, we present an efficient variant of beam search that improves performance and reduces run-times by an order of magnitude, making the model suitable for real-time applications.

The use of Recurrent Neural Networks is not as extensive as Feedforward Neural Networks. Training algorithms for Recurrent Neural Networks, based on the error gradient, are very unstable in their search for a minimum and require much computational time when the number of neurons is high. The problems surrounding the application of these methods have driven us to develop new training tools. In this paper, we present a Real-Coded Genetic Algorithm that uses the appropriate operators for this encoding type to train Recurrent Neural Networks. We describe the algorithm and we also experimentally compare our Genetic Algorithm with the Real-Time Recurrent Learning algorithm to perform the fuzzy grammatical inference.

Semantic matching is of central importance to many natural language tasks [2, 28]. A successful matching algorithm needs to adequately model the internal structures of language objects and the interaction between them. As a step toward this goal, we propose convolutional neural network models for matching two sentences, by adapting the convolutional strategy in vision and speech. The proposed models not only nicely represent the hierarchical structures of sentences with their layerby-layer composition and pooling, but also capture the rich matching patterns at different levels. Our models are rather generic, requiring no prior knowledge on language, and can hence be applied to matching tasks of different nature and in different languages. The empirical study on a variety of matching tasks demonstrates the efficacy of the proposed model on a variety of matching tasks and its superiority to competitor models.

In this paper, a new representation of neural tensor networks is presented. Recently, state-of-the-art neural tensor networks have been introduced to complete RDF knowledge bases. However, mathematical model representation of these networks is still a challenging problem, due to tensor parameters. To solve this problem, it is proposed that these networks can be represented as two-layer perceptron network. To complete the network topology, the traditional gradient based learning rule is then developed. It should be mentioned that for tensor networks there have been developed some learning rules which are complex in nature due to the complexity of the objective function used. Indeed, this paper is aimed to show that the tensor network can be viewed and represented by the two-layer feedforward neural network in its traditional form. The simulation results presented in the paper easily verify this claim.

Several strategies are described that overcome limitations of basic network models as steps towards the design of large connectionist speech recognition systems. The two major areas of concern are the problem of time and the problem of scaling. Speech signals continuously vary over time and encode and transmit enormous amounts of human knowledge. To decode these signals, neural networks must be able to use appropriate representations of time and it must be possible to extend these nets to almost arbitrary sizes and complexity within finite resources. The problem of time is addressed by the development of a Time-Delay Neural Network; the problem of scaling by Modularity and Incremental Design of large nets based on smaller subcomponent nets. It is shown that small networks trained to perform limited tasks develop time invariant, hidden abstractions that can subsequently be exploited to train larger, more complex nets efficiently. Using these techniques, phoneme recognition networks of increasing complexity can be constructed that all achieve superior recognition performance.


Question: Modeling compositional structure using multiplicative recurrent neural networks.

This paper presents a novel approach to modeling semantic compositionality through the use of recursive matrix-vector spaces in a recursive neural network (RNN) framework. Existing single-word vector space models have excelled in capturing lexical information but fall short in representing the compositional meaning of phrases and sentences, limiting their ability to understand language at a deeper level. To address this, we propose an RNN model that can learn compositional vector representations for phrases and sentences, regardless of their syntactic type or length. Our model assigns a vector and a matrix to each node in a parse tree. The vector encapsulates the inherent meaning of the constituent, whereas the matrix represents how it modifies the meaning of adjacent words or phrases. This matrix-vector RNN is capable of learning the semantics of operators in both propositional logic and natural language. The model achieves state-of-the-art performance across a range of experiments, including the prediction of fine-grained sentiment distributions for adverb-adjective combinations, classification of sentiment labels in movie reviews, and categorization of semantic relationships such as cause-effect or topic-message between nouns, leveraging the syntactic path connecting them. This innovative approach to modeling compositionality significantly enhances the model's understanding and processing of language.

We propose to exploit the potential of multiple word clusterings in class-based recurrent neural network (RNN) language models for ensemble RNN language modeling. By varying the clustering criteria and the space of word embedding, different word clusterings are obtained to define different word/class factorizations. For each such word/class factorization, several base RNNLMs are learned, and the word prediction probabilities of the base RNNLMs are then combined to form an ensemble prediction. We use a greedy backward model selection procedure to select a subset of models and combine these models for word prediction. The proposed ensemble language modeling method has been evaluated on Penn Treebank test set as well as Wall Street Journal (WSJ) Eval 92 and 93 test sets, where it improved test set perplexity and word error rate over the state-of-the-art single RNNLMs as well as multiple RNNLMs produced by varying RNN learning conditions.

A major challenge in computer vision is scaling activity understanding to the long tail of complex activities without requiring collecting large quantities of data for new actions. The task of video retrieval using natural language descriptions seeks to address this through rich, unconstrained supervision about complex activities. However, while this formulation offers hope of leveraging underlying compositional structure in activity descriptions, existing approaches typically do not explicitly model compositional reasoning. In this work, we introduce an approach for explicitly and dynamically reasoning about compositional natural language descriptions of activity in videos. We take a modular neural network approach that, given a natural language query, extracts the semantic structure to assemble a compositional neural network layout and corresponding network modules. We show that this approach is able to achieve state-of-the-art results on the DiDeMo video retrieval dataset.

We propose an application of SeqGAN, generative adversarial networks for discrete sequence generation, for creating polyphonic musical sequences. Instead of monophonic melody generation suggested in the original work, we present an efficient representation of polyphony MIDI file that captures chords and melodies with dynamic timings simultaneously. The network can create sequences that are musically coherent. We also report that careful tuning of reinforcement learning signals of the model are crucial for general application.

In this paper, we extend the deep long short-term memory (DL-STM) recurrent neural networks by introducing gated direct connections between memory cells in adjacent layers. These direct links, called highway connections, enable unimpeded information flow across different layers and thus alleviate the gradient vanishing problem when building deeper LSTMs. We further introduce the latency-controlled bidirectional LSTMs (BLSTMs) which can exploit the whole history while keeping the latency under control. Efficient algorithms are proposed to train these novel networks using both frame and sequence discriminative criteria. Experiments on the AMI distant speech recognition (DSR) task indicate that we can train deeper LSTMs and achieve better improvement from sequence training with highway LSTMs (HLSTMs). Our novel model obtains 43.9/47.7% WER on AMI (SDM) dev and eval sets, outperforming all previous works. It beats the strong DNN and DLSTM baselines with 15.7% and 5.3% relative improvement respectively.

We present the Neural Physics Engine (NPE), a framework for learning simulators of intuitive physics that naturally generalize across variable object count and different scene configurations. We propose a factorization of a physical scene into composable object-based representations and a neural network architecture whose compositional structure factorizes object dynamics into pairwise interactions. Like a symbolic physics engine, the NPE is endowed with generic notions of objects and their interactions; realized as a neural network, it can be trained via stochastic gradient descent to adapt to specific object properties and dynamics of different worlds. We evaluate the efficacy of our approach on simple rigid body dynamics in two-dimensional worlds. By comparing to less structured architectures, we show that the NPE’s compositional representation of the structure in physical interactions improves its ability to predict movement, generalize across variable object count and different scene configurations, and infer latent properties of objects such as mass.

This paper compares the performance of two recurrent neural network models learning the grammar of simple English sentences and introduces a method for creating useful negative training examples from a set of grammatical sentences. The learning task is cast as a classification problem as opposed to a prediction problem; networks are taught to classify a given sequence as grammatical or ungrammatical. The two neural network models compared are the simple architecture proposed by Elman in [1], and the Long Short-Term Memory (LSTM) network from [4]. For comparison, a Naive Bayes (NB) classifier is also trained with bigrams to accomplish this task. Only the Elman network learns the training data better than NB, and neither network generalizes better than NB on sentences longer than the training sentences. The LSTM network does not generalize as well as the Elman network on a simple regular grammar, but shows comparable ability to generalize knowledge of English Grammar.

Sophisticated gated recurrent neural network architectures like LSTMs and GRUs have been shown to be highly effective in a myriad of applications. We develop an un-gated unit, the statistical recurrent unit (SRU), that is able to learn long term dependencies in data by only keeping moving averages of statistics. The SRU’s architecture is simple, un-gated, and contains a comparable number of parameters to LSTMs; yet, SRUs perform favorably to more sophisticated LSTM and GRU alternatives, often outperforming one or both in various tasks. We show the efficacy of SRUs as compared to LSTMs and GRUs in an unbiased manner by optimizing respective architectures’ hyperparameters in a Bayesian optimization scheme for both synthetic and realworld tasks.

Definition: Convolution-Based Memory Models are a mathematical model of neural storage of complex data structures using distributed representations. Data structures stored range from lists of pairs through sequences, trees and networks.

Several mechanisms to focus attention of a neural network on selected parts of its input or memory have been used successfully in deep learning models in recent years. Attention has improved image classification, image captioning, speech recognition, generative models, and learning algorithmic tasks, but it had probably the largest impact on neural machine translation. Recently, similar improvements have been obtained using alternative mechanisms that do not focus on a single part of a memory but operate on all of it in parallel, in a uniform way. Such mechanism, which we call active memory, improved over attention in algorithmic tasks, image processing, and in generative modelling. So far, however, active memory has not improved over attention for most natural language processing tasks, in particular for machine translation. We analyze this shortcoming in this paper and propose an extended model of active memory that matches existing attention models on neural machine translation and generalizes better to longer sentences. We investigate this model and explain why previous active memory models did not succeed. Finally, we discuss when active memory brings most benefits and where attention can be a better choice.

Although NeuroCRF, an augmented Conditional Random Fields (CRF) model whose feature function is parameterized as a Feed-Forward Neural Network (FF NN) on word embeddings, has soundly outperformed traditional linear-chain CRF on many sequence labeling tasks, it is held back by the fact that FF NNs have a fixed input length and therefore cannot take advantage of the full input sentence. We propose to address this issue by replacing the FF NN with a Long Short-Term Memory (LSTM) NN, which can summarize an input of arbitrary length into a fixed dimension representation. The resulting model obtains F1=89.28 on WikiNER dataset, a significant improvement over the NeuroCRF baseline’s F1=87.58, which is already a highly competitive result.


Question: Using multiplicative recurrent neural networks to model compositional structures.

This paper introduces a novel methodology for phrase-level sentiment analysis, focusing on accurately distinguishing between neutral and polar expressions within text. Once this initial classification is made, the system further refines its analysis by disambiguating the polarity of the identified polar expressions. This two-step process enables the system to effectively discern the contextual polarity of a significant portion of sentiment-laden phrases, resulting in notably superior performance compared to baseline approaches. By recognizing the nuanced context surrounding these expressions, the system showcases its advanced capability to interpret and analyze sentiment at a detailed level, significantly enhancing the precision of sentiment analysis tasks.

Deep neural networks for machine comprehension typically utilizes only word or character embeddings without explicitly taking advantage of structured linguistic information such as constituency trees and dependency trees. In this paper, we propose structural embedding of syntactic trees (SEST), an algorithm framework to utilize structured information and encode them into vector representations that can boost the performance of algorithms for the machine comprehension. We evaluate our approach using a state-of-the-art neural attention model on the SQuAD dataset. Experimental results demonstrate that our model can accurately identify the syntactic boundaries of the sentences and extract answers that are syntactically coherent over the baseline methods.

Distributed word representations have recently been proven to be an invaluable resource for NLP. These representations are normally learned using neural networks and capture syntactic and semantic information about words. Information about word morphology and shape is normally ignored when learning word representations. However, for tasks like part-of-speech tagging, intra-word information is extremely useful, specially when dealing with morphologically rich languages. In this paper, we propose a deep neural network that learns character-level representation of words and associate them with usual word representations to perform POS tagging. Using the proposed approach, while avoiding the use of any handcrafted feature, we produce stateof-the-art POS taggers for two languages: English, with 97.32% accuracy on the Penn Treebank WSJ corpus; and Portuguese, with 97.47% accuracy on the Mac-Morpho corpus, where the latter represents an error reduction of 12.2% on the best previous known result.

Recurrent Neural Networks (RNNs) and their variants, such as Long-Short Term Memory (LSTM) networks, and Gated Recurrent Unit (GRU) networks, have achieved promising performance in sequential data modeling. The hidden layers in RNNs can be regarded as the memory units, which are helpful in storing information in sequential contexts. However, when dealing with high dimensional input data, such as video and text, the input-to-hidden linear transformation in RNNs brings high memory usage and huge computational cost. This makes the training of RNNs very difficult. To address this challenge, we propose a novel compact LSTM model, named as TR-LSTM, by utilizing the low-rank tensor ring decomposition (TRD) to reformulate the input-to-hidden transformation. Compared with other tensor decomposition methods, TRLSTM is more stable. In addition, TR-LSTM can complete an end-to-end training and also provide a fundamental building block for RNNs in handling large input data. Experiments on real-world action recognition datasets have demonstrated the promising performance of the proposed TR-LSTM compared with the tensortrain LSTM and other state-of-the-art competitors.

In this paper, we propose a probabilistic parsing model that defines a proper conditional probability distribution over nonprojective dependency trees for a given sentence, using neural representations as inputs. The neural network architecture is based on bi-directional LSTMCNNs, which automatically benefits from both wordand character-level representations, by using a combination of bidirectional LSTMs and CNNs. On top of the neural network, we introduce a probabilistic structured layer, defining a conditional log-linear model over nonprojective trees. By exploiting Kirchhoff’s Matrix-Tree Theorem (Tutte, 1984), the partition functions and marginals can be computed efficiently, leading to a straightforward end-to-end model training procedure via back-propagation. We evaluate our model on 17 different datasets, across 14 different languages. Our parser achieves state-of-the-art parsing performance on nine datasets.

Traditional convolutional neural network (CNN) based query classification uses linear feature mapping in its convolution operation. The recurrent neural network (RNN), differs from a CNN in representing word sequence with their ordering information kept explicitly. We propose using a deep long-short-term-memory (DLSTM) based feature mapping to learn feature representation for CNN. The DLSTM, which is a stack of LSTM units, has different order of feature representations at different depth of LSTM unit. The bottom LSTM unit equipped with input and output gates, extracts the first order feature representation from current word. To extract higher order nonlinear feature representation, the LSTM unit at higher position gets input from two parts. First part is the lower LSTM unit’s memory cell from previous word. Second part is the lower LSTM unit’s hidden output from current word. In this way, the DLSTM captures the nonlinear nonconsecutive interaction within n-grams. Using an architecture that combines a stack of the DLSTM layers with a tradition CNN layer, we have observed new state-of-the-art query classification accuracy on benchmark data sets for query classification.

It is a known fact that training recurrent neural networks for tasks that have long term dependencies is challenging. One of the main reasons is the vanishing or exploding gradient problem, which prevents gradient information from propagating to early layers. In this paper we propose a simple recurrent architecture, the Fourier Recurrent Unit (FRU), that stabilizes the gradients that arise in its training while giving us stronger expressive power. Specifically, FRU summarizes the hidden states h along the temporal dimension with Fourier basis functions. This allows gradients to easily reach any layer due to FRU’s residual learning structure and the global support of trigonometric functions. We show that FRU has gradient lower and upper bounds independent of temporal dimension. We also show the strong expressivity of sparse Fourier basis, from which FRU obtains its strong expressive power. Our experimental study also demonstrates that with fewer parameters the proposed architecture outperforms other recurrent architectures on many tasks.

In existing convolutional neural networks (CNNs), both convolution and pooling are locally performed for image regions separately, no contextual dependencies between different image regions have been taken into consideration. Such dependencies represent useful spatial structure information in images. Whereas recurrent neural networks (RNNs) are designed for learning contextual dependencies among sequential data by using the recurrent (feedback) connections. In this work, we propose the convolutional recurrent neural network (C-RNN), which learns the spatial dependencies between image regions to enhance the discriminative power of image representation. The C-RNN is trained in an end-to-end manner from raw pixel images. CNN layers are firstly processed to generate middle level features. RNN layer is then learned to encode spatial dependencies. The C-RNN can learn better image representation, especially for images with obvious spatial contextual dependencies. Our method achieves competitive performance on ILSVRC 2012, SUN 397, and MIT indoor.

We propose a new fast purely discriminative algorithm for natural language parsing, based on a “deep” recurrent convolutional graph transformer network (GTN). Assuming a decomposition of a parse tree into a stack of “levels”, the network predicts a level of the tree taking into account predictions of previous levels. Using only few basic text features which leverage word representations from Collobert and Weston (2008), we show similar performance (in F1 score) to existing pure discriminative parsers and existing “benchmark” parsers (like Collins parser, probabilistic context-free grammars based), with a huge speed advantage.

Semantic slot filling is one of the most challenging problems in spoken language understanding (SLU). In this paper, we propose to use recurrent neural networks (RNNs) for this task, and present several novel architectures designed to efficiently model past and future temporal dependencies. Specifically, we implemented and compared several important RNN architectures, including Elman, Jordan, and hybrid variants. To facilitate reproducibility, we implemented these networks with the publicly available Theano neural network toolkit and completed experiments on the well-known airline travel information system (ATIS) benchmark. In addition, we compared the approaches on two custom SLU data sets from the entertainment and movies domains. Our results show that the RNN-based models outperform the conditional random field (CRF) baseline by 2% in absolute error reduction on the ATIS benchmark. We improve the state-of-the-art by 0.5% in the Entertainment domain, and 6.7% for the movies domain.

Although NeuroCRF, an augmented Conditional Random Fields (CRF) model whose feature function is parameterized as a Feed-Forward Neural Network (FF NN) on word embeddings, has soundly outperformed traditional linear-chain CRF on many sequence labeling tasks, it is held back by the fact that FF NNs have a fixed input length and therefore cannot take advantage of the full input sentence. We propose to address this issue by replacing the FF NN with a Long Short-Term Memory (LSTM) NN, which can summarize an input of arbitrary length into a fixed dimension representation. The resulting model obtains F1=89.28 on WikiNER dataset, a significant improvement over the NeuroCRF baseline’s F1=87.58, which is already a highly competitive result.