Question: Adiabatic charging of capacitors through switched capacitor converters with various target voltages.

This paper explores the theoretical boundaries of two-phase switched-capacitor (SC) dc-dc converters, delving into the attainable performance metrics. For a specified number of capacitors, k, the paper enumerates the comprehensive range of achievable dc conversion ratios. Notably, the maximum step-up or step-down ratio is determined by the kth Fibonacci number, providing a clear benchmark for design purposes. Additionally, the paper establishes a cap on the number of switches necessary in any SC circuit, which is calculated as 3k^2. These findings have significant practical implications, including the potential for component reduction in specific applications and the capability to develop SC converters capable of maintaining output voltage regulation and high efficiency across a broad spectrum of input voltage fluctuations. The established limits on output resistance and efficiency serve as valuable tools for the selection and comparative analysis of SC converters, facilitating informed decision-making in engineering design.

**Question**: A hybrid algorithm for the bi-objective two-machine flow shop problem

**Documents**: A hybrid genetic algorithm for the bi-objective flexible flow shop problem
This paper presents a hybrid genetic algorithm for the bi-objective flexible flow shop problem. The algorithm combines a genetic algorithm with a local search procedure to improve the solutions found. The genetic algorithm is used to generate an initial population of solutions, which are then refined by the local search procedure. The local search procedure uses a set of neighborhood structures to explore the solution space and improve the solutions found by the genetic algorithm. The algorithm is tested on a set of benchmark instances and compared to other algorithms from the literature. The results show that the hybrid genetic algorithm is able to find solutions that are competitive with the best solutions found by other algorithms.

**Similar Documents**: This study introduces a hybrid genetic algorithm tailored for addressing the bi-objective flexible flow shop problem, integrating genetic algorithm principles with a local search mechanism to enhance solution quality. Initially, a genetic algorithm creates a population of potential solutions, which are subsequently optimized through the local search process. The local search employs a collection of neighborhood structures to navigate the solution space more effectively, refining solutions generated by the genetic algorithm. The algorithm's performance is evaluated against benchmark instances and compared with alternative methods documented in the literature. Experimental outcomes demonstrate that the proposed hybrid genetic algorithm yields competitive solutions, matching or surpassing the best results obtained by existing algorithms.

A High Efficiency two-stage AC/DC converter is presented and discussed in this paper, which can be applied in on-board charger for electric vehicles. The front-end converter is a high efficiency ZVS half bridge power factor correction (PFC) topology with only one device in the current flowing path. Owing to the high voltage stress of the half bridge rectifier, the SiC MOSFET is applied to reduce the conduction and switching losses and achieve high frequency. The downstream stage is a LLC resonant DC/DC converter. The peak efficiency reaches 97.5%.

This paper proposes a novel soft-switching interleaved boost converter composed of two shunted elementary boost conversion units and an auxiliary inductor. This converter is able to turn on both the active power switches at zero voltage to reduce their switching losses and evidently raise the conversion efficiency. Since the two parallel-operated elementary boost units are identical, operation analysis and design for the converter module becomes quite simple. A laboratory test circuit is built, and the circuit operation shows satisfactory agreement with the theoretical analysis. The experimental results show that this converter module performs very well with the output efficiency as high as 95%.

For pulsed power applications special pulsed voltage converters are needed. This paper presents an approach for a pulsed power converter, which generates bipolar output voltages up to 10 kV with extremely fast voltage slopes and high repetition rates. The topology is based on a H-bridge with a 10 kV dc-link. The output voltage and current of the pulsed voltage converter are adapted for the operation with Dielectric Barrier Discharge. To avoid the use of spark gaps due to their limited lifetime and thus to the lifetime of the converter, series stacked MOSFETs are used to realize a switch with a high blocking voltage. A balancing network for the series stacked MOSFETs is introduced as well as an adequate gate drive circuit. A matching method for the capacitive load is described, to achieve a maximum voltage slope at this capacitive load. To validate the theoretical considerations a prototype and measurements are presented.

A new pulsed-power generator based on a Marx generator (MG) is proposed in this paper with a reduced number of semiconductor components and with a more efficient load-supplying process. The main idea is to charge two groups of capacitors in parallel through an inductor and take advantage of the resonant phenomenon in charging each capacitor up to twice as the input-voltage level. In each resonant half-cycle, one of those capacitor groups is charged; eventually, the charged capacitors will be connected in series, and the summation of the capacitor voltages are appeared at the output of a pulsed-power converter. This topology can be considered as a modified MG, which works based on the resonant concept. The simulated models of this converter have been investigated in a MATLAB/Simulink platform, and a laboratory prototype has been implemented in a laboratory. The simulation and test results verify the operation of the proposed topology in different switching modes.

In this paper, four different power conversion methods (voltage control, duty-cycle control, frequency control and phase-shift control) are compared for wireless power transfer applications by considering the energy transfer efficiency, electromagnetic interference, stability, and implementation complexity. The phase-shift control is found to be the optimal scheme with good efficiency and low electromagnetic interference. Its constant frequency feature is also within the framework of the new international wireless charging standard called ‘Qi’. A high system efficiency of 72% for 5W wireless charging applications has been achieved practically.

This paper addresses voltage control of distributed dc power systems. DC power systems have been discussed as a result of the introduction of renewable, small-scale power generation units. Also, telecommunication power systems featuring UPS properties might benefit from a broader introduction of dc power systems. Droop control is utilized to distribute the load between the source converters. In order to make the loading of the source converters equal, in per unit, the voltage control algorithm for each converter has to be designed to act similar. The dc side capacitor of each converter, needed for filtering, is also determined as a consequence. The root locus is investigated for varying dc bus impedance. It is found that the risk of entering converter over-modulation is a stronger limitation than stability, at least for reasonable dc bus cable parameters. The stationary and dynamic properties during load variations are also investigated.

This paper discusses applying various switching strategies such as conventional complementary and proportional methods on a specific converter. This converter was recently presented and it can provide high voltage gain and has current ripple cancelation ability at a preselected duty-cycle. The input current ripple is zero when the converter works at a special duty-cycle. But load disturbance leads to output voltage changes that maybe cause the duty-cycle to deviate from its preselected value. In this situation, the input current ripple cannot be completely canceled. The proposed proportional strategy is an optimized method for minimizing input current ripple at other operating duty-cycles and also it provides a voltage gain lower than conventional complementary strategy. Here, the converter's performance is analyzed under two switching strategies and the effect of various strategies has been investigated on converter parameters including output voltage and input current ripple. These considerations are verified by implementing a 100-W prototype of the proposed converter in laboratory.

A new multiple-loop design technique for high-performance low-dropout (LDO) regulator designs has been proposed and successfully implemented in many commercial products for portable smart phone and tablet PC applications. The proposed LDO is composed of five loops that allows designers to obtain a good tradeoff between quiescent current and other performances, such as undershoot, overshoot, and so on. A total of one bandgap reference and 38 LDOs (including n-type and p-type LDOs, which will be named NLDO and PLDO, respectively) were integrated in one power-management IC chip for powering an application processor inside mobile devices. The proposed LDO has been fabricated based on 0.13-<inline-formula> <tex-math notation="LaTeX">$\mu {m}$ </tex-math></inline-formula> CMOS process and supplies various current capacities from 50 to 600 mA; 38 LDOs have been designed and supply different output voltage levels from 0.7 to 3.0 V. One of the proposed NLDOs consumes <inline-formula> <tex-math notation="LaTeX">$14~\mu {A}$ </tex-math></inline-formula> of the quiescent current and features under 56/24 mV of undershoot/overshoot at VOUT =1<italic>V</italic> as the load current steps up from 0 to 300 mA with 300 mA/1 <inline-formula> <tex-math notation="LaTeX">$\mu {s}$ </tex-math></inline-formula> on a 1-<inline-formula> <tex-math notation="LaTeX">$\mu {F}$ </tex-math></inline-formula> output capacitor. The measured output load and line regulations are 1.8 and 0.4 mV, respectively. The measured integrated output noise from 10 Hz to 100 kHz at <inline-formula> <tex-math notation="LaTeX">$\text{I}_{\mathrm{ LOAD}}=10\%$ </tex-math></inline-formula> of maximum current shows 80 <inline-formula> <tex-math notation="LaTeX">$\mu {V}$ </tex-math></inline-formula> rms. The package chip size is <inline-formula> <tex-math notation="LaTeX">$6.25 \times 6.25$ </tex-math></inline-formula> mm2 with 169 balls.

This paper presents the design of a highly efficient CMOS level shifter qc-level shifter. Unlike many recent level shifters, the proposed qc-level shifter does not use bootstrap capacitors to minimize active area. When implemented on a 65nm CMOS technology, under the large capacitive loading condition (2pF), qc-level shifter has a lower active area (94%), and energy-delay product (21.4%) than the reference bootstrap level shifter circuit (ts-level shifter). In comparison to a conventional shifter (c-level shifter)the corresponding reductions are 9.5% and 55%, respectively. Also qc-level shifter has very small effective input capacitance in comparison with ts-level shifter as it does not need a bootstrap capacitor connected to its input.

Switching regulators with ripple-based control (i.e., ¿ripple regulators¿) are conceptually simple, have fast transient responses to both line and load perturbations, and some versions operate with a switching frequency that is proportional to the load current under the discontinuous conduction mode. These characteristics make the ripple regulators well-suited, especially for power management applications in computers and portable electronic devices. Ripple regulators also have some drawbacks, including (in some versions) a poorly defined switching frequency, noise-induced jitter, inadequate dc regulation, and a tendency for fast-scale instability. This paper presents an overview of the various ripple-based control techniques, discusses their merits and limitations, and introduces techniques for reducing the noise sensitivity and the sensitivity to capacitor parameters, improving the frequency stability and the dc regulation, and avoiding fast-scale instability.


Question: Adiabatic charging of capacitors through switched capacitor converters with various target voltages.

This paper introduces an analytical framework for evaluating and optimizing the performance of switched-capacitor (SC) DC-DC converters. The method, which has been validated through simulation and experimental data, focuses on determining the steady-state operation by analyzing the output impedance. This straightforward approach facilitates the optimization of capacitor sizes to adhere to constraints like total capacitance or energy storage limits. Additionally, it allows for the optimization of switch sizes under conditions such as total switch conductance or total switch volt-ampere (V-A) product limits. These optimizations enable a comparative analysis among various SC topologies and with traditional magnetic-based DC-DC converter circuits, tailored to different application scenarios. Notably, the study reveals that a ladder-type SC converter outperforms conventional magnetic-based converters in terms of conduction loss for medium to high conversion ratios, showcasing the potential superiority of SC converters in specific applications.

In this paper, four different power conversion methods (voltage control, duty-cycle control, frequency control and phase-shift control) are compared for wireless power transfer applications by considering the energy transfer efficiency, electromagnetic interference, stability, and implementation complexity. The phase-shift control is found to be the optimal scheme with good efficiency and low electromagnetic interference. Its constant frequency feature is also within the framework of the new international wireless charging standard called ‘Qi’. A high system efficiency of 72% for 5W wireless charging applications has been achieved practically.

This paper presents an energy-efficient SAR ADC which adopts reduced MSB cycling step with dual sampling of the analog signal. By sampling and holding the analog signal asymmetrically at both input sides of comparator, the MSB cycling step can be hidden by hold mode. Benefits from this technique, not only the total capacitance of DAC is reduced by half, but also the average switching energy is reduced by 68% compared with conventional SAR ADC. Moreover, switching energy distribution is more uniform over entire output code compared with previous works.

A 1.5-V, 10-bit, 14.3-MS/s pipeline analog-to-digital converter was implemented in a 0.6m CMOS technology. Emphasis was placed on observing device reliability constraints at low voltage. MOS switches were implemented without lowthreshold devices by using a bootstrapping technique that does not subject the devices to large terminal voltages. The converter achieved a peak signal-to-noise-and-distortion ratio of 58.5 dB, maximum differential nonlinearity of 0.5 least significant bit (LSB), maximum integral nonlinearity of 0.7 LSB, and a power consumption of 36 mW.

Harvesting energy from ambient energy sources such as solar and thermal gradients is one solution to address the dramatic increase in energy consumption of personal electronics. In this paper, an ultra low quiescent current charger and battery management IC that can efficiently extract energy from solar panels and thermoelectric generators to charge batteries and super capacitors is presented. While previous works on energy harvesting [1-5] report efficient DC-DC converters to extract the energy from the harvester, not all of them provide complete battery management functionality for various chemistries. Also, maximum power point tracking (MPPT) which is critical in energy harvesting applications is not provided in some of the published work. In this paper, a charger and battery management IC with 330nA quiescent current is presented. The IC can cold start from 330mV and 5μW of input power. The charger achieves efficiency greater than 80% at single cell solar voltages of 0.5V. A low quiescent current battery management architecture involving sampled circuits, sub regulated rails and clock gating is demonstrated.

A novel three-level (3L) bidirectional converter (BDC) is proposed in this paper. Compared with the traditional BDC, the inductor of the 3L BDC can be reduced significantly so that the dynamic response is greatly improved. Hence, the proposed converter is very suitable for fuel-cell/battery hybrid power systems. In addition, the voltage stress on the switch of the proposed converter is only half of the voltage on the high-voltage side, so it is also suitable for high-voltage applications. The operation principle and the implementation of the control circuit are presented in detail. This paper also proposes a novel bidirectional soft-start control strategy for the BDC. A 1-kW prototype converter is built to verify the theoretical analysis.

The narrow duty cycle in the buck converter limits its application for high-step-down dc-dc conversion. With a simple structure, the tapped-inductor buck converter shows promise for extending the duty cycle. However, the leakage inductance causes a huge turn-off voltage spike across the top switch. Also, the gate drive for the top switch is not simple due to its floating source connection. This paper solves all these problems by modifying the tapped-inductor structure. A simple lossless clamp circuit can effectively clamp the switch turn-off voltage spike and totally recover the leakage energy. Experimental results for 12V-to-1.5V and 48V-to-6V dc-dc conversions show significant improvements in efficiency.

Switching regulators with ripple-based control (i.e., ¿ripple regulators¿) are conceptually simple, have fast transient responses to both line and load perturbations, and some versions operate with a switching frequency that is proportional to the load current under the discontinuous conduction mode. These characteristics make the ripple regulators well-suited, especially for power management applications in computers and portable electronic devices. Ripple regulators also have some drawbacks, including (in some versions) a poorly defined switching frequency, noise-induced jitter, inadequate dc regulation, and a tendency for fast-scale instability. This paper presents an overview of the various ripple-based control techniques, discusses their merits and limitations, and introduces techniques for reducing the noise sensitivity and the sensitivity to capacitor parameters, improving the frequency stability and the dc regulation, and avoiding fast-scale instability.

This paper addresses voltage control of distributed dc power systems. DC power systems have been discussed as a result of the introduction of renewable, small-scale power generation units. Also, telecommunication power systems featuring UPS properties might benefit from a broader introduction of dc power systems. Droop control is utilized to distribute the load between the source converters. In order to make the loading of the source converters equal, in per unit, the voltage control algorithm for each converter has to be designed to act similar. The dc side capacitor of each converter, needed for filtering, is also determined as a consequence. The root locus is investigated for varying dc bus impedance. It is found that the risk of entering converter over-modulation is a stronger limitation than stability, at least for reasonable dc bus cable parameters. The stationary and dynamic properties during load variations are also investigated.

The series resonant converter is analyzed in steady state, and for constant switching frequency the output current and voltage characteristics are found to be ellipses. The converter operating point can then be easily obtained by superimposing a load line on these elliptical characteristics. Peak resonant capacitor voltage and inductor current are also plotted in the output plane and are dependent to first order only on output current. When peak voltage and current are plotted in this manner, the dependence of component stresses on operating point is clearly revealed. The output characteristics are modified to include the effect of transistor and diode voltage drops, and experimental verification is presented.

This paper proposes a novel soft-switching interleaved boost converter composed of two shunted elementary boost conversion units and an auxiliary inductor. This converter is able to turn on both the active power switches at zero voltage to reduce their switching losses and evidently raise the conversion efficiency. Since the two parallel-operated elementary boost units are identical, operation analysis and design for the converter module becomes quite simple. A laboratory test circuit is built, and the circuit operation shows satisfactory agreement with the theoretical analysis. The experimental results show that this converter module performs very well with the output efficiency as high as 95%.


Question: Adiabatic charging of capacitors through switched capacitor converters with various target voltages.

This paper presents an overview of the advancements in Switched-Capacitor (SC) Power Electronics Circuits, a field that has seen significant growth in the past decade. The focus on developing switching-mode converters devoid of inductors and transformers has been driven by the demands for lightweight, compact size, and high power density in power supplies, particularly for portable electronic devices like smartphones and PDAs. SC converters, with their capability to achieve substantial voltage conversion ratios, are emerging as a viable solution for the 21st century's power electronics challenges, including high-efficiency conversion with low EMI emissions, significant voltage step-down for integrated circuits, and steep voltage step-up for sectors such as automotive and telecom.

The tutorial covers the key outcomes in SC-converter research and design, highlighting the significant role of the Circuits and Systems (CAS) Society in contributing to this area. Researchers in this domain predominantly belong to the CAS Society's technical committee, with numerous groundbreaking contributions published in CAS publications. The rationale behind this is rooted in the CAS Society's expertise in circuit theory, which in the 1950s, led to the development of active and SC filters as alternatives to traditional magnetic devices in passive filters.

The evolution of SC converters addressed the power electronics designer's challenge of miniaturizing the power supply by eliminating inductors and transformers. The initial hurdle of low efficiency in charging capacitors was overcome through a decade of research, culminating in the development of high-efficiency SC energy-processing circuits.

The first SC converters, developed by a Japanese research team, aimed to regulate a DC voltage from an unregulated source. These initial DC-DC converters were soon complemented by AC-DC, DC-AC, and AC-AC converters, showcasing the potential for high power density in a hybrid technology format. The foundational principle of SC converters involves using a control element to cyclically alter the converter's topology, adjusting the durations of the switching states for regulation. This approach enables the converter to maintain a constant output voltage despite fluctuations in input voltage or load conditions.

In summary, the advancement in SC converter technology represents a significant milestone in power electronics, offering solutions for miniaturization and efficiency that are crucial for modern electronic systems. The contributions from the CAS Society have been pivotal in shaping this field, demonstrating the potential for further innovation in energy processing and power supply design.

In this paper, a new low-energy switching technique with low common-mode voltage variation is proposed for successive-approximation analog-to-digital converters (SA-ADCs). In the proposed scheme, not only the switching energy consumed within the first three comparisons is less than zero, but also other comparisons are made with the low-power monotonic method. Therefore, the switching energy of the capacitive array, including the consumed energy during the sampling phase, is reduced by 90.68% compared with the conventional counterpart. Moreover, since the variation of the input common-mode voltage of the employed comparator is only 0.125Vref, where Vref is the reference voltage of the ADC, the required comparator’s performance can be much more relaxed leading to more power saving. Post-layout simulation results of a 10-bit 1-MS/s SA-ADC in a 0.18-μm CMOS technology show a signal-to-noise-and-distortion ratio (SNDR) of 61 dB, a spurious-free dynamic range (SFDR) of 79.8 dB, and an effective number of 9.84 bits. The ADC consumes 35.3 μW with a 1.8-V supply and achieves a Figure-of-Merit (FoM) of 38.5 fJ/conversion-step.

This paper investigates the soft start-up for high frequency LLC resonant converter with optimal trajectory control. Two methods are proposed to realize soft start-up for high frequency LLC converter by commercial low-cost microcontrollers (MCU). Both methods can achieve soft start-up with minimum stress and optimal energy delivery. One method is mixed-signal implementation by sensing resonant tank to minimize the digital delay. Another method is digital implementation with look-up table. Experimental results are demonstrated on 500kHz 1kW 400V/12V LLC converter.

This paper presents a multi-MHz buck regulator for portable applications using an auto-selectable peak- and valley-current control (ASPVCC) scheme. The proposed ASPVCC scheme can enable the current-mode buck regulator to reduce the settling-time requirement of the current sensing by two times. In addition, the dynamically biased shunt feedback technique is employed to improve the sensing speed and the sensing accuracy of both the peak and valley current sensors. With both ASPVCC scheme and advanced current sensors, the buck regulator can thus operate at high switching frequencies with a wide range of duty ratios for reducing the required off-chip inductance. The proposed current-mode buck regulator was fabricated in a standard 0.35-μm CMOS process and occupies a small chip area of 0.54 mm2. Measurement results show that the buck regulator can deliver a maximum output current of 500 mA, operate at 5 MHz with a wide duty-ratio range of about 0.6, use a small-value off-chip inductor of 1 μH, and achieve the peak power efficiency of 91%.

In the transportation field, the empty weight of the vehicle has to be as light as possible in order to have the highest energetic efficiency. For onboard storage electrical systems, DC/DC converters are usually used to control the power flows in the vehicle. This paper focuses on two different DC/DC converter applications: high current/low voltage on the one hand and low current/high voltage on the other hand. For all power mobile applications, a standard dc/dc converter is not realizable due to mass and volume constraints. As the inductor is usually the biggest and heaviest component in the system, it is important to develop a new structure allowing the reduction of its size and weight. The chosen technology is based on interleaved channels operating in discontinuous mode, also using reverse conduction of power MOSFET transistors. For both of the cases described above, a prototype has been built composed of 8 and 4 channels respectively. The first converter is used for power assistance (250 A/18 V) using the MOS reverse conduction property. In this particular case, efficiency is up to 96.3 %. The second is developed for a mobile solar application (3.8 A/270 V). Efficiencies are up to 98.4 %

A voltage-to-time converter (VTC) is presented for use in a time-based analog-to-digital converter (ADC). The converter runs with a 5 GHz input clock to provide a maximum conversion rate of 5 GS/s. A novel architecture enables the VTC to provide an adjustable linear delay versus input voltage characteristic. The circuit is realized in a 90nm CMOS process. After calibration, the converter achieves better than 3.7 effective bits for input frequencies up to 1.75 GHz, making it suitable for use in a time-based ADC with up to 4-bit resolution.

AC/DC converters for worldwide deployment or multiple applications are often required to operate with single-or three-phase AC connections at a variety of voltages. Electric vehicle charging is an excellent example: a typical AC/DC converter optimized for a 400 V three-phase European connection will only produce about 35% of rated power when operating on a 240 V single-phase connection in North America. This paper proposes a reconfigurable AC-DC converter with integral DC-DC stage that allows the single-phase AC power to roughly double compared to a standard configuration, up to 2/3 of the DC/DC stage rating. The system is constructed with two conventional back-to-back three-phase two-level bridges. When operating in single-phase mode, rather than leaving the DC side under-utilized, our proposed design repurposes one of the DC phase legs and the unused AC phase to create two parallel single-phase connections. This roughly doubles the AC output current and hence doubles the single phase AC power.

High voltage gain dc-dc converters are required in many industrial applications such as photovoltaic and fuel cell energy systems, high-intensity discharge lamp (HID), dc back-up energy systems, and electric vehicles. This paper presents a novel input-parallel output-series boost converter with dual coupled inductors and a voltage multiplier module. On the one hand, the primary windings of two coupled inductors are connected in parallel to share the input current and reduce the current ripple at the input. On the other hand, the proposed converter inherits the merits of interleaved series-connected output capacitors for high voltage gain, low output voltage ripple, and low switch voltage stress. Moreover, the secondary sides of two coupled inductors are connected in series to a regenerative capacitor by a diode for extending the voltage gain and balancing the primary-parallel currents. In addition, the active switches are turned on at zero current and the reverse recovery problem of diodes is alleviated by reasonable leakage inductances of the coupled inductors. Besides, the energy of leakage inductances can be recycled. A prototype circuit rated 500-W output power is implemented in the laboratory, and the experimental results shows satisfactory agreement with the theoretical analysis.

The amount of stored energy is an important issue in the low-droop and high-precision pulse generation. Pulse forming network (PFN) simulating an open-ended transmission line seems to be useful due to its minimum stored energy. The drawbacks of the PFN include low-output pulse quality and constant pulsewidth, which limit their applications. Another limitation of generating rectangular pulses using PFN is that half of the charged voltage appears in the output when delivering the entire stored energy to the matched load. This paper proposes a structure of PFN-based pulse generator that overcomes the above-mentioned limitations. For continuous pulsewidth adjustment, a lossless scheme is presented. By increasing the ratio of the load magnitude to the characteristic impedance, the amplitude of the output pulse voltage would be approximately equal to that of the charged voltage. Therefore, the semiconductor switches can be utilized in the PFN pulse generator easily. In addition, the pulse quality is enhanced and faster rising time is obtained in comparison with the conventional PFNs. In this proposal, the stored energy quantity is limited to three times the energy needed for the output pulse in the case of  $R_{L}=10Z_{0}$ . To verify the proposed structure, a prototype is simulated using PSPICE software, and the experimental results are presented.

Corresponding Author: Michelle Lim Sern Mi Institute of Microengineering and Nanoelectronics (IMEN), UKM, 43600 Bangi, Selangor, Malaysia Email: mlsm_2002@yahoo.com Abstract: This paper reviews CMOS based charge pump topologies used within autonomous embedded micro-systems. These charge pump structures have evolved from its simplistic diode-tied, single-branches with major threshold drops to exponential type, dual-branches with sophisticated gate and substrate control for lower voltage operation. Published charge pumps are grouped based on architecture, operation principles and pump optimization techniques with their pros and cons compared and results contrasted. The various charge pump topologies and schemes used are considered based on pumping efficiency, power efficiency, charge transferability, circuit complexity, pumping capacitors, form factor and minimum supply voltages with an optimum load. This article concludes with an overview of suitable techniques and recommendations that will aid a designer in selecting the most suitable charge pump topology especially for low ambient micro energy harvesting applications.

A new active clamping zero-voltage switching (ZVS) pulse-width modulation (PWM) current-fed half-bridge converter (CFHB) is proposed in this paper. Its active clamping snubber (ACS) can not only absorb the voltage surge across the turned-off switch, but also achieve the ZVS of all power switches. Moreover, it can be applied to all current-fed power conversion topologies and its operation as well as structure is very simple. Since auxiliary switches in the snubber circuit are switched in a complementary way to main switches, an additional PWM IC is not necessary. In addition, it does not need any clamp winding and auxiliary circuit besides additional two power switches and one capacitor while the conventional current-fed half bridge converter has to be equipped with two clamp windings, two ZVS circuits, and two snubbers. Therefore, it can ensure the higher operating frequency, smaller-sized reactive components, lower cost of production, easier implementation, and higher efficiency. The operational principle, theoretical analysis, and design considerations are presented. To confirm the operation, validity, and features of the proposed circuit, experimental results from a 200-W, 24-200Vdc prototype are presented.


Question: Adiabatic charging of capacitors through switched capacitor converters with multiple target voltage levels.

This paper introduces a comprehensive analysis of a novel class of switched-capacitor resonant converters that utilize a minimalistic design approach, incorporating just two transistors. The distinctive feature of these circuits is their operation under zero-current switching conditions, thereby eliminating switching losses entirely. This not only enhances efficiency but also extends the operational lifespan of the components. Additionally, the circuits offer a versatile range of voltage conversion capabilities, encompassing both fractional and multiple voltage conversion ratios, as well as inverted voltage transformations. This flexibility makes them suitable for a wide array of applications, particularly in scenarios requiring precise and efficient voltage control.

This paper presents a multi-objective Model Predictive Control (MPC) for grid-tied 4-cell 15-level Packed U Cells (PUC) inverter. This challenging topology is characterized by high power quality with reduced number of components compared to conventional multilevel inverters. Compared to traditional PI controller, MPC is attracting more interest due to its good dynamic response and high accuracy of reference tracking, through the minimization of a flexible user-defined cost function. For the presented PUC topology, the grid current should be jointly controlled with the capacitors' voltages for ensuring proper operation of the inverter, which leads to an additional requirement of pre-charge circuits for the capacitors in case of using PI current controller (or using additional PI controllers). The proposed MPC achieves grid current injection, low current THD, unity PF, while balancing the capacitors' voltages.

This paper proposes new step-up DC-DC converters with high ratio capability. The proposed DC-DC converters are derived using combination of boost converters and buck-boost converters. A high ratio capability is achieved by parallel input and series output combination so that the efficiency is better than the one that is achieved by using a conventional boost converter. A method to reduce the input and output ripples are also proposed in this paper. Simulated and experimental results are included to show the validity of the proposed converters.

Harvesting energy from ambient energy sources such as solar and thermal gradients is one solution to address the dramatic increase in energy consumption of personal electronics. In this paper, an ultra low quiescent current charger and battery management IC that can efficiently extract energy from solar panels and thermoelectric generators to charge batteries and super capacitors is presented. While previous works on energy harvesting [1-5] report efficient DC-DC converters to extract the energy from the harvester, not all of them provide complete battery management functionality for various chemistries. Also, maximum power point tracking (MPPT) which is critical in energy harvesting applications is not provided in some of the published work. In this paper, a charger and battery management IC with 330nA quiescent current is presented. The IC can cold start from 330mV and 5μW of input power. The charger achieves efficiency greater than 80% at single cell solar voltages of 0.5V. A low quiescent current battery management architecture involving sampled circuits, sub regulated rails and clock gating is demonstrated.

In this paper, a new current source gate drive circuit is proposed for high efficiency synchronous buck VRMs. The proposed circuit achieves quick turn-on and turn-off transition times to reduce switching loss and conduction loss in power MOSFETS. The driver circuit consists of two sets of four control switches and two very small inductors (typically 50 nH-300 nH each at 1 MHz). It drives both the control MOSFET and synchronous MOSFET in synchronous buck VRMs. An analysis, design procedure, optimization procedure and experimental results are presented for the proposed circuit. Experimental results demonstrate an efficiency of 86.6% at 15 A load and 81.9% at 30 A load for 12 V input and 1.3 V output at 1 MHz.

This work proposes a 12 b 10 MS/s 0.11 lm CMOS successive-approximation register ADC based on a C-R hybrid DAC for low-power sensor applications. The proposed C-R DAC employs a 2-step split-capacitor array of upper seven bits and lower five bits to optimize power consumption and chip area at the target speed and resolution. A VCM-based switching method for the most significant bit and reference voltage segments from an insensitive R-string for the last two least significant bits minimize the number of unit capacitors required in the C-R hybrid DAC. The comparator accuracy is improved by an open-loop offset cancellation technique in the first-stage pre-amp. The prototype ADC in a 0.11 lm CMOS process demonstrates the measured differential nonlinearity and integral nonlinearity within 1.18 LSB and 1.42 LSB, respectively. The ADC shows a maximum signal-to-noise-and-distortion ratio of 63.9 dB and a maximum spurious-free dynamic range of 77.6 dB at 10 MS/s. The ADC with an active die area of 0.34 mm consumes 1.1 mW at 1.0 V and 10 MS/s, corresponding to a figure-of-merit of 87 fJ/conversion-step.

This paper presents a multi-input battery charging system that is capable of increasing the charging efficiency of lithium-ion (Li-ion) batteries. The proposed battery charging system consists of three main building blocks: a pulse charger, a step-down dc–dc converter, and a power path controller. The pulse charger allows charging via a wall outlet or an energy harvesting system. It implements charge techniques that increase the battery charge efficiency of a Li-ion battery. The power path controller (PPC) functions as a power monitor and selects the optimal path for charging either via an energy harvesting system or an ac adapter. The step-down dc–dc converter provides an initial supply voltage to start up the energy harvesting system. The integrated circuit design is implemented on a CMOS 0.18 μm technology process. Experimental results verify that the proposed pulse charger reduces the charging time of 100 mAh and 45 mAh Li-ion batteries respectively by 37.35% and 15.56% and improves the charge efficiency by 3.15% and 3.27% when compared to the benchmark constant current-constant voltage charging technique. The step-down dc–dc converter has a maximum efficiency of 90% and the operation of the PPC is also verified by charging the battery via a thermoelectric energy harvesting system.

The series resonant converter is analyzed in steady state, and for constant switching frequency the output current and voltage characteristics are found to be ellipses. The converter operating point can then be easily obtained by superimposing a load line on these elliptical characteristics. Peak resonant capacitor voltage and inductor current are also plotted in the output plane and are dependent to first order only on output current. When peak voltage and current are plotted in this manner, the dependence of component stresses on operating point is clearly revealed. The output characteristics are modified to include the effect of transistor and diode voltage drops, and experimental verification is presented.

In this paper, a novel positive level shifter is designed for high speed interfacing between the digital circuits operating at 1.2V and the memory core circuits operating at high voltages in flash memories. The proposed design is also validated for a wide range of output voltage levels and has been optimized to perform efficiently across the different process corners and temperatures. For the targeted design of 1.2V and 33 MHz input signal to 4V output signal, the level shifter has a symmetric switching speed with a rise time and fall time of 2ns and 1.89ns respectively while the average power consumption of 0.056mW is obtained in the typical condition. Simulation results are compared with various other conventional positive level shifters at different process corners and temperatures.

This paper addresses voltage control of distributed dc power systems. DC power systems have been discussed as a result of the introduction of renewable, small-scale power generation units. Also, telecommunication power systems featuring UPS properties might benefit from a broader introduction of dc power systems. Droop control is utilized to distribute the load between the source converters. In order to make the loading of the source converters equal, in per unit, the voltage control algorithm for each converter has to be designed to act similar. The dc side capacitor of each converter, needed for filtering, is also determined as a consequence. The root locus is investigated for varying dc bus impedance. It is found that the risk of entering converter over-modulation is a stronger limitation than stability, at least for reasonable dc bus cable parameters. The stationary and dynamic properties during load variations are also investigated.

This paper presents an energy-efficient SAR ADC which adopts reduced MSB cycling step with dual sampling of the analog signal. By sampling and holding the analog signal asymmetrically at both input sides of comparator, the MSB cycling step can be hidden by hold mode. Benefits from this technique, not only the total capacitance of DAC is reduced by half, but also the average switching energy is reduced by 68% compared with conventional SAR ADC. Moreover, switching energy distribution is more uniform over entire output code compared with previous works.


Question: Question Passage Inquiry & Pathway Encoding Question-Passage Linkage Passage Intra-Matching Word Character Response Forecasting

This paper presents a novel global logbilinear regression model for generating word representations, which we name Global Vectors for Word Representation (GloVe). Unlike previous methods, our model explicitly captures the origin of fine-grained semantic and syntactic regularities in vector arithmetic, making the underlying model properties transparent. By combining the strengths of global matrix factorization and local context window techniques, GloVe efficiently utilizes statistical information by focusing on the nonzero elements in a word-word cooccurrence matrix, rather than processing the entire sparse matrix or individual context windows from a large corpus. This approach results in a vector space with a meaningful substructure, as evidenced by its impressive performance of 75% on a recent word analogy task. Furthermore, GloVe outperforms competing models in similarity tasks and named entity recognition, showcasing its superiority in capturing the intricacies of language.

This study compared the nature of text comprehension as measured by multiple-choice format and open-ended format questions. Participants read a short text while explaining preselected sentences. After reading the text, participants answered open-ended and multiple-choice versions of the same questions based on their memory of the text content. The results indicated that performance on open-ended questions was correlated with the quality of self-explanations, but performance on multiple-choice questions was correlated with the level of prior knowledge related to the text. These results suggest that open-ended and multiple-choice format questions measure different aspects of comprehension processes. The results are discussed in terms of dual process theories of text comprehension.

Vector-space word representations have been very successful in recent years at improving performance across a variety of NLP tasks. However, common to most existing work, words are regarded as independent entities without any explicit relationship among morphologically related words being modeled. As a result, rare and complex words are often poorly estimated, and all unknown words are represented in a rather crude way using only one or a few vectors. This paper addresses this shortcoming by proposing a novel model that is capable of building representations for morphologically complex words from their morphemes. We combine recursive neural networks (RNNs), where each morpheme is a basic unit, with neural language models (NLMs) to consider contextual information in learning morphologicallyaware word representations. Our learned models outperform existing word representations by a good margin on word similarity tasks across many datasets, including a new dataset we introduce focused on rare words to complement existing ones in an interesting way.

Question answering (QA) has become a popular way for humans to access billion-scale knowledge bases. QA systems over knowledge bases produce accurate and concise answers. The key of QA over knowledge bases is to map the question to a certain substructure in the knowledge base. To do this, KBQA (Question Answering over Knowledge Bases) uses a new kind of question representation: templates, learned from a million scale QA corpora. For example, for questions about a city’s population, KBQA learns templates such as What’s the population of $city?, How many people are there in $city?. It learns overall 1171303 templates for 4690 relations. Based on these templates, KBQA effectively and efficiently supports binary factoid questions or complex questions.

In a conversation or a dialogue process, attention and intention play intrinsic roles. This paper proposes a neural network based approach that models the attention and intention processes. It essentially consists of three recurrent networks. The encoder network is a word-level model representing source side sentences. The intention network is a recurrent network that models the dynamics of the intention process. The decoder network is a recurrent network that produces responses to the input from the source side. It is a language model that is dependent on the intention and has an attention mechanism to attend to particular source side words, when predicting a symbol in the response. The model is trained end-toend without labeling data. Experiments show that this model generates natural responses to user inputs.

There are two main types of word representations: low-dimensional embeddings and high-dimensional distributional vectors, in which each dimension corresponds to a context word. In this paper, we initialize an embedding-learning model with distributional vectors. Evaluation on word similarity shows that this initialization significantly increases the quality of embeddings for rare words.

Automatic question generation (QG) is a useful yet challenging task in NLP. Recent neural network-based approaches represent the state-of-the-art in this task, but they are not without shortcomings. Firstly, these models lack the ability to handle rare words and the word repetition problem. Moreover, all previous works optimize the cross-entropy loss, which can induce inconsistencies between training (objective) and testing (evaluation measure). In this paper, we present a novel deep reinforcement learning based framework for automatic question generation. The generator of the framework is a sequence-to-sequence model, enhanced with the copy mechanism to handle the rare-words problem and the coverage mechanism to solve the word repetition problem. The evaluator model of the framework evaluates and assigns a reward to each predicted question. The overall model is trained by learning the parameters of the generator network which maximizes the reward. Our framework allows us to directly optimize any task-specific score including evaluation measures such as BLEU, GLEU, ROUGE-L, etc., suitable for sequence to sequence tasks such as QG. Our comprehensive evaluation shows that our approach significantly outperforms state-of-the-art systems on the widely-used SQuAD benchmark in both automatic and human evaluation.

Although NeuroCRF, an augmented Conditional Random Fields (CRF) model whose feature function is parameterized as a Feed-Forward Neural Network (FF NN) on word embeddings, has soundly outperformed traditional linear-chain CRF on many sequence labeling tasks, it is held back by the fact that FF NNs have a fixed input length and therefore cannot take advantage of the full input sentence. We propose to address this issue by replacing the FF NN with a Long Short-Term Memory (LSTM) NN, which can summarize an input of arbitrary length into a fixed dimension representation. The resulting model obtains F1=89.28 on WikiNER dataset, a significant improvement over the NeuroCRF baseline’s F1=87.58, which is already a highly competitive result.

Existing knowledge-based question answering systems often rely on small annotated training data. While shallow methods like relation extraction are robust to data scarcity, they are less expressive than the deep meaning representation methods like semantic parsing, thereby failing at answering questions involving multiple constraints. Here we alleviate this problem by empowering a relation extraction method with additional evidence from Wikipedia. We first present a neural network based relation extractor to retrieve the candidate answers from Freebase, and then infer over Wikipedia to validate these answers. Experiments on the WebQuestions question answering dataset show that our method achieves an F1 of 53.3%, a substantial improvement over the state-of-the-art.

Nowadays, neural networks play an important role in the task of relation classification. In this paper, we propose a novel attention-based convolutional neural network architecture for this task. Our model makes full use of word embedding, part-of-speech tag embedding and position embedding information. Word level attention mechanism is able to better determine which parts of the sentence are most influential with respect to the two entities of interest. This architecture enables learning some important features from task-specific labeled data, forgoing the need for external knowledge such as explicit dependency structures. Experiments on the SemEval-2010 Task 8 benchmark dataset show that our model achieves better performances than several stateof-the-art neural network models and can achieve a competitive performance just with minimal feature engineering.

While natural languages are compositional, how state-of-the-art neural models achieve compositionality is still unclear. We propose a deep network, which not only achieves competitive accuracy for text classification, but also exhibits compositional behavior. That is, while creating hierarchical representations of a piece of text, such as a sentence, the lower layers of the network distribute their layer-specific attention weights to individual words. In contrast, the higher layers compose meaningful phrases and clauses, whose lengths increase as the networks get deeper until fully composing the sentence.


Question: Question Encoding Passage Question Matching Self-Matching of the Passage Word Character Answer Prediction

This paper presents Structural Embedding of Syntactic Trees (SEST), a novel framework designed to integrate structured linguistic information, such as constituency and dependency trees, into deep neural network models for machine comprehension. Unlike conventional approaches that solely rely on word or character embeddings, SEST explicitly leverages syntactic structure to enhance vector representations. This incorporation of syntactic information significantly boosts the performance of machine comprehension algorithms. To validate the efficacy of SEST, we applied it to a cutting-edge neural attention model and tested it on the SQuAD dataset. Our findings reveal that SEST enables the model to precisely pinpoint syntactic boundaries within sentences and extract answers that are syntactically consistent, outperforming baseline methods. This advancement underscores the importance of structured linguistic data in improving the accuracy of machine comprehension systems.

We propose Neural Responding Machine (NRM), a neural networ k-based response generator for Short-Text Conversation. NRM takes the general encoder-de coder framework: it formalizes the generation of response as a decoding process based on the lat ent representation of the input text, while both encoding and decoding are realized with recurren t n ural networks (RNN). The NRM is trained with a large amount of one-round conversation dat a collected from a microblogging service. Empirical study shows that NRM can generate gramma tically correct and content-wise appropriate responses to over 75% of the input text, outperf orming state-of-the-arts in the same setting, including retrieval-based and SMT-based models.

Joint representation learning of text and knowledge within a unified semantic space enables us to perform knowledge graph completion more accurately. In this work, we propose a novel framework to embed words, entities and relations into the same continuous vector space. In this model, both entity and relation embeddings are learned by taking knowledge graph and plain text into consideration. In experiments, we evaluate the joint learning model on three tasks including entity prediction, relation prediction and relation classification from text. The experiment results show that our model can significantly and consistently improve the performance on the three tasks as compared with other baselines.

Human generates responses relying on semantic and functional dependencies, including coreference relation, among dialogue elements and their context. In this paper, we investigate matching a response with its multi-turn context using dependency information based entirely on attention. Our solution is inspired by the recently proposed Transformer in machine translation (Vaswani et al., 2017) and we extend the attention mechanism in two ways. First, we construct representations of text segments at different granularities solely with stacked self-attention. Second, we try to extract the truly matched segment pairs with attention across the context and response. We jointly introduce those two kinds of attention in one uniform neural network. Experiments on two large-scale multi-turn response selection tasks show that our proposed model significantly outperforms the state-of-the-art models.

We consider the ROC story cloze task (Mostafazadeh et al., 2016) and present several findings. We develop a model that uses hierarchical recurrent networks with attention to encode the sentences in the story and score candidate endings. By discarding the large training set and only training on the validation set, we achieve an accuracy of 74.7%. Even when we discard the story plots (sentences before the ending) and only train to choose the better of two endings, we can still reach 72.5%. We then analyze this “ending-only” task setting. We estimate human accuracy to be 78% and find several types of clues that lead to this high accuracy, including those related to sentiment, negation, and general ending likelihood regardless of the story context.

Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering. One such architecture, the dynamic memory network (DMN), obtained high accuracy on a variety of language tasks. However, it was not shown whether the architecture achieves strong results for question answering when supporting facts are not marked during training or whether it could be applied to other modalities such as images. Based on an analysis of the DMN, we propose several improvements to its memory and input modules. Together with these changes we introduce a novel input module for images in order to be able to answer visual questions. Our new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the bAbI-10k text question-answering dataset without supporting fact supervision.

This paper describes an ensemble system submitted as part of the LSDSem Shared Task 2017 the Story Cloze Test. The main conclusion from our results is that an approach based on semantic similarity alone may not be enough for this task. We test various approaches and compare them with two ensemble systems. One is based on voting and the other on logistic regression based classifier. Our final system is able to outperform the previous state of the art for the Story Cloze test. Another very interesting observation is the performance of sentiment based approach which works almost as well on its own as our final en-

Knowing the quality of reading comprehension (RC) datasets is important for the development of natural-language understanding systems. In this study, two classes of metrics were adopted for evaluating RC datasets: prerequisite skills and readability. We applied these classes to six existing datasets, including MCTest and SQuAD, and highlighted the characteristics of the datasets according to each metric and the correlation between the two classes. Our dataset analysis suggests that the readability of RC datasets does not directly affect the question difficulty and that it is possible to create an RC dataset that is easy to read but difficult to answer.

Most tasks in natural language processing can be cast into question answering (QA) problems over language input. We introduce the dynamic memory network (DMN), a unified neural network framework which processes input sequences and questions, forms semantic and episodic memories, and generates relevant answers. Questions trigger an iterative attention process which allows the model to condition its attention on the result of previous iterations. These results are then reasoned over in a hierarchical recurrent sequence model to generate answers. The DMN can be trained end-to-end and obtains state of the art results on several types of tasks and datasets: question answering (Facebook’s bAbI dataset), sequence modeling for part of speech tagging (WSJ-PTB), coreference resolution (Quizbowl dataset) and text classification for sentiment analysis (Stanford Sentiment Treebank). The model relies exclusively on trained word vector representations and requires no string matching or manually engineered features.

This paper presents the results of systematic experimentation on the impact in duplicate question detection of different types of questions across both a number of established approaches and a novel, superior one used to address this language processing task. This study permits to gain a novel insight on the different levels of robustness of the diverse detection methods with respect to different conditions of their application, including the ones that approximate real usage scenarios.

In this paper, we propose a bi-attention, a multi-layer attention and an attention mechanism and convolution neural network based text representation and classification model (ACNN). The bi-attention have two attention mechanism to learn two context vectors, forward RNN with attention to learn forward context vector c⃗ $\overrightarrow {\mathbf {c}}$ and backward RNN with attention to learn backward context vector c⃖ $\overleftarrow {\mathbf {c}}$, and then concatenation c⃗ $\overrightarrow {\mathbf {c}}$ and c⃖ $\overleftarrow {\mathbf {c}}$ to get context vector c. The multi-layer attention is the stack of the bi-attention. In the ACNN, the context vector c is obtained by the bi-attention, then the convolution operation is performed on the context vector c, and the max-pooling operation is used to reduce the dimension. After max-pooling operation the text is converted to low-dimensional sentence vector m. Finally, the Softmax classifier be used for text classification. We test our model on 8 benchmarks text classification datasets, and our model achieved a better or the same performance compare with the state-of-the-art methods.


Question: Question Passage Inquiry & Passage Encoding Question-Passage Alignment Passage Self-Alignment Word Character Response Forecasting

In this work, we explore a novel approach to reading comprehension that involves understanding complex biological processes described in text. Unlike much of the current research which focuses on extracting factual information from large, redundant corpora, our method targets a specific task that necessitates deep reasoning about a single document. The challenge is to accurately answer questions based on a detailed comprehension of the relationships between entities and events within a biological process described in the text. Our solution involves two primary steps: first, we predict a comprehensive, structured representation of the biological process as depicted in the paragraph. This structure captures the intricate interactions and dependencies between the various components of the process. Subsequently, we translate each question into a formal query, which is then executed against the predicted structure. Our results show that utilizing these predicted structures for question answering significantly outperforms baseline methods that rely on more superficial textual representations. This approach not only enhances the precision of answers but also underscores the importance of a deep understanding of text for effective reading comprehension tasks.

We report on a series of experiments with convolutional neural networks (CNN) trained on top of pre-trained word vectors for sentence-level classification tasks. We show that a simple CNN with little hyperparameter tuning and static vectors achieves excellent results on multiple benchmarks. Learning task-specific vectors through fine-tuning offers further gains in performance. We additionally propose a simple modification to the architecture to allow for the use of both task-specific and static vectors. The CNN models discussed herein improve upon the state of the art on 4 out of 7 tasks, which include sentiment analysis and question classification.

Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks. But, these tasks only evaluate lexical semantics indirectly. In this paper, we study whether it is possible to utilize distributed representations to generate dictionary definitions of words, as a more direct and transparent representation of the embeddings’ semantics. We introduce definition modeling, the task of generating a definition for a given word and its embedding. We present several definition model architectures based on recurrent neural networks, and experiment with the models over multiple data sets. Our results show that a model that controls dependencies between the word being defined and the definition words performs significantly better, and that a characterlevel convolution layer designed to leverage morphology can complement word-level embeddings. Finally, an error analysis suggests that the errors made by a definition model may provide insight into the shortcomings of word embeddings.

Neural network architectures with memory and attention mechanisms exhibit certain reasoning capabilities required for question answering. One such architecture, the dynamic memory network (DMN), obtained high accuracy on a variety of language tasks. However, it was not shown whether the architecture achieves strong results for question answering when supporting facts are not marked during training or whether it could be applied to other modalities such as images. Based on an analysis of the DMN, we propose several improvements to its memory and input modules. Together with these changes we introduce a novel input module for images in order to be able to answer visual questions. Our new DMN+ model improves the state of the art on both the Visual Question Answering dataset and the bAbI-10k text question-answering dataset without supporting fact supervision.

The skip-thought model has been proven to be effective at learning sentence representations and capturing sentence semantics. In this paper, we propose a suite of techniques to trim and improve it. First, we validate a hypothesis that, given a current sentence, inferring the previous and inferring the next sentence provide similar supervision power, therefore only one decoder for predicting the next sentence is preserved in our trimmed skip-thought model. Second, we present a connection layer between encoder and decoder to help the model to generalize better on semantic relatedness tasks. Third, we found that a good word embedding initialization is also essential for learning better sentence representations. We train our model unsupervised on a large corpus with contiguous sentences, and then evaluate the trained model on 7 supervised tasks, which includes semantic relatedness, paraphrase detection, and text classification benchmarks. We empirically show that, our proposed model is a faster, lighter-weight and equally powerful alternative to the original skip-thought model.

Self-attention is a method of encoding sequences of vectors by relating these vectors to each-other based on pairwise similarities. These models have recently shown promising results for modeling discrete sequences, but they are non-trivial to apply to acoustic modeling due to computational and modeling issues. In this paper, we apply self-attention to acoustic modeling, proposing several improvements to mitigate these issues: First, self-attention memory grows quadratically in the sequence length, which we address through a downsampling technique. Second, we find that previous approaches to incorporate position information into the model are unsuitable and explore other representations and hybrid models to this end. Third, to stress the importance of local context in the acoustic signal, we propose a Gaussian biasing approach that allows explicit control over the context range. Experiments find that our model approaches a strong baseline based on LSTMs with networkin-network connections while being much faster to compute. Besides speed, we find that interpretability is a strength of selfattentional acoustic models, and demonstrate that self-attention heads learn a linguistically plausible division of labor.

Text embedding has gained a lot of interests in text classification area. This paper investigates the popular neural document embedding method Paragraph Vector as a source of evidence in document ranking. We focus on the effects of combining knowledge-based with knowledge-free document embeddings for text classification task. We concatenate these two representations so that the classification can be done more accurately. The results of our experiments show that this approach achieves better performances on a popular dataset.

In this paper we show that a simple beam approximation of the joint distribution between attention and output is an easy, accurate, and efficient attention mechanism for sequence to sequence learning. The method combines the advantage of sharp focus in hard attention and the implementation ease of soft attention. On five translation and two morphological inflection tasks we show effortless and consistent gains in BLEU compared to existing attention mechanisms.

Machine learning models for question-answering (QA), where given a question 1 and a passage, the learner must select some span in the passage as an answer, are 2 known to be brittle. By inserting a single nuisance sentence into the passage, an 3 adversary can fool the model into selecting the wrong span. A promising new 4 approach for QA decomposes the task into two stages: (i) select relevant sentences 5 from the passage; and (ii) select a span among those sentences. Intuitively, if 6 the sentence selector excludes the offending sentence, then the downstream span 7 selector will be robust. While recent work has hinted at the potential robustness 8 of two-stage QA, these methods have never, to our knowledge, been explicitly 9 combined with adversarial training. This paper offers a thorough empirical in10 vestigation of adversarial robustness, demonstrating that although the two-stage 11 approach lags behind single-stage span selection, adversarial training improves its 12 performance significantly, leading to an improvement of over 22 points in F1 score 13 over the adversarially-trained single-stage model. 14

Although NeuroCRF, an augmented Conditional Random Fields (CRF) model whose feature function is parameterized as a Feed-Forward Neural Network (FF NN) on word embeddings, has soundly outperformed traditional linear-chain CRF on many sequence labeling tasks, it is held back by the fact that FF NNs have a fixed input length and therefore cannot take advantage of the full input sentence. We propose to address this issue by replacing the FF NN with a Long Short-Term Memory (LSTM) NN, which can summarize an input of arbitrary length into a fixed dimension representation. The resulting model obtains F1=89.28 on WikiNER dataset, a significant improvement over the NeuroCRF baseline’s F1=87.58, which is already a highly competitive result.

Conventional QA systems cannot answer to the questions composed of two or more sentences. Therefore, we aim to construct a QA system that can answer such multiple-sentence questions. As the first stage, we propose a method for classifying multiple-sentence questions into question types. Specifically, we first extract the core sentence from a given question text. We use the core sentence and its question focus in question classification. The result of experiments shows that the proposed method improves F-measure by 8.8% and accuracy by 4.4%.


Question: Question Passage Inquiry & Passage Encoding Question-Passage Correlation Passage Self-Referencing Word Character Answer Forecasting

Introducing SQuAD: A Large-Scale Machine Comprehension Challenge with 100,000+ Questions
This paper debuts the Stanford Question Answering Dataset (SQuAD), a comprehensive reading comprehension dataset that includes over 100,000 questions formulated by human contributors based on a selection of Wikipedia articles. A distinctive feature of SQuAD is that the answer to each question is a specific text segment extracted from the related passage. To gain insights into the nature of reasoning required to address these questions, we conduct a detailed analysis utilizing dependency and constituency tree structures. To establish a baseline, we construct a robust logistic regression model that achieves an F1 score of 51.0%, marking a notable enhancement from a rudimentary baseline (20%). Nonetheless, human performance (86.8%) significantly outperforms our model, underscoring the dataset's potential as a stimulating challenge for future research endeavors. SQuAD is available without cost at https://stanford-qa.com, offering a valuable resource for the development and evaluation of machine comprehension systems.

Question answering (QA) systems often consist of several components such as Named Entity Disambiguation (NED), Relation Extraction (RE), and Query Generation (QG). In this paper, we focus on the QG process of a QA pipeline on a large-scale Knowledge Base (KB), with noisy annotations and complex sentence structures. We therefore propose SQG, a SPARQL Query Generator with modular architecture, enabling easy integration with other components for the construction of a fully functional QA pipeline. SQG can be used on large open-domain KBs and handle noisy inputs by discovering a minimal subgraph based on uncertain inputs, that it receives from the NED and RE components. This ability allows SQG to consider a set of candidate entities/relations, as opposed to the most probable ones, which leads to a significant boost in the performance of the QG component. The captured subgraph covers multiple candidate walks, which correspond to SPARQL queries. To enhance the accuracy, we present a ranking model based on Tree-LSTM that takes into account the syntactical structure of the question and the tree representation of the candidate queries to find the one representing the correct intention behind the question. SQG outperforms the baseline systems and achieves a macro F1-measure of 75% on the LC-QuAD dataset.

We create a new NLI test set that shows the deficiency of state-of-the-art models in inferences that require lexical and world knowledge. The new examples are simpler than the SNLI test set, containing sentences that differ by at most one word from sentences in the training set. Yet, the performance on the new test set is substantially worse across systems trained on SNLI, demonstrating that these systems are limited in their generalization ability, failing to capture many simple inferences.

The mathematical representation of semantics is a key issue for Natural Language Processing (NLP). A lot of research has been devoted to finding ways of representing the semantics of individual words in vector spaces. Distributional approaches—meaning distributed representations that exploit co-occurrence statistics of large corpora—have proved popular and successful across a number of tasks. However, natural language usually comes in structures beyond the word level, with meaning arising not only from the individual words but also the structure they are contained in at the phrasal or sentential level. Modelling the compositional process by which the meaning of an utterance arises from the meaning of its parts is an equally fundamental task of NLP. This dissertation explores methods for learning distributed semantic representations and models for composing these into representations for larger linguistic units. Our underlying hypothesis is that neural models are a suitable vehicle for learning semantically rich representations and that such representations in turn are suitable vehicles for solving important tasks in natural language processing. The contribution of this thesis is a thorough evaluation of our hypothesis, as part of which we introduce several new approaches to representation learning and compositional semantics, as well as multiple state-of-the-art models which apply distributed semantic representations to various tasks in NLP. Part I focuses on distributed representations and their application. In particular, in Chapter 3 we explore the semantic usefulness of distributed representations by evaluating their use in the task of semantic frame identification. Part II describes the transition from semantic representations for words to compositional semantics. Chapter 4 covers the relevant literature in this field. Following this, Chapter 5 investigates the role of syntax in semantic composition. For this, we discuss a series of neural network-based models and learning mechanisms, and demonstrate how syntactic information can be incorporated into semantic composition. This study allows us to establish the effectiveness of syntactic information as a guiding parameter for semantic composition, and answer questions about the link between syntax and semantics. Following these discoveries regarding the role of syntax, Chapter 6 investigates whether it is possible to further reduce the impact of monolingual surface forms and syntax when attempting to capture semantics. Asking how machines can best approximate human signals of semantics, we propose multilingual information as one method for grounding semantics, and develop an extension to the distributional hypothesis for multilingual representations. Finally, Part III summarizes our findings and discusses future work.

Search logs contain examples of frequently occurring patterns of user reformulations of queries. Intuitively, the reformulation "San Francisco" -- "San Francisco 49ers" is semantically similar to "Detroit" -- "Detroit Lions". Likewise, "London" -- "things to do in London" and "New York" -- "New York tourist attractions" can also be considered similar transitions in intent. The reformulation "movies" -- "new movies" and "york" -- "New York", however, are clearly different despite the lexical similarities in the two reformulations. In this paper, we study the distributed representation of queries learnt by deep neural network models, such as the Convolutional Latent Semantic Model, and show that they can be used to represent query reformulations as vectors. These reformulation vectors exhibit favourable properties such as mapping semantically and syntactically similar query changes closer in the embedding space. Our work is motivated by the success of continuous space language models in capturing relationships between words and their meanings using offset vectors. We demonstrate a way to extend the same intuition to represent query reformulations. Furthermore, we show that the distributed representations of queries and reformulations are both useful for modelling session context for query prediction tasks, such as for query auto-completion (QAC) ranking. Our empirical study demonstrates that short-term (session) history context features based on these two representations improves the mean reciprocal rank (MRR) for the QAC ranking task by more than 10% over a supervised ranker baseline. Our results also show that by using features based on both these representations together we achieve a better performance, than either of them individually.

How to model a pair of sentences is a critical issue in many NLP tasks such as answer selection (AS), paraphrase identification (PI) and textual entailment (TE). Most prior work (i) deals with one individual task by fine-tuning a specific system; (ii) models each sentence’s representation separately, rarely considering the impact of the other sentence; or (iii) relies fully on manually designed, task-specific linguistic features. This work presents a general Attention Based Convolutional Neural Network (ABCNN) for modeling a pair of sentences. We make three contributions. (i) The ABCNN can be applied to a wide variety of tasks that require modeling of sentence pairs. (ii) We propose three attention schemes that integrate mutual influence between sentences into CNNs; thus, the representation of each sentence takes into consideration its counterpart. These interdependent sentence pair representations are more powerful than isolated sentence representations. (iii) ABCNNs achieve state-of-the-art performance on AS, PI and TE tasks. We release code at: https://github.com/yinwenpeng/Answer_Selection.

Answering questions according to multi-modal context is a challenging problem as it requires a deep integration of different data sources. Existing approaches only employ partial interactions among data sources in one attention hop. In this paper, we present the Holistic Multi-modal Memory Network (HMMN) framework which fully considers the interactions between different input sources (multi-modal context, question) in each hop. In addition, it takes answer choices into consideration during the context retrieval stage. Therefore, the proposed framework effectively integrates multi-modal context, question, and answer information, which leads to more informative context retrieved for question answering. Our HMMN framework achieves state-of-the-art accuracy on MovieQA dataset. Extensive ablation studies show the importance of holistic reasoning and contributions of different attention strategies.

We introduce a general strategy for improving neural sequence generation by incorporating knowledge about the future. Our decoder combines a standard sequence decoder with a ‘soothsayer’ prediction function Q that estimates the outcome in the future of generating a word in the present. Our model draws on the same intuitions as reinforcement learning, but is both simpler and higher performing, avoiding known problems with the use of reinforcement learning in tasks with enormous search spaces like sequence generation. We demonstrate our model by incorporating Q functions that incrementally predict what the future BLEU or ROUGE score of the completed sequence will be, its future length, and the backwards probability of the source given the future target sequence. Experimental results show that future prediction yields improved performance in abstractive summarization and conversational response generation and the state-of-the-art in machine translation, while also enabling the decoder to generate outputs that have specific properties.

• Neural network language model and distributed representation for words (Vector representation) • Capture syntactic and remantic regularities in language • Outperform state-of-the-art

We address the task of computing vector space representations for the meaning of word occurrences, which can vary widely according to context. This task is a crucial step towards a robust, vector-based compositional account of sentence meaning. We argue that existing models for this task do not take syntactic structure sufficiently into account. We present a novel structured vector space model that addresses these issues by incorporating the selectional preferences for words’ argument positions. This makes it possible to integrate syntax into the computation of word meaning in context. In addition, the model performs at and above the state of the art for modeling the contextual adequacy of paraphrases.