Question: Innovative technologies facilitating the Internet of Health Things.

This article focuses on the Internet of Things (IoT), an emerging concept that hinges on the integration of various technologies and communication solutions. The IoT's core components include identification and tracking systems, along with both wired and wireless sensor and actuator networks. Enhanced communication protocols, which are shared with the Next Generation Internet, and distributed intelligence for smart objects are also critical elements. Given the IoT's interdisciplinary nature, it requires collaborative efforts across multiple domains, including telecommunications, informatics, electronics, and social sciences. This survey is designed for newcomers to the IoT field, aiming to guide and inspire contributions to its advancement. The paper presents various perspectives on the IoT paradigm and thoroughly examines the enabling technologies. It is evident that the research community still faces significant challenges, which are discussed in depth. This work is published under the copyright of 2010 Elsevier B.V., with all rights reserved.

In the context of the Internet of Health Things (IoHT), this survey explores the integration of technologies that underpin this transformative paradigm. Key enablers include identification and tracking systems, alongside wired and wireless sensor and actuator networks specifically tailored for healthcare applications. Enhanced communication protocols, shared with the Next Generation Internet, and distributed intelligence for smart medical devices are pivotal. The interdisciplinary nature of IoHT necessitates collaboration across various fields, such as telemedicine, health informatics, medical electronics, and health policy. This comprehensive review is intended for researchers and practitioners entering the IoHT domain, providing insights and guiding contributions to its development. The paper highlights various perspectives on the IoHT concept and critically assesses the enabling technologies. It is clear that the research community confronts major hurdles, which are explored in detail. The publication is copyrighted by 2010 Elsevier B.V., with all rights reserved.

Technology is great growling engine of the change and Internet of Things (IoT) is the backbone of such revolutionary engines. Basically, in the real world the things having sensor capability, sufficient power supply and connectivity to internet makes field like Internet of Things (IoT) possible. For such rapid growing technology, it is the necessity to have very light, inexpensive and minimum bandwidth protocol like Message Queuing Telemetry Transport (MQTT) Protocol. Because of such non-established protocol it is easy for the clients to publish or/and subscribe the desire topic through the host acting as server of the network also known to be the broker. In this paper, it is shown that communication between the low power ESP8266 WiFi as client with the clients on smartphones and laptop using an MQTT protocol becomes easier and more reliable. The WiFi enabled ESP8266 board interfaces with DHT11 sensor and LDR sensor to monitor the ambient condition and according to the light intensity level the brightness level of 8∗8 Neopixel matrix is controlled. The adafruit.io is the MQTT server i.e, broker which also provides the facility of monitoring system through the dashboard. The clients on smartphone and laptop subscribes to the topics of temperature, humidity and light intensity level gets the respective update. The main objective of the paper is to implement the idea for the smart home appliance, street light system for the smart cities, fire alert systems, whether monitoring system and so.

To many people, home is a sanctuary. For those people who need special medical care, they may need to be pulled out of their home to meet their medical needs. As the population ages, the percentage of people in this group is increasing and the effects are expensive as well as unsatisfying. We hypothesize that many people with disabilities can lead independent lives in their own homes with the aid of athome automated assistance and health monitoring. In order to accomplish this, robust methods must be developed to collect relevant data and process it dynamically and adaptively to detect and/or predict threatening long-term trends or immediate crises. The main objective of this paper is to investigate techniques for using agent-based smart home technologies to provide this at-home health monitoring and assistance. To this end, we have developed novel inhabitant modeling and automation algorithms that provide remote health monitoring for caregivers. Specifically, we address the following technological challenges: 1) identifying lifestyle trends, 2) detecting anomalies in current data, and 3) designing a reminder assistance system. Our solution approaches are being tested in simulation and with volunteers at the UTA’s MavHome site, an agent-based

We are on the threshold of the explosive growth in the global Internet-of-Things (IoT) market. Comparing with the legacy human-centric applications, machine type communication scenarios exhibit totally different characteristics, such as low throughput, delay insensitivity, occasional transmission, and deep coverage. Meanwhile, it also requires the terminal devices to be cheap enough and sustain long battery life. These demands hasten the prosperity of low power wide area (LPWA) technologies. Narrowband IoT (NB-IoT) is the newest Long Term Evolution (LTE) specification ratified by the third generation partner project as one of the LPWA solutions to achieve the objectives of super coverage, low power, low cost, and massive connection. Working in the licensed frequency band, it is designed to reuse and coexist with the existing LTE cellular networks, which endows it with outstanding advantages in decreasing network installation cost and minimizing product time-to-market. In this backdrop, it has been extensively regarded as one of the most promising technologies toward the IoT landscape. However, as a new LTE standard, there are still a lot of challenges that need to overcome. This paper surveys its evolutions, technologies, and issues, spanning from performance analysis, design optimization, combination with other leading technologies, to implementation and application. The goal is to deliver a holistic understanding for the emerging wireless communication system, in helping to spur further research in accelerating the broad use of NB-IoT.

Connected health care has several applications including telecare medicine information system, personally controlled health records system, and patient monitoring. In such applications, user authentication can ensure the legality of patients. In user authentication for such applications, only the legal user/patient himself/herself is allowed to access the remote server, and no one can trace him/her according to transmitted data. Chang et al. proposed a uniqueness-and-anonymity-preserving remote user authentication scheme for connected health care (Chang et al., J Med Syst 37:9902, 2013). Their scheme uses the user’s personal biometrics along with his/her password with the help of the smart card. The user’s biometrics is verified using BioHashing. Their scheme is efficient due to usage of one-way hash function and exclusive-or (XOR) operations. In this paper, we show that though their scheme is very efficient, their scheme has several security weaknesses such as (1) it has design flaws in login and authentication phases, (2) it has design flaws in password change phase, (3) it fails to protect privileged insider attack, (4) it fails to protect the man-in-the middle attack, and (5) it fails to provide proper authentication. In order to remedy these security weaknesses in Chang et al.’s scheme, we propose an improvement of their scheme while retaining the original merit of their scheme. We show that our scheme is efficient as compared to Chang et al.’s scheme. Through the security analysis, we show that our scheme is secure against possible attacks. Further, we simulate our scheme for the formal security verification using the widely-accepted AVISPA (Automated Validation of Internet Security Protocols and Applications) tool to ensure that our scheme is secure against passive and active attacks. In addition, after successful authentication between the user and the server, they establish a secret session key shared between them for future secure communication.

The Internet of Things (IoT) could be described as the pervasive and global network which aids and provides a system for the monitoring and control of the physical world through the collection, processing and analysis of generated data by IoT sensor devices. It is projected that by 2020 the number of connected devices is estimated to grow exponentially to 50 billion. The main drivers for this growth are our everyday devices such as cars, refrigerators, fans, lights, mobile phones and other operational technologies including the manufacturing infrastructures which are now becoming connected systems across the world. It is apparent that security will pose a fundamental enabling factor for the successful deployment and use of most IoT applications and in particular secure routing among IoT sensor nodes thus, mechanisms need to be designed to provide secure routing communications for devices enabled by the IoT technology. This survey analyzes existing routing protocols and mechanisms to secure routing communications in IoT, as well as the open research issues. We further analyze how existing approaches ensure secure routing in IoT, their weaknesses, threats to secure routing in IoT and the open challenges and strategies for future research work for a better secure IoT routing.

This work is based on the Internet of Things and ZigBee wireless sensor network technology. A kind of smart home design based on ZigBee wireless sensor network was proposed in this paper. Texas Instruments MCU device LM3S9B96, which is the ARM Cortex-M3 based controllers, was used in this system. The entire system is running on the μC/OS-II embedded real-time multitasking operating system. Users can access this system by a dynamic webpage of LwIP TCP/IP protocol stack or GSM SMS. Using this system users can conveniently know the environment parameters of home, such as temperature, humidity, meter readings, light, and control the home electronic equipments, such as light, aircondition, heater, by ZigBee wireless sensor network.

The Internet of Things (IoT) has connected an incredible diversity of devices in novel ways, which has enabled exciting new services and opportunities. Unfortunately, IoT systems also present several important challenges to developers. This paper proposes a vision for how we may build IoT systems in the future by reconceiving IoT's fundamental unit of construction not as a "thing", but rather as a widely and finely distributed "microservice" already familiar to web service engineering circles. Since IoT systems are quite different from more established uses of microservice architectures, success of the approach depends on adaptations that enable them to met the key challenges that IoT systems present. We argue that a microservice approach to building IoT systems can combine in a mutually enforcing way with patterns for microservices, API gateways, distribution of services, uniform service discovery, containers, and access control. The approach is illustrated using two case studies of IoT systems in personal health management and connected autonomous vehicles. Our hope is that the vision of a microservices approach will help focus research that can fill in current gaps preventing more effective, interoperable, and secure IoT services and solutions in a wide variety of contexts.

Internet of Things (IoT) infrastructure within the physical library environment is the basis for an integrative, hybrid approach to digital resource recommenders. The IoT infrastructure provides mobile, dynamic wayfinding support for items in the collection, which includes features for location-based recommendations. The evaluation and analysis herein clarified the nature of users’ requests for recommendations based on their location, and describes subject areas of the library for which users request recommendations. The results indicated that users of IoTbased recommendation are interested in a broad distribution of subjects, with a short-head distribution from this collection in American and English Literature. A long-tail finding showed a diversity of topics that are recommended to users in the library book stacks with IoT-powered recommendations.

In this paper, we propose a Simple Wireless transmission System using common approach Sensor Platform called The wireless based Patient Sensor platform (WSP, Sensor Node) which has remote access capability. The goals of the WSP are to establish: Standard sensor node (System on module), a common software .The proposed platform architecture (Sensor Node) offers flexibility, easy customization for different vital parameter collecting and sending. An prototype has been established based on wireless communication channel. Wireless lan (IEEE .802.15.4)) has been used as communication channel on our prototype (Sensor node). Desire sensor information (vital parameter) can be viewed remotely, and also vital parameter can be adjusted to meet demand.

With the recent advancement in technologies such as embedded system, wireless distributed sensor, light weight material, smart cognitive radio networks, cloud computing, higher efficiency and ultra-low emission internal combustion engines, intelligent converter, high performance battery and fuel cell technology, the production of smarter, safer, energy efficient and zero emission vehicles is possible in near future. Apart from vehicle technologies, other factors such as road users', well maintained road infrastructure, well maintained vehicles, drivers' attitudes and law and enforcement are also important to be considered and they should work together in order to make our world natural resources can be preserved and maintain cleaner environment and produce sustainable mobility. This paper will discuss the impacts of IoT and Big Data and other emerging technologies mentioned above to the automotive industry. It will include discussion on education, economy, advanced technology, environment, safety and energy.


Question: Innovative technologies facilitating the Internet of Health Things ecosystem.

This paper introduces a groundbreaking technique for embedding stealthy hardware Trojans at the dopant level, a critical concern for governments, industries, and the scientific community alike. The primary worry revolves around the potential for malicious manipulation of integrated circuits during the manufacturing process, particularly when fabrication occurs in foreign countries. Despite the absence of reported instances of hardware Trojans in real-world scenarios, the lack of knowledge regarding their practical implementation and appearance remains a significant gap. Herein, we present an exceptionally covert method for integrating hardware Trojans below the gate level, focusing on the impact on the security of the targeted device. Our approach eschews the addition of extra circuitry to the design, opting instead to insert Trojans by altering the dopant polarity of existing transistors. This modification ensures that the circuit appears entirely legitimate at all wiring layers, encompassing every metal and polysilicon level, thereby making our family of Trojans highly resistant to detection methods, including fine-grained optical inspection and comparisons against "golden chips". To substantiate the efficacy of our methodology, we have successfully implanted Trojans into two distinct designs: a digital post-processing module derived from Intel's cryptographically secure Random Number Generator (RNG) design, as utilized in Ivy Bridge processors, and a side-channel resistant SBox implementation. We further investigate the detectability of these Trojans and their implications for security.

eHealth is the provision of healthcare with the inclusion of telecommunication techniques. This project looks at the construction of a simple device that will be capable of transferring the data of a patient's vital signs to a remote device wirelessly. The necessity of this project is to alleviate the difficulty that is encountered by medical experts in monitoring multiple patients simultaneously. This project will enable them to observe patients without having to be physically present at their bedside, be it in the hospital or in their home. A patient's body temperature, heart rate and electrocardiography (ECG) are transferred wirelessly through an agent such as Bluetooth technology.

Internet of Things (IoT) are being adopted for industrial and manufacturing applications such as manufacturing automation, remote machine diagnostics, prognostic health management of industrial machines and supply chain management. CloudBased Manufacturing is a recent on-demand model of manufacturing that is leveraging IoT technologies. While Cloud-Based Manufacturing enables on-demand access to manufacturing resources, a trusted intermediary is required for transactions between the users who wish to avail manufacturing services. We present a decentralized, peer-to-peer platform called BPIIoT for Industrial Internet of Things based on the Block chain technology. With the use of Blockchain technology, the BPIIoT platform enables peers in a decentralized, trustless, peer-to-peer network to interact with each other without the need for a trusted intermediary.

Internet of things is a promising research due to its importance in many commerce, industry, and education applications. Recently, new applications and research challenges in numerous areas of Internet of things are fired. In this paper, we discuss the history of Internet of things, different proposed architectures of Internet of things, research challenges and open problems related to the Internet of things. We also introduce the concept of Internet of things database and discuss about the future vision of Internet of things. These are the manuscript preparation guidelines used as a standard template for all journal submissions. Author must follow these instructions while preparing/modifying these guidelines.

In this conceptual paper, we review the definitions, characteristics, and architecture of the Internet of Things (IoT) concept. We then identify the deficiencies of the IoT concept, analyze them, and discover the issue of lack of reasoning and intelligence in the IoT concept. We propose a solution to augment the IoT with intelligent software agents resulting in a new concept called the Agents of Things (AoT). The paper presents the AoT architecture and a scenario for one application of the proposed concept. Finally, it discusses the benefits of implementing AoT concept to solve real world issues and the future work.

Wireless Body Area Network (WBAN) is a collection of wireless sensor nodes which can be placed within the body or outside the body of a human or a living person which in result observes or monitors the functionality and adjoining situations of the body. Utilizing a Wireless Body Area Network, the patient encounters a greater physical versatility and is never again constrained to remain in the hospital. As the Wireless Body Area Network sensor devices is being utilized for gathering the sensitive data and possibly will run into antagonistic situations, they require complicated and very secure security medium or structure to avoid the vitriolic communications within the system. These devices represent various security and privacy protection of touchy and private patient medical data. The medical records of patients are a significant and an unsolved situation due to which a changing or exploitation to the system is possible. In this research paper, we first present an overview of WBAN, how they utilized for healthcare monitoring, its architecture then highlight major security and privacy requirements and assaults at different network layer in a WBAN and we finally talk about various cryptographic algorithms and laws for providing solution of security and privacy in WBAN. Keywords—E-Health; privacy; security; wireless body area networks

Internet evolution has been recently related with some aspect of user empowerment, mostly in terms of content distribution, and this has been ultimately accelerated by the fast-paced introduction and expansion of wireless technologies. Hence, the Internet should start to be seen as a communications infrastructure able to support the integration of a myriad of embedded and personal wireless objects. This way a future Internet will support the interaction between users’ social, physical and virtual sphere. This position paper aims to raise some discussion about the technology required to ensure an efficient interaction between the physical, social and virtual worlds by extending the Internet by means of interconnected objects. Namely, it is argued that an efficient interaction between the physical, social and virtual worlds requires the development of a data-centric architecture based on IP-driven opportunisitc networking able to make useful data available to people when and where they really need it, augmenting their social and environmental awareness.

The impact of air quality on health and on life comfort is well established. In many societies, vulnerable elderly and young populations spend most of their time indoors. Therefore, indoor air quality monitoring (IAQM) is of great importance to human health. Engineers and researchers are increasingly focusing their efforts on the design of real-time IAQM systems using wireless sensor networks. This paper presents an end-to-end IAQM system enabling measurement of CO₂, CO, SO₂, NO₂, O₃, Cl₂, ambient temperature, and relative humidity. In IAQM systems, remote users usually use a local gateway to connect wireless sensor nodes in a given monitoring site to the external world for ubiquitous access of data. In this work, the role of the gateway in processing collected air quality data and its reliable dissemination to end-users through a web-server is emphasized. A mechanism for the backup and the restoration of the collected data in the case of Internet outage is presented. The system is adapted to an open-source Internet-of-Things (IoT) web-server platform, called Emoncms, for live monitoring and long-term storage of the collected IAQM data. A modular IAQM architecture is adopted, which results in a smart scalable system that allows seamless integration of various sensing technologies, wireless sensor networks (WSNs) and smart mobile standards. The paper gives full hardware and software details of the proposed solution. Sample IAQM results collected in various locations are also presented to demonstrate the abilities of the system.

This article provides an overview of body area sensor networks (BASNs), an application of wireless technology that changed health care to suit the comfort of the population. There are various applications of BASNs and attempts have been made to make the human body a channel for wireless communication. BASNs employ a three-tiered architectural system that requires various technical requirements for its optimal and efficient operation. Energy consumption is one of the major issues that is currently being addressed through self-harvesting and many other techniques. Even with the benefits at hand, there are various issues such as interference and eavesdropping that BASNs have to tackle. Biometrics is a widely used solution. Researchers are also working on various ambitious projects that deal with improving deep brain simulation, heart regulation, drug delivery, and prosthetic actuation to use BASN effectively.

This article presents a methodology based on design thinking and user experience design methods for creating what we call `people-aware' IoT applications, where user needs, not technological opportunities, drive the development. This methodology is divided into 7 steps: discovery, capturing, research, design, prototype, evaluate and refine. The tools used include conventional user experience procedures such as problem identification, group brainstorming, surveys, or interviews, mixed with more IoT-specific design specificities. The results of the methodology include well-described and user-oriented scenarios meeting user's needs and also a complete toolbox to assist the implementation and the testing of abovementioned scenarios in an IoT perspective. The article describes the methodology in detail with the help of a use case conducted in a business environment available for the project that leads to the identification and partial design of concrete people-aware IoT applications in the context of a smart meeting room.

The growing interest in the Internet of Things (IoT) has resulted in a number of wide-area deployments of IoT subnetworks, where multiple heterogeneous wireless communication solutions coexist: from multiple access technologies such as cellular, WiFi, ZigBee, and Bluetooth, to multi-hop ad-hoc and MANET routing protocols, they all must be effectively integrated to create a seamless communication platform. Managing these open, geographically distributed, and heterogeneous networking infrastructures, especially in dynamic environments, is a key technical challenge. In order to take full advantage of the many opportunities they provide, techniques to concurrently provision the different classes of IoT traffic across a common set of sensors and networking resources must be designed. In this paper, we will design a software-defined approach for the IoT environment to dynamically achieve differentiated quality levels to different IoT tasks in very heterogeneous wireless networking scenarios. For this, we extend the Multinetwork INformation Architecture (MINA), a reflective (self-observing and adapting via an embodied Observe-Analyze-Adapt loop) middleware with a layered IoT SDN controller. The developed IoT SDN controller originally i) incorporates and supports commands to differentiate flow scheduling over task-level, multi-hop, and heterogeneous ad-hoc paths and ii) exploits Network Calculus and Genetic Algorithms to optimize the usage of currently available IoT network opportunities. We have applied the extended MINA SDN prototype in the challenging IoT scenario of wide-scale integration of electric vehicles, electric charging sites, smart grid infrastructures, and a wide set of pilot users, as targeted by the Artemis Internet of Energy and Arrowhead projects. Preliminary simulation performance results indicate that our approach and the extended MINA system can support efficient exploitation of the IoT multinetwork capabilities.


Question: Movement, feeling, and sympathy in the aesthetic encounter.

This paper proposes a comprehensive neural hypothesis that integrates the cognitive processes underlying an individual's ability to comprehend the actions and emotions of others. Central to our hypothesis is the assertion that the mirror neuron system plays a pivotal role in facilitating the experiential understanding of others' actions. This system, when activated, enables individuals to resonate with and mimic the actions they observe, thus fostering a sense of shared experience. Similarly, for the experiential understanding of emotions, a distinct yet related mechanism comes into play. Here, the activation of viscero-motor centers, which are involved in the regulation of internal bodily states, allows individuals to empathetically resonate with the emotional states of others. This dual mechanism, involving both the mirror neuron system and the viscero-motor centers, provides a neurobiological foundation for the social cognition processes that underpin empathy and the ability to relate to others on an emotional level.

Recent empirical studies have implicated the use of the motor system during action observation, imitation and social interaction. In this paper, we explore the computational parallels between the processes that occur in motor control and in action observation, imitation, social interaction and theory of mind. In particular, we examine the extent to which motor commands acting on the body can be equated with communicative signals acting on other people and suggest that computational solutions for motor control may have been extended to the domain of social interaction.


Question: Movement, feeling, and sympathy in the aesthetic encounter.

This study explores the neurophysiological mechanisms underlying the observation of motor events in primates. Neurons within the rostral region of the inferior premotor cortex have been found to exhibit activity during the execution of goal-oriented hand movements, such as grasping, holding, and tearing. Our research further reveals that these neurons also respond when the monkey observes similar hand movements being performed by human experimenters. The observed movements that trigger neuronal activity are those that are meaningful and contextually relevant, such as handling food or manipulating objects. Importantly, the observed movements must bear a clear resemblance to the movements the monkey itself performs; only actions that are identical to those controlled by a given neuron are capable of activating it. This suggests that premotor neurons have the capacity to not only retrieve movements based on sensory stimuli, but also according to the semantic content of the observed actions, potentially linking motor behavior with cognitive understanding and empathy.

More and more, robots are expected to interact with humans in a social, easily understandable manner, which presupposes effective use of robot affect. This chapter provides a brief overview of research advances into this important aspect of human-robot interaction. Keywords: human-robot interaction, affective robotics, robot behavior I. Introduction
  and
  Motivation Humans possess an amazing capability of attributing life and affect to inanimate objects (Reeves and Nass 96, Melson et al 09). Robots take this to the next level, even beyond that of virtual characters due to their embodiment and situatedness. They offer the opportunity for people to bond with them by maintaining a physical presence in their world, in some ways comparable to other beings, such as fellow humans and pets. This raises a broad range of questions in terms of the role of affect in human-robot interaction (HRI), which will be discussed in this article: • What is the role of affect for a robot and in what ways can it add value and risk to humanrobot relationships? Can robots be companions, friends, even intimates to people? • Is it necessary for a robot to actually experience emotion in order to convey its internal state to a person? Is emotion important in enhancing HRI and if so when and where? • What approaches, theories, representations, and experimental methods inform affective HRI research? Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 2014 2. REPORT TYPE 3. DATES COVERED 00-00-2014 to 00-00-2014 4. TITLE AND SUBTITLE Affect in Human-Robot Interaction 5a. CONTRACT NUMBER

Recent empirical studies have implicated the use of the motor system during action observation, imitation and social interaction. In this paper, we explore the computational parallels between the processes that occur in motor control and in action observation, imitation, social interaction and theory of mind. In particular, we examine the extent to which motor commands acting on the body can be equated with communicative signals acting on other people and suggest that computational solutions for motor control may have been extended to the domain of social interaction.

In line with the sensorimotor contingency theory, we investigate the problem of the perception of space from a fundamental sensorimotor perspective. Despite its pervasive nature in our perception of the world, the origin of the concept of space remains largely mysterious. For example in the context of artificial perception, this issue is usually circumvented by having engineers pre-define the spatial structure of the problem the agent has to face. We here show that the structure of space can be autonomously discovered by a naive agent in the form of sensorimotor regularities, that correspond to so called compensable sensory experiences: these are experiences that can be generated either by the agent or its environment. By detecting such compensable experiences the agent can infer the topological and metric structure of the external space in which its body is moving. We propose a theoretical description of the nature of these regularities and illustrate the approach on a simulated robotic arm equipped with an eye-like sensor, and which interacts with an object. Finally we show how these regularities can be used to build an internal representation of the sensor's external spatial configuration.

Emotions are very important for human intelligence. For example, emotions are closely related to the appraisal of the internal bodily state and external stimuli. This helps us to respond quickly to the environment. Another important perspective in human intelligence is the role of emotions in decision-making. Moreover, the social aspect of emotions is also very important. Therefore, if the mechanism of emotions were elucidated, we could advance toward the essential understanding of our natural intelligence. In this study, a model of emotions is proposed to elucidate the mechanism of emotions through the computational model. Furthermore, from the viewpoint of partner robots, the model of emotions may help us to build robots that can have empathy for humans. To understand and sympathize with people’s feelings, the robots need to have their own emotions. This may allow robots to be accepted in human society. The proposed model is implemented using deep neural networks consisting of three modules, which interact with each other. Simulation results reveal that the proposed model exhibits reasonable behavior as the basic mechanism of emotion.

The increasing use of wearable devices for tourism purposes sets the stage for a critical discussion on technological mediation in tourism experiences. This paper provides a theoretical reflection on the phenomenon of embodiment relation in technological mediation and then assesses the embodiment of wearable augmented reality technology in a tourism attraction. The findings suggest that technology embodiment is a multidimensional construct consisting of ownership, location, and agency. These support the concept of technology withdrawal, where technology disappears as it becomes part of human actions, and contest the interplay of subjectivity and intentionality between humans and technology in situated experiences such as tourism. It was also found that technology embodiment affects enjoyment and enhances experience with tourism attractions.


Question: Movement, feeling, and compassion in the aesthetic encounter.

This study employs functional magnetic resonance imaging (fMRI) to pinpoint brain regions that become active during the observation of actions performed by others, providing insights into the neural mechanisms underlying motion, emotion, and empathy in aesthetic experiences. The research involved presenting participants with a variety of actions, including those related to objects and those unrelated, executed using different body parts such as the mouth, hand, and foot. The findings revealed a somatotopically organized activation pattern in the premotor cortex during the observation of both object-related and non-object-related actions, mirroring the somatotopic organization of the primary motor cortex homunculus. Additionally, when participants observed object-related actions, there was an extra somatotopically organized activation in the posterior parietal lobe. This suggests that when we observe actions, our brains automatically generate an internal representation of the observed action in the premotor cortex. For object-related actions, this representation is further refined in the parietal lobe, as if the observer were actually interacting with the objects. These results extend the concept of the action observation/execution matching system, or mirror system, by demonstrating its involvement in multiple somatotopically organized motor circuits, beyond just the ventral premotor cortex. This broader perspective enhances our understanding of how motion, emotion, and empathy are processed during aesthetic experiences, indicating that the brain's response to observed actions is multifaceted and involves various interconnected regions.

In this paper, we introduce a new approach for applying haptic feedback technology to interpersonal communication. We present the design of our prototype inTouch system which provides a physical link between users separated by distance.

In this paper, we report a study that examines the relationship between image-based computational analyses of web pages and users' aesthetic judgments about the same image material. Web pages were iteratively decomposed into quadrants of minimum entropy (quadtree decomposition) based on low-level image statistics, to permit a characterization of these pages in terms of their respective organizational symmetry, balance and equilibrium. These attributes were then evaluated for their correlation with human participants' subjective ratings of the same web pages on four aesthetic and affective dimensions. Several of these correlations were quite large and revealed interesting patterns in the relationship between low-level (i.e., pixel-level) image statistics and design-relevant dimensions.


Question: Movement, feeling, and sympathy in the aesthetic encounter.

This paper explores the concept of "intentional attunement," a neurophysiological perspective on social cognition that is particularly relevant in understanding the experiences of individuals with autism. Intentional attunement is a form of direct, experiential comprehension of others, achieved by simulating their behaviors as intentional experiences through the activation of shared neural systems. These systems underpin the correlation between the actions, emotions, and sensations experienced by others and by ourselves. This simulation mechanism, often referred to as embodied simulation, involves the evocation of internal representations of body states in the observer when confronted with social stimuli. These representations are akin to the observer performing a similar action or experiencing a similar emotion or sensation, creating a shared neural state between two different bodies, which transforms the "objectual other" into "another self." Mirror neuron systems are proposed as the neural correlate of this mechanism, playing a crucial role in social cognition. A disruption in this intentional attunement, potentially due to a deficiency in embodied simulation, may underlie some of the social impairments observed in individuals with autism. This perspective sheds light on the neurophysiological basis of social interaction and empathy, highlighting the importance of embodied simulation in facilitating a deeper understanding of others' intentions and emotions.

This essay reviews theory and research regarding the “Michelangelo phenomenon,” which describes the manner in which close partners shape one another’s dispositions, values, and behavioral tendencies. Individuals are more likely to exhibit movement toward their ideal selves to the degree that their partners exhibit affirming perception and behavior, exhibiting confidence in the self ’s capacity and enacting behaviors that elicit key features of the self ’s ideal. In turn, movement towards the ideal self yields enhanced personal well-being and couple well-being. We review empirical evidence regarding this phenomenon and discuss self and partner variables that contribute to the process.

Emotions are very important for human intelligence. For example, emotions are closely related to the appraisal of the internal bodily state and external stimuli. This helps us to respond quickly to the environment. Another important perspective in human intelligence is the role of emotions in decision-making. Moreover, the social aspect of emotions is also very important. Therefore, if the mechanism of emotions were elucidated, we could advance toward the essential understanding of our natural intelligence. In this study, a model of emotions is proposed to elucidate the mechanism of emotions through the computational model. Furthermore, from the viewpoint of partner robots, the model of emotions may help us to build robots that can have empathy for humans. To understand and sympathize with people’s feelings, the robots need to have their own emotions. This may allow robots to be accepted in human society. The proposed model is implemented using deep neural networks consisting of three modules, which interact with each other. Simulation results reveal that the proposed model exhibits reasonable behavior as the basic mechanism of emotion.


Question: Movement, feeling, and sympathy in the aesthetic encounter.

In the intricate landscape of primate behavior, particularly human behavior, the comprehension and interpretation of actions performed by others hold significant evolutionary and social importance. This understanding is not only vital for survival but also serves as the cornerstone of social organization and cultural development through imitation learning. Unlike many other species, humans possess the unique ability to learn by observing and replicating the actions of others, a trait that underpins the rich tapestry of human culture. Central to this capacity is the mirror-neuron system, a neurophysiological mechanism that plays a pivotal role in both action understanding and the acquisition of skills through imitation.

In this review, we delve into the intricate details of mirror neurons, first in non-human primates and subsequently in humans. We explore the functional attributes of mirror neurons in monkeys, highlighting their ability to fire both when an animal performs an action and when it observes the same action being executed by another individual. This dual activation is a fundamental characteristic that supports the theory of action understanding and empathy.

Shifting our focus to humans, we analyze the distinct features of the human mirror-neuron system, emphasizing those aspects that may account for our superior imitative abilities. These include the complexity and specificity of human mirror neurons in processing a wide range of actions and their associated emotions, which contribute to our empathetic responses and social interactions. We also discuss the potential link between the mirror-neuron system and language, exploring how this neural mechanism might have facilitated the evolution of linguistic communication in humans.

This review aims to provide a comprehensive overview of the mirror-neuron system, its role in action understanding and imitation learning, and its implications for the development of human culture and language. Through this exploration, we gain deeper insights into the neural underpinnings of human behavior and the mechanisms that enable us to connect, learn, and evolve as a species.

This essay reviews theory and research regarding the “Michelangelo phenomenon,” which describes the manner in which close partners shape one another’s dispositions, values, and behavioral tendencies. Individuals are more likely to exhibit movement toward their ideal selves to the degree that their partners exhibit affirming perception and behavior, exhibiting confidence in the self ’s capacity and enacting behaviors that elicit key features of the self ’s ideal. In turn, movement towards the ideal self yields enhanced personal well-being and couple well-being. We review empirical evidence regarding this phenomenon and discuss self and partner variables that contribute to the process.

While traditional HCI research emphasizes usability based on models of cognition, user experience (UX) focuses on affect and emotion through the provision of positive interactive experiences. Providing affective cues, such as apologetic on-screen display messages, appears to be a way to influence users' affective states as well as their perceptions toward an information retrieval system. A study was designed to determine whether users' affect and perceptions differ between three types of systems: neutral, apologetic, and non-apologetic. Our results revealed that the users perceived the apologetic system as more aesthetically appealing and usable than the neutral or non-apologetic system. The result also showed that users' frustration was the lowest when using the apologetic system. We discuss the implications of these results in designing a more experience-centered system.


Question: An implicit segmentation technique for the recognition of handwritten character strings.

This article outlines an innovative Hidden Markov Model (HMM)-based methodology aimed at recognizing off-line unconstrained handwritten words, particularly within extensive vocabularies. Following a preprocessing phase, a word image is partitioned into discrete components, including letters and pseudoletters, which are subsequently represented by dual feature sequences of uniform length. These sequences are characterized by an alternating pattern of shape-symbols and segmentation-symbols, both of which are meticulously modeled. The word model is assembled by concatenating relevant letter models, which are fundamentally composed of basic HMMs. An interpolation technique, rooted in HMMs, is employed to optimally amalgamate the dual feature sets. The paper explores two distinct rejection strategies, contingent upon the certainty that the word image is part of the lexicon. The efficacy of this approach is substantiated through experiments conducted on real-world data, illustrating its potential for proficient handwritten word recognition. Keywords: Handwriting analysis, preprocessing techniques, segmentation strategies, feature extraction, hidden Markov models, word recognition, rejection methodologies.

We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1 % error rate and about a 9% reject rate on zipcode digits provided by the U.S. Postal Service.

Arabic Optical Character Recognition (OCR) as Web Services represents a major challenge for handwritten document recognition. A variety of approaches, methods, algorithms and techniques have been proposed in order to build powerful Arabic OCR web services. Unfortunately, these methods could not succeed in achieving this mission in case of large large quantity Arabic handwritten documents. Intensive experiments and observations revealed that some of the existing approaches and techniques are complementary and can be combined to improve the recognition rate. Designing and implementing these recent sophisticated complementary approaches and techniques as web services are commonly complex; they require strong computing power to reach an acceptable recognition speed especially in case of large quantity documents. One of the possible solutions to overcome this problem is to benefit from distributed computing architectures such as cloud computing. This paper describes the design and implementation of Arabic Handwriting Recognition as a web service (AHRweb service) based on the complementary approach K-Nearest Neighbor (KNN) /Support Vector Machine (SVM) (K-NN/SVM) via Amazon Elastic MapReduce (EMR) model. The experiments were conducted on a cloud computing environment with a real large scale handwriting dataset from the Institute for Communications Technology (IFN)/ Ecole Nationale d’Ingénieur de Tunis (ENIT) IFN/ENIT database. The J-Sim (Java Simulator) was used as a tool to generate and analyze statistical results. Experimental results show that Amazon Elastic MapReduce (EMR) model constitutes a very promising framework for enhancing large AHRweb service performances.

In this paper, we study oracle character recognition and general sketch recognition. First, a data set of oracle characters, which are the oldest hieroglyphs in China yet remain a part of modern Chinese characters, is collected for analysis. Second, typical visual representations in shape- and sketch-related works are evaluated. We analyze the problems suffered when addressing these representations and determine several representation design criteria. Based on the analysis, we propose a novel hierarchical representation that combines a Gabor-related low-level representation and a sparse-encoder-related mid-level representation. Extensive experiments show the effectiveness of the proposed representation in both oracle character recognition and general sketch recognition. The proposed representation is also complementary to convolutional neural network (CNN)-based models. We introduce a solution to combine the proposed representation with CNN-based models, and achieve better performances over both approaches. This solution has beaten humans at recognizing general sketches.

Threshold based segmentation methods can effectively separate objects from the background. The conventional Otsu's method can achieve good result only when the histogram of an image has two distinct peaks. In this paper, an iterative thresholding based on 2D improved Otsu method using a novel threshold value recognition function is proposed which is used to find the optimum threshold value in different types of histograms and separate into two classes. For the two classes separated by the above threshold value, mean values are computed. Based on the threshold value and the two mean values, the histogram of an image is divided iteratively into three regions, namely, the Foreground (FG) region, Background (BG) region and To-Be-Processed (TBP) region. The pixels greater than the larger mean belongs to FG region, the pixel which are lesser than the smaller mean belongs to BG region. The pixel that lies between the two mean values belongs to the TBP region and only this region is processed at next iteration, the FG and BG region are not processed further. Again for that TBP region, the novel threshold value recognition function is used to find a new threshold value and then the two mean values are computed to separate into three classes, namely, the FG, BG and TBP region. Then the new TBP region is processed in a similar manner iteratively until the two mean values computed at successive iterations are not changed. Finally all the previously determined foreground regions are combined separately and the background regions are combined separately to give the segmented image. The experimental results show that the proposed method performs well in detecting the objects and has better anti-noise performance compared with the existing methods.

This paper addresses the problem of segmenting an image into regions. We define a predicate for measuring the evidence for a boundary between two regions using a graph-based representation of the image. We then develop an efficient segmentation algorithm based on this predicate, and show that although this algorithm makes greedy decisions it produces segmentations that satisfy global properties. We apply the algorithm to image segmentation using two different kinds of local neighborhoods in constructing the graph, and illustrate the results with both real and synthetic images. The algorithm runs in time nearly linear in the number of graph edges and is also fast in practice. An important characteristic of the method is its ability to preserve detail in low-variability image regions while ignoring detail in high-variability regions.

Due to the complex structure and handwritten deformation, the offline handwritten Chinese characters recognition has been one of the most challenging problems. In this paper, an offline handwritten Chinese character recognition tool has been developed based on the Tesseract open source OCR engine. The tool mainly contributes on the following two points: First, a handwritten Chinese character features library is generated, which is independent of a specific user's writing style, Second, by preprocessing the input image and adjusting the Tesseract engine, multiple candidate recognition results are output based on weight ranking. The recognition accuracy rate of this tool is above 88% for both known user test set and unknown user test set. It has shown that the Tesseract engine is feasible for offline handwritten Chinese character recognition to a certain degree.

This paper presents a complete Optical Character Recognition (OCR) system for camera captured image/graphics embedded textual documents for handheld devices. At first, text regions are extracted and skew corrected. Then, these regions are binarized and segmented into lines and characters. Characters are passed into the recognition module. Experimenting with a set of 100 business card images, captured by cell phone camera, we have achieved a maximum recognition accuracy of 92.74%. Compared to Tesseract, an open source desktop-based powerful OCR engine, present recognition accuracy is worth contributing. Moreover, the developed technique is computationally efficient and consumes low memory so as to be applicable on handheld devices.

A real time static isolated gesture recognition application using a Hidden Markov Model approach with features extracted from gestures' silhouettes is presented. Nine different hand poses with various degrees of rotation are considered. The system, both simple and effective, uses color images of the hands to be recognized directly from the camera and is capable of processing 23 frames per second on a Quad Core Intel Processor.

Sketch recognition aims to automatically classify human hand sketches of objects into known categories. This has become increasingly a desirable capability due to recent advances in human computer interaction on portable devices. The problem is nontrivial because of the sparse and abstract nature of hand drawings as compared to photographic images of objects, compounded by a highly variable degree of details in human sketches. To this end, we present a method for the representation and matching of sketches by exploiting not only local features but also global structures of sketches, through a star graph based ensemble matching strategy. Different local feature representations were evaluated using the star graph model to demonstrate the effectiveness of the ensemble matching of structured features. We further show that by encapsulating holistic structure matching and learned bag-of-features models into a single framework, notable recognition performance improvement over the state-of-the-art can be observed. Extensive comparative experiments were carried out using the currently largest sketch dataset released by Eitz et al. [15], with over 20,000 sketches of 250 object categories generated by AMT (Amazon Mechanical Turk) crowd-sourcing.

CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart) are increasingly used in many applications for machine and human identification. Compared with traditional English and digital characters based CAPTCHAs, Chinese characters contain more complicated characters which greatly enhance difficulty of automatic recognition. To solve that problem, we proposed a Convolution Neural Network (CNN) based approach. This approach greatly improves the recognition accuracy of Chinese Character CAPTCHAs with distortion, rotation and background noise. Our experiment results show that this approach achieves more than 95% accuracy for single character and 84% accuracy for three types of Chinese Character CAPTCHAs with four characters. This encouraging result indicates that deep neural network is useful in complicated structure perception of Chinese Character CAPTCHAs.


Question: An implicit segmentation technique for the recognition of handwritten character strings.

This paper introduces an extensive database specifically designed for the advancement of off-line handwriting recognition systems. The database is built upon the Lancaster-Oslo/Bergen (LOB) corpus, a collection of written texts that have been utilized to create forms filled out by various individuals in their own handwriting. As of December 1998, the database encompasses 556 forms, each representing the unique handwriting of approximately 250 distinct writers. Notably, the database is comprised of complete English sentences, providing a rich resource for the development and evaluation of handwriting recognition algorithms. Its primary emphasis is on techniques that incorporate linguistic understanding beyond the basic lexicon level, enabling more sophisticated analysis. This linguistic knowledge can be automatically extracted from the corpus itself or sourced from external linguistic resources, enhancing the recognition capabilities of the system. The database, along with the accompanying preprocessing and text segmentation methods, offers a valuable tool for researchers aiming to improve the accuracy and efficiency of handwriting recognition systems, particularly in recognizing handwritten strings of characters.

A primary preprocessing block of almost any typical OCR system is binarization, through which it is intended to remove unwanted part of the input image, and only keep a binarized and cleaned-up version for further processing. The binarization step does not, however, always perform perfectly, and it can happen that binarization artifacts result in important information loss, by for instance breaking or deforming character shapes. In historical documents, due to a more dominant presence of noise and other sources of degradations, the performance of binarization methods usually deteriorates; as a result the performance of the recognition pipeline is hindered by such preprocessing phases. In this paper, we propose to skip the binarization step by directly training a 1D Long Short Term Memory (LSTM) network on gray-level text lines. We collect a large set of historical Fraktur documents, from publicly available online sources, and form train and test sets for performing experiments on both gray-level and binarized text lines. In order to observe the impact of resolution, the experiments are carried out on two identical sets of low and high resolutions. Overall, using gray-level text lines, the 1D LSTM network can reach 24% and 19% lower error rates on the low- and high-resolution sets, respectively, compared to the case of using binarization in the recognition pipeline.

We present a system for capturing ink strokes written with ordinary pen and paper using a fast camera with a frame rate comparable to a stylus digitizer. From the video frames, ink strokes are extracted and used as input to an online handwriting recognition engine. A key component in our system is a pen up/down detection model for detecting the contact of the pen-tip with the paper in the video frames. The proposed model consists of feature representation with convolutional neural networks and classification with a recurrent neural network. We also use a high speed tracker with kernelized correlation filters to track the pen-tip. For training and evaluation, we collected labeled video data of users writing English and Japanese phrases from public datasets, and we report on character accuracy scores for different frame rates in the two languages.

A document image is composed of a variety of physical entities or regions such as text blocks, lines, words, figures, tables, and background. We could also assign functional or logical labels such as sentences, titles, captions, author names, and addresses to some of these regions. The process of document structure and layout analysis tries to decompose a given document image into its component regions and understand their functional roles and relationships. The processing is carried out in multiple steps, such as preprocessing, page decomposition, structure understanding, etc. We will look into each of these steps in detail in the following sections. Document images are often generated from physical documents by digitization using scanners or digital cameras. Many documents, such as newspapers, magazines and brochures, contain very complex layout due to the placement of figures, titles, and captions, complex backgrounds, artistic text formatting, etc. (see Figure 1). A human reader uses a variety of additional cues such as context, conventions and information about language/script, along with a complex reasoning process to decipher the contents of a document. Automatic analysis of an arbitrary document with complex layout is an extremely difficult task and is beyond the capabilities of the state-of-the-art document structure and layout analysis systems. This is interesting since documents are designed to be effective and clear to human interpretation unlike natural images.

The Programming by Demonstration (PbD) technique aims at teaching a robot to accomplish a task by learning from a human demonstration. In a manipulation context, recognizing the demonstrator's hand gestures, specifically when and how objects are grasped, plays a significant role. Here, a system is presented that uses both hand shape and contact-point information obtained from a data glove and tactile sensors to recognize continuous human-grasp sequences. The sensor fusion, grasp classification, and task segmentation are made by a hidden Markov model recognizer. Twelve different grasp types from a general, task-independent taxonomy are recognized. An accuracy of up to 95% could be achieved for a multiple-user system.

The problem of efficient, interactive foreground/background segmentation in still images is of great practical importance in image editing. Classical image segmentation tools use either texture (colour) information, e.g. Magic Wand, or edge (contrast) information, e.g. Intelligent Scissors. Recently, an approach based on optimization by graph-cut has been developed which successfully combines both types of information. In this paper we extend the graph-cut approach in three respects. First, we have developed a more powerful, iterative version of the optimisation. Secondly, the power of the iterative algorithm is used to simplify substantially the user interaction needed for a given quality of result. Thirdly, a robust algorithm for "border matting" has been developed to estimate simultaneously the alpha-matte around an object boundary and the colours of foreground pixels. We show that for moderately difficult examples the proposed method outperforms competitive tools.

Off-line recognition requires transferring the text under consideration into an image file. This represents the only available solution to bring the printed materials to the electronic media. However, the transferring process causes the system to lose the temporal information of that text. Other complexities that an off-line recognition system has to deal with are the lower resolution of the document and the poor binarisation, which can contribute to readability when essential features of the characters are deleted or obscured. Recognising Arabic script presents two additional challenges: orthography is cursive and letter shape is context sensitive. Certain character combinations form new ligature shapes, which are often font-dependent. Some ligatures involve vertical stacking of characters. Since not all letters connect, word boundary location becomes an interesting problem, as spacing may separate not only words, but also certain characters within a word. Various techniques have been implemented to achieve high recognition rates. These techniques have tackled different aspects of the recognition system. This review is organised into five major sections, covering a general overview, Arabic writing characteristics, Arabic text recognition system, Arabic OCR software and conclusions.

We address the problem of query segmentation: given a keyword query, the task is to group the keywords into phrases, if possible. Previous approaches to the problem achieve reasonable segmentation performance but are tested only against a small corpus of manually segmented queries. In addition, many of the previous approaches are fairly intricate as they use expensive features and are difficult to be reimplemented.
 The main contribution of this paper is a new method for query segmentation that is easy to implement, fast, and that comes with a segmentation accuracy comparable to current state-of-the-art techniques. Our method uses only raw web n-gram frequencies and Wikipedia titles that are stored in a hash table. At the same time, we introduce a new evaluation corpus for query segmentation. With about 50,000 human-annotated queries, it is two orders of magnitude larger than the corpus being used up to now.

-This, paper introduces a new method for describing signature images. The description involves features of the signatures and relations among its parts organized in a hierarchical structure. The description is used to classify the signatures and globally analyse their structure. Description Segmentation Feature extraction Classification Signature analysis I. I N T R O D U C T I O N Although a lot of research on signatures has been reported in the past 20 years (t) they are mainly confined to verification in which a set of goal oriented features is used to give the decision about the authenticity of the claimed signature. However, it is interesting to be able to access a signature database using a compact and meaningful description derived from its pictorial content. Constructing such a signature database, requires an automatic description of signature images. Researches on signature description are very limited and appeared implicitly in verification researches. In on-line systems, Lamarche and Plamondon (z) reconsidered the theoretical model of the handwriting process introduced by Verdenberg and Koster <3) to describe the pen tip speed and orientation and used this description for feature extraction. In off-line systems, Nagel and Rosenfeld (̀ ~ modified the model introduced by Eden (~) for describing upright handwriting by adding the slant parameter and used the parameters of the modified model as a basis for choosing their features for verification, and Brocklehurst (6) used data reduction techniques to encode the signatures into a list of 7 features (called encoded description) and used them for verification. In all cases, neither a structural description of the signature as a 2-D image nor a conceptualization of the features was proposed. In this paper, we introduce a method for signature description and a reliable approach that can describe digitized signature images by this method. The description uses a hierarchical approach, i.e. the signature is at the top level and each lower level gives more detail about the higher one. Since the range of the natural variation of signature writing will change with time as found by Kapoor <~ and Evett, (s) and the ratios of the writing parameters for different samples of the same individual's writing remain essentially constant over a wide range of the writing surface and space available, ('~ features and relations of o u r description are extracted in a relative way (e.g. the length is measured by its ratio to the width, and so on) to attempt to make it insensitive to the natural variation in the genuine signatures. The signature is described using a global description suitable for purposes like classifying the signature data and studying its nature, and a local description convenient for applications which require more detailed information about the signature like signature analysis or similarity oriented retrieval. The global description is represented by a character string, and the local description is represented by a tree structure. In Section 2, the general approach for describing signature images is introduced, and Section 3 introduces the practical method of the description. Section 4 presents the application of the global description to classifying and analysing the signature data, and finally, Section 5 contains the closing remarks. 2. SIGNATURE DESCRIPTION Signatures are a special case of handwriting in which special characters and flourishes are available, and the ratio between the middle zone width and the letter heights is sometimes abnormal. In many cases, the signature is not readable even by a human. Therefore, we deal with the signature as an image that has the principal zones (upper, middle and lower zones) of the handwriting, not as strokes, letters and words. As indicated by Rosenfeld, (9) description of a picture involves properties of the picture or its parts and relationships among them. Our description involves features like slant, and relations among signature properties like the comparison between the widths of the zones. The description is not reversible, that is, the signature cannot be reconstructed from its description. However, the description involves sufficient information to characterize the signature and its parts. Features and relations are extracted quantitatively, and then mapped into a conceptual

Let X and Y be any two strings of finite length. The problem of transforming X to Y using the edit operations of substitution, deletion, and insertion has been extensively studied in the literature. The problem can be solved in quadratic time if the edit operations are extended to include the operation of transposition of adjacent characters, and is NP-complete if the characters can be edited repeatedly. In this paper we consider the problem of transforming X to Y when the set of edit operations is extended to include the squashing and expansion operations. Whereas in the squashing operation two (or more) contiguous characters of X can be transformed into a single character of Y, in the expansion operation a single character in X may be expanded into two or more contiguous characters of Y. These operations are typically found in the recognition of cursive script. A quadratic time solution to the problem has been presented. This solution is optimal for the infinite-alphabet case. The strategy to compute the sequence of edit operations is also presented.

Full end-to-end text recognition in natural images is a challenging problem that has received much attention recently. Traditional systems in this area have relied on elaborate models incorporating carefully hand-engineered features or large amounts of prior knowledge. In this paper, we take a different route and combine the representational power of large, multilayer neural networks together with recent developments in unsupervised feature learning, which allows us to use a common framework to train highly-accurate text detector and character recognizer modules. Then, using only simple off-the-shelf methods, we integrate these two modules into a full end-to-end, lexicon-driven, scene text recognition system that achieves state-of-the-art performance on standard benchmarks, namely Street View Text and ICDAR 2003.


Question: An implicit segmentation technique for the identification of handwritten character strings.

This correspondence introduces an innovative method for the rapid and automated two-class clustering of two-dimensional data, while preserving key characteristics of the input data. The proposed method stands out for its analytical, deterministic, unsupervised, and noniterative nature, offering a quick solution for clustering tasks. Unlike other methods that demand the calculation of an n-by-n dissimilarity matrix, this method operates with a computation time linear to the data size (n), significantly speeding up the process. Moreover, it eliminates the need for guessing initial values, making it ideal for applications requiring fast, automatic hierarchical clustering or any scenario that necessitates the swift two-class clustering of 2-D data. The method's versatility is further demonstrated through its potential extension to higher-dimensional data clustering, as exemplified by a 3-D clustering case study.

The development of this method has broad implications, particularly in the realm of machine learning and pattern recognition, where efficient clustering techniques are crucial. Its ability to handle data in higher dimensions without sacrificing speed or accuracy opens up new possibilities for data analysis and classification tasks across various fields, including but not limited to, bioinformatics, image processing, and customer segmentation in business analytics. The method's automatic and deterministic nature ensures consistency and reliability in clustering outcomes, which is essential for applications where decisions are based on the clustering results.

This paper proposes a method to recognize stroke orders of on-line handwritten Chinese characters based on analyzing both spatial and temporal information. A novel control-point-based analysis method is presented for spatial information analysis to match strokes with various shapes and styles. Its computation complexity is much lower than image correlation method and is suitable for applications on mobile devices. For temporal information analysis, Hidden Markov Model is adopted and a proposed rectification method is integrated to find the optimal pair-wise matching result of stroke sequences. Experimental results proved the effectiveness of the proposed method. The verification rate is 99.6% on the test set.

A method is presented for automated segmentation of vessels in two-dimensional color images of the retina. This method can be used in computer analyses of retinal images, e.g., in automated screening for diabetic retinopathy. The system is based on extraction of image ridges, which coincide approximately with vessel centerlines. The ridges are used to compose primitives in the form of line elements. With the line elements an image is partitioned into patches by assigning each image pixel to the closest line element. Every line element constitutes a local coordinate frame for its corresponding patch. For every pixel, feature vectors are computed that make use of properties of the patches and the line elements. The feature vectors are classified using a kNN-classifier and sequential forward feature selection. The algorithm was tested on a database consisting of 40 manually labeled images. The method achieves an area under the receiver operating characteristic curve of 0.952. The method is compared with two recently published rule-based methods of Hoover et al. and Jiang et al. . The results show that our method is significantly better than the two rule-based methods (p<0.01). The accuracy of our method is 0.944 versus 0.947 for a second observer.

In this paper, a new algorithm for vehicle license plate identification is proposed, on the basis of a novel adaptive image segmentation technique (sliding concentric windows) and connected component analysis in conjunction with a character recognition neural network. The algorithm was tested with 1334 natural-scene gray-level vehicle images of different backgrounds and ambient illumination. The camera focused in the plate, while the angle of view and the distance from the vehicle varied according to the experimental setup. The license plates properly segmented were 1287 over 1334 input images (96.5%). The optical character recognition system is a two-layer probabilistic neural network (PNN) with topology 108-180-36, whose performance for entire plate recognition reached 89.1%. The PNN is trained to identify alphanumeric characters from car license plates based on data obtained from algorithmic image processing. Combining the above two rates, the overall rate of success for the license-plate-recognition algorithm is 86.0%. A review in the related literature presented in this paper reveals that better performance (90% up to 95%) has been reported, when limitations in distance, angle of view, illumination conditions are set, and background complexity is low

In this paper, we propose a new unsupervised approach for word segmentation. The core idea of our approach is a novel word induction criterion called WordRank, which estimates the goodness of word hypotheses (character or phoneme sequences). We devise a method to derive exterior word boundary information from the link structures of adjacent word hypotheses and incorporate interior word boundary information to complete the model. In light of WordRank, word segmentation can be modeled as an optimization problem. A Viterbistyled algorithm is developed for the search of the optimal segmentation. Extensive experiments conducted on phonetic transcripts as well as standard Chinese and Japanese data sets demonstrate the effectiveness of our approach. On the standard Brent version of BernsteinRatner corpora, our approach outperforms the state-ofthe-art Bayesian models by more than 3%. Plus, our approach is simpler and more efficient than the Bayesian methods. Consequently, our approach is more suitable for real-world applications.

We present a novel single-shot text detector that directly outputs word-level bounding boxes in a natural image. We propose an attention mechanism which roughly identifies text regions via an automatically learned attentional map. This substantially suppresses background interference in the convolutional features, which is the key to producing accurate inference of words, particularly at extremely small sizes. This results in a single model that essentially works in a coarse-to-fine manner. It departs from recent FCN-based text detectors which cascade multiple FCN models to achieve an accurate prediction. Furthermore, we develop a hierarchical inception module which efficiently aggregates multi-scale inception features. This enhances local details, and also encodes strong context information, allowing the detector to work reliably on multi-scale and multi-orientation text with single-scale images. Our text detector achieves an F-measure of 77% on the ICDAR 2015 benchmark, advancing the state-of-the-art results in [18, 28]. Demo is available at: http://sstd.whuang.org/.

In languages that use diacritical characters, if these special signs are stripped-off from a word, the resulted string of characters may not exist in the language, and therefore its normative form is, in general, easy to recover. However, this is not always the case, as presence or absence of a diacritical sign attached to a base letter of a word which exists in both variants, may change its grammatical properties or even the meaning, making the recovery of the missing diacritics a difficult task, not only for a program but sometimes even for a human reader. We describe and evaluate an accurate knowledge-based system for automatic recovering the missing diacritics in MSOffice documents written in Romanian. For the rare cases when the system is not able to reliably make a decision, it either provides the user a list of words with their recovery suggestions, or probabilistically choose one of the possible changes, but leaves a trace (a highlighted comment) on each word the modification of which was uncertain.

Most text detection methods hypothesize texts are horizontal or multi-oriented and thus define quadrangles as the basic detection unit. However, text in the wild is usually perspectively distorted or curved, which can not be easily tackled by existing approaches. In this paper, we propose a deep character embedding network (CENet) which simultaneously predicts the bounding boxes of characters and their embedding vectors, thus making text detection a simple clustering task in the character embedding space. The proposed method does not require strong assumptions of forming a straight line on general text detection, which provides flexibility on arbitrarily curved or perspectively distorted text. For character detection task, a dense prediction subnetwork is designed to obtain the confidence score and bounding boxes of characters. For character embedding task, a subnet is trained with contrastive loss to project detected characters into embedding space. The two tasks share a backbone CNN from which the multi-scale feature maps are extracted. The final text regions can be easily achieved by a thresholding process on character confidence and embedding distance of character pairs. We evaluated our method on ICDAR13, ICDAR15, MSRA-TD500, and Total Text. The proposed method achieves state-of-the-art or comparable performance on all of the datasets, and shows a substantial improvement in the irregular-text datasets, i.e. Total-Text.

We tackle the problem of one-shot segmentation: finding and segmenting a previously unseen object in a cluttered scene based on a single instruction example. We propose a novel dataset, which we call cluttered Omniglot. Using a baseline architecture combining a Siamese embedding for detection with a U-net for segmentation we show that increasing levels of clutter make the task progressively harder. Using oracle models with access to various amounts of ground-truth information, we evaluate different aspects of the problem and show that in this kind of visual search task, detection and segmentation are two intertwined problems, the solution to each of which helps solving the other. We therefore introduce MaskNet, an improved model that attends to multiple candidate locations, generates segmentation proposals to mask out background clutter and selects among the segmented objects. Our findings suggest that such image recognition models based on an iterative refinement of object detection and foreground segmentation may provide a way to deal with highly cluttered scenes.

Due to the complex structure and handwritten deformation, the offline handwritten Chinese characters recognition has been one of the most challenging problems. In this paper, an offline handwritten Chinese character recognition tool has been developed based on the Tesseract open source OCR engine. The tool mainly contributes on the following two points: First, a handwritten Chinese character features library is generated, which is independent of a specific user's writing style, Second, by preprocessing the input image and adjusting the Tesseract engine, multiple candidate recognition results are output based on weight ranking. The recognition accuracy rate of this tool is above 88% for both known user test set and unknown user test set. It has shown that the Tesseract engine is feasible for offline handwritten Chinese character recognition to a certain degree.

Data augmentation is usually used by supervised learning approaches for offline writer identification, but such approaches require extra training data and potentially lead to overfitting errors. In this study, a semi-supervised feature learning pipeline was proposed to improve the performance of writer identification by training with extra unlabeled data and the original labeled data simultaneously. Specifically, we proposed a weighted label smoothing regularization (WLSR) method for data augmentation, which assigned the weighted uniform label distribution to the extra unlabeled data. The WLSR method could regularize the convolutional neural network (CNN) baseline to allow more discriminative features to be learned to represent the properties of different writing styles. The experimental results on well-known benchmark datasets (ICDAR2013 and CVL) showed that our proposed semisupervised feature learning approach could significantly improve the baseline measurement and perform competitively with existing writer identification approaches. Our findings provide new insights into offline write identification.


Question: An implicit segmentation technique for the recognition of handwritten character strings.

This paper explores the utilization of hidden Markov models (HMMs) in the domain of continuous speech recognition, offering a comprehensive review and analysis. HMMs are contextualized as an evolution from dynamic programming, their predecessor in speech recognition technology. A holistic perspective is adopted, merging linguistic decoding and acoustic matching within a singular, optimized network search structure. Innovations in recognition architectures are highlighted, with a particular focus on the Viterbi beam search algorithm, a cornerstone in real-time speech recognition for its efficiency. The methodologies for estimating probabilities linked to an HMM model are scrutinized, alongside an in-depth look at the HMM-supervised training approach. The paper concludes with a series of case studies, showcasing the effectiveness of HMM-based systems in speech recognition, underscoring their successful implementation in various applications.

This paper describes the Chinese handwriting recognition competition held at the 12th International Conference on Document Analysis and Recognition (ICDAR 2013). This third competition in the series again used the CASIA-HWDB/OLHWDB databases as the training set, and all the submitted systems were evaluated on closed datasets to report character-level correct rates. This year, 10 groups submitted 27 systems for five tasks: classification on extracted features, online/offline isolated character recognition, online/offline handwritten text recognition. The best results (correct rates) are 93.89% for classification on extracted features, 94.77% for offline character recognition, 97.39% for online character recognition, 88.76% for offline text recognition, and 95.03% for online text recognition, respectively. In addition to the test results, we also provide short descriptions of the recognition methods and brief discussions on the results.

Individual optical character recognition (OCR) engines vary in the types of errors they commit in recognizing text, particularly poor quality text. By aligning the output of multiple OCR engines and taking advantage of the differences between them, the error rate based on the aligned lattice of recognized words is significantly lower than the individual OCR word error rates. This lattice error rate constitutes a lower bound among aligned alternatives from the OCR output. Results from a collection of poor quality mid-twentieth century typewritten documents demonstrate an average reduction of 55.0% in the error rate of the lattice of alternatives and a realized word error rate (WER) reduction of 35.8% in a dictionary-based selection process. As an important precursor, an innovative admissible heuristic for the A* algorithm is developed, which results in a significant reduction in state space exploration to identify all optimal alignments of the OCR text output, a necessary step toward the construction of the word hypothesis lattice. On average 0.0079% of the state space is explored to identify all optimal alignments of the documents.

We present an efficient method for character skew rectification in scene text images. The method is based on a novel skew estimators, which exploit intuitive glyph properties and which can be efficiently computed in a linear time. The estimators are evaluated on a synthetically generated data (including Latin, Cyrillic, Greek, Runic scripts) and real scene text images, where the skew rectification by the proposed method improves the accuracy of a state-of-the-art scene text recognition pipeline.

Recognition and retrieval of historical handwritten material is an unsolved problem. We propose a novel approach to recognizing and retrieving handwritten manuscripts, based upon word image classification as a key step. Decision trees with normalized pixels as features form the basis of a highly accurate AdaBoost classifier, trained on a corpus of word images that have been resized and sampled at a pyramid of resolutions. To stem problems from the highly skewed distribution of class frequencies, word classes with very few training samples are augmented with stochastically altered versions of the originals. This increases recognition performance substantially. On a standard corpus of 20 pages of handwritten material from the George Washington collection the recognition performance shows a substantial improvement in performance over previous published results (75% vs 65%). Following word recognition, retrieval is done using a language model over the recognized words. Retrieval performance also shows substantially improved results over previously published results on this database. Recognition/retrieval results on a more challenging database of 100 pages from the George Washington collection are also presented.

Scene text detection has been made great progress in recent years. The detection manners are evolving from axisaligned rectangle to rotated rectangle and further to quadrangle. However, current datasets contain very little curve text, which can be widely observed in scene images such as signboard, product name and so on. To raise the concerns of reading curve text in the wild, in this paper, we construct a curve text dataset named CTW1500, which includes over 10k text annotations in 1,500 images (1000 for training and 500 for testing). Based on this dataset, we pioneering propose a polygon based curve text detector (CTD) which can directly detect curve text without empirical combination. Moreover, by seamlessly integrating the recurrent transverse and longitudinal offset connection (TLOC), the proposed method can be end-to-end trainable to learn the inherent connection among the position offsets. This allows the CTD to explore context information instead of predicting points independently, resulting in more smooth and accurate detection. We also propose two simple but effective post-processing methods named nonpolygon suppress (NPS) and polygonal non-maximum suppression (PNMS) to further improve the detection accuracy. Furthermore, the proposed approach in this paper is designed in an universal manner, which can also be trained with rectangular or quadrilateral bounding boxes without extra efforts. Experimental results on CTW-1500 demonstrate our method with only a light backbone can outperform state-of-the-art methods with a large margin. By evaluating only in the curve or non-curve subset, the CTD + TLOC can still achieve the best results. Code is available at https://github.com/Yuliang-Liu/Curve-Text-Detector.

Surveying a suite of algorithms that offer a solution to managing large document archives.

This paper analyzes, compares, and contrasts technical challenges, methods, and the performance of text detection and recognition research in color imagery. It summarizes the fundamental problems and enumerates factors that should be considered when addressing these problems. Existing techniques are categorized as either stepwise or integrated and sub-problems are highlighted including text localization, verification, segmentation and recognition. Special issues associated with the enhancement of degraded text and the processing of video text, multi-oriented, perspectively distorted and multilingual text are also addressed. The categories and sub-categories of text are illustrated, benchmark datasets are enumerated, and the performance of the most representative approaches is compared. This review provides a fundamental comparison and analysis of the remaining problems in the field.

Pattern recognition is a tough problem for computers, although humans are wired for it. Pattern recognition is becoming increasingly important in the age of automation and information handling and retrieval. This paper reviews possible application areas of Pattern recognition. Author covers various sub-disciplines of pattern recognition based on learning methods, such as supervised, unsupervised, semi-supervised learning and key research areas such as grammar induction. Novel solutions to these possible problems could be well deployed for character recognition, speech analysis, man and machine diagnostics, person identification, industrial inspection and so on. The paper concludes with brief discussion on open issues that need to be addressed by future researchers.

Building a complete personalized Chinese font library for an ordinary person is a tough task due to the existence of huge amounts of characters with complicated structures. Yet, existing automatic font generation methods still have many drawbacks. To address the problem, this paper proposes an end-to-end learning system, DCFont, to automatically generate the whole GB2312 font library that consists of 6763 Chinese characters from a small number (e.g., 775) of characters written by the user. Our system has two major advantages. On the one hand, the system works in an end-to-end manner, which means that human interventions during offline training and online generating periods are not required. On the other hand, a novel deep neural network architecture is designed to solve the font feature reconstruction and handwriting synthesis problems through adversarial training, which requires fewer input data but obtains more realistic and high-quality synthesis results compared to other deep learning based approaches. Experimental results verify the superiority of our method against the state of the art.

Arabic Optical Character Recognition (OCR) as Web Services represents a major challenge for handwritten document recognition. A variety of approaches, methods, algorithms and techniques have been proposed in order to build powerful Arabic OCR web services. Unfortunately, these methods could not succeed in achieving this mission in case of large large quantity Arabic handwritten documents. Intensive experiments and observations revealed that some of the existing approaches and techniques are complementary and can be combined to improve the recognition rate. Designing and implementing these recent sophisticated complementary approaches and techniques as web services are commonly complex; they require strong computing power to reach an acceptable recognition speed especially in case of large quantity documents. One of the possible solutions to overcome this problem is to benefit from distributed computing architectures such as cloud computing. This paper describes the design and implementation of Arabic Handwriting Recognition as a web service (AHRweb service) based on the complementary approach K-Nearest Neighbor (KNN) /Support Vector Machine (SVM) (K-NN/SVM) via Amazon Elastic MapReduce (EMR) model. The experiments were conducted on a cloud computing environment with a real large scale handwriting dataset from the Institute for Communications Technology (IFN)/ Ecole Nationale d’Ingénieur de Tunis (ENIT) IFN/ENIT database. The J-Sim (Java Simulator) was used as a tool to generate and analyze statistical results. Experimental results show that Amazon Elastic MapReduce (EMR) model constitutes a very promising framework for enhancing large AHRweb service performances.