Discuss Japan > Archives > Society > Diversity Opens the Path to Innovation
LinkedInTumblrDeliciousYahoo Bookmarks
Society ,No.46  May 18, 2018

Diversity Opens the Path to Innovation


Asakawa Chieko, Fellow, IBM Research-Tokyo

I joined IBM Research—Tokyo in 1985 as the only visually impaired researcher at a time when there were very few female researchers at the company. Since then I have brought a diversity perspective to my work in accessibility research, one of the fields in Human Computer Interaction (HCI). Aiming to optimize Braille library creation and sharing, I participated in the research and development of digital Braille editors, dictionary systems, and Braille information sharing networks after joining the company. I was able to move the research forward because my visual impairment meant that I understood the value of digitizing Braille. Starting in the mid-1990s, I worked on a text-to-speech reader for the Internet. This idea also emerged out of the needs of the visually impaired, but since then it has spread in ways I never expected. At present, I am working on new assistive technologies using smartphones, the Internet of Things (IoT), Artificial Intelligence (AI), and other rapidly developing technologies. In this article, I consider the role of diversity as I have experienced it through these projects.

(1) Information Accessibility

I lost my sight in a swimming pool accident when I was a student at junior high school. So, I have experienced the problems with social participation for the visually impaired from the perspective of a young person. Generally speaking, young people with visual impairments encounter two great barriers to receiving an education and participating in society. One is the information barrier and the other one is the movement barrier. When I lost my sight, there were no personal computers, no Internet, and no smartphones. The only way to read was Braille books created by punching holes in paper. There were few Braille books and they rarely included any of the texts required to obtain a high school education. Since translating books to Braille is time-consuming, several months would pass between requesting and obtaining a text required for university classes. These experiences inspired me to start the Braille digitizing project after joining IBM. With digitalization, it became possible to delete characters, edit texts, and the Braille translation work could be shared among several people over a network. In addition, Braille book data could be downloaded and printed on a Braille printer anywhere in Japan. It became possible to search the text and portable electronic Braille dictionaries were produced. [i] These technologies changed education for the visually impaired in important ways.

The amount of available information expanded with the digitalization of Braille, but information sources were still limited to Braille and recorded books. Then, the Web came on the scene in the mid-1990s. Since the Web was still a new technology at the time, it was only used by engineers and a few other users. When I first accessed the Web with the help of other researchers at IBM Research, I was convinced that the vast amounts of text and voice information would become a new information source for the visually impaired. So, I started to research and develop a voice browser for the Web combined with a voice synthesizer engine.[ii] Later, the research was turned into a product and became the Home Page Reader, the popular de facto standard. Gradually, as the need to access the Web using voice became widely recognized, voice access was incorporated into the international standard for the Web as a mandatory item, and compatibility with a diverse range of needs (access methods, input devices, screen size, etc.) became a major focus of Web development. In addition, the websites of federal agencies in the United States have to be accessible in a variety of ways in line with the 1998 amendments to Section 508 of the US Rehabilitation Act.

As a result, the development of ICT has vastly improved information accessibility for the visually impaired. The sources of information for the visually impaired have grown exponentially from Braille on paper to digital Braille, and then the Internet. This has also had a great impact on technology standards and government legislation.

(2) Voice Synthesizer Evolution and Diversity

It is not well known that the visually impaired played a major role in the development of the technologies for voice synthesizers. The history of voice synthesizer technologies dates back to research and development that began in the 1960s, but the first voices had a robotic quality. When personal computers became popular in the 1980s, general users had more opportunities to hear the synthetic voices, but the voice quality was still a long way from the human voice. However, voice synthesizer technology was indispensable to the visually impaired when using personal computers. Voice synthesizers were used on a daily basis to read textual information and to create text using word processors. So, with the exception of some special applications, the visually impaired were almost the only users of voice synthesizer technologies in the 1980s and 1990s. When I developed the Home Page Reader in 1997, many able-bodied people commented that they could not understand the voice but the visually impaired had no problem. The voice synthesizer was revolutionary in the sense that it expanded the sources of information, and the quality of the sound was not an issue at all. The visually impaired had used voice synthesizer technologies continually from the days when the sound quality lacked clarity and, unexpectedly, they also played a role in the development of voice synthesizer technologies by providing feedback to developers. Now, in 2017, voice synthesizer technologies exist all around us. They are used everywhere including car navigation systems, smartphones, at train stations, and at airports. I think it would have been difficult to develop the technologies without the efforts of the visually impaired who persevered and continued using them from the 1980s to the 2000s. 

The examples of technologies that were developed and became widespread after emerging out of the needs of people with impairments are too numerous to mention. If you peruse history, you will find that the telephone was originally invented in the process of developing a communication tool for the hearing impaired. Keyboards were allegedly developed as a means for people with upper limb impairments to write. Character recognition was first used in text reading devices for the visually impaired. Voice recognition technologies were developed as a method for the hearing impaired to converse by voice. A major goal of self-driving cars was to develop cars that could be operated by the visually impaired by 2010. The perspectives of diversity and the extreme needs imposed by not being able to see or hear have triggered the creation and development of new technologies.

(3) AI for the Visually Impaired

When I was a child, I watched a television program that featured a bird-shaped robot that assisted a boy going into battle. The robot sat on the boy’s shoulder and whispered into his ear, telling him about everything from an approaching opponent to the weather. When I lost my sight, I remembered that program and wished for a bird robot. Of course, this robot was simply science fiction drawn in the 1960s. However, as the age of artificial intelligence and the Internet of Things approaches, I think that robot is within the range of what technology can do. Researchers refer to AI technologies that get close to people like that bird robot did as cognitive assistants. They can help people with impairments or the elderly by using AI to compensate for or amplify lost or weakened sensory or cognitive functions. There is ongoing research and development worldwide into cognitive assistants, which are a new concept in accessibility technologies using AI.   

With the help of cognitive assistants, the visually impaired will not only be able to recognize obstacles at street crossings, traffic lights, and on the sidewalks, but they will probably be able to recognize the information they need to independently negotiate stairways, escalators, and elevators. Cognitive assistants should also be able to recognize the ages and expressions of conference participants and to communicate the information as necessary. By memorizing everything that the elderly see, they could also serve as tools to supplement the memory. A cognitive assistant is a technology that will always be at your side ready to provide assistance as needed.

Four groups of technologies are indispensable to make cognitive assistants reality. We have localization technologies. To assist the user in the day-to-day environment, it is necessary to measure indoor and outdoor location with a high degree of accuracy. Since GPS technologies lack accuracy and cannot be used indoors, we are developing technologies to measure location with a high degree of accuracy using Wi-Fi, Bluetooth Low Energy (BLE) beacons, and image processing technologies. The NavCog system, developed in collaboration with Carnegie Mellon University, uses BLE beacons to measure position with an accuracy of one to two meters. The NavCog system has been installed in the three buildings of the School of Computer Science at Carnegie Mellon University to guide users to their destinations with the help of a highly accurate navigation system that identifies classrooms and labs inside the building.

Next, there are recognition technologies—image processing technology being the most important one when implementing cognitive assistants. If visually impaired persons are able to recognize people, expressions, products, structures inside buildings (stairs, escalators, elevators, doors, etc.), obstacles, as well as people, objects, and the environment, they will be able to obtain the information they need to participate in society in a timely fashion. This would be a change similar to the one that made information accessible.

Knowledge is necessary to make use of the outcomes of recognition. Recognition of products, calorie information, and social media reputation is a given, but it may also be possible to make cognitive assistants more relevant by using knowledge about the individual such as behavior history or health information. Lastly, there is interaction technology. Voice interaction is a given, but there is also potential for cognitive assistants that can be used seamlessly in daily life with the help of glasses-style interfaces for always-on recognition, or gesture interfaces. It is also important to broaden the field of application beyond devices such as smartphones and wearable technologies to robot technologies.

(4) Open Source and Open Data

The technologies needed for cognitive assistants are varied and have the added dimension of a showcase for integrating AI technologies. It is something that a single organization would find difficult to achieve and that can only be implemented by marshaling the technologies of universities and the private sector. To implement such integration, open source is likely to have an important role in the future. Today, many companies and universities use TensorFlow, Google’s machine learning library. Inception, the object recognition engine based on Deep Learning running on TensorFlow, is an example of the rise in the use of open source. Aiming to popularize measurement technologies, we made NavCog open source while developing a reusable form. [iii] We hope you will make use of it.

Open data is another important issue. Indoor mapping information is necessary to achieve indoor navigation. However, indoor mapping information is normally not available to the public as it is the property of the building owner. Depending on the country, there are also completely different arrangements for managing outdoor mapping. Huge amounts of image data of product packaging is required for learning purposes to recognize and read a candy bar or other product in a store. However, the manufacturer owns the copyright to such image data and it is not possible to use it freely. As we move into the AI age, we will need new rules for open data. To facilitate reading with NavCog, we are considering setting up an open server to register information in our immediate vicinity such as store information, sales information, signboards, and information about places where there are lots of people.   

To make cognitive assistants reality we cannot avoid the issue of open data. Moreover, it is no exaggeration to say that open data is an issue that society as a whole should engage with as we move toward using AI technologies. As history has shown, the needs of people with impairments will trigger and facilitate research and development of open data and, eventually, artificial intelligence. Doubtlessly, this development will add to the list of precedents where diversity has opened up a new future.  


We often hear about the importance of diversity in innovation. However, it is difficult to cite examples. This article introduces historical examples based on my own experience. In the process of developing information accessibility, a variety of technologies were created and popularized. To make cognitive assistants reality it will be necessary to develop more varied technologies. Every day I sense the beginnings of great innovation. I hope that readers of this article will develop a better understanding of innovation through diversity.

Translated from ”Tokushu I: Jenda to kagaku no atarashii torikumi Tayosei ga hiraku inobeishon (Special feature: New efforts for gender and science ―Diversity Opens the Path to Innovation),” Gakujutsu no Doko (TRENDS IN THE SCIENCES), November 2017, pp.24-28. (Courtesy of Japan Science Support Foundation) [November 2017]

[i] Asakawa Chieko. “A Braille English/Japanese Dictionary System” [in Japanese]. Journal of Information Processing 34 (8), 1844-1852, 1993.

[ii] Asakawa Chieko, Takashi Itoh. User Interface of a Home Page Reader. Proceedings of the Third International ACM Conference on Assistive Technologies, 149-156, ACM, 1998.

[iii] NavCog. http://facebook.com/navcog