Skip navigation
Taylor Ripke at the computer

Computer with a key difference

Faculty-student team creates system for people with physical disabilities

Contact: Gary H. Piatek


​When you walk into any coffee shop, college library or office, you see fingers flying across computer keyboards.

But what if you don't have fingers, or can't use the ones you have? How do you communicate? How do you fit in?

That's the issue Central Michigan University computer science faculty member Tony Morelli and graduate assistant Taylor Ripke, of McBain, Michigan, are tackling. They're refining a computer system that has a projected image in place of a keyboard and a monitor that doubles as a camera.

 

Morelli recently demonstrated the system at an assistive technology conference and trade show at California State University, Northridge.

How it works

The "keyboard" is a flat pad onto which a few isolated characters and commands are projected — not the entire alphabet. By moving a finger, hand or the end of an arm, the user chooses characters, tabs and spaces. When the user indicates the empty space between two of the projected letters, say A and G, the letters within that range will appear and can be selected.

The computer's camera sees the choices and adds them to the screen as the user "types."

"I've never seen that many people interested in something so quickly." — Tony Morelli, CMU computer science faculty

"It doesn't matter what the person's hand looks like or how it functions, as long as it's in view of the camera. Any person can use the system as long as they can move their finger, hand or arm to the letter they want," Ripke said.

Growth of an idea

In the late 1990s at Purdue University, Morelli was part of a service-learning senior project that was to address human, community and environmental needs. He knew people whose children with motor impairments wanted to use computers, and his project group decided to try to help.

He found a computer touchpad and experimented. They used the ending of a nursery rhyme as a signal for a child to push a green button on the pad to continue, and he substituted a green sheet of paper for the button. The idea was that once the child learned to hit the sheet of paper, it would be cut in half, and the child would learn to hit smaller and smaller targets as steps on the way toward using a keyboard.

"We didn't quite get that far," he said.

But the idea of helping people with disabilities who couldn't use a keyboard stuck in his head.

Later, when Morelli saw the computer built by Hewlett-Packard called HP Sprout, with a touch pad in place of a standard keyboard, the idea of an accessible keyboard came back to him. He contacted HP, and it lent him some Sprouts.

He sketched out a plan and gave it to his undergraduate student at the time, Brad Wojcik, to implement for his senior project. Wojcik programmed the first version of the system, conducted a study where people would try it using a finger and fist, and got good results, Morelli said.

cut-ripke.jpg

Taylor Ripke uses a fist to "type" without a keyboard.

Changing hands

Morelli chose Ripke to help advance the project because of his expertise in computer vision.

"There are few people who have his ability to handle this processing."

Now the team is thinking of other ways the interactive technology can be used, perhaps including people who are visually impaired.

"We're looking at possibly adding sound so when a person hovers over a letter, it is announced out loud," Ripke said. "Once they learn where the letters are and the sound, they will be able to create words quickly."

Morelli and Ripke would like to develop the system for elementary school classroom use.

"The technology is very new," Ripke said. "We are just beginning to explore its capabilities."

Hopeful horizon

The system received rave reviews at the California conference, Morelli said.

"Everyone was blown away by the capabilities of the system," he said. "I've never seen that many people interested in something so quickly."

The bulk of those interested, he said, were K-12 special education teachers with students who have physical impairments.

"People were asking, 'When can I get this?'"

Morelli's unsure.

The next advancements would take more development time, testing and maintenance, he said.

"I see how many lives this could potentially affect, but my time is spread across other potentially life-changing projects. I'm not exactly sure what I'm going to do next on this one. I'm torn," he said.

He is considering polishing up the project a bit and then turning it over for someone to take it to the finish line.

"It really needs support," he said.


Photo Associator

Article Photo Title

Photo Title required.

Photo for News Home

Select File
{{vm.homeFile.fileName}}
Upload
Use This One

Photo for News Feeds

Select File
{{vm.feedFile.fileName}}
Upload
Use This One