http://youtu.be/rlopidricZ0
We all have something that makes us anxious, but if you get stressed at the prospect of social interactions, help may be at hand.
A new faculty member at the University of Rochester in western New York has created a computer program that offers people the chance to practice social interactions in their own home, and get feedback on how they come across.
The system is called the MACH program (My Automated Conversation coacH) and it uses an animated character that can listen, react, and respond to a user in real time.
Mary is the face of the current MACH program, and as you talk to her she blinks, nods, and moves her head as if she’s really listening to what you’re saying.
“She’s moving her head and that’s not happening in a random way. She’s actually looking at my head movement and finding out what’s my head orientation and she wants to mirror those behaviors. It feels like she’s paying attention, as opposed to I’m talking to a computer program,” says creator Eshan Hoque.
The system came about after Professor Hoque received requests from people who struggled with social interactions, but were unsure about how and where to seek help.
“We think that the best way you can get help on your social interaction is to get help from another human,” Hoque says.
“So if you have that ability go and do it. But, some people may have a fear of social stigma, or they may not want to go and take that extra step to get help. Or sometimes when you do desire help, the help may not be available.”
That, says Hoque, is where an automated conversation coach can come in handy. The program records your interactions with Mary through a web cam and then crunches the data about both your verbal and non-verbal communication skills.
“So it’s analyzing every single frame of the video to see where exactly you smile. It’s analyzing every single word you’re saying, So it’s doing speech recognition, and not only what you’re saying, but how you’re saying it.”
While you talk to Mary your head movements, smiles, voice tone and volume, degree of eye contact and use of filler words like ‘um’ are all recorded and evaluated by the system.
She’ll also call you out if you don’t pay enough attention to her or let your eyes wander for too long, saying things like; “Are you still there? I can’t find you.”
At the end of the interaction, users get see the breakdown of their nonverbal behavior during the conversation.
And, Hoque says, that can be revealing.
“As you get all that information, it reveals a lot of things about you. Some students told us they didn’t realize they didn’t smile at all. Even though they felt that they appeared friendly, when they looked at the graph there was no sign of smile.”
As a proof of concept for the system, Hoque chose to program it to mimic a scenario most of us have to go through eventually – a job interview, where Mary is the interviewer.
To assess how effective the MACH program is at helping to make social interaction more fluid, Hoque first had 90 undergraduate students undergo a mock interview with a real career counselor.
Students then sat through a session with Mary, and after seeing the feedback about their interactions, they underwent a second mock interview to see if the experience had altered their behavior.
“What we noticed was that students that interacted with Mary demonstrated better skills in dimensions like overall improvement, they sound excited about the job, they appear friendly. So this is a situation where you can see that they interact with the system and they were able to reflect that behavior as they interact with the career counselor,” says Hoque.
The next step, he says, is to create a variety of scenarios that could help people in other situations outside the job interview setting.
“I’d love to try this out in the context of helping people who may have some social difficulties, or Asperger’s syndrome. So that’s something we’re exploring. Other possibilities are helping people with public speaking.”
Hoque says the system could also be adapted to train customer service representatives or help individuals with post-traumatic stress disorder assimilate back into society.
He’s hoping the system will be available online as a free service within the next two years.
