When he spoke to us on Skype about his new project building the bridge between disability and electronic composition, Andreas Refsgaard, a Danish man of barely 30, showed a happy and easy smile. He has every right: for his final degree project, he developed over a period of only eight weeks a system to enable seriously handicapped people to enter the world of electronic music production. Based on a facial recognition system and linked to the Ableton and Max for Live (one of its packs), the initiative is still at the prototype stage but could quickly inspire others.
While supporting disabled access to music is no new concept (we all remember the Finnish punk group PKN, who reached the semi-final of Eurovision 2015 and were composed solely of middle-aged sufferers of trisomy 21), it had never been taken to such an extreme level of disability; Andreas’ invention is aimed at people of whom some have lost almost all control of their limbs. And although similar systems of facial recognition have already given disabled people access to artistic expression, notably through drawing and digital writing, electronic music production remains a possibility practically unexplored.
Breaking the boundaries of creative expression
Andreas Refsgaard has just completed his studies in digital design, a term which includes the development of new interactive interfaces between man and machine. While for a long time he found himself immersed in rock, the student is also very fond of electronic music: ‘my ex-girlfriend belonged to the Danish and European techno scenes, she was a DJ and a producer. She taught me to love electronic music, and also to compose with Ableton and Max for Live. Previously, I would play guitar in my bedroom, so I discovered a whole new way to see musical creation.’
But why did he look to disability for his final degree project? Andreas remembers: ‘Every night after school I used to take care of a young boy affected by dystrophy, a nervous illness which gradually paralyses muscles through the body. Each thing the child was unable to do, I had to do for him. I realised how much he was limited, in particular in terms of his artistic expression.’ From the episode, he came to the conclusion that sowed the seed of his project: ‘There are already plenty of solutions to the difficulties of daily life, but the fact remains that they are not given many ways to artistically express themselves.’
An interface which adapts to the level of impairment
The system invented by Andreas is very simple to use and relies on the recognition of facial movement, like that of the eyes, the mouth and the eyebrows. Notes arranged in a circle are controlled by the movement of the pupils, and their pitch adjusted by a raising or lowering of the eyebrows. Three effects complete the package, you simply open your mouth to varying degrees.
That’s not all: thanks to the Ableton interface, the user can customise that of Eye Conductor at their leisure, with access to several different instruments and effects, and change the number of notes on the screen depending on their ability to move their pupils. The possibilities are infinite, as his short presentation video proves:
As for the cost of this device? ‘Money has not really been a problem,‘ the young man asserts, explaining that he worked with his computer’s webcam, the cheapest eye tracking technology on the market, and software that was either free or of his own design. Along the way he was met with a couple of compatibility issues, like ‘a few eye tracking components, which belonged to the research subjects, conflicting with the interface‘, but nothing which threatened the fair-haired boy’s motivation.
Because for Andreas, the greatest difficulty lies elsewhere: ‘It’s adapting my project to the users in question that brought the most problems. I designed the device sat in front of my PC, without taking it into account that certain disabled people could have their heads leaning for example. I deliberately chose not to linger on that detail due to the little time that I had.‘
‘Some care centers never got back to me’
At the last minute, unexpected setbacks. Andreas explains, ‘a little disappointed’ by the reactions of some of those responsible for disabled people: ‘I contacted lots of schools and different centers . Some never got back to me, and others remained skeptical as to the project’s feasibility. I can’t understand their reticence.’ But the young Dane is keen to note that ‘today the same centers seem interested and understand that the handicapped would enjoy the experience.‘ He was enthusiastic about the success of his project: ‘My subjects couldn’t speak, but to see their expressions gave me no doubt that they immensely enjoyed the experience.‘
Andreas’ project was a roaring success with the disabled users. (©Andreas Refsgaard)
Since the end of this incredible project, Andreas took himself into Amsterdam exile for a six month internship in a completely different field. He still doesn’t know if he will have the time to make good on all the improvements he is thinking of bringing to the system. ‘I would like to put the project online, so that everybody could access my interface via a simple URL address – on the condition of course that they have an eye tracking device like ‘Eye Tracker’. Or otherwise to make it an independent program, not one subordinate to another like Ableton.’
‘I hope people will think the project is a good one and that they’ll propose their help and ideas to me’, concludes Andreas, who hopes that his initiative will feed the creativity of other technicians, computer engineers and artists as keen as he is. If you liked the concept and you feel the urge to contribute as a volunteer, Andreas wants your CV now. Don’t be afraid, he speaks perfect English.