Design & Art Direction
Selected Interaction Design, User Experience, UI, Design Research and Art Direction
Monitoring multiple sclerosis is incredibly difficult; the neuro-degenerative condition develops slowly over time at an inconsistent pace, giving even the most skilled neurologists a major clinical challenge.
Sensor-based recordings of human movements are becoming increasingly important for the assessment of motor symptoms in neurological disorders. ASSESS MS is a movement recording and analysis system being developed to automate the classification of motor dysfunction in patients with multiple sclerosis using depth-sensing computer vision and artificial intelligence. It aims to provide a more consistent and finer-grained measurement of motor dysfunction than currently possible.
ASSESS MS brings together doctors’ expertise—and humanity—with exacting computational data to provide a more robust approach to tracking the disease.
The biotech company Novartis came to Microsoft because they had seen the Kinect system for Xbox game consoles and thought the body movement sensors might be helpful for tracking the progression of multiple sclerosis. That, in turn, could help them more consistently judge whether a treatment they were proposing was working.
Microsoft Research gathered experts in machine learning to create algorithms that could more accurately measure the subtle changes in movement that can occur as MS progresses. And also a team of human-computer interaction researchers, including me, who designed a specialised system that would work in a real-world hospital setting, where medical professionals with no specialised technical skills would be asked to operate the machine in cramped exam rooms.
To try to quantify the progress of multiple sclerosis, doctors have developed a standard set of tests they perform, like asking a patient to touch their nose or sit with their arms outstretched. Traditionally, doctors watch the patient and then use a rating scale to determine how strong the patient's symptoms are.
Depth cameras from the Kinect gaming system and artificial intelligence give qualitative status reports, which provide essential data to supplement doctors’ informed decisions about treatment.
The system is usable and supports the acquisition of highly standardised data. We are currently carrying out further research on the visualisation of the machine learning results for health professionals. We are particularly interested in how temporal representations can enable health professionals to gain new insights into the motor ability of their patients.
This collaboration between Microsoft Research’s Cambridge, UK, lab and Novartis holds the promise of helping doctors more quickly bring better treatments to patients who are suffering from MS, by offering more consistent analysis of whether the treatments are helping patients.