nose keyboard with machine learning

A keyboard that can be played with facial gestures on the browser sounds sooooooo kewl (does anybody still say that?)

So, I worked on a keyboard from the p5.js code examples and merged it with PoseNet's VideoMusic model.

I think this can be a small therapeutic tool for a person's who is weak but can move their neck.

It would also be interesting to have a physical component to this, like a thumb piano, or mbira:

Reminds me of this project I worked on at PBS: http://grownups.pbskids.org/africa/piano/haveflash.html -- http://grownups.pbskids.org/africa/piano/piano.html

 

mbira
mbira

 

Try it out here: