BCI/Silent speech recognition software


#1

Hello all,
Few months ago I had interest with machine learning and I spent some time to study. During the study my target was developing existing matlab examples in C/C++ or java. So finally I have a working application using neural network alghoritms. It has lot of parameters and calculations but works quite fast. I have added multicore and OpenCL support. And of course opensource:)
Well when its finish I said maybe I can do something with my muse device. So I wrote 2 additional software that one of them does listen udp data which is provided from Muse-IO then picks a random word from given list and shows user and then user thinks about the word and blinks so application records specified channel (alpha,theta etc… by regex which fits osc path) and creates a input data file for prediction application.

Then prediction application runs a training session, calculates errors on prediction and provides another file to use for live prediction.

Then the last application which is a client for live prediction,reads prediction file which is just created and waits user input. So you just think about word then you blink. Prediction success rate not much, but after many trials during the same session application can predict some words continuesly when I think. Maybe my eye movements effects, breathing, heart rate so I can not say it is a real or not:)

Currently there is just support for linux (I think should work on mac too, it is using posix threads)
Here you can find instructions and source codes:

Recording app:
https://github.com/buraksarac/OscRecord

Prediction app:
https://github.com/buraksarac/NeuralNetwork

Client app:
https://github.com/buraksarac/OscClient

I am very new on BCI area and I am not aware of existing solutions but I thought it would be a good shot. Even neuralnetwork application can be used not just for EEG but image recognition,spam analysis etc…

All those 3 application can be put in one application but for prediction you need to do some try and see way because since you have more rows then prediction takes time and different parameters can behave better. So what I am doing now: I prepare another application that picks a path from a list and trys different settings like brut force, so my target is finding the best channel combination,osc path(s) and settings for most successfull prediction.

As many of you have good experience and that is possible some muse developers will read this I will be waiting for your feedback:)

Thank you
Burak


#2

Wow! Super cool stuff! I am definitely going to check this out soon. :slight_smile:


#3

Yeah, sounds like a lot of fun, keep us posted!


#4

Hello All,
I apologize for late response and thank you for your inputs:) I know not very well coded but works. So I have done a brut force to find a proper setting. I have recorded 2 muse session. In first session I have choose 5 word (hello, world, how, are, you) then I blink and think about the word 10 times per each. In the second session I repeat words as a sentence 8 time (so total 40). Then I prepare 61 path (i.e. alpha_absolute, delta_absolute, alpha_absolute and beta_absolute etc…) and tested them between 5 to 13 scanrange. So after automation complete even I didnt reach 50% :slight_smile: but it wasnt my target I wanted to compare paths between each other. Please find below results, first column is total correct in %, second column total correct, third column total wrong, fourth columnd has sequence (as a sentence => hello world how are you?), 5th column is scan range, sixth column is result file and last column is osc path. Seems like alpha_absolute and delta absolute together were best. Also which preset would you like to advice for running muse to test prediction? Do I really need research framework for that?

Thank you
Burak

I couldnt upload txt so it is on google docs:
https://docs.google.com/document/d/1VoOh3kCSun0v0t-V2PZ983j607swS0_NDDtZqf40nmg/edit?usp=sharing