Mac/linux SDK in Python

[dr-mephesto]

Hi,

I was really annoyed that the NIA didnt come mac software, seeing as it should be easy to implement due to the NIA device acting as a HID.

Seems I was wrong...

The NIA acts as a HID, but it doesnt follow the protocol the way Mac's expect, and so you cant just access the data easily. But I used libusb, access via pyUSB to get to the NIA directly, so now we have a reader for Mac. I have also written the code to analyze brain waves and filter out some of the electrical noise etc. I still have to write the code to detect eye movement, although that should be easy as it has a very characteristic pattern.

What I don't know how to do it write a keyboard emulator, so that detected brain waves and glances can be used to generate simulated key presses, so as to control games and other software. Im guessing you can use Scripting Bridge for this, but it may not be fast enough. I will look into it later.

Anyway, here is a screenshot of the alpha code, let me know if anyone wants the software. It doesnt do much for now, but you can practice alpha/beta wave stuff etc.

It shows brain-fingers at the top; the bottom left is a fourier transform updated every 100ms from the bottom up, so you can see all the brainwaves from 8 to 40 hertz simultaneously for the past few second (more useful than brain fingers in my opinion), and the bottom right shows the current waveform, filtered of frequencies over 45Hz. It should work on Linux as well, with a few python eggs.

Anyway, let me know what you think.
Dave

*EDIT 1st May* So you dont have to read the whole thread

The Python SDK framework is more or less finished. This SDK lets you access brainfinger data, clean fourier data, and filtered waveform data. Yet to be implemented are glance and muscle data, but these can be found in the waveform data.

Download the latest version from:
http://code.google.com/p/pynia/downloads/list

to generate keystrokes in MacOSX, first inport the applescript interface, "from appscript import *", then use something like this in the main loop:

fingers = nia.fingers()
if fingers[0] > 0.8:
  app('System Events').keystroke('x')

This would generate an "x" keypress when the alpha 1 brainfinger reached 80% of maximum.

[CmdrRickHunter]

I don't know much about Mac, but if the functionality exists, its buggy. X, on its own, explicitly doesn't support injection of keyboard events. On linux, you actually have to become superuser, and dump the data directly in from the keyboard device. This means Apple will have had to hack it on top...

I'd say your best bet would be to find some open source software which supports auto-type, and steal their code! They'll have worked out any bugs

[dr-mephesto]

It seems generating keystrokes is actually pretty easy with applescript and its python hook appscript.

eg:

from appscript import *
import os, time

while 1:
	time.sleep (1)
	app('System Events').keystroke('x')

generates x's every second in any active window. So, it should be pretty easy to map eye-movements etc to keystrokes.


I have released the alpha version of pyNIA today, at:

pynia.googlecode.com
have the NIA plugged in before you run it.

So, anyone have some cool idea's for features to implement?

[Unregistered]

That's very cool stuff!

[TheBlack Box]

Very cool indeed.

I will play with it, within the next few days and give feedback.

Big thanks for your work !

About suggestions:

I often wondered if it might be helpful, to allow combined channels .... brainfinger chords ... or things like Min(Beta2, Beta1) ... or pretty much just allowing a scripting language ... to write a function to combine brain-fingers into a meta-finger for the input of a joystick or switch.

Another line of thought goes into combining nia with additional sensors and OpenEEG.

Cheers,

-------------------------------------------------------------

Feedback:

(requests)
1. code.google or sourceforge page
2. Linux-friendly packaging

(hints)
1. To my knowledge there is no written law anywhere that says:
"BCI configuration-tools MUST look like early 80s arcade-games"

(props)
1. A very promising first alpha-version ... i hope you will have the time and drive to
continue your fine work !

Let us know in this thread of any specific help you could need.

[dr-mephesto]

No worries, I got it sorted.
You can now download the source code (all 8k including comments) from:

pynia.googlecode.com

To use it, download this and the expanded picture archive, and be sure to have the python packages listed installed. It should work on Linux, Mac and Windows. I'm not a linux fan, so I have no idea how to package the thing into a exe or app like I did for Mac.

BTW, when I decided to write it, it was to learn how to program in Python. I have only been programming for a few months, so any advice from proficient python programmers would be most welcome.

Anyway, the code has been commented, and should be pretty easy to follow. There are a few tricky bits, like getting arrays into a format pyglet can handle to display as images, but nothing too hard to figure out.

I don't have so much time now (I have to write a damned thesis), so I wont be making any real changes for the next few months, but anyone who is even a bit interested could learn some basic python and get the code to jump through hoop without too much trouble. Just let me know of any changes you make, and I will upload them to the project site. Also, if you have any question, just ask on this thread, and I will be happy to explain how the code works in more detail.

to theBlackBox:
80's arcade games rocked! Alien Syndrome and Operation Wolf are timeless classics! :thumbs: Maybe a Mame interface should be the next pyNIA project?

[TheBlack Box]

Note to Ubuntu-users:

The package python-pyglet is outdated.

To get pynia running, you will want to get a more recent pyglet and install that directly (Soruce-release ... at the bottom of This Page)

[TheBlack Box]

I had to include "import sys" to get over "NameError: global name 'sys' is not defined"

Starting with sudo, to avoid "USBError: error submitting URB: Operation not permitted" (on my standard Ubuntu installation)

Now i am at "USBError: error submitting URB: Device or resource busy"

But the application starts fine and shows pretty much what is to be expected with that error-message ... no signal ...
and i feel the urge to insert a coin somewhere

I will look into it more later ... and finally will have to learn python too, it seems.

wxPython is supposed to be pretty nice btw ...

[dr-mephesto]

yeah, I knew there would be problems porting to another platform...

I get the "NameError: global name 'sys' is not defined" error if the NIA is not plugged in before I run the program. The first thing to do it test if the packages installed correctly.

Try this code:

import usb
busses = usb.busses()
for bus in busses:
  devices = bus.devices
  for dev in devices:
    print "Device:", dev.filename
    print "  Device class:",dev.deviceClass
    print "  idVendor:",dev.idVendor
    print "  idProduct:",dev.idProduct

If libusb and pyusb are installed correctly, it should list all you USB devices, including the NIA. Let me know how it goes.

Yeah, I was thinking of using wxpython, but I wanted to try making games in the future, so I thought it would be a good time to learn pyglet too feel free to port the code to wxpython though. As you can see, it should be pretty easy.

[TheBlack Box]

This seems to be the nia:

Device: 007
Device class: 0
idVendor: 4660
idProduct: 0

So pyusb should be working fine. (4660 = 0x1234)

I cant blame you for not being a fan of Linux, when it comes to usb-devices.

-------------------------------

Comment on Python:

I am impressed how little code this actually is.
Once i got the basics running on linux, it should be fairly easy to start experimentation with some hardcoded nia-profile alternatives for testing.

Specially for Alpha+Beta based movement, it will be interesting to see if performance can be tweaked ... EITHER strafe-left OR strafe-right .. depending on which signal is stronger ... reducing the amount of contradictory events that are being fired paralelly... same for forward/backward ... jump/crouch

[dr-mephesto]

you will just have to play around with the pyusb until you can get the NIA attached

sorry the code is so disorganized, I was hacking the thing around for ages trying new stuff, so the classes and functions are all messed up. I will do a code tidy up this week, so that the NiaData class returns nice data and images data separately. Then it should be trivial to do the stuff you are talking about, or use wxpython.

So, once you have your linux usb problem sorted, switch to the new code I will upload, and you should be good to go.

[Xavian]

I'm impressed with the NIA detective work documented on these forums. The NIA hardware is designed for easy access, so its not surprising to see dedicated tinkerers have found ways to read from the device. Seeing Python demo code appearing via dr-mephesto's contribution is especially interesting since IronPython was used for prototyping and unit testing the original internal NIA communication layer. Nice work.

[dr-mephesto]

well, its not fully sorted out... I found something weird on reading data. Using my python script, I check the device every ~2ms, and pick up usually 7 or 8 data points from the device buffer. Over a second I seem to count about 3910 samples. I would have expected a round number or 4096. So either somehow I am losing data somewhere, I have a bug, or the device clock is a bit weird.

If I plot the data, it looks great, so I am probably doing something right But the weird number of samples is a concern. Does anyone have any clues for me?

Also, the NIA has a bulk-write interface, and as far as I know, no one here has figured out how to use it, and I am a bit hesitant about squirting random data in there and breaking my NIA.

[TheBlack Box]

Tested pyusb (0.41) with several other example-applications and it seems to be running fine.

Worked with Ubuntus udev-rules to get the device-permissions under control.

Without the "import sys" i cant get pynia to start at all. (NameError: global name 'sys' is not defined)

And when working around issues to the best of my knowledge ... i run into a wall that says:

read_bytes = self.handle.interruptRead(0x81,64,1);
USBError: error submitting URB: Device or resource busy

Besides adding the "import sys", i didnt change the pynia-code at all yet
and it seems to be happy with the device it locates ... until these reading errors occur.

Anyone got further hints for me ?


--------------------------------------------------------

Anyone got it to run under linux yet ? Any special issues that had to be resolved ?

[Xavian]

read_bytes = self.handle.interruptRead(0x81,64,1);
USBError: error submitting URB: Device or resource busy

Besides adding the "import sys", i didnt change the pynia-code at all yet
and it seems to be happy with the device it locates ... until these reading errors occur.

Anyone got further hints for me ?

I've played around with this before with my own code base of libusb on Ubuntu. I've just recently dusted it off, but I looked at the code and I wrap the interruptRead() in a try...except Python clause kinda like this (pseudocode):

while True:
    try:
        data = device.interruptRead(endpoint.address, endpoint.maxPacketSize, reasonable_timeout)
        # insert packet decoding and other behavior here
    # todo: catch USBError exception and figure out whether it was a timeout
    # or something else and handle accordingly
    exception:
        continue

The interruptRead behavior seems buggy and its been a while since I've revisited it. I think setting a timeout is important, but getting it to work correctly is another issue.

One thing to note is that its highly unlikely that Python can read data at the rate the NIA streams it, that would likely require a Linux C library API that deals with the full-speed stream, averages samples and reduces the rate to something digestible by a scripting language.

Even at lower resolution there is more than enough data to work with.