Cracking the code (WIP)


#1

I am working on a streaming app for Muse written solely in .NET
I’m trying to interpret the compressed packet data, but I’m a little stuck.

Compressed data real-world example:
Assume that we’ve already received an uncompressed packet, in this example, we’ll say the values are CH1: 512, CH2: 512, CH3: 512, CH4: 512
One full compressed data packet (I was wearing the Muse at the time, so values should be “normal”):
192,4,8,32,192,0,1,45,62,206,189,252,141,99,12,146,30,254,135,220,199,18,166,42,78,186,243,51,218,81 ,149,56,139,158,49,118,245,43,224,180,237,16,233,2 33,87,168

Break it down:
Header at index 0: 192: 11000000 = 0xE 0x0
The 0xE means that this is an uncompressed EEG packet, and the 0x0 is unused, so we ignore

Next 5 bytes: 4,8,32,192,0: 00000100 00001000 00100000 11000000 00000000
Regroup these into 10-bit blocks: 0000010000 0010000010 0000110000 0000000000
Regroup each of these into 4- and 6-bit blocks: 0000 010000 0010 000010 0000 110000 0000 000000
Each 4-bit block represents the quantisation flags for each of the 4 channels (see documentation for explanation of this)
Quotients CH1: 0000 = 1, CH2: 0010 = 4, CH3: 0000 = 1, CH4: 0000 = 1
Each 6-bit block represents the median for each of the 4 channels (used for the truncated binary encoding of the remainder)
Medians CH1: 010000 = 16, CH2: 000010 = 2, CH3: 110000 = 48, CH4: 000000 = 0

Next 2 bytes: 1,45: 00000001 00101101
Interpretted as a 16-bit integer giving the length of the samples in the packet: 301 bits = 37.6 bytes = 38 bytes

Next 38 bytes (data): 62,206,189,252,141,99,12,146,30,254,135,220,199,18,166,42,78,186,243,51,218,81,149,56,139,158,49,118,245,43,224,180,237,16,233,233,87,168
This is the Golomb-encoded, 16 * 4 sample-delta.
Each sample delta starts with the quotient (encoded in either unary where value <= 14, else Elias)

First data byte: 62: 00111110 - this is where I’m stuck. I’d expect every value of the quotient to start with a ‘1’ bit, whether in unary or Elias encoding (unless we were skipping the samples, but that shouldn’t be the case here, since the median for CH1 is not 0), but this byte starts with a ‘0’. It’s not clear to me from the documentation how I can proceed from here. Do I ignore the first 2 bits, and proceed with a quotient of 5?

If anyone can offer thoughts or suggestions, I’m very keen to hear them
Thanks!


Bluetooth Connectivity on Linux
#2

Hassmeer, check out my c# code here,

http://forum.choosemuse.com/forum/developer-forum/3344-my-source-code-for-direct-dot-net-connection-to-muse

I think your problem is the reversal of litlendian bytes.


#3

Or maybe Hassmeer has missed the part that says: " Elias encoding starts with a unary encoded value of 15. This is followed by Elias encoding of the number, "
I’m still very confused with the whole thing, but I’m sure your code will help a lot.

Great job David !

And many thanks for sharing !


#4

Hi,

Just a few quick comments on this.

192 is actually 0xC0, which is Compression, not uncompressed. 0xE0 is 11100000.

When you parse the Medians it is 10 bits per median, but they are unfortunately not packed the way you assumed. I apologize I’m not sure this is properly documents.

Byte 1: [B]00000100[/B]
Byte 2: 000010[B]00[/B]
Byte 3: [B]0010[/B]0000
Byte 4: 11[B]000000[/B]
Byte 5: 00000000

I’ve highlighted every second median in bold.

This actually makes for Medians:
[1] 00 0000 0100
[2] 00 0000 0010
[3] 00 0000 0010
[4] 00 0000 0011

Where the least significant bits come first, the most significant bits come in the following packet.

Your length calculation should be correct.

With regards to the 62 at the start of the packet, I’m less certain the cause of that. I’ve never seen that before. It should start with a series of 1’s as you’ve stated. Perhaps there’s an off by one error somewhere, a miss reference or some other anomaly collecting the data. I hope this helps.


#5

David, congratulations on getting it running! You have the honour of being first to publish :slight_smile:
A tip for moving ahead with your code: think about implementing an abstract factory design pattern for processing the packets. Each packet type should contain its own methods for processing the data coming through (think Separation of Concerns). This frame-of-mind will help you move away from a procedural programming style (which is a nightmare to maintain), into a more OO style. Also, “test, Test, TEST!”

Farough, thank you for the clarification. Yes, the 0xE0 was a transpositional error (oops), I meant 0xC0. I implemented your way of processing the Medians, and it works! When I run the packet through my algorithm, I get the right number of bits being processed, and the deltas appear to be reasonable. I will try and publish my solution tonight


#6

Eduardo, I will try and write a post later about the process from start to finish, with some (pseudo) code once I know what I’m doing is correct. I’d like to also implement, with your permission, your idea of retransmitting the bluetooth signal once it’s been been harvested by my app, so we can use Calm, and record the signal concurrently. I haven’t tried so I’m not sure if this is even possible… Will the Calm app be happy to use the signal coming from a source other than the Muse, assuming that the signal being received is identical? Is this even allowed by the ToS?


#7

Thanks Hashmeer, sure design patterns is the way to go, it is just our first working proff of concept,
Also If you want to use my code in a slow system and need real time, better have it optimized with bit shift functions, and maybe a stack instead of dual byte buffers.


#8

I’m glad you got it running. I apologize I misspoke earlier regarding the 62 byte being your first packet.

Obviously you realized your mistake in order to get it to run properly, but just for completeness sake I will post the correct response here now.

Your first Median is 4. Therefore the remainders are 0 to 3 which is all the states, 0 = 00, 1 = 01, 2 = 10 and 3 = 11.

Therefore with 62, 206
0011 1110 1100 1110

The first 0 is the Quotient of 0.
Then 01 is remainder of 1.
Lastly the 1 is a sign of +1.
Making this difference (04 + 1)+1 = 1.

This would be followed by Q = 1110, 3. R = 11, 3. S = 0, -1. Diff = (34 +3)-1 = -15.

And this process would continue as such. After 16 differences you would switch to your second channel and median, similarly the third and fourth.

Once you have calculated all the difference you would apply the respective quantizations.

Hope that is clearer.