Monday, October 8, 2007

Compression?

I've attempted an experimental compression idea, although it doesn't do very well.

Here's an overview of my thoughts. I would take a stream of dense encrypted data, using my OTP pad, then find all matches and large matches and use a structure to handle it. it goes through several rotations, before it just comes up kinda dead. The way i was considering using it, although it seems to be failing, is it would use the following format.

1bit - 1 for encoded, 0 for skip and comeback later at different window.
length is not known ahead of time, so it's kinda... eh?
12bits - lookup table back
2bits - 2-5 bytes long match
= 15bits.

Each 0 on the first bit, increases the bitsize by one, so if you do 6 rotations, eventually it's 21bits, shy of 3 bytes.

in small chunks, (4k and less) it's 1/2 the numbers are 1 byte, 100-200 are 2 byte, and the rest are 3-4 byte (1-3 overall).

In order to test it's usefulness i've attempted to do a bit count, using it's best compression outputting, to express how big it would be when done. when it days 8000bits original, it comes back 10,000-12,000 bits. Ocationally i get it closer or equal to the original value, but at that point there's no compression or it is simply far too much work (16Meg of data for a 24-26bit data structure hoping for 3-6 byte matches). It doesn't seem to work

I will consider different forms of compression to add in the cipher, but dense data just equals, probability of 0 and 1 is 50% each.. Equally. No matter how many levels it goes, it is about even and when tested in very large and different sized numbers, it's very equal and nearly exactly what the value should be.

Era

No comments: