Over the course of the last several weeks, i have learned java, and after being part of a team larger than myself, i have come to a sudden realization. I have been doing things wrong, and i must apologize.
So in the upcoming version, which may be a few months yet, i'm making several large changes and will worry about improving code to a large degree another time.
1) Eclipse CDT IDE. Before i've entirely build the cipher in a basic text editor, and that brings many annoyances when you're needing to look up functions, parameter orders, names, and descriptions. It's been a wonderful tool so far, and makes refactoring and building code easier. I'll be putting in more descriptive and appropriate names to all variables.
2) Doxygen for inline documentation. My cipher is under-documented, and after reading several good articles, references and suggestions, i'm going to go completely through and re-do all of my comments from over brief to in-depth when appropriate. As well as have doxygen generate my docs later.
3) I've tried before to over optimize my code, and i realize after looking at a couple of the functions, they are kind of a mess. This isn't what i was going for. I'm going to rebuild them for readability and simplicity, rather than speed. I figure, computers are getting faster, and compilers are getting smarter, i will instead break it into something everyone can read and maintain, and leave the cycle saving for the compiler.
4) Unit testing. I'm met a wonderful thing called JUnits, which once understood enough, they make sense. I'm systematically making a test for each file i'm updating with commenting, with a unit test, which when all combined will help in the long run. And let my earlier test.c be.
Estimated next release, August sometime.
Era Scarecrow
Saturday, May 24, 2008
Wednesday, April 2, 2008
Snake Oil
After my first release in a long time. Version 2.4.7, a couple of individuals have called semeir as having a large chance of being 'snake oil'. However as much as i dis-agree, i am forced to agree they are correct that i don't have enough statistical data to prove my cipher is 'secure'. However, from my own tests it's working, however i'll let the data speak for itself.
They've also claimed that declaring i'm using a OTP (One Time Pad) or claiming i have one is bad practice and misleading. It's true it's not a true One Time Pad, (working on pencil and paper with a true list of random data) But nothing is truly random anyways. If you consider that a One Time Pad uses a random stream of data for it's security; that's exactly what my OTP functions do, they double as multiple parallel RNG's if you want them that way.
http://lwn.net/Articles/274857/
http://www.geocities.com/rtcvb32/zips/dh.tbz2.zip <- PRNG tests with Diehard, 1Gig
Next bits of data come from this, /apps/misc_tests/otp.c uses a 9 blocksize by default and it's number of returns before a repeat is:
9 - 10,625,324,586,456,701,730,816 Ints (38,654,705,664 -TeraBytes)
I Used a 4Gig random data block; the results are pretty astounding.
However that doesn't stop you from trying it out and using it as a 'Experimental alternative'. If you feel it's not secure enough for you, that's ok.
Era
They've also claimed that declaring i'm using a OTP (One Time Pad) or claiming i have one is bad practice and misleading. It's true it's not a true One Time Pad, (working on pencil and paper with a true list of random data) But nothing is truly random anyways. If you consider that a One Time Pad uses a random stream of data for it's security; that's exactly what my OTP functions do, they double as multiple parallel RNG's if you want them that way.
http://lwn.net/Articles/274857/
http://www.geocities.com/rtcvb32/zips/dh.tbz2.zip <- PRNG tests with Diehard, 1Gig
Next bits of data come from this, /apps/misc_tests/otp.c uses a 9 blocksize by default and it's number of returns before a repeat is:
9 - 10,625,324,586,456,701,730,816 Ints (38,654,705,664 -TeraBytes)
I Used a 4Gig random data block; the results are pretty astounding.
1's 17,179,831,574 - 49.999890540493652224540710449219Currently, it will take a couple years before the code is accepted, and tests proving it's worth using. There are so many advantages in it already, however slower and using more resources than other ciphers, it still does quite well.
0's 17,179,906,794 - 50.000109459506347775459289550781
(50% within 1/10000th for being 1:2508.253 the size of the full length)
However that doesn't stop you from trying it out and using it as a 'Experimental alternative'. If you feel it's not secure enough for you, that's ok.
Era
Wednesday, March 12, 2008
Compression Idea #4
I've had a multitude of ideas, but recently i've decided to put one to the test.
I know yet again i drift from encryption, but this may be worth it.
I've considered the idea, that you take data, turn it into a puzzle with some redundancy, and then remove all the data you can, and compress the remainder that you can rebuild with. With this in mind i'm talking of something similar to Sudoku, although it may be more than 9x9, it may be more like 25x25 or something, too large for humans (shy of very very intelligent and bored people) doing. The exact method of this is uncertain as of yet. But i'll give you some information what i got so far.
1) I've successfully created a virtual sudoku environment, which works since it looks and feels like soduko, but can increase, but due to testing and building purposes can't go beyond the 9x9 yet.
2) I've programmed only one method (Called method1), which does the simplest of calculations and adds on the more obvious solutions, using this alone can solve puzzled rated easy, and maybe intermediate. I haven't fully tested, but the results were so astounding for a first run with no debugging i was surprised.
There are more methods to add, i think in the range of 4-5 more. When successfully able to do any puzzle on the go in seconds, the algorithm is ready for the next stages. Which i will outline now.
1) Build solver algorithm
2) Build converter from/to sudoku
3) Build number remover, which selectively removes numbers as many as it can, till it can do no more. (Expert level hopefully)
4) Experiment with compression until proper method is made that will hopefully compress the data down enough to compensate for the data, and make the whole venture worth it.
This is all going in hopes that the puzzle compressed is smaller then the original data was to build the puzzle in the first place. But with only one method, results for a solver (even if compression fails) will be promising enough to complete the project.
Era
I know yet again i drift from encryption, but this may be worth it.
I've considered the idea, that you take data, turn it into a puzzle with some redundancy, and then remove all the data you can, and compress the remainder that you can rebuild with. With this in mind i'm talking of something similar to Sudoku, although it may be more than 9x9, it may be more like 25x25 or something, too large for humans (shy of very very intelligent and bored people) doing. The exact method of this is uncertain as of yet. But i'll give you some information what i got so far.
1) I've successfully created a virtual sudoku environment, which works since it looks and feels like soduko, but can increase, but due to testing and building purposes can't go beyond the 9x9 yet.
2) I've programmed only one method (Called method1), which does the simplest of calculations and adds on the more obvious solutions, using this alone can solve puzzled rated easy, and maybe intermediate. I haven't fully tested, but the results were so astounding for a first run with no debugging i was surprised.
There are more methods to add, i think in the range of 4-5 more. When successfully able to do any puzzle on the go in seconds, the algorithm is ready for the next stages. Which i will outline now.
1) Build solver algorithm
2) Build converter from/to sudoku
3) Build number remover, which selectively removes numbers as many as it can, till it can do no more. (Expert level hopefully)
4) Experiment with compression until proper method is made that will hopefully compress the data down enough to compensate for the data, and make the whole venture worth it.
This is all going in hopes that the puzzle compressed is smaller then the original data was to build the puzzle in the first place. But with only one method, results for a solver (even if compression fails) will be promising enough to complete the project.
Era
Wednesday, March 5, 2008
OTP (One Time Pad) Improved!
I've done some major improvement on the program, and have a working windows GUI using AutoHotKeys. I may go away from this eventually, but i see no reason just yet.
A little history. Originally, the OTP was a simple RNG you could have multiple streams with and working. However it only returned 3Billion ints, and you could only seed it once.
New version, still in testing stages but has no problems so far, you can seed it as many ints as it's blocksize will let, and of course, the returnable INT's before it repeats or starts over is as this list shows.
Note, i am not sure if Decabyte is right, but it's the term i'm using.
k 2^10 (1024)
Mb 2^20 (1,048,576)
Gb 2^30 (1,073,741,824)
Tb 2^40 (1,099,511,627,776)
Pb 2^50 (1,125,899,906,842,624)
1 - 32 (256 bytes)
2 - 32,768 (32k)
3 - 12,582,912 (49,512k)
4 - 4,294,967,296 (16,384Mb)
5 - 1,374,389,534,720 (5,120 - Gigabytes)
6 - 422,212,465,065,984 (1,572,864 -Gigabytes)
7 - 126,100,789,566,373,888 (458,752 -Terabytes)
8 - 36,893,488,147,419,103,232 (134,217,728 -TeraBytes)
9 - 10,625,324,586,456,701,730,816 (38,654,705,664 -TeraBytes)
10 - 3,022,314,549,036,572,936,765,440 (10,995,116,277,760 -TeraBytes)
11 - 851,083,777,008,698,938,993,147,904 (3,023,656,976,384 -Petabytes)
12 - 237,684,487,542,793,012,780,631,851,008 (70,368,744,177,664 -Petabytes)
13 - 65,917,831,211,867,928,877,828,566,679,552 (234,187,180,623,265,792 -Petabytes)
These are figured characteristics, usually the returns are using 2xored returns, so it's half the values, but still more than enough for any normal message, probably enough to use on your drive to randomize it before you put a encrypted FS on it.
I will have more later.
Era
A little history. Originally, the OTP was a simple RNG you could have multiple streams with and working. However it only returned 3Billion ints, and you could only seed it once.
New version, still in testing stages but has no problems so far, you can seed it as many ints as it's blocksize will let, and of course, the returnable INT's before it repeats or starts over is as this list shows.
Note, i am not sure if Decabyte is right, but it's the term i'm using.
k 2^10 (1024)
Mb 2^20 (1,048,576)
Gb 2^30 (1,073,741,824)
Tb 2^40 (1,099,511,627,776)
Pb 2^50 (1,125,899,906,842,624)
1 - 32 (256 bytes)
2 - 32,768 (32k)
3 - 12,582,912 (49,512k)
4 - 4,294,967,296 (16,384Mb)
5 - 1,374,389,534,720 (5,120 - Gigabytes)
6 - 422,212,465,065,984 (1,572,864 -Gigabytes)
7 - 126,100,789,566,373,888 (458,752 -Terabytes)
8 - 36,893,488,147,419,103,232 (134,217,728 -TeraBytes)
9 - 10,625,324,586,456,701,730,816 (38,654,705,664 -TeraBytes)
10 - 3,022,314,549,036,572,936,765,440 (10,995,116,277,760 -TeraBytes)
11 - 851,083,777,008,698,938,993,147,904 (3,023,656,976,384 -Petabytes)
12 - 237,684,487,542,793,012,780,631,851,008 (70,368,744,177,664 -Petabytes)
13 - 65,917,831,211,867,928,877,828,566,679,552 (234,187,180,623,265,792 -Petabytes)
These are figured characteristics, usually the returns are using 2xored returns, so it's half the values, but still more than enough for any normal message, probably enough to use on your drive to randomize it before you put a encrypted FS on it.
I will have more later.
Era
Sunday, February 10, 2008
Future of Semeir 2.X.X
Current plans for semeir are as follows. I have a couple different versions i was working on, where i will merge all changes.
Upon completion of that, the primary goal is to finish upgrading the current kernel module for the linux kernel. Since 99% of the code for the kernel is merely the api plugin to make it work. The remainder of it is tested in user-mode, using normal streams and tools to ensure it's working.
Currently limitations of key-length is unknown, which can be as short as 0 bytes (default) or greater than 64k. And Blocksizes are in the range of 1 byte to 15k, using key counts from 1 to 511. The larger the key and the larger the number of keys, the slower the encryption.
Plugins for the kernel, will include several options, which are as follows.
1) Blocksize options: 8, 16, 32, 64, 128, 256, and 512 bytes. Since 512 is the smallest block size, i won't force it higher, since it only gets more complicated beyond that level.
2) EBC, Encryption will be tested and fully implimented.
3) Options for OTP encryption (Recommended 1 key per filesystem, and as a compressed read-only only!, like cramfs or squashfs)
4) Offer special multi-pass option for generating keys using OTP streams XOR'd together. Special advantages is a more secure, more complicated key system, and if split between multiple parties, all need to enter their codes to access the partition.
Further features and notes will be submitted as they are worked on. The current beta version of semeir is 2.4.6, however 2.5.0 will be the next full release.
Upon completion of that, the primary goal is to finish upgrading the current kernel module for the linux kernel. Since 99% of the code for the kernel is merely the api plugin to make it work. The remainder of it is tested in user-mode, using normal streams and tools to ensure it's working.
Currently limitations of key-length is unknown, which can be as short as 0 bytes (default) or greater than 64k. And Blocksizes are in the range of 1 byte to 15k, using key counts from 1 to 511. The larger the key and the larger the number of keys, the slower the encryption.
Plugins for the kernel, will include several options, which are as follows.
1) Blocksize options: 8, 16, 32, 64, 128, 256, and 512 bytes. Since 512 is the smallest block size, i won't force it higher, since it only gets more complicated beyond that level.
2) EBC, Encryption will be tested and fully implimented.
3) Options for OTP encryption (Recommended 1 key per filesystem, and as a compressed read-only only!, like cramfs or squashfs)
4) Offer special multi-pass option for generating keys using OTP streams XOR'd together. Special advantages is a more secure, more complicated key system, and if split between multiple parties, all need to enter their codes to access the partition.
Further features and notes will be submitted as they are worked on. The current beta version of semeir is 2.4.6, however 2.5.0 will be the next full release.
Sunday, January 6, 2008
Encryption Thoughts #2
Continuing earlier thoughts, I have a pre/post encryption algorythmn, that completely changes the internal structure as it encrypts/decrypts; Using very unique and simple algorythmns. If it starts getting complicated it slows the algorythmn down, the number of pre/post peices alter in every step how it changes, so only by having the original key/RNG string could you duplicate it.
Earlier i commented on using prime numbers, I which i highly suggest in use of this process, since it can greatly cut down on the number required subkeys to be considered 'secure', and thereby increase encryption. 2,3,5,7,11,13, each increasing by 1 till it passes of it's max, it then turns around and starts at 1, or some other predetermined number (which could be random). which totals at 30,030 combinations, but i have seen primes in the hundreds; increasing complexity, even for 8bits of combinations very complex.
However, for the main key itself, primes are not applicable, or useful, since you might select the Alphabet only, or a combination, or all of them at random. Since the length is unique, it's internal counters and rythmns follow different rules, using odd numbers for even keys, and even for odd keys; And failure to follow the rules can remove anywhere from 1/4 to 1/2 of it's potential. The stepping rules are different as well, adding/subtracting any number.
There are further ways to greatly increase complexity and strength, however i have made limits under certain rules. Following these rules, i have tried to even out at high strength without high memory/CPU cost.
1) The bit sizes for the codes never exceed 12 bits, and never go under 8 bits.
Reasons: 8bits has much complexity, but not as much as i'd like. 12 is the inbetween number between 8/16, allowing for middle areas (3bytes for example) to increase greatly in key combinations and complexity. If Memory requirements increase too large, it will be difficult to implement in chips and devices. Memory used for keys is as follows.
8bit 4K per key, 6K for speedups
9bit 8K per key, 12K
10bit 16K per key, 24K
11bit 32K per key, 48K
12bit 64K per key, 96K
Using a 32bit key, straight up, in memory would require some 48Gigs, i consider it too long to create, work with, and not worth it in exchange for it's encryption abilities. (2^32 * 4 * 3) 2^32 * 4 for actual key combinations, and 2 or more for the keys, and 1 for pre/post encryption. With this in mind, i perfer to use several smaller keys to create the complexity without memory consumption requirements.
Era
Earlier i commented on using prime numbers, I which i highly suggest in use of this process, since it can greatly cut down on the number required subkeys to be considered 'secure', and thereby increase encryption. 2,3,5,7,11,13, each increasing by 1 till it passes of it's max, it then turns around and starts at 1, or some other predetermined number (which could be random). which totals at 30,030 combinations, but i have seen primes in the hundreds; increasing complexity, even for 8bits of combinations very complex.
However, for the main key itself, primes are not applicable, or useful, since you might select the Alphabet only, or a combination, or all of them at random. Since the length is unique, it's internal counters and rythmns follow different rules, using odd numbers for even keys, and even for odd keys; And failure to follow the rules can remove anywhere from 1/4 to 1/2 of it's potential. The stepping rules are different as well, adding/subtracting any number.
There are further ways to greatly increase complexity and strength, however i have made limits under certain rules. Following these rules, i have tried to even out at high strength without high memory/CPU cost.
1) The bit sizes for the codes never exceed 12 bits, and never go under 8 bits.
Reasons: 8bits has much complexity, but not as much as i'd like. 12 is the inbetween number between 8/16, allowing for middle areas (3bytes for example) to increase greatly in key combinations and complexity. If Memory requirements increase too large, it will be difficult to implement in chips and devices. Memory used for keys is as follows.
8bit 4K per key, 6K for speedups
9bit 8K per key, 12K
10bit 16K per key, 24K
11bit 32K per key, 48K
12bit 64K per key, 96K
Using a 32bit key, straight up, in memory would require some 48Gigs, i consider it too long to create, work with, and not worth it in exchange for it's encryption abilities. (2^32 * 4 * 3) 2^32 * 4 for actual key combinations, and 2 or more for the keys, and 1 for pre/post encryption. With this in mind, i perfer to use several smaller keys to create the complexity without memory consumption requirements.
Era
Subscribe to:
Posts (Atom)