Subject: [Classpathx-crypto] on testing (long)

From: "Raif S. Naffah" <[email-masked]>
Date: Sun, 23 Jun 2002 17:32:15 +1000

hello there,

test.cipher.TestOfSerpent is a model for testing a cipher's correctness, 
and i would like us to think on how we can use this pattern for testing 
all our symmetric key block ciphers.

things to consider:

* we should also test for the cloneability of our ciphers.  this can be 
as as easy as:

    a. initialise a cipher instance,
    b. encrypt one block,
    c. clone the cipher,
    d. encrypt a second block with both the cipher and its clone,
    e. verify that the result from both is the same.
    f. repeat the exercise with decryption.

* the TestOfSerpent uses test vector values that are hard-coded in the 
class.  should we (a) continue doing that, (b) use something similar to 
TestOfNistVectors where the test vector values are read from files in a 
pre-designated location, or (c) combine both approaches?

having hard-coded values that must be processed correctly for the test 
to pass, works around the fact that TestOfNistVectors does not fail if 
it cannot locate the files it is expecting.

besides, having hard-coded values inside the test-case, allows 
application of the (more or less) same pattern to non-NIST ciphers.

i'd go with (c).

* if we go with (b) or (c), how many hard-coded test vectors should we 
use?  --all of them is out of the question because of size and speed. 
theoretically one value should be enough, but 3 is more "re-assuring."

* should there be any special criteria for selecting the agreed number 
of test-vectors? or throwing a dice would suffice?

* once we have coded those TestOfxxx for our ciphers.  should we still 
keep the selfTest() method in the API and implementations?

in its actual form the setlfTest() for the ciphers only test for 
symmetry, which may be true even for incorrectly implemented ciphers. 
in other words it's almost useless!

we should change it to either (a) something similar to the selfTest() of 
hash algorithms, (b) something similar to the TestOfxxx discussed above 
with some hard-coded test vectors, or (c) get rid of it altogether!

* if we get rid of the selfTest() method in the API, how can we 
guarrantee that noone/nothing would replace our "tested" implementations 
with their (probably) insecure/leaking ones?  would a signed jar be enough?

* the test vectors generated by, at least, the NistKat and NistMCT tools 
should be included in the distribution, since they are used by the 
TestOfNistVectors, which (a) if amended to be more rigourous, and (b) 
passes all its tests for the test vector files, would constitute a 
comprehensive "bill-of-health" for the correctness of a cipher's 
implementation.

relying on the NistKat and NistMCT tools to generate them in-situ is not 
a valid option IMO.  a conscientious user would download these test 
vectors from a different source (NIST itself, the home page of the 
algorithm designers, or a trustworthy site/distribution), and only then 
run the TestOfNistVectors over them.

so am i contradicting myself? no.  am i putting more stringent 
conditions on what to distribute and how to distribute it? may be.

let me explain.

+ the NistKat and NistMCT tools are used only for (a) generating test 
vectors for new cipher implementation, and (b) for bootstraping the process.

+ once the test vector files are generated, they are "certified" (by a 
process which i still dont know how).  the end result would be a 
signed-jar for each of the NIST-compatible algorithms; eg. aes-tv.jar 
for the AES test vectors.

+ the gnu-crypto.jar is signed.

+ the TestOfNistVectors, or similar class, would use the cipher 
implementation from the gnu-crypto signed jar with the test vectors from 
the cipher's test vectors signed jar to assert the correctness of that 
cipher.

does it make sense?

* whatever we come with from the discussion of the previous point, a 
third jar, grouping the test vectors, and separate from the 
gnu-crypto.jar and gnu-crypto-test.jar should be considered.

comments and suggestions are welcome + cheers;
rsn

---

From: Casey Marshall <[email-masked]>
Date: Mon, 24 Jun 2002 00:35:42 -0700 (PDT)

On Sun, 23 Jun 2002, Raif S. Naffah wrote:

> [...]

I would go with (c), too. As far as the number of test cases to hard-code,
I would recommend *at least* two of each of the monte-carlo tests, so there
is at least one change of key. This is because if a test only does the
first monte-carlo test, a problem with endianess *could* sneak by, since
the all-zero key looks the same regardless of endianness (this did happen
early on when I was working on the Serpent implementation, when I only
checked the all-zero key).

For the known-answer tests I would recommend using texts/keys where a bit
is set somewhere in each byte/nybble/word, to try to cover a range, not
just the first few.

Other than that I would simply use a Comfortable Round Number.

> with some hard-coded test vectors, or (c) get rid of it altogether!

I would go for encrypting a known text, comparing the ciphertext with a
known answer, then decrypting it to see if it results in the plaintext.

-- 
Casey Marshall < [email-masked] > http://metastatic.org/

---

From: "Raif S. Naffah" <[email-masked]>
Date: Mon, 24 Jun 2002 20:38:48 +1000

> checked the all-zero key).

good point!

> Other than that I would simply use a Comfortable Round Number.

let's say 5 values for each test, which makes a total of 30 --including 
the KAT ones.  the position of the bits for the latter, should be 
selected at random.

> known answer, then decrypting it to see if it results in the plaintext.

so a symmetry test but with known values.  yup; sounds good.

how do you want to split the work?

cheers;
rsn

---

From: Casey Marshall <[email-masked]>
Date: Tue, 25 Jun 2002 00:53:07 -0700 (PDT)

> the KAT ones.  the position of the bits for the latter, should be

I think for the KA tests the number of tests should depend on the length of
the key/text, by (say) using the word 0x80000000 at all the possible word
(dynner, 32 bit) aligned offsets, e.g.

   8000000000000000000000000000000000000000
   0000000080000000000000000000000000000000
   ...etc.

just because it is seems less arbitrary and spans the space of keys/texts.

> how do you want to split the work?

I could do the TestOfXxxx classes, maybe make a general superclass that
does the work, and subclasses that simply fill in the test vectors.

---

From: "Raif S. Naffah" <[email-masked]>
Date: Tue, 25 Jun 2002 19:03:40 +1000

> (dynner, 32 bit) aligned offsets, e.g.

i'm not sure.  if the cipher is not correctly implemented, it's highly 
unlikely to pass some KATs and fails others.

even if this is the case, a cipher implementation should be deemed 
correct iff it passes *all* the tests!

> does the work, and subclasses that simply fill in the test vectors.

excellent;  a base test-case for the TestOfxxx ciphers would be great!

i'll do the selfTest() implementations.

ps. checkout the latest TestOfSerpent and Util before you start.

cheers;
rsn

---

From: Casey Marshall <[email-masked]>
Date: Thu, 27 Jun 2002 04:13:33 -0700 (PDT)

Patch for the test classes is attached. It consists of a base test class
and the five TestOf* classes modified to make use of it.

> correct iff it passes *all* the tests!

Absolutely. I decided to just use sequential texts (shift one bit to the
right each time) instead of random/arbitrary ones. It is simpler to program
and to produce tests, and the monte carlo tests exercise the algorithms
enough that errors are unlikely to sneak by.

Cheers + goodnight,

---

From: "Raif S. Naffah" <[email-masked]>
Date: Fri, 28 Jun 2002 22:23:42 +1000

> enough that errors are unlikely to sneak by.

i havent had a chance to look at your code yet --was busy getting my cvs 
  client working again-- but i'll checkin this stuff along with mine 
during the weekend.

the log entries will be shorter than usual since i wont be using wincvs 
for some time; unless someone can point out how to get vim configured to 
be the editor for use with the cvs cmd-line client tool.

> Cheers + goodnight,

goodnight + have a nice weekend;
rsn

---

From: "Raif S. Naffah" <[email-masked]>
Date: Fri, 28 Jun 2002 22:39:34 +1000

Raif S. Naffah wrote:
> ...

found it!

cheers;
rsn

---

From: Nic Ferrier <[email-masked]>
Date: 28 Jun 2002 13:46:50 +0100

> be the editor for use with the cvs cmd-line client tool.

the env var: EDITOR needs to be set to the path of your editor (in
this case, vim).

On windoze machines you sometimes have trouble with CVS not finding
the pserver password file but that shouldn't matter if you're using
ssh.

What else is there?

Nic

---

From: "Raif S. Naffah" <[email-masked]>
Date: Fri, 28 Jun 2002 22:48:30 +1000

> the env var: EDITOR needs to be set to the path of your editor (in

yes.  i shouldve done man cvs!

> the pserver password file but that shouldn't matter if you're using

world peace? ;-)

thanks buddy + cheers;
rsn

---

From: "Raif S. Naffah" <[email-masked]>
Date: Sun, 30 Jun 2002 00:21:37 +1000

A. where we're at
-------------------

earlier i wrote:
 > ...
 > test.cipher.TestOfSerpent is a model for testing a cipher's 
correctness...

today's checkin concludes that part which leaves the library in the
following state:

* hash and cipher algorithms have a selfTest() method which is
guarranteed to be called by the respective Factories when asked to
provide a specific implementation.

* the selfTest() method when invoked for the first time, compares the
result of processing a designated input against a knwon pre-computed
output value.  when the two values agree, a "validity" flag is set to true.

* the cached value of the validity flag is re-used in recurrent 
invocations of the selfTest() methods.

* all the TestOfxxx cipher test-cases now follow a similar pattern which 
consists of:

    a. test the validity;
    b. test the cloneability;
    c. run 5 tests each of KAT VK, VT, MCT ECB Encryption and
       Decryption for one block size and each key-size in the set
       formed by the intersection of (i) the cipher's supported key
       sizes and (ii) the set of the following 3 values: 128, 192
       and 256.

* in addition, the top-level test harness, runs a general verification 
test if previously generated test vectors are found in a pre-deisngated 
location (tv/nist/...).  this test assumes the input test vector files 
follow a naming and have a specific format described by NIST --and hence 
may have been generated by 3rd parties using other tools than this library.

* the NIST tools, have been changed to allow generation of similar test 
vector files.

B. where do we go from here
-----------------------------

1. review the hash TestOfxxx and see if a similar pattern can be applied 
and/or is it worth it.

2. include in the build process the generation, in a separate jar, of 
the (at least) the NIST test vectors.

3. ensure that all the build tools (ant, top-level make, and preferably 
automakejar, as well as gcj/make):

    a. run the same tests,
    b. build the same deliverables.

ant's build.xml should be the reference.

once this list is done then we're ready for the first public release.

C. what's next for the near future
------------------------------------

1. do we go for the signed test-vector jars?
2. do we (PGP) sign our distribution?
3. can we get the library integrated with the gcj release --from 
Olivier's tests it sounded like our JCE adapters fill a missing niche in 
their distribution?

D. what's next for the medium-term future
-------------------------------------------

1. start the adapters for the javax.security SPIs.
2. what to do with ASN.1 related stuff?
3. work out a process for selecting new algorithms to implement.

comments, suggestions, volunteers are more than welcome ;-)

cheers;
rsn

---

From: Nic Ferrier <[email-masked]>
Date: 29 Jun 2002 22:50:30 +0100

> comments, suggestions, volunteers are more than welcome ;-)

Well done. Excellent work.

I hope soon to have some more time to sort out the automakejar
issues. I have a Windoze pc at work so I'll install cygwin on it and
see if I can get it all working.

I would help with the GCJ issues but my GCJ is broken right now /8-<

Nic

---

From: "Raif S. Naffah" <[email-masked]>
Date: Sun, 30 Jun 2002 13:54:53 +1000

> Well done. Excellent work.

thanks.

> I would help with the GCJ issues but my GCJ is broken right now /8-<

is there a particular person you'd recommend to talk to about 
integrating gnu.crypto with the GNU java classes (java.security.* packages)?

cheers;
rsn

---

From: Nic Ferrier <[email-masked]>
Date: 30 Jun 2002 09:17:40 +0100

> integrating gnu.crypto with the GNU java classes (java.security.* packages)?

You should just pitch on the Classpath list.

Do you guys have a JCA/JCE implementation?

Nic

---

From: "Raif S. Naffah" <[email-masked]>
Date: Mon, 01 Jul 2002 18:52:32 +1000

> You should just pitch on the Classpath list.

ok.

> Do you guys have a JCA/JCE implementation?

no we dont; but we have implementations of MD5, SHA, DSA implementatios 
and adapters to hook them into a standard JDK --or compliant in the 
sense of what GNU classpath is developing; i.e. the java.* packages.

code is also available for including ASN.1 DER classes for x.509 
certificates and the like.

we also plan on providing adapters for use by JCE/JCA implementations 
from Sun (jdk1.4) or cleanroom ones (bouncycastle, cryptix, etc...) for 
pre jdk1.4 --the javax.security.* packages.

cheers;
rsn