Subject: [Classpathx-crypto] PATCH: NIST cipher tester

From: Casey Marshall <[email-masked]>
Date: Mon, 17 Jun 2002 16:01:28 -0700 (PDT)

Hello,

I've put together a (fairly simple) cipher testing program that will read
in the standard test vector files that the NIST AES submissions required;
all that is involved in testing an algorithm foo with a 128 bit block size
is putting all the test vectors in a directory `tv/foo-128', then
instantiating the class test.cipher.NISTCipherTest. The class loads test
files with the `getResource' method of java.lang.Class, so the vector
files can simply be put into gnu-crypto-test.jar.

The patch only adds the class itself, and updates the build scripts and
test.cipher.AllTests. I didn't include the test files for the three AES
submitted ciphers in Classpathx, because they're probably far too big for
this mailing list.

I've also noted that in the file `Makefile.in' there is a line that says
`gnu/crypto/mac/UMash32.java' which I'm guessing should have read
`gnu/crypto/mac/UMac32.java'.

Cheers,

-- 
Casey Marshall < [email-masked] > http://metastatic.org/
diff -Naurw crypto/Makefile.in crypto.1/Makefile.in
--- crypto/Makefile.in	Fri Jun  7 22:47:45 2002
+++ crypto.1/Makefile.in	Sun Jun 16 19:52:46 2002
@@ -48,7 +48,7 @@
 	$(SOURCEDIR)/gnu/crypto/mac/IMac.java \
 	$(SOURCEDIR)/gnu/crypto/mac/MacFactory.java \
 	$(SOURCEDIR)/gnu/crypto/mac/UHash32.java \
-	$(SOURCEDIR)/gnu/crypto/mac/UMash32.java \
+	$(SOURCEDIR)/gnu/crypto/mac/UMac32.java \
 	\
 	$(SOURCEDIR)/gnu/crypto/mode/BaseMode.java \
 	$(SOURCEDIR)/gnu/crypto/mode/CTR.java \
@@ -100,8 +100,10 @@
 	$(SOURCEDIR)/test/cipher/TestOfKhazad.java \
 	$(SOURCEDIR)/test/cipher/TestOfNullCipher.java \
 	$(SOURCEDIR)/test/cipher/TestOfRijndael.java \
+	$(SOURCEDIR)/test/cipher/TestOfSerpent.java \
 	$(SOURCEDIR)/test/cipher/TestOfSquare.java \
 	$(SOURCEDIR)/test/cipher/TestOfTwofish.java \
+	$(SOURCEDIR)/test/cipher/NISTCipherTest.java \
 	\
 	$(SOURCEDIR)/test/hash/AllTests.java \
 	$(SOURCEDIR)/test/hash/TestOfHashFactory.java \
@@ -142,9 +144,18 @@
 	$(SOURCEDIR)/test/sig/rsa/TestOfRSACodec.java \
 	$(SOURCEDIR)/test/sig/rsa/TestOfRSAKeyGeneration.java \
 	$(SOURCEDIR)/test/sig/rsa/TestOfRSAPSSSignature.java \
+	\
+	tv/rijndael-128/cbc_d_m.txt tv/rijndael-128/cbc_e_m.txt \
+	tv/rijndael-128/ecb_d_m.txt tv/rijndael-128/ecb_e_m.txt \
+	tv/rijndael-128/ecb_vk.txt tv/rijndael-128/ecb_vt.txt \
+	tv/serpent-128/cbc_d_m.txt tv/serpent-128/cbc_e_m.txt \
+	tv/serpent-128/ecb_d_m.txt tv/serpent-128/ecb_e_m.txt \
+	tv/serpent-128/ecb_vk.txt tv/serpent-128/ecb_vt.txt \
+	tv/twofish-128/cbc_d_m.txt tv/twofish-128/cbc_e_m.txt \
+	tv/twofish-128/ecb_d_m.txt tv/twofish-128/ecb_e_m.txt \
+	tv/twofish-128/ecb_vk.txt tv/twofish-128/ecb_vt.txt
 
 PKGFILES =
-
 
 #a mangling of the SOURCEFILES list above
 #This is not quite perfect, we've had to hardcode the
diff -Naurw crypto/build.xml crypto.1/build.xml
--- crypto/build.xml	Fri Jun  7 22:51:15 2002
+++ crypto.1/build.xml	Sun Jun 16 19:16:50 2002
@@ -115,7 +115,9 @@
    </target>
 
    <target name="jar-test" depends="compile-test">
-      <jar jarfile="${test.jar}" basedir="${classes.dir}" includes="test/**"/>
+      <jar jarfile="${test.jar}" basedir="${classes.dir}" includes="test/**">
+         <fileset dir="${basedir}" includes="tv/**"/>
+      </jar>
    </target>
 
    <target name="compile-test" depends="jar">
diff -Naurw crypto/source/test/cipher/AllTests.java crypto.1/source/test/cipher/AllTests.java
--- crypto/source/test/cipher/AllTests.java	Sat Jun  8 17:09:09 2002
+++ crypto.1/source/test/cipher/AllTests.java	Sun Jun 16 19:03:46 2002
@@ -73,6 +73,10 @@
       result.addTest(TestOfSquare.suite());

       result.addTest(TestOfTwofish.suite());

+      result.addTest(new NISTCipherTest("rijndael", 16));
+      result.addTest(new NISTCipherTest("serpent", 16));
+      result.addTest(new NISTCipherTest("twofish", 16));
+
       return result;

    }

diff -Naurw crypto/source/test/cipher/NISTCipherTest.java crypto.1/source/test/cipher/NISTCipherTest.java
--- crypto/source/test/cipher/NISTCipherTest.java	Wed Dec 31 16:00:00 1969
+++ crypto.1/source/test/cipher/NISTCipherTest.java	Mon Jun 17 15:22:39 2002
@@ -0,0 +1,310 @@
+package test.cipher;
+
+// ---------------------------------------------------------------------------
+// $Id$
+//
+// Copyright (C) 2002  The Free Software Foundation, Inc.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program; if not, write to the
+//
+//    Free Software Foundation, Inc.,
+//    59 Temple Place, Suite 330,
+//    Boston, MA  02111-1307
+//    USA
+//
+// As a special exception, if you link this library with other files to
+// produce an executable, this library does not by itself cause the
+// resulting executable to be covered by the GNU General Public License.
+// This exception does not however invalidate any other reasons why the
+// executable file might be covered by the GNU General Public License.
+//
+// ---------------------------------------------------------------------------
+
+import java.util.HashMap;
+
+import java.io.InputStream;
+import java.io.InputStreamReader;
+import java.io.LineNumberReader;
+
+import java.net.URL;
+
+import junit.framework.TestCase;
+
+import gnu.crypto.cipher.CipherFactory;
+import gnu.crypto.cipher.IBlockCipher;
+import gnu.crypto.util.Util;
+
+/**
+ * <p>A generic cipher conformance test against NIST-style test vectors. To
+ * implement a test for a particular cipher, include the files `ecb_vt.txt',
+ * `ecb_vk.txt', `ecb_e_m.txt', `ecb_d_m.txt', `cbc_e_m.txt', and
+ * `cbc_d_m.txt' in the directory `tv/ciphername-blocksize'.</p>
+ *
+ * <p>That is, to test Rijndael with a 128 bit block size, put the appropriate
+ * (and appropriately named) files into the directory `tv/rijndael-128'. Then
+ * create an instance of this class with e.g.</p>
+ *
+ * <blockquote><code>test = new NISTCipherTest("rijndael", 16);</code>
+ * </blockquote>
+ *
+ * <p>And use <code>test</code> as your test case.</p>
+ *
+ * <p>Note that a full-conformance test will likely take a while to finish
+ * (it would have to do 48,001,152 encryptions or decryptions).</p>
+ *
+ * <p>References:</p>
+ * <ol>
+ *    <li><a href="http://csrc.nist.gov/encryption/aes/katmct/katmct.htm">Known
+ *    Answer Tests and Monte Carlo Tests for AES Submissions</a> for an
+ *    explanation of the tests and the format of the resulting files.</li>
+ * </ol>
+ *
+ * @version $Revision$ 
+ */
+public class NISTCipherTest extends TestCase {
+
+   // Constants and variables
+   // ------------------------------------------------------------------------
+
+   // file names.
+   protected static final String ECB_VK = "ecb_vk.txt";
+   protected static final String ECB_VT = "ecb_vt.txt";
+   protected static final String ECB_E_M = "ecb_e_m.txt";
+   protected static final String ECB_D_M = "ecb_d_m.txt";
+   protected static final String CBC_E_M = "cbc_e_m.txt";
+   protected static final String CBC_D_M = "cbc_d_m.txt";
+
+   protected static final int ENCRYPTION = 0;
+   protected static final int DECRYPTION = 1;
+
+   protected IBlockCipher cipher;
+   protected HashMap attrib;
+   protected URL ecb_vk;
+   protected URL ecb_vt;
+   protected URL ecb_e_m;
+   protected URL ecb_d_m;
+   protected URL cbc_e_m;
+   protected URL cbc_d_m;
+
+   // Constructors
+   // ------------------------------------------------------------------------
+
+   public NISTCipherTest(String algorithm, int blockSize) {
+      super(algorithm);
+      cipher = CipherFactory.getInstance(algorithm);
+      attrib = new HashMap();
+      attrib.put(cipher.CIPHER_BLOCK_SIZE, new Integer(blockSize));
+      attrib.put(cipher.KEY_MATERIAL, new byte[cipher.defaultKeySize()]);
+      try {
+         cipher.init(attrib);
+      } catch (Exception e) { }
+   }
+
+   // Instance methods.
+   // ------------------------------------------------------------------------
+
+   public void setUp() {
+      String prefix = "/tv/" + cipher.name().toLowerCase() + "/";
+      ecb_vk = NISTCipherTest.class.getResource(prefix + ECB_VK);
+      ecb_vt = NISTCipherTest.class.getResource(prefix + ECB_VT);
+      ecb_e_m = NISTCipherTest.class.getResource(prefix + ECB_E_M);
+      ecb_d_m = NISTCipherTest.class.getResource(prefix + ECB_D_M);
+      cbc_e_m = NISTCipherTest.class.getResource(prefix + CBC_E_M);
+      cbc_d_m = NISTCipherTest.class.getResource(prefix + CBC_D_M);
+   }
+
+   public void runTest() {
+      String s = "Conformance(" + cipher.name() + "): ";
+      try {
+         if (ecb_vk != null) {
+            VarTest(ecb_vk.openStream());
+         }
+         if (ecb_vt != null) {
+            VarTest(ecb_vt.openStream());
+         }
+         if (ecb_e_m != null) {
+            MCTestECB(ecb_e_m.openStream(), ENCRYPTION);
+         }
+         if (ecb_d_m != null) {
+            MCTestECB(ecb_d_m.openStream(), DECRYPTION);
+         }
+         if (cbc_e_m != null) {
+            MCTestCBC(cbc_e_m.openStream(), ENCRYPTION);
+         }
+         if (cbc_d_m != null) {
+            MCTestCBC(cbc_d_m.openStream(), DECRYPTION);
+         }
+      } catch (Exception e) {
+         fail(s + e.getMessage());
+      }
+   }
+
+   // Own methods.
+   // ------------------------------------------------------------------------
+
+   /** Variable-key and variable-text test. */
+   protected void VarTest(InputStream tvIn) throws Exception {
+      LineNumberReader in = new LineNumberReader(new InputStreamReader(tvIn));
+      String line;
+      byte[] key = new byte[cipher.defaultKeySize()];
+      byte[] pt = new byte[cipher.currentBlockSize()];
+      byte[] ct = new byte[cipher.currentBlockSize()];
+      byte[] ect = new byte[cipher.currentBlockSize()];
+      while ((line = in.readLine()) != null) {
+         if (line.startsWith("KEYSIZE=")) {
+            int ks = Integer.parseInt(line.substring(line.indexOf('=')+1));
+            key = new byte[ks / 8];
+         } else if (line.startsWith("PT=")) {
+            stringToBytes(pt, line.substring(line.indexOf('=')+1));
+         } else if (line.startsWith("KEY=")) {
+            stringToBytes(key, line.substring(line.indexOf('=')+1));
+            attrib.put(cipher.KEY_MATERIAL, key);
+         } else if (line.startsWith("CT=")) {
+            stringToBytes(ect, line.substring(line.indexOf('=')+1));
+            cipher.reset();
+            cipher.init(attrib);
+            cipher.encryptBlock(pt, 0, ct, 0);
+            assertTrue(Util.areEqual(ct, ect));
+         }
+	 // Other lines are ignored.
+      }
+      in.close();
+   }
+
+   /** Electronic codebook mode monte carlo test. */
+   protected void MCTestECB(InputStream tvIn, int mode) throws Exception {
+      LineNumberReader in = new LineNumberReader(new InputStreamReader(tvIn));
+      String line;
+      byte[] key = new byte[cipher.defaultKeySize()];
+      byte[] pt = new byte[cipher.currentBlockSize()];
+      byte[] ct = new byte[cipher.currentBlockSize()];
+      byte[] et = new byte[cipher.currentBlockSize()];
+      while ((line = in.readLine()) != null) {
+         if (line.startsWith("KEYSIZE=")) {
+            int ks = Integer.parseInt(line.substring(line.indexOf('=')+1));
+            key = new byte[ks / 8];
+         } else if (line.startsWith("PT=")) {
+            if (mode == DECRYPTION) {
+               stringToBytes(et, line.substring(line.indexOf('=')+1));
+               cipher.reset();
+               cipher.init(attrib);
+               for (int i = 0; i < 10000; i++) {
+                  cipher.decryptBlock(ct, 0, pt, 0);
+                  System.arraycopy(pt, 0, ct, 0, pt.length);
+               }
+               assertTrue(Util.areEqual(pt, et));
+            } else {
+               stringToBytes(pt, line.substring(line.indexOf('=')+1));
+            }
+         } else if (line.startsWith("KEY=")) {
+            stringToBytes(key, line.substring(line.indexOf('=')+1));
+            attrib.put(cipher.KEY_MATERIAL, key);
+         } else if (line.startsWith("CT=")) {
+            if (mode == ENCRYPTION) {
+               stringToBytes(et, line.substring(line.indexOf('=')+1));
+               cipher.reset();
+               cipher.init(attrib);
+               for (int i = 0; i < 10000; i++) {
+                  cipher.encryptBlock(pt, 0, ct, 0);
+                  System.arraycopy(ct, 0, pt, 0, ct.length);
+               }
+               assertTrue(Util.areEqual(ct, et));
+            } else {
+               stringToBytes(ct, line.substring(line.indexOf('=')+1));
+            }
+         }
+	 // Other lines are ignored.
+      }
+      in.close();
+   }
+
+   /** Cipher block chaining mode monte carlo test. */
+   protected void MCTestCBC(InputStream tvIn, int mode) throws Exception {
+      LineNumberReader in = new LineNumberReader(new InputStreamReader(tvIn));
+      String line;
+      byte[] key = new byte[cipher.defaultKeySize()];
+      byte[] pt = new byte[cipher.currentBlockSize()];
+      byte[] ct = new byte[cipher.currentBlockSize()];
+      byte[] et = new byte[cipher.currentBlockSize()];
+      byte[] last = new byte[cipher.currentBlockSize()];
+      byte[] iv = new byte[cipher.currentBlockSize()];
+      while ((line = in.readLine()) != null) {
+         if (line.startsWith("KEYSIZE=")) {
+            int ks = Integer.parseInt(line.substring(line.indexOf('=')+1));
+            key = new byte[ks / 8];
+            if (mode == ENCRYPTION) {
+               for (int i = 0; i < ct.length; i++) {
+                  ct[i] = 0;
+               }
+            } else {
+               for (int i = 0; i < pt.length; i++) {
+                  pt[i] = 0;
+               }
+            }
+         } else if (line.startsWith("PT=")) {
+            if (mode == DECRYPTION) {
+               stringToBytes(et, line.substring(line.indexOf('=')+1));
+               cipher.reset();
+               cipher.init(attrib);
+               for (int i = 0; i < 10000; i++) {
+                  cipher.decryptBlock(ct, 0, pt, 0);
+                  for (int j = 0; j < pt.length; j++) {
+                     pt[j] ^= iv[j];
+                  }
+                  System.arraycopy(ct, 0, iv, 0, ct.length);
+                  System.arraycopy(pt, 0, ct, 0, pt.length);
+               }
+               assertTrue(Util.areEqual(pt, et));
+            } else {
+               stringToBytes(pt, line.substring(line.indexOf('=')+1));
+            }
+         } else if (line.startsWith("KEY=")) {
+            stringToBytes(key, line.substring(line.indexOf('=')+1));
+            attrib.put(cipher.KEY_MATERIAL, key);
+         } else if (line.startsWith("CT=")) {
+            if (mode == ENCRYPTION) {
+               stringToBytes(et, line.substring(line.indexOf('=')+1));
+               cipher.reset();
+               cipher.init(attrib);
+               for (int i = 0; i < 10000; i++) {
+                  for (int j = 0; j < pt.length; j++) {
+                     pt[j] ^= iv[j];
+                  }
+                  System.arraycopy(ct, 0, last, 0, ct.length);
+                  cipher.encryptBlock(pt, 0, ct, 0);
+                  System.arraycopy(ct, 0, iv, 0, ct.length);
+                  System.arraycopy(last, 0, pt, 0, last.length);
+               }
+               assertTrue(Util.areEqual(ct, et));
+            } else {
+               stringToBytes(ct, line.substring(line.indexOf('=')+1));
+            }
+         } else if (line.startsWith("IV=")) {
+            stringToBytes(iv, line.substring(line.indexOf('=')+1));
+         }
+	 // Other lines are ignored.
+      }
+      in.close();
+   }
+
+ // Class methods.
+   // -----------------------------------------------------------------------
+
+   /** Convert a hexadecimal string to a byte array. */
+   protected static void stringToBytes(byte[] buf, String str) {
+      for (int i = 0; i < buf.length && i * 2 < str.length(); i++) {
+         buf[i] = (byte) Integer.parseInt(str.substring(i*2, i*2+2), 16);
+      }
+   }
+}

---

From: "Raif S. Naffah" <[email-masked]>
Date: Tue, 18 Jun 2002 20:53:17 +1000

hello Casey,

thanks for your contribution :-)  it's a nice addition to the test suite 
of this library.

may i rename the "NISTCipherTest" to "TestOfNISTVectors" ?  it looks 
nicer with the other classes + i think conveys better what it is 
supposed to do.

...and yes, you guessed right; it is UMac32!

i'll generate (the test vectors) and test the new class before i check 
in the lot with all the corrections/amendements.

cheers;
rsn

Casey Marshall wrote:
> ...