寫程式是良心事業: 2016

2016年5月1日星期日

Java JCE - AES Encryption & Decryption @2016-05-01 (English Version)

Foreword:

Why I write this article ? In my original thinking, I believe that there is a lot of blogs talk about Java AES example. However, I still found some sample code is not clearly to explain the issues of these sample. Therefore, I am afraid someone may really adopt these samples in their production system.

In addition, in order to simplify the explanation, the following sample will ignore the exception handle.

Content:

Some similar sample code that you often find on internet is as the following:

class AES_DEFAULT {  
 public static byte[] Encrypt(SecretKey secretKey, String msg) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); //: default is AES/ECB/PKCS5Padding
  cipher.init(Cipher.ENCRYPT_MODE, secretKey); 
  System.out.println("AES_DEFAULT IV:"+cipher.getIV());
  System.out.println("AES_DEFAULT Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes());  
  System.out.println("Encrypted result and base64 encoded：" + Base64.getEncoder().encodeToString(byteCipherText));
  return byteCipherText;  
 }  

 public static byte[] Decrypt(SecretKey secretKey, byte[] cipherText) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey);  
  byte[] decryptedText = cipher.doFinal(cipherText);  
  String strDecryptedText = new String(decryptedText);
  System.out.println("Decrypted result：" + strDecryptedText);
  return decryptedText;  
 }  

 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(128,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_DEFAULT.Encrypt(secretKey, "I am PlainText!!");
  AES_DEFAULT.Decrypt(secretKey, cipher);  
 }
}

The above sample code can work well for encryption and decryption, but there are some issues:

The first issue is located at the line 4. It uses the ECB cipher mode, however this cipher mode is not a secure cipher mode for AES, because it will cause the cipher block is the same if the input plain-text is the same. You could refer to the Wiki.
The second issue is located at the line 25. We should not use the 128 bits length as the AES KEY. Now is 2016, the recommended length of AES key is at least 256 bits.
The third issue is the line 8. It use the msg.getBytes( ). This style will be fine if the program is running at the same platform or machine. However, your program will run at different platform/machine, and you will find the default charset may be different for different platform/machine. Therefore, this may cause the decrypted result is not as you expected.

The recommended implementation is as the following sample:

Important warring: You should select the correct cipher mode, for example: CCM or GCM mode. (Update 2022/03/04) The reason is the CBC mode is vulnerable to padding oracle attacks.

class AES_CBC_PKCS5PADDING {
 
 public static byte[] Encrypt(SecretKey secretKey, byte[] iv, String msg) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CBC_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CBC_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes("UTF-8"));
  System.out.println("Encrypted result and base64 encoded：" + Base64.getEncoder().encodeToString(byteCipherText));

  return byteCipherText;
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] cipherText, byte[] iv) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  byte[] decryptedText = cipher.doFinal(cipherText);
  String strDecryptedText = new String(decryptedText);
  System.out.println("Decrypted result：" + strDecryptedText);
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_CBC_PKCS5PADDING.Encrypt(secretKey, iv, "I am PlainText!!");
  AES_CBC_PKCS5PADDING.Decrypt(secretKey, cipher, iv);  
 }
}

As you seeing at line 3, we adopt the CBC cipher mode with PKCS5 Padding. You could refer to the Padding for the detail. At line 8, we directly invoke msg.getBytes("UTF-8") to avoid some charset issues. Of course, you could specify it use ANSI , and it is still work well if your plain-text contain ANSI only,

Here, allow me to remind you. The recommended length of AES key is 256 bits. In addition, you should always generate a new IV to encrypt if you use the same AES key to encrypt data.

You could refer the below sample code to generate the required Secret Key and IV. In addition, you also need to provide the same IV to the decryption side.

  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

Sometime, you may see the following error message, the reason is the default Oracle's Java Runtime can not generate the AES 256 bits keys because the policy issue.

Exception in thread "main" java.security.InvalidKeyException: Illegal key size or default parameters
 at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1026)
 at javax.crypto.Cipher.implInit(Cipher.java:801)
 at javax.crypto.Cipher.chooseProvider(Cipher.java:864)
 at javax.crypto.Cipher.init(Cipher.java:1249)
 at javax.crypto.Cipher.init(Cipher.java:1186)

You need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files if you have the same problem at your environment. One more thing that you may need to check is the policy file version should be same as the JRE version at your environment. (The Android will not have this issue)

Finally, you may ask common questions as the followings：

How to encrypt a stream file with large file size (EX:Video or Audio)？
How to randomly access specific block data of the encrypted file ?

The possible solution is as below ：

The answer for the first one question is easy. Just study the Java Cipher API doc carefully, then you can invoke the update method of Cipher with multiple times , and invoke the doFinal method at the end.
The answer for the 2nd question is to change the cipher mode as the CTR cipher mode. In addition, you need to write a calculate IV method for the target block that you want.

You could refer to the following implementation:
(Note: This sample code does not testing well, please don't adopt it on production system.)

class AES_CTR_PKCS5PADDING {
 private static final int BLOCK_SIZE = 16;
 
 public static void Encrypt(SecretKey secretKey, byte[] iv, File plainTextFile, File encryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CTR_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CTR_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(plainTextFile);
    OutputStream out = new FileOutputStream(encryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] cipherBytes = cipher.update(buf, 0 , readBytes);
    out.write(cipherBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] iv, File cipherTextFile, File decryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  
  if(!decryptedFile.exists()){
   decryptedFile.createNewFile(); //: Here, it may be fail if ...
  }
  
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(cipherTextFile);
    OutputStream out = new FileOutputStream(decryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] decryptedBytes = cipher.update(buf, 0 , readBytes);
    out.write(decryptedBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
  
 public static byte[] DecryptPartial(SecretKey secretKey, byte[] iv, File cipherTextFile, int blockIndex, int blockCount ) throws Exception{
  final int offset = blockIndex * BLOCK_SIZE;
  final int bufSize = blockCount * BLOCK_SIZE;

  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, calculateIVForBlock(new IvParameterSpec(iv), blockIndex ));

  byte[] decryptedBytes = new byte[bufSize];
  try (FileInputStream in = new FileInputStream(cipherTextFile)){
   byte inputBuf[] = new byte[bufSize];
   in.skip(offset);
   int readBytes = in.read(inputBuf);
   decryptedBytes = cipher.update(inputBuf, 0, readBytes);
  }
  return decryptedBytes;
 } 

 private static IvParameterSpec calculateIVForBlock(final IvParameterSpec iv,
         final long blockIndex) {  
     final BigInteger biginIV = new BigInteger(1, iv.getIV());
     final BigInteger blockIV = biginIV.add(BigInteger.valueOf(blockIndex));
     final byte[] blockIVBytes = blockIV.toByteArray();

     // Normalize the blockIVBytes as 16 bytes for IV
     if(blockIVBytes.length == BLOCK_SIZE){
      return new IvParameterSpec(blockIVBytes);
     }
     if(blockIVBytes.length > BLOCK_SIZE ){
      // For example: if the blockIVBytes length is 18, blockIVBytes is [0],[1],...[16],[17]
      // We have to remove [0],[1] , so we change the offset = 2
      int offset = blockIVBytes.length - BLOCK_SIZE;
      return new IvParameterSpec(blockIVBytes, offset, BLOCK_SIZE);
     }
     else{
      // For example: if the blockIVBytes length is 14, blockIVBytes is [0],[1],...[12],[13]
      // We have to insert 2 bytes at head
      final byte[] newBlockIV = new byte[BLOCK_SIZE]; //: default set to 0 for 16 bytes
      int offset = blockIVBytes.length - BLOCK_SIZE;
      System.arraycopy(blockIVBytes, 0, newBlockIV, offset, blockIVBytes.length);
      return new IvParameterSpec(newBlockIV);
     }
 }
 
 private static void createTestFile(String path) throws Exception{
  File test = new File(path);  
  try(FileOutputStream out = new FileOutputStream(test)){

   StringBuffer buf = new StringBuffer(16);

   int blockCount = 100000;
   for(int i = 0 ; i < blockCount ; i ++){
    buf.append(i);
    int size = buf.length();
    for(int j = 0; j < (14-size); j++ ){
     buf.append('#');
    }
    out.write(buf.toString().getBytes());
    out.write("\r\n".getBytes());
    buf.delete(0, 16);
   }   
  }  
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);
  
  {
   String originalFile = "~/PlainText.txt";
   String encryptedFile = "~/CipherText.enc"; 
   String deryptedFile = "~/Decrypted.txt";   

   AES_CTR_PKCS5PADDING.createTestFile(originalFile); //: Create Testing Data
   
   AES_CTR_PKCS5PADDING.Encrypt(secretKey, iv, new File(originalFile), new File(encryptedFile));
   AES_CTR_PKCS5PADDING.Decrypt(secretKey, iv, new File(encryptedFile), new File(deryptedFile));
   byte[] ret = AES_CTR_PKCS5PADDING.DecryptPartial(secretKey, iv, new File(encryptedFile), 100, 10);   
   System.out.println(new String(ret));
  }
 }

Final:

I don't talk about the GCM cipher mode here. In principle, you don't need the GCM if you don't need the authentication

Reference:

* https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_.28ECB.29
* https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html

Java JCE - AES 的 Encryption & Decryption @2016-05-01

前言：

都已經2016年了，本來不打算寫這篇的，但偶然發現網路上有多個中文部落格，甚至是論壇上分享或是討論 Java 的 AES 的程式，都沒有討論一些問題，深怕一堆人看到這種範例程式就放到你開發的系統上。另外，為了簡化程式來說明，以下的程式並不考慮Exception處理方式。

內容：

先來看網路上常見的程式的寫法：

class AES_DEFAULT {  
 public static byte[] Encrypt(SecretKey secretKey, String msg) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); //: 等同 AES/ECB/PKCS5Padding
  cipher.init(Cipher.ENCRYPT_MODE, secretKey); 
  System.out.println("AES_DEFAULT IV:"+cipher.getIV());
  System.out.println("AES_DEFAULT Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes());  
  System.out.println("加密結果的Base64編碼：" + Base64.getEncoder().encodeToString(byteCipherText));
  return byteCipherText;  
 }  

 public static byte[] Decrypt(SecretKey secretKey, byte[] cipherText) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey);  
  byte[] decryptedText = cipher.doFinal(cipherText);  
  String strDecryptedText = new String(decryptedText);
  System.out.println("解密結果：" + strDecryptedText);
  return decryptedText;  
 }  

 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(128,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_DEFAULT.Encrypt(secretKey, "I am PlainText!!");
  AES_DEFAULT.Decrypt(secretKey, cipher);  
 }
}

上面這種寫法，這個程式對於加解密的運作是正常的，但會有潛在的3個問題：

第 1 個問題在第 4 行這個用法所使用的 Cipher mode 是 ECB，也就是比較不安全的方式。原因是 ECB 對於相同的資料加密後的結果會是一樣的，有興趣可以參考(Wiki上的那張企鵝圖)。如果你的應用是每次加密時 secret key 都是重新產生的，而且需要被加密的資料每次都完全不同的時候，各自的資料內容本身也是異質性相當高，如果採用這種做法也沒有太大的問題，但還是不建議。因為以加密的應用來說，常見的對象就是檔案或是運用在傳輸加密。而這兩種方式，大部分都會有相同的資料。以檔案來說，相同類型的檔案你用binary編輯器打開檔案，你就可以觀察到檔案的前面都會有雷同的資料。通訊協定更是如此，例如：HTTP通訊協定。
第 2 個問題在第 25 行 secret key 長度的問題，不應該使用 128 bits 長度，強度太弱。
第 3 個問題在第 8 行的 msg.getBytes( )，不同的作業系統所使用的預設 charset 可能是不同的。這樣的做法，有可能會造成不同平台在加密時的行為不如你所預期，也就是解密後的內容可能跟你當初要加密的資料不同。

比較建議的寫法是採用下面這種：

Important warring: You should select the correct cipher mode, for example: CCM or GCM mode. (Update 2022/03/04) The reason is the CBC mode is vulnerable to padding oracle attacks.

class AES_CBC_PKCS5PADDING {
 
 public static byte[] Encrypt(SecretKey secretKey, byte[] iv, String msg) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CBC_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CBC_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes("UTF-8"));
  System.out.println("加密結果的Base64編碼：" + Base64.getEncoder().encodeToString(byteCipherText));

  return byteCipherText;
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] cipherText, byte[] iv) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  byte[] decryptedText = cipher.doFinal(cipherText);
  String strDecryptedText = new String(decryptedText);
  System.out.println("解密結果：" + strDecryptedText);
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_CBC_PKCS5PADDING.Encrypt(secretKey, iv, "I am PlainText!!");
  AES_CBC_PKCS5PADDING.Decrypt(secretKey, cipher, iv);  
 }
}

在第 3 行明確指定採用 CBC 的 cipher mode，並且指定 Padding 方式，有興趣可參考這篇 Padding 運作方式。在第 8 行，直接指定用 msg.getBytes("UTF-8")，避免一些問題，如果需要加密的內容都是純ANSI，你可以指定用 ANSI 就可以。

此處，還是要強調 AES的加密 Secret Key 的長度建議至少要 256 bit 以上，用同一把 Secret Key 做加密時候，應該都要產生新的 IV 來加密。可以參考下面這種寫法來產生所需要的 AES Secret Key 和 IV。另外，IV 在解密時候，也要用當初加密使用的相同ＩＶ才可以。

  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

另外，由於 Oracle 官方預設標準的Java執行環境沒法支援產生 AES 256 bits 長度的 secret key。所以，你會遇到下面這種錯訊息：

Exception in thread "main" java.security.InvalidKeyException: Illegal key size or default parameters
 at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1026)
 at javax.crypto.Cipher.implInit(Cipher.java:801)
 at javax.crypto.Cipher.chooseProvider(Cipher.java:864)
 at javax.crypto.Cipher.init(Cipher.java:1249)
 at javax.crypto.Cipher.init(Cipher.java:1186)

這表示你的執行環境需要安裝 Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files。這裡要特別注意，你務必要安裝跟執行環境JVM相同版本的JCE Policy，否則可能會遇到一些怪異的現象。(Android執行環境不在此限)

最後，常問遇到的問題就是：

如果檔案很大的Stream類型的檔案(如:Video or Audio)要如何做到加密？
想要隨意位置讀取已經加密的內容要怎麼做？

建議的做法：

第 1 個問題，很簡單請看清楚 Cipher 的 API doc，採用多次呼叫 update 的方法，最後再呼叫 doFinal 方法即可。
第 2 個問題，只要將 Cipher mode 改為 CTR 即可，另外如果是要隨機存取某個 Block 的資料，必須要自己重新計算那個 Block 開始的 IV。

請參考類似下面的寫法：
(Note: This sample code does not testing well, please don't adopt it on production system.)

class AES_CTR_PKCS5PADDING {
 private static final int BLOCK_SIZE = 16;
 
 public static void Encrypt(SecretKey secretKey, byte[] iv, File plainTextFile, File encryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CTR_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CTR_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(plainTextFile);
    OutputStream out = new FileOutputStream(encryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] cipherBytes = cipher.update(buf, 0 , readBytes);
    out.write(cipherBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] iv, File cipherTextFile, File decryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  
  if(!decryptedFile.exists()){
   decryptedFile.createNewFile(); //: Here, it may be fail if ...
  }
  
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(cipherTextFile);
    OutputStream out = new FileOutputStream(decryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] decryptedBytes = cipher.update(buf, 0 , readBytes);
    out.write(decryptedBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
  
 public static byte[] DecryptPartial(SecretKey secretKey, byte[] iv, File cipherTextFile, int blockIndex, int blockCount ) throws Exception{
  final int offset = blockIndex * BLOCK_SIZE;
  final int bufSize = blockCount * BLOCK_SIZE;

  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, calculateIVForBlock(new IvParameterSpec(iv), blockIndex ));

  byte[] decryptedBytes = new byte[bufSize];
  try (FileInputStream in = new FileInputStream(cipherTextFile)){
   byte inputBuf[] = new byte[bufSize];
   in.skip(offset);
   int readBytes = in.read(inputBuf);
   decryptedBytes = cipher.update(inputBuf, 0, readBytes);
  }
  return decryptedBytes;
 } 

 private static IvParameterSpec calculateIVForBlock(final IvParameterSpec iv,
         final long blockIndex) {  
     final BigInteger biginIV = new BigInteger(1, iv.getIV());
     final BigInteger blockIV = biginIV.add(BigInteger.valueOf(blockIndex));
     final byte[] blockIVBytes = blockIV.toByteArray();

     // Normalize the blockIVBytes as 16 bytes for IV
     if(blockIVBytes.length == BLOCK_SIZE){
      return new IvParameterSpec(blockIVBytes);
     }
     if(blockIVBytes.length > BLOCK_SIZE ){
      // For example: if the blockIVBytes length is 18, blockIVBytes is [0],[1],...[16],[17]
      // We have to remove [0],[1] , so we change the offset = 2
      int offset = blockIVBytes.length - BLOCK_SIZE;
      return new IvParameterSpec(blockIVBytes, offset, BLOCK_SIZE);
     }
     else{
      // For example: if the blockIVBytes length is 14, blockIVBytes is [0],[1],...[12],[13]
      // We have to insert 2 bytes at head
      final byte[] newBlockIV = new byte[BLOCK_SIZE]; //: default set to 0 for 16 bytes
      int offset = blockIVBytes.length - BLOCK_SIZE;
      System.arraycopy(blockIVBytes, 0, newBlockIV, offset, blockIVBytes.length);
      return new IvParameterSpec(newBlockIV);
     }
 }
 
 private static void createTestFile(String path) throws Exception{
  File test = new File(path);  
  try(FileOutputStream out = new FileOutputStream(test)){

   StringBuffer buf = new StringBuffer(16);

   int blockCount = 100000;
   for(int i = 0 ; i < blockCount ; i ++){
    buf.append(i);
    int size = buf.length();
    for(int j = 0; j < (14-size); j++ ){
     buf.append('#');
    }
    out.write(buf.toString().getBytes());
    out.write("\r\n".getBytes());
    buf.delete(0, 16);
   }   
  }  
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);
  
  {
   String originalFile = "~/PlainText.txt";
   String encryptedFile = "~/CipherText.enc"; 
   String deryptedFile = "~/Decrypted.txt";   

   AES_CTR_PKCS5PADDING.createTestFile(originalFile); //: Create Testing Data
   
   AES_CTR_PKCS5PADDING.Encrypt(secretKey, iv, new File(originalFile), new File(encryptedFile));
   AES_CTR_PKCS5PADDING.Decrypt(secretKey, iv, new File(encryptedFile), new File(deryptedFile));
   byte[] ret = AES_CTR_PKCS5PADDING.DecryptPartial(secretKey, iv, new File(encryptedFile), 100, 10);   
   System.out.println(new String(ret));
  }
 }

最後：

這邊我沒有提到另外一種 GCM 的 Cipher Mode，原則上，如果你沒有 Authentication 的需要時候，就不需要用到 GCM。

Reference:

* https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_.28ECB.29
* https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html

2016年1月11日星期一

MySQL - FEDERATED

Why use MySQL FEDERATED ?

Sometime that you need to access the remote server's table at local database, however you don't want to duplicated the data at two databases (remote and local). Therefore, you have another choice - MySQL FEDERATED storage engine.

In addition, you can grant the specific permission for specific account at remote database for this purpose.
Whatever, create a local table with federated storage engine, it just likes a local proxy table; any SQL statement will be executed at remote table.

How to do ?

Check the MySQL engine:
- mysql> show engines;

+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+

+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+

| MyISAM | YES | MyISAM storage engine | NO | NO | NO |

| CSV | YES | CSV storage engine | NO | NO | NO |

| MRG_MYISAM | YES | Collection of identical MyISAM tables | NO | NO | NO |

| BLACKHOLE | YES | /dev/null storage engine (anything you write to it disappears) | NO | NO | NO |

| MEMORY | YES | Hash based, stored in memory, useful for temporary tables | NO | NO | NO |

| PERFORMANCE_SCHEMA | YES | Performance Schema | NO | NO | NO |

| ARCHIVE | YES | Archive storage engine | NO | NO | NO |

| InnoDB | DEFAULT | Supports transactions, row-level locking, and foreign keys | YES | YES | YES |

+--------------------+---------+----------------------------------------------------------------+--------------+------+------------+

Enable federated.
- Use mysql console to install ha_federated.so
  - mysql>install plugin federated soname 'ha_federated.so';
- Modify my.cnf. Add ‘federated’ under [mysqld]

[mysqld]

federated

Configure Remote MySQL Server
- Grant permission

mysql> GRANT ALL on DB_NAME.* TO user@‘local_host' IDENTIFIED BY ‘your password';

For example:

mysql> GRANT ALL on ViewDBTest2.* TO root@'172.16.66.143' IDENTIFIED BY '111111';

Configure Inbound rule of Remote MySQL Server

shell> sudo /sbin/iptables -A INPUT -i eth0 -p tcp --destination-port 3306 -j ACCEPT

Create View Table at Local Database

Create Table `Local_Table_Name` (

      `admin_id` int(11) NOT NULL AUTO_INCREMENT,
      `email` varchar(255) NOT NULL,
      `password` varchar(128) NOT NULL,
      `role` varchar(16) NOT NULL,
      PRIMARY KEY (`admin_id`),
      UNIQUE KEY `UK_jl20d0ecx48g7qwy1dxe2akre` (`email`)
) ENGINE=FEDERATED

DEFAULT CHARSET=utf8 COLLATE=utf8_general_ci

CONNECTION='mysql://user:password@remote_mysql_host:port/DB_NAME/Remote_Table_Name';

Note: The columns definition in Local_Table_Name and Remote_Table_Name should be the same!

Something you need to know

The federated has some performance issue at the Oracle MySQL.

The data is not existed at the local table; any SQL statement is executing at remote table.

The schema changed of remote table would be an issue, because the local table will not be notified.

Reference:

https://dev.mysql.com/doc/refman/5.7/en/federated-storage-engine.html

2016年1月10日星期日

OCSP & CRL 介紹

這篇主要是要談論在 PKI 架構下，如何查詢一張 X.509 憑證 ( Certificate ) 的有效性？

舉例：

瀏覽器 (Browser) 如何驗證支援 SSL 連線的伺服器所使用的憑證 Certificate 是有效的。
自行開發的應用程式，如何透過程式碼來查詢某張憑證是否已經被 CA 單位所撤銷 (Revoke)。

首先，我們先定義"有效的憑證"：

該憑證是經由公認的 CA 或是執行的系統所信任的 CA 發行的憑證。
憑證的是否在有效期間內。
該憑證沒有被發行憑證的 CA 所撤銷 ( Revoked )。

我們會分別以下面的主題來解釋 OCSP 和 CRL 這兩個在 PKI 架構的規範。

基本概念介紹
實務上的應用
優點缺點

CRL 基本概念介紹

CRL 是 Certificate Revocation List 的縮寫，顧名思義就是紀錄被CA所撤銷的憑證清單。實際上他是一個發行憑證的 CA 用來對外公告，這個 CA 所發行的憑證，雖然憑證上的日期仍然是有效的，但有這些憑證當中，哪些是因為某些因素而被撤銷的憑證。被憑證撤銷的理由 (Reason) 很多，RFC 的規格上定義了幾個常見的理由如下：

unspecified (0)
keyCompromise (1)
CACompromise (2)
affiliationChanged (3)
superseded (4)
cessationOfOperation (5)
certificateHold (6)
removeFromCRL (8)
privilegeWithdrawn (9)
AACompromise (10)

實際上，CRL 就是一個檔案，任何人都可以根據憑證中所指定的URI位置取得這份檔案。如果你透過工具來檢視憑證，我們可透過 OpenSSL 指令來檢視憑證內容

shell>openssl x509 -in cert.crt -text

如果發出這張憑證單位的 CA 有提供 CRL 資訊，你可以在 X.509 的 V3 Extension 找到一個
"X509v3 CRL Distribution Points" 的資訊，它會顯示一個或是一個以上的 URI，如下圖紅色標示的部分：

當你將那段URI的位置，透過工具如 wget or curl 或是瀏覽器來取得這個 CRL 檔案時，檔案的內容就可能紀錄著類似藍色箭頭所指示的內容，裡面會包含哪些憑證已經在何時被撤銷。每張憑證都具有一個 Serial Number, 他就像是憑證的身分證號碼，由發行的CA單位所管理，這些 Serial Number 必須是在這個CA單位中是唯一。然而，CRL 的檔案通常都是用DER或是PEM的編碼過的，所以你無法用一般的編輯器直接來檢視他的內容。此外，它會包含CA所簽署的簽章(Signature)資訊，如此才能證明這個CRL檔案是由 CA 單位所公告的資訊。我們可透過 OpenSSL 指令來檢視一個CRL檔案的內容

shell>openssl crl -in crl.der -text -noout

你就可以看到，類似下圖所示的內容：

此外，你可以在最下方看見一個簽章的資訊，例如下圖：

它說明了這個 CRL 檔案是由 sha256withRSAEncryption 的演算法所簽署的簽章 (Signature)，因此取得這個CRL的檔案後，要拿它來判斷是否有憑證被撤銷之前，應該要驗證這個 CRL 檔案的簽章，以確保這個CRL檔案是沒有被竄改過的。那要如何驗證呢？這時要檢視 CRL extension 中的 Authority Key Identifier。這表示這個簽章要使用 Authority Key Identifier 所指定的 Public Key 來驗證。如果發行憑證的 CA 沒有特別針對 CRL 而另外指定用於 signing CRL 專屬的 certificate 時，這個 Authority Key Identifier 通常就是發行 CA 的 public key id。因此你就可以透過 CA certificate 來驗證這個CRL的簽章是否正確。

CRL extensions:

X509v3 Authority Key Identifier:

keyid:51:68:FF:90:AF:02:07:75:3C:CC:D9:65:64:62:A2:12:B8:59:72:3B

實務上的應用

一般來說，程式(例如: Browser 或自行開發會使用到憑證的應用程式)會根據憑證在 X.509 的 V3 Extension 找到一個"X509v3 CRL Distribution Points" 的資訊去取得這份 CRL 檔案。然而，CRL 檔案通常是由 CA 單位來定期發佈, 而 CRL的檔案中會記錄著這份CRL的有效期間，以及下次更新的時間。因此，要使用CRL 的應用程式，可以根據 CRL 的有效期間來決定是否重新取得這份CRL檔案，或是只接使用在本地端的 CRL 檔案即可。

CRL的優點

CA 實作CRL是相對容易的，只需要定期將被撤銷的憑證資訊更新到 CRL 之中即可。
CRL 取得容易，一般來說，需要 CRL 的應用程式可以直接透過 HTTP 就取得這個檔案。

CRL的缺點

CRL 檔案的內容可能非常大，如果 CA 單位長時間下來撤銷許多發行的憑證時，這個 CRL 檔案內容就會非常的大，可能高達25MB以上。有興趣的讀者可以參考這篇Digging Into Certificate Revocation Listsj文章。
CRL 檔案資訊不夠即時，可能造成安全上的空窗期。因為CRL檔案是每隔一段時間才公布，因此在下次公布CRL檔案之前，有憑證被撤銷時，要等到下次公布才能得到這個資訊。
如果程式無法下載 CRL 檔案，則應用程式可能會直接信任它想要確認的憑證。

Base CRL 和 Delta CRL

由於，CRL 本身在設計上有這樣的缺點，因此後來提出另一個的解決方式稱為 Delta CRL。

Delta CRL 是指記錄部分的被撤銷的憑證資訊，而不是全部的被撤銷的憑證資訊。並且縮短了 Delta CRL 的發佈時間。舉例，原本的 CRL 發佈時間是 10 天發佈 1 次，而這個 CRL 又被稱為 Base CRL。而 Delta CRL 的發佈週期就比較短一點，可能只要 2 天就發佈一次。因此，Delta CRL 只會記錄在這 2 天內有被撤銷的憑證資訊，並且在 Delta CRL 檔案中會記錄著他的 Base CRL 是哪一個版號。而應用程式只要每2天重新抓取這個 Delta CRL檔案，加上原本的 Base CRL檔案就可以得到完整的 CRL 資訊。

請參考下方這張圖，Base CRL 檔案中會記錄著一個 X509v3 Freshest CRL的資訊，而上面所記錄的 URI位置就是 Delta CRL檔案的位置。當你下載這個 Deltra CRL檔案後，即可發現 Delta CRL 檔案結構和 Base CRL一樣，只是 X509v3 Delta CRL Indicator 標記這個 CRL 檔案是一個 Delta CRL 檔案，此外，它記錄著是根據哪一個 Base CRL的版號，當作比對的基礎，而產生這份 Delta CRL。

OCSP

基本概念介紹

OCSP 是 Online Certificate Status Protocol 的縮寫，它目的在於提供線上動態查詢憑證的狀態。OCSP 本身提供了應用程式更多的彈性，來根據使用上的需要來查詢憑證的狀態。簡單的說就是應用程式能夠根據憑證的序號，發出OCSP請求到憑證上所記錄的 OCSP URI 來查詢憑證的狀態。我們可以參考下面這張概念圖：

由上圖，可以看到左右兩邊橘色部分是有關數位簽章的資訊，右邊是 OCSP 的 Response 的格式示意圖，而橘色的部分是指這個 OCSP Response 內容的簽章。接收端應該要驗證這個簽章無誤後，才能夠信任這個 OCSP 的回應內容。而左邊，是發出 OCSP 請求的客戶端把請求的內容也算出了一個簽章資訊。但，並不是強制性的，這完全是根據實作 OCSP responder來決定是否要強制要求，發出 OCSP 請求的客戶端一定要產生這個簽章。

當客戶端發出請求給 OCSP Responder時候，這時可以根據憑證的狀態有3種回答：

Good - 表示憑證的狀態是在有效期內，而且沒有被撤銷
Revoked - 表示憑證已經被撤銷
Unknown - 表示 OCSP Responder 無法認得這張憑證

實務上的應用

實務上就是大部份知名的瀏覽器都支援 OCSP 的實作，來驗證瀏覽的網頁伺服器所使用的HTTPS的憑證是否有效。瀏覽器連上某個網站時候，它會根據伺服器所使用的憑證內容所記錄的 OCSP Responder 位置，發出 OCSP 請求去詢問這張憑證目前的狀態，來決定是否信任這個網站。

OCSP 優點

應用程式可以一次只問一張憑證的狀態即可，不需要像 CRL 把所有的撤銷的憑證資訊都下載下來。
降低了發佈的時間延遲，縮短了這方面的問題。

OCSP 缺點

對 OCSP Responder 的流量會相當大
如果遇到不支援 OCSP 的客戶端或是連不到 OCSP Responder的情形，會造成客戶端直接信任伺服器憑證的狀況，那就會讓駭客能夠偽裝成伺服器來進行攻擊

OCSP Stapling

由於 OCSP 原本的設計方式是，會由一個 OCSP Responder 來負責回答客戶端的 OCSP 請求。然而在實務上瀏覽器為了避免要發兩次請求到不同的位置，而造成延遲，另一方面也是為了減輕 OCSP Responder的負擔，讓瀏覽器不需要每次都到 OCSP Responder來詢問。因此，發展出了另一個改良版本的做法，稱作 OCSP Stapling。它是指由伺服器端同時提供 OCSP 回應的做法。這樣瀏覽器連接到這個網站時，就不需要再發另一個請求到另一個位置的 OCSP Responder，可以在同一個連線當中就可以得到這張伺服器所使用的憑證是否還是有效。

當然，要支援 OCSP Stapling 是需要付出代價的，那就是你所架設的伺服器必須要能夠支援。目前常見的網頁伺服器都有支援，如：Windows, Apache, Nginx 等等。

Reference

CRL - RFC-5280 - https://www.ietf.org/rfc/rfc5280.txt
OCSP -RFC-6960 - https://tools.ietf.org/html/rfc6960
Digging Into Certificate Revocation Lists - http://community.websense.com/blogs/securitylabs/archive/2013/07/10/digging-into-certificate-revocation-lists.aspx
OCSP Stapling - https://blog.gslin.org/archives/2014/06/21/4786/ocsp-stapling/
Enable OCSP Stapling - https://www.digicert.com/enabling-ocsp-stapling.htm

訂閱：文章 (Atom)

2016年5月1日 星期日

Java JCE - AES Encryption & Decryption @2016-05-01 (English Version)

Foreword:

Content:

Final:

Reference:

Java JCE - AES 的 Encryption & Decryption @2016-05-01

前言：

內容：

Reference:

2016年1月11日 星期一

MySQL - FEDERATED

Why use MySQL FEDERATED ?

How to do ?

Something you need to know

Reference:

https://dev.mysql.com/doc/refman/5.7/en/federated-storage-engine.html

2016年1月10日 星期日

OCSP & CRL 介紹

CRL 基本概念介紹

實務上的應用

CRL的優點

CRL的缺點

Base CRL 和 Delta CRL

OCSP

基本概念介紹

實務上的應用

OCSP 優點

OCSP 缺點

OCSP Stapling

Reference

2016年5月1日星期日

2016年1月11日星期一

2016年1月10日星期日