顯示具有 Encryption 標籤的文章。 顯示所有文章
顯示具有 Encryption 標籤的文章。 顯示所有文章

2016年5月1日 星期日

Java JCE - AES Encryption & Decryption @2016-05-01 (English Version)

Foreword:

Why I write this article ?  In my original thinking, I believe that there is a lot of blogs talk about Java AES example. However, I still found some sample code is not clearly to explain the issues of these sample. Therefore, I am afraid someone may really adopt these samples in their production system.

In addition, in order to simplify the explanation, the following sample will ignore the exception handle.

Content:

Some similar sample code that you often find on internet is as the following:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class AES_DEFAULT {  
 public static byte[] Encrypt(SecretKey secretKey, String msg) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); //: default is AES/ECB/PKCS5Padding
  cipher.init(Cipher.ENCRYPT_MODE, secretKey); 
  System.out.println("AES_DEFAULT IV:"+cipher.getIV());
  System.out.println("AES_DEFAULT Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes());  
  System.out.println("Encrypted result and base64 encoded：" + Base64.getEncoder().encodeToString(byteCipherText));
  return byteCipherText;  
 }  

 public static byte[] Decrypt(SecretKey secretKey, byte[] cipherText) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey);  
  byte[] decryptedText = cipher.doFinal(cipherText);  
  String strDecryptedText = new String(decryptedText);
  System.out.println("Decrypted result：" + strDecryptedText);
  return decryptedText;  
 }  

 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(128,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_DEFAULT.Encrypt(secretKey, "I am PlainText!!");
  AES_DEFAULT.Decrypt(secretKey, cipher);  
 }
}

The above sample code can work well for encryption and decryption, but there are some issues:
  1. The first issue is located at the line 4. It uses the ECB cipher mode, however this cipher mode is not a secure cipher mode for AES, because it will cause the cipher block is the same if the input plain-text is the same. You could refer to the Wiki. 
  2. The second issue is located at the line 25. We should not use the 128 bits length as the AES KEY. Now is 2016, the recommended length of AES key is at least 256 bits.
  3. The third issue is the line 8. It use the msg.getBytes( ). This style will be fine if the program is running at the same platform or machine. However, your program will run at different platform/machine, and you will find the default charset may be different for different platform/machine. Therefore, this may cause the decrypted result is not as you expected.  

The recommended implementation is as the following sample:

Important warring: You should select the correct cipher mode,  for example: CCM or GCM mode. (Update 2022/03/04) The reason is the CBC mode is vulnerable to padding oracle attacks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
class AES_CBC_PKCS5PADDING {
 
 public static byte[] Encrypt(SecretKey secretKey, byte[] iv, String msg) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CBC_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CBC_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes("UTF-8"));
  System.out.println("Encrypted result and base64 encoded：" + Base64.getEncoder().encodeToString(byteCipherText));
  return byteCipherText;
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] cipherText, byte[] iv) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  byte[] decryptedText = cipher.doFinal(cipherText);
  String strDecryptedText = new String(decryptedText);
  System.out.println("Decrypted result：" + strDecryptedText);
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_CBC_PKCS5PADDING.Encrypt(secretKey, iv, "I am PlainText!!");
  AES_CBC_PKCS5PADDING.Decrypt(secretKey, cipher, iv);  
 }
}
As you seeing at line 3,  we adopt the CBC cipher mode with PKCS5 Padding. You could refer to the  Padding for the detail.  At line 8, we directly invoke msg.getBytes("UTF-8") to avoid some charset issues. Of course,  you could specify it use ANSI , and it is still work well if your plain-text contain ANSI only,

Here, allow me to remind you. The recommended length of AES key is 256 bits. In addition, you should always generate a new IV to encrypt if you use the same AES key to encrypt data.

You could refer the below sample code to generate the required Secret Key and IV. In addition, you also need to provide the same IV to the decryption side.

  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

Sometime, you may see the following error message, the reason is the default Oracle's Java Runtime can not generate the AES 256 bits keys because the policy issue.

1
2
3
4
5
6
Exception in thread "main" java.security.InvalidKeyException: Illegal key size or default parameters
 at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1026)
 at javax.crypto.Cipher.implInit(Cipher.java:801)
 at javax.crypto.Cipher.chooseProvider(Cipher.java:864)
 at javax.crypto.Cipher.init(Cipher.java:1249)
 at javax.crypto.Cipher.init(Cipher.java:1186)

You need to install the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files if you have the same problem at your environment. One more thing that you may need to check is the policy file version should be same as the JRE version at your environment. (The Android will not have this issue)

Finally,  you may ask common questions as the followings：
  1. How to encrypt a stream file with large file size (EX:Video or Audio)？
  2. How to randomly access specific block data of the encrypted file ?
The possible solution is as below ：
  • The answer for the first one question is easy. Just study the Java Cipher API doc carefully, then you can invoke the update method of Cipher with multiple times , and invoke the doFinal method at the end.
  • The answer for the 2nd question is to change the cipher mode as the CTR cipher mode. In addition, you need to write a calculate IV method for the target block that you want.
You could refer to the following implementation:
(Note: This sample code does not testing well, please don't adopt it on production system.)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
class AES_CTR_PKCS5PADDING {
 private static final int BLOCK_SIZE = 16;
 
 public static void Encrypt(SecretKey secretKey, byte[] iv, File plainTextFile, File encryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CTR_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CTR_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(plainTextFile);
    OutputStream out = new FileOutputStream(encryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] cipherBytes = cipher.update(buf, 0 , readBytes);
    out.write(cipherBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] iv, File cipherTextFile, File decryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  
  if(!decryptedFile.exists()){
   decryptedFile.createNewFile(); //: Here, it may be fail if ...
  }
  
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(cipherTextFile);
    OutputStream out = new FileOutputStream(decryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] decryptedBytes = cipher.update(buf, 0 , readBytes);
    out.write(decryptedBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
  
 public static byte[] DecryptPartial(SecretKey secretKey, byte[] iv, File cipherTextFile, int blockIndex, int blockCount ) throws Exception{
  final int offset = blockIndex * BLOCK_SIZE;
  final int bufSize = blockCount * BLOCK_SIZE;

  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, calculateIVForBlock(new IvParameterSpec(iv), blockIndex ));

  byte[] decryptedBytes = new byte[bufSize];
  try (FileInputStream in = new FileInputStream(cipherTextFile)){
   byte inputBuf[] = new byte[bufSize];
   in.skip(offset);
   int readBytes = in.read(inputBuf);
   decryptedBytes = cipher.update(inputBuf, 0, readBytes);
  }
  return decryptedBytes;
 } 

 private static IvParameterSpec calculateIVForBlock(final IvParameterSpec iv,
         final long blockIndex) {  
     final BigInteger biginIV = new BigInteger(1, iv.getIV());
     final BigInteger blockIV = biginIV.add(BigInteger.valueOf(blockIndex));
     final byte[] blockIVBytes = blockIV.toByteArray();

     // Normalize the blockIVBytes as 16 bytes for IV
     if(blockIVBytes.length == BLOCK_SIZE){
      return new IvParameterSpec(blockIVBytes);
     }
     if(blockIVBytes.length > BLOCK_SIZE ){
      // For example: if the blockIVBytes length is 18, blockIVBytes is [0],[1],...[16],[17]
      // We have to remove [0],[1] , so we change the offset = 2
      int offset = blockIVBytes.length - BLOCK_SIZE;
      return new IvParameterSpec(blockIVBytes, offset, BLOCK_SIZE);
     }
     else{
      // For example: if the blockIVBytes length is 14, blockIVBytes is [0],[1],...[12],[13]
      // We have to insert 2 bytes at head
      final byte[] newBlockIV = new byte[BLOCK_SIZE]; //: default set to 0 for 16 bytes
      int offset = blockIVBytes.length - BLOCK_SIZE;
      System.arraycopy(blockIVBytes, 0, newBlockIV, offset, blockIVBytes.length);
      return new IvParameterSpec(newBlockIV);
     }
 }
 
 private static void createTestFile(String path) throws Exception{
  File test = new File(path);  
  try(FileOutputStream out = new FileOutputStream(test)){

   StringBuffer buf = new StringBuffer(16);

   int blockCount = 100000;
   for(int i = 0 ; i < blockCount ; i ++){
    buf.append(i);
    int size = buf.length();
    for(int j = 0; j < (14-size); j++ ){
     buf.append('#');
    }
    out.write(buf.toString().getBytes());
    out.write("\r\n".getBytes());
    buf.delete(0, 16);
   }   
  }  
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);
  
  {
   String originalFile = "~/PlainText.txt";
   String encryptedFile = "~/CipherText.enc"; 
   String deryptedFile = "~/Decrypted.txt";   

   AES_CTR_PKCS5PADDING.createTestFile(originalFile); //: Create Testing Data
   
   AES_CTR_PKCS5PADDING.Encrypt(secretKey, iv, new File(originalFile), new File(encryptedFile));
   AES_CTR_PKCS5PADDING.Decrypt(secretKey, iv, new File(encryptedFile), new File(deryptedFile));
   byte[] ret = AES_CTR_PKCS5PADDING.DecryptPartial(secretKey, iv, new File(encryptedFile), 100, 10);   
   System.out.println(new String(ret));
  }
 }

Final:

I don't talk about the GCM cipher mode here. In principle, you don't need the GCM if you don't need the authentication

Reference:

* https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_.28ECB.29
* https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html

張貼者： 於 沒有留言:
標籤： , , , , , , , , , , , ,

Java JCE - AES 的 Encryption & Decryption @2016-05-01

前言：

都已經2016年了，本來不打算寫這篇的，但偶然發現網路上有多個中文部落格，甚至是論壇上分享或是討論 Java 的 AES 的程式，都沒有討論一些問題，深怕一堆人看到這種範例程式就放到你開發的系統上。另外，為了簡化程式來說明，以下的程式並不考慮Exception處理方式。

內容：

先來看網路上常見的程式的寫法：
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
class AES_DEFAULT {  
 public static byte[] Encrypt(SecretKey secretKey, String msg) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); //: 等同 AES/ECB/PKCS5Padding
  cipher.init(Cipher.ENCRYPT_MODE, secretKey); 
  System.out.println("AES_DEFAULT IV:"+cipher.getIV());
  System.out.println("AES_DEFAULT Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes());  
  System.out.println("加密結果的Base64編碼：" + Base64.getEncoder().encodeToString(byteCipherText));
  return byteCipherText;  
 }  

 public static byte[] Decrypt(SecretKey secretKey, byte[] cipherText) throws Exception  
 {  
  Cipher cipher = Cipher.getInstance("AES"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey);  
  byte[] decryptedText = cipher.doFinal(cipherText);  
  String strDecryptedText = new String(decryptedText);
  System.out.println("解密結果：" + strDecryptedText);
  return decryptedText;  
 }  

 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(128,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_DEFAULT.Encrypt(secretKey, "I am PlainText!!");
  AES_DEFAULT.Decrypt(secretKey, cipher);  
 }
}
上面這種寫法，這個程式對於加解密的運作是正常的，但會有潛在的3個問題：


  1. 第 1 個問題在第 4 行 這個用法所使用的 Cipher mode 是 ECB，也就是比較不安全的方式。原因是 ECB 對於相同的資料加密後的結果會是一樣的，有興趣可以參考(Wiki上的那張企鵝圖)。 如果你的應用是每次加密時 secret key 都是重新產生的，而且需要被加密的資料每次都完全不同的時候，各自的資料內容本身也是異質性相當高，如果採用這種做法也沒有太大的問題，但還是不建議。因為以加密的應用來說，常見的對象就是檔案或是運用在傳輸加密。而這兩種方式，大部分都會有相同的資料。以檔案來說，相同類型的檔案你用binary編輯器打開檔案，你就可以觀察到檔案的前面都會有雷同的資料。通訊協定更是如此，例如：HTTP通訊協定。
  2. 第 2 個問題在第 25 行 secret key 長度的問題，不應該使用 128 bits 長度，強度太弱。
  3. 第 3 個問題在第 8 行的 msg.getBytes( )，不同的作業系統所使用的預設 charset 可能是不同的。這樣的做法，有可能會造成不同平台在加密時的行為不如你所預期，也就是解密後的內容可能跟你當初要加密的資料不同。


比較建議的寫法是採用下面這種：

Important warring: You should select the correct cipher mode,  for example: CCM or GCM mode. (Update 2022/03/04) The reason is the CBC mode is vulnerable to padding oracle attacks.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
class AES_CBC_PKCS5PADDING {
 
 public static byte[] Encrypt(SecretKey secretKey, byte[] iv, String msg) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CBC_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CBC_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte[] byteCipherText = cipher.doFinal(msg.getBytes("UTF-8"));
  System.out.println("加密結果的Base64編碼：" + Base64.getEncoder().encodeToString(byteCipherText));

  return byteCipherText;
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] cipherText, byte[] iv) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  byte[] decryptedText = cipher.doFinal(cipherText);
  String strDecryptedText = new String(decryptedText);
  System.out.println("解密結果：" + strDecryptedText);
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

  byte[] cipher = AES_CBC_PKCS5PADDING.Encrypt(secretKey, iv, "I am PlainText!!");
  AES_CBC_PKCS5PADDING.Decrypt(secretKey, cipher, iv);  
 }
}
在第 3 行明確指定採用 CBC 的 cipher mode，並且指定 Padding 方式，有興趣可參考這篇 Padding 運作方式。在第 8 行，直接指定用 msg.getBytes("UTF-8")，避免一些問題，如果需要加密的內容都是純ANSI，你可以指定用 ANSI 就可以。

此處，還是要強調 AES的加密 Secret Key 的長度建議至少要 256 bit 以上，用同一把 Secret Key 做加密時候，應該都要產生新的 IV 來加密。可以參考下面這種寫法來產生所需要的 AES Secret Key 和 IV。另外，IV 在解密時候，也要用當初加密使用的相同ＩＶ才可以。

  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);

另外，由於 Oracle 官方預設標準的Java執行環境沒法支援產生 AES 256 bits 長度的 secret key。所以，你會遇到下面這種錯訊息：

1
2
3
4
5
6
Exception in thread "main" java.security.InvalidKeyException: Illegal key size or default parameters
 at javax.crypto.Cipher.checkCryptoPerm(Cipher.java:1026)
 at javax.crypto.Cipher.implInit(Cipher.java:801)
 at javax.crypto.Cipher.chooseProvider(Cipher.java:864)
 at javax.crypto.Cipher.init(Cipher.java:1249)
 at javax.crypto.Cipher.init(Cipher.java:1186)

這表示你的執行環境需要安裝 Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files。這裡要特別注意，你務必要安裝跟執行環境JVM相同版本的JCE Policy，否則可能會遇到一些怪異的現象。(Android執行環境不在此限)

最後，常問遇到的問題就是：

  1. 如果檔案很大的Stream類型的檔案(如:Video or Audio)要如何做到加密？
  2. 想要隨意位置讀取已經加密的內容要怎麼做？

建議的做法：


  • 第 1 個問題，很簡單請看清楚 Cipher 的 API doc，採用多次呼叫 update 的方法，最後再呼叫 doFinal 方法即可。
  • 第 2 個問題，只要將 Cipher mode 改為 CTR 即可，另外如果是要隨機存取某個 Block 的資料，必須要自己重新計算那個 Block 開始的 IV。
請參考類似下面的寫法：
(Note: This sample code does not testing well, please don't adopt it on production system.)

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
class AES_CTR_PKCS5PADDING {
 private static final int BLOCK_SIZE = 16;
 
 public static void Encrypt(SecretKey secretKey, byte[] iv, File plainTextFile, File encryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.ENCRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  System.out.println("AES_CTR_PKCS5PADDING IV:"+cipher.getIV());
  System.out.println("AES_CTR_PKCS5PADDING Algoritm:"+cipher.getAlgorithm());
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(plainTextFile);
    OutputStream out = new FileOutputStream(encryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] cipherBytes = cipher.update(buf, 0 , readBytes);
    out.write(cipherBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
 
 public static void Decrypt(SecretKey secretKey, byte[] iv, File cipherTextFile, File decryptedFile) throws Exception{
  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, new IvParameterSpec(iv));    
  
  if(!decryptedFile.exists()){
   decryptedFile.createNewFile(); //: Here, it may be fail if ...
  }
  
  byte buf[] = new byte[4096];
  try (InputStream in = new FileInputStream(cipherTextFile);
    OutputStream out = new FileOutputStream(decryptedFile);){
   int readBytes = in.read(buf);   
   while(readBytes > 0){
    byte[] decryptedBytes = cipher.update(buf, 0 , readBytes);
    out.write(decryptedBytes);
    readBytes = in.read(buf);
   }
   cipher.doFinal();
  }
 }
  
 public static byte[] DecryptPartial(SecretKey secretKey, byte[] iv, File cipherTextFile, int blockIndex, int blockCount ) throws Exception{
  final int offset = blockIndex * BLOCK_SIZE;
  final int bufSize = blockCount * BLOCK_SIZE;

  Cipher cipher = Cipher.getInstance("AES/CTR/PKCS5PADDING"); 
  cipher.init(Cipher.DECRYPT_MODE, secretKey, calculateIVForBlock(new IvParameterSpec(iv), blockIndex ));

  byte[] decryptedBytes = new byte[bufSize];
  try (FileInputStream in = new FileInputStream(cipherTextFile)){
   byte inputBuf[] = new byte[bufSize];
   in.skip(offset);
   int readBytes = in.read(inputBuf);
   decryptedBytes = cipher.update(inputBuf, 0, readBytes);
  }
  return decryptedBytes;
 } 

 private static IvParameterSpec calculateIVForBlock(final IvParameterSpec iv,
         final long blockIndex) {  
     final BigInteger biginIV = new BigInteger(1, iv.getIV());
     final BigInteger blockIV = biginIV.add(BigInteger.valueOf(blockIndex));
     final byte[] blockIVBytes = blockIV.toByteArray();

     // Normalize the blockIVBytes as 16 bytes for IV
     if(blockIVBytes.length == BLOCK_SIZE){
      return new IvParameterSpec(blockIVBytes);
     }
     if(blockIVBytes.length > BLOCK_SIZE ){
      // For example: if the blockIVBytes length is 18, blockIVBytes is [0],[1],...[16],[17]
      // We have to remove [0],[1] , so we change the offset = 2
      int offset = blockIVBytes.length - BLOCK_SIZE;
      return new IvParameterSpec(blockIVBytes, offset, BLOCK_SIZE);
     }
     else{
      // For example: if the blockIVBytes length is 14, blockIVBytes is [0],[1],...[12],[13]
      // We have to insert 2 bytes at head
      final byte[] newBlockIV = new byte[BLOCK_SIZE]; //: default set to 0 for 16 bytes
      int offset = blockIVBytes.length - BLOCK_SIZE;
      System.arraycopy(blockIVBytes, 0, newBlockIV, offset, blockIVBytes.length);
      return new IvParameterSpec(newBlockIV);
     }
 }
 
 private static void createTestFile(String path) throws Exception{
  File test = new File(path);  
  try(FileOutputStream out = new FileOutputStream(test)){

   StringBuffer buf = new StringBuffer(16);

   int blockCount = 100000;
   for(int i = 0 ; i < blockCount ; i ++){
    buf.append(i);
    int size = buf.length();
    for(int j = 0; j < (14-size); j++ ){
     buf.append('#');
    }
    out.write(buf.toString().getBytes());
    out.write("\r\n".getBytes());
    buf.delete(0, 16);
   }   
  }  
 }
 
 public static void main(String args[]) throws Exception{
  KeyGenerator keyGen = KeyGenerator.getInstance("AES");
  keyGen.init(256,new SecureRandom( ) );
  SecretKey secretKey = keyGen.generateKey();
  byte[] iv = new byte[16]; 
  SecureRandom prng = new SecureRandom();
  prng.nextBytes(iv);
  
  {
   String originalFile = "~/PlainText.txt";
   String encryptedFile = "~/CipherText.enc"; 
   String deryptedFile = "~/Decrypted.txt";   

   AES_CTR_PKCS5PADDING.createTestFile(originalFile); //: Create Testing Data
   
   AES_CTR_PKCS5PADDING.Encrypt(secretKey, iv, new File(originalFile), new File(encryptedFile));
   AES_CTR_PKCS5PADDING.Decrypt(secretKey, iv, new File(encryptedFile), new File(deryptedFile));
   byte[] ret = AES_CTR_PKCS5PADDING.DecryptPartial(secretKey, iv, new File(encryptedFile), 100, 10);   
   System.out.println(new String(ret));
  }
 }



最後：

這邊我沒有提到另外一種 GCM 的 Cipher Mode，原則上，如果你沒有 Authentication 的需要時候，就不需要用到 GCM。

Reference:

* https://en.wikipedia.org/wiki/Block_cipher_mode_of_operation#Electronic_Codebook_.28ECB.29
* https://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html

張貼者： 於 3 則留言:
標籤： , , , , , , , , , , , , , , ,

2013年8月18日 星期日

Python M2Crypto - RSA 的 Encrypt, Decrypt, Sign and Verify

        這裡要介紹一些RSA基本知識，以及如何產生一把 RSA 的非對稱式(asymmetric) 公開金鑰 (public key)和 私有金鑰(private key)。並且用一個簡單的範例程式來解釋如何用RSA來進行加密解密和簽章驗證的應用。

RSA的基本知識和重要名詞

        RSA是一種asymmetric演算法，它是透過一組public key和private key來進行Encrypt/Decrypt或是Sign/Verify。它的優點是：資料交換過程當中，雙方只需要拿到對方的public key即可加密資料，然後將加密後的密文(cipher)傳送給對方。只有正確的接收者才會有private key能夠解出密文(cipher)。缺點是，加密或是解密的效能較差不適合用於大量的資料加密。RSA的private key的建議長度最少是1024 bits以上，才具有基本的安全性, private key長度越長所需要的加解密運算成本越高,相對也會更加安全。

Key Length

        RSA 的Key Length 決定了RSA被破解的強度，長度越長基本上越難以被破解。但是，不是永遠不能破解，只是目前還沒有提出有效的破解方法。

Public Exponent

        公開指數是為了滿足RSA演算法所需要的一個整數，而且必須是質數Exponent ，然而很多密碼學的函式庫所內建的RSA Public Exponent都是65537的質數。有個討論在探討RSA Public Exponent 選擇 3 是否不夠安全的問題，有興趣的可以參考這網頁。無論如何，基本上選擇 65537的質數應該是一個較為安全且建議的作法。雖然選擇過大的質數會影響解密和驗證的效率，但是除非是要應用在運算能力非常弱的環境上，否則選擇小的質數當作Public Exponent應該不建議的，特別是Padding Mode很差的情形。但是，若選用合適的Padding Mode時，即使選擇3當作 RSA Public Exponent對於安全性並沒有太大的不同。

Padding Mode

        RSA基本上必須透過隨機的Padding方式，以確保即使每次都加密相同明文(Plan-Text)時候，不會產生完全相同的密文(Cipher-Text)。而OpenSSL支援 4 種 Padding Modes，以下截取自OpenSSL官方文件。而M2Crypto目前只支援 RSA_PKCS1_PADDING 和 RSA_PKCS1_OAEP_PADDING，根據官方文件上的建議就是使用 RSA_PKCS1_OAEP_PADDING。
• RSA_PKCS1_PADDING
• RSA_PKCS1_OAEP_PADDING
• RSA_SSLV23_PADDING
• RSA_NO_PADDING

RSA Encrypt / Decrypt 的基本範例程式解說

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
import M2Crypto
import M2Crypto.BN as BN

def generate_keypair_as_pem(key_len, exponent):
    def empty_callback():
        pass

    rsa = M2Crypto.RSA.gen_key(key_len, exponent, empty_callback)
    # Get RSA Public Key in PEM format
    buf = M2Crypto.BIO.MemoryBuffer('')
    rsa.save_pub_key_bio(buf)
    public_key = buf.getvalue()

    # Get Private Key in PEM format
    buf = M2Crypto.BIO.MemoryBuffer('')
    rsa.save_key_bio(buf, None)
    private_key = buf.getvalue() # RSA Private Key
    
    return (public_key, private_key)

if __name__ == '__main__':
    keylen = 1024         # 1024 bits
    exponent = 65537  
    padding = M2Crypto.RSA.pkcs1_oeap_padding
    
    # Generate RSA key-pair in PEM files for public key and private key 
    public_key, private_key = generate_keypair_as_pem(keylen, exponent)
    message = 'This is a plain text data'
        
    # Use public key to encrypt 'message'
    buf = M2Crypto.BIO.MemoryBuffer('')
    buf.write(public_key)
    rsa1 = M2Crypto.RSA.load_pub_key_bio(buf)
    cipher_message = rsa1.public_encrypt(message, padding)

    # Use private key to decrypt 'cipher_message'
    rsa2 = M2Crypto.RSA.load_key_string(private_key)
    plaintext_message = rsa2.private_decrypt(cipher_message, padding)

  1. 使用RSA演算法之前，必須產生 RSA 金鑰(Public Key and Private Key Pair)
  2. 第 22 和 23 行指定 key長度為 1024 bits，選用的public exponent 為 65537 
  3. 產生 RSA Key-Pair 在第 8 行，透過 M2Crypto.RSA.gen_key 函數並指定 key 長度和 public exponent。第 3 個參數基本上只要給 empty_callback 即可，若是沒有給時，你呼叫這個函數會在 standard output 中出現類似....++的符號。它只是用來表示初始化  RSA key-pair 的進度狀態。此時我們可以取得 RSA 的 instance，存在 rsa 變數中。
  4. 分別取得 RSA public key 和 private key。M2Crypto 提供的方法，會將 Public key 和Private key 轉成 PEM 格式。
  5. 第 10 ～12 行是取得 RSA public key 的方式，第 15 ～17 行是取得RSA private key的方式。差別在於 rsa.save_key_bio(buf, None) 的第 2 個參數設定為 None的原因，是希望直接取得真正的 RSA private key 而不要再經由 aes_128_cbc的方式來保護。否則執行到這段程式碼時，系統會在 console 要求使用者輸入 passphrase 。細節請參M2Crypto官方文件。 但是，一般來說如果是要存在系統中某個地方時候，是會透過另一種加密演算法來保護這把 RSA private key。
  6. 第 31 ～33 行是載入 RSA public key 取得 RSA的 instance 存在變數 rsa1 中，第 34 行呼叫  rsa1.public_encrypt(message, padding)來加密明文(Plain Text)，其中指定 padding mode 為 OEAP Padding 。
  7. 第 37 ～38 透過 RSA private key 解開密文(Cipher Text)，也必須指定相同的 padding mode 。


RSA Sign and Verify Signature 的基本範例程式解說

以下這個範例，是 sender 以安全的方式傳送一個 message 給 receiver 的 RSA 常見的應用。為了方便解釋，以下我們稱 A 為 sender, Ｂ為 receiver 。它大致上有 7 個步驟如下：

  1. Ａ與Ｂ各自產生一個 RSA的 key-pair (private and public key)
  2. Ａ與Ｂ交換自己的 public key。
  3. Ａ使用Ｂ的 public key來加密訊息產生 Cipher message。
  4. Ａ再使用自己的 private key 來為 Cipher message 產生 Signature。
  5. 然後 Ａ 把 Signature 和 Cipher message 傳送給 B。
  6. B 必須用 Ａ的 public key 來驗證所收到的 Signature 是否正確。
  7. B 再用自己的 private key 來解開 Cipher Message，即可得到Ａ傳送的訊息內容。 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
import M2Crypto
import M2Crypto.BN as BN

def generate_keypair_as_pem(key_len, exponent):
    def empty_callback():
        pass

    rsa = M2Crypto.RSA.gen_key(key_len, exponent, empty_callback)
    # Get RSA Public Key in PEM format
    buf = M2Crypto.BIO.MemoryBuffer('')
    rsa.save_pub_key_bio(buf)
    public_key = buf.getvalue()

    # Get Private Key in PEM format
    buf = M2Crypto.BIO.MemoryBuffer('')
    rsa.save_key_bio(buf, None)
    private_key = buf.getvalue() # RSA Private Key
    
    return (public_key, private_key)

def get_data_digest(data):
    msg_digest = M2Crypto.EVP.MessageDigest('sha256')
    msg_digest.update (data)
    digest =  msg_digest.digest()
    return digest

def generate_secure_msg(A_private_key, B_public_key, message):
    padding = M2Crypto.RSA.pkcs1_oaep_padding
    buf = M2Crypto.BIO.MemoryBuffer('')
    buf.write(B_public_key)
    rsa1 = M2Crypto.RSA.load_pub_key_bio(buf)
    cipher_message = rsa1.public_encrypt(message, padding)
    # Use A's private key to sign the 'cipher_message'
    digest1 = get_data_digest(cipher_message)
    rsa2 = M2Crypto.RSA.load_key_string(A_private_key)
    signature = rsa2.sign(digest1, 'sha256')
    return cipher_message, signature

def read_secure_msg(A_public_key, B_private_key, cipher_message, signature):
    try:
        # Use A's public key to verify 'signature'
        buf = M2Crypto.BIO.MemoryBuffer('')
        buf.write(A_public_key)
        rsa3 = M2Crypto.RSA.load_pub_key_bio(buf)                
        # Verify
        digest2 = get_data_digest(cipher_message)
        rsa3.verify(digest2, signature, 'sha256')
        # Use B's private key to decrypt 'cipher_message'
        rsa4 = M2Crypto.RSA.load_key_string(B_private_key)        
        padding = M2Crypto.RSA.pkcs1_oaep_padding
        plaintext_message = rsa4.private_decrypt(cipher_message, padding)
        return plaintext_message
    except Exception as err:        
        print 'Verify Fail:%r'% err
        raise 

if __name__ == '__main__':
    keylen = 1024         # 1024 bits
    exponent = 65537
    padding = M2Crypto.RSA.pkcs1_oaep_padding
    
    # Generate RSA key-pair in PEM files for public key and private key 
    A_pub_key, A_priv_key = generate_keypair_as_pem(keylen, exponent)
    
    # Generate RSA key-pair in PEM files for public key and private key 
    B_pub_key, B_priv_key = generate_keypair_as_pem(keylen, exponent)

    # A is sender, B is receiver
    msg = 'A want to send this message to B'

    # Sender's behavior
    cipher_msg, signature = generate_secure_msg(A_priv_key, B_pub_key, msg)

    # Receiver's behavior
    plain_text = read_secure_msg(A_pub_key, B_priv_key, cipher_msg, signature)

附註：
        實務上 RSA 通常都會搭配 AES 做 secure key 的保護，純粹是根據你所要保護的資料大小以及應用所著重的特點而有不同，此外 RSA 並不適合用來加密大量資料。

Reference:

張貼者： 於 沒有留言:
標籤： , , , , , ,

2013年8月17日 星期六

Python M2Crypto 常見的應用範例程式

Introduction

        M2Crypto 是一個讓Python開發人員能夠在透過API呼叫的方式來使用OpenSSL的套件。簡單來說，使用M2Crypto重新包裝過後的API，讓開發人員寫Python程式時，不再需要使用command line方式呼叫openssl來處理Encrypt/Decrypt/Sign/Verify...等等的功能。然而，M2Crypto對於一般不熟悉密碼學的開發人員而言，使用上會有一點點的進入門檻，特別是對於OpenSSL本身也不熟的開發人員，當然如果你已經是一個資深的開發人員也瞭解密碼學，則應該不會有這方面的困擾。

        M2Crypto在國外的論壇上或是網站上，都可找到一些針對不同應用的範例程式。我本身覺得這些應用在專案中是很常見，所以本文章會稍作整理，也當作自己工作上的學習紀錄。以下，我將會將整理一些之前專案中需要有用到M2Crypto的應用，但是不會深入探討各種密碼學的演算法細節，若你需要詳細的演算法請到 wikipedia 研讀。

本文章會談到以下這些應用：
  1. AES 的 Encrypt 與 Decrypt
  2. RSA 的 Encrypt, Decrypt, Sign and Verify
  3. 產生 Certificate Sign Request 與 Sign Certificate  
  4. SMIME Encrypt/Decrypt/Sign/Verify
  5. 完整的 RSA 與 AES 搭配的範例
針對以上這些M2Crypt的基本應用，每篇文章都會有基本知識的介紹以及範例程式。


張貼者： 於 沒有留言:
標籤： , , , , , , , , , , , , , ,
訂閱： 文章 (Atom)