In this lab, we shall see how public/private key encryption system works. Public/Private key encryption is a method which is used when you have to share data between third parties in a secure fashion. This system requires every person to have a key pair (One which is private to them, and one which is a public key that they share with others). Let’s understand the scenario with an example.

Suppose, Mr. A wants to send a file to Mr. B. Then, Mr. A should encrypt his message with Mr. B’s public key so that only Mr. B can decrypt the message (with the help of private key corresponding to the public key). This method of encrypting a message with the public key and decrypting it with its private key is called asymmetric encryption.

For this lab, we need the following:

  • An instance of Kali Linux running on virtual box
  • OpenSSL – preinstalled in Kali Linux

Let’s start the exercise.

First, we need to create a key pair. A key pair contains both private and public keys. In this exercise, we shall see how to encrypt a message with our public key and then decrypting it with our private key.

1. Open the terminal on your Kali Linux VM and go to your $home directory and enter the following command to create the key pair.

openssl genrsa -out myprivatekey.pem 1024

[Note: This command creates an RSA key pair of 1024 bits with it’s name as myprivatekey.pem]

2. Next, we shall extract the public key from this keypair. This can be done with the following command.

openssl rsa -in myprivatekey.pem -out mypublickey.pem -outform PEM -pubout

[Note: This command takes the input file as myprivatekey.pem and gives mypublickey.pem as the output. We specify that the key pair we are using is RSA encrypted. Also, PEM is the file format we would want the output file to be generated in.]

3. This public key can be shared with anybody with whom we want to have a secure communication channel. We can share the public key generated in the above step through an email/cloud storage etc.

4. Let’s assume, Bob receives the public key and has a document which needs to be sent to us. He then uses the public key to encrypt his document. This can be done in the following way.

  • First, let’s create a text file on behalf of Bob.
  • Run the command: echo “My account password is: test@123” > secret.txt
  • The above command should create a file named secret.txt with the contents between the double quotes. Let’s check the file in the below screenshot.

  • Next, let’s encrypt the file on behalf of Bob. This can be achieved by running the following command.

    openssl rsautl -encrypt -inkey mypublickey.pem -pubin -in secret.txt -out message.enc

[Notes: The above command creates a encrypted file called message.enc. The command uses one of the RSA utility (rsautl) to encrypt the file by specifying the public key and the secret.txt file that needs to be encrypted into message.enc]

  • Let’s check the message.enc file which will be sent to us from Bob. As you can see in the screenshot below, the message is non-readable to anyone having the copy of this file. Unless it is decrypted using the private key.

5. Bob shares the encrypted file with us over an email/cloud storage channel. Let’s decrypt the file message.enc with our private key and see if we are indeed the intended recipients.

  • We can run the following command to decrypt the file. After running the command let’s verify the contents of the file.

    openssl rsautl -decrypt -inkey myprivatekey.pem -in message.enc -out


[Note: The above command uses the RSA utility (rsautl) to decrypt the file using the myprivatekey.pem and gives the output file as decryptedMessage.txt from the message.enc which was encrypted by Bob. Below is the screenshot of the command and the decrypted contents.]

Understanding performance of RSA and AES encryption:

For this part of the exercise we will have a 16-byte string, for example, let’s create a text file.

echo “secret: 98765432” > confidential.txt

Confidential.txt contains a 16-byte information. We will be doing the following actions on this file.

  • Encrypt the file with RSA key pair.
  • Encrypt the file with a 128 bit AES key

At each stage, we will compare the time taken for the action. Let’s do this with a simple shell script. The shell script looks as shown below in the screenshot. On running the shell script, our observations are noted as below:

Script using RSA keypair:

Script using AES key:

1. Time taken to encrypt a 16-byte file with RSA key pair = ~0.018 seconds

2. Time taken to encrypt a 16-byte file with AES key of 128 bit = ~0.013 seconds

This shows us that AES is faster than RSA. We can perform this benchmarking by running the following commands and observing the output.

openssl speed rsa

openssl speed aes

Understanding digital signatures:

In this section let’s focus on understanding the digital signatures.

Let’s use the same file we created earlier in this section. We shall see how a digital signature of a file changes based on the content of the file. Digital signatures are like unique for each entity. They provide authenticity to the data in hand. If the digital signatures of the data in hand don’t match then, we can safely assume that the data is tampered with. Let’s have a look at an example:

openssl dgst -sha256 confidential.txt > hash.txt

[Note: This command outputs the digital signature of confidential.txt into hash.txt as seen in the below screenshot. Dgst is called the digest command in openssl, which is used to obtain the digital signature of the file.]

Ethical Hacking Training – Resources (InfoSec)

Next, let’s change the content of the file – confidential.txt and check the hash value.

As you can see, on changing the content of the file, we have a new hash value that confirms to the new data in the file.

Key Takeaways:

With this, we have completed the exercise. Now we should be able to understand the following:

  • Creating key pair with OpenSSL
  • Extracting public key from the key pair
  • Using any public key to encrypt the message.
  • Using our private key to decrypt any message that was encrypted using our public key.
  • We have understood the performance of AES and RSA methods.
  • We have also understood how the has value varies based on the input.


  • RSA operation error: This error is caused when the message to be encrypted is of a very large size.

    ▪ To solve this either increase the key size in Step 1 from 1024 to something very large. (Not recommended)

    ▪ Alternatively, you could reduce the size of the input that is supposed to be encrypted. This approach is best suggested to encrypt messages like passwords, credit card data, etc.