Hacking

Cryptography fundamentals for hackers

What is cryptography?

Cryptography literally means “secret writing.” It’s the science of protecting sensitive data from being read (or modified) by unauthorized parties such as eavesdroppers, wiretappers and so on. 

While ancient encryption algorithms have largely been broken, modern cryptography is based on strong mathematical principles and has been subjected to intensive study by professional cryptographers. As a result, cryptography, when used correctly, can be an effective defense for data privacy. However, even a small mistake in design or implementation can give an ethical hacker the necessary opening to break open the encryption and read the encrypted data.

Earn two pentesting certifications at once!

Enroll in one boot camp to earn both your Certified Ethical Hacker (CEH) and CompTIA PenTest+ certifications — backed with an Exam Pass Guarantee.

View Pricing

Fundamentals of cryptography

Before diving into the details of how cryptography works, it’s important to understand the terminology. Technically, cryptography (creation of codes) is a subfield of cryptology (the study of codes), but both terms are often used interchangeably. The other subfield of cryptology, cryptanalysis, focuses on trying to break encryption algorithms.

Every good encryption algorithm has at least three main components: the plaintext, the ciphertext and the secret key.

The plaintext is the message that will be protected by the encryption algorithm. While this message is often the secret that a hacker is trying to steal, it may also be under the control of the hacker. Most encryption algorithms are protected against known plaintext attacks, where the attacker knows the plaintext and is trying to learn the secret key.

The ciphertext is the result of encrypting the plaintext with a secret key. A ciphertext should be essentially indistinguishable from a random number. This is accomplished by creating encryption algorithms with high confusion (making each bit of ciphertext dependent on several bits of the secret key) and diffusion (ensuring that flipping one bit of the plaintext flips an average of half of the bits of the ciphertext).

The secret key is a piece of data that should be known only to authorized users of the protected data. It is provided as an argument to the encryption algorithm (along with the plaintext) to create the ciphertext. 

According to Kerckhoff’s Principle, the secret key should be the only secret component of an encryption algorithm. Relying on security by obscurity (e.g., keeping the encryption algorithm or parts of it secret) is poor cryptography. Relying on security by obscurity is one of the mistakes that made the Enigma cipher breakable.

Types of cryptography

There are a few different ways in which cryptographic algorithms can be classified. Some of the major ones include asymmetric versus symmetric and block versus stream. Knowledge of these classifications can be useful for ethical hacking since different types of cryptography, if misused, can be vulnerable to attack.

Symmetric and asymmetric

One of the main ways to differentiate between different encryption algorithms is based on whether they use symmetric or asymmetric encryption keys. A symmetric encryption algorithm uses the same secret key for both encryption and decryption, while an asymmetric algorithm uses two related keys: a private key and a public key.

Symmetric encryption algorithms are generally better for bulk data encryption, but they have one main shortcoming: both sides need to have a copy of the same shared secret key. A good cryptographic implementation will use asymmetric cryptography to set up a channel to share a symmetric encryption key for bulk encryption. However, if a symmetric key is sent in cleartext or embedded in software, intercepting it can allow an ethical hacker to decrypt and read all of the protected data.

Asymmetric encryption uses a related public and private key. With asymmetric cryptography (also called public key cryptography), the main vulnerability of the protocol is to quantum computers, which can break it easily.

However, public key cryptography can also face the authentication problem. A user’s public key can be used to encrypt a message to them or verify a digital signature generated using their associated private key. This assumes that the public key in possession of and in use by the software is the correct one. If public key authenticity is not checked appropriately, substituting an ethical hacker’s public key may allow them to decrypt messages and generate digital signatures accepted by the software.

Block and stream

The other main distinction between types of cryptographic algorithms is whether they are block or stream ciphers. As the name suggests, a block cipher performs encryption and decryption on fixed-size blocks of data. A stream cipher, on the other hand, generates a stream of pseudorandom output bits that are exclusive-ored (XORed) with the bits of the plaintext to produce the ciphertext.

The main issue with block encryption algorithms is that they don’t have a built-in mechanism for encrypting data longer than the size of a single block. To fix this, block cipher modes of operation were developed to define a protocol for encryption of multi-block plaintexts. However, some of these leak information about the encrypted data, which can be exploited by an ethical hacker.

Stream ciphers are designed to encrypt plaintexts of arbitrary length, since they encrypt a bit at a time. However, this bit-level encryption can also make the system vulnerable to bit flip errors. If an application doesn’t test for the integrity of the encrypted data (through a MAC or checksum), a hacker can flip bits without detection. Also, the most well-known stream cipher (RC4) has vulnerabilities that can leak data about the secret key if not used properly.

Cryptography for ethical hacking

The widespread use of modern cryptography is good for privacy and security but also complicates the work of an ethical hacker. Data encryption can conceal other flaws in a system if the hacker can’t properly analyze data at rest or in transit to identify them.

During an ethical hacking exercise, it’s important to inspect the use of cryptography for any mistakes that may allow a hacker to break the encryption. It may also be a good idea to request access to unencrypted data as well, in order to enable analysis of the system for other flaws hidden by encryption.

Become a Certified Ethical Hacker, guaranteed!

Get training from anywhere to earn your Certified Ethical Hacker (CEH) Certification — backed with an Exam Pass Guarantee.

Learn More

Sources

1. Padding oracles and the decline of CBC-mode cipher suites, The Cloudflare Blog
2. Stream Cipher Attack: Bit-Flipping Attack, Total Web Security
3. CVE-2015-2808, CVE