The best way to keep a secret is never to write it down and tell it to no one. Failing that, encryption lets you protect secrets from prying eyes.

Encryption is conceptually straightforward: convert readable information called plaintext into another form, called ciphertext, that can't be read by anyone who doesn't have the key for converting the ciphertext back to its original form. Because the idea is so simple, it's no surprise that people have been using encryption since almost the beginning of written history: An Egyptian used non-standard hieroglyphs back in 1900 B.C., and a Mesopotamian tablet from 1500 B.C. concealed the valuable recipe for a pottery glaze.

Julius Caesar used encryption when he sent messages to his generals, and encryption has remained an important tool of the military ever since. The Germans in World War II relied on the Enigma machine to encrypt messages, and the Allies' breaking the code helped shorten the war.

Encryption has become, if anything, even more important in today's digitized world, where sensitive personally identifying information, valuable intellectual property and national security secrets are stored online, and the Internet means files can be accessed from anywhere. Militaries, businesses and ordinary consumers all depend on encryption to protect their information and identities from malware attacks and other unauthorized access. Cryptography provides four functions in modern computing:

  1. Authentication: Encryption enables the use of SSL certificates that allow a server to prove its identity.
  2. Non-repudiation: Encryption allows the creation of digital signatures that can prove a user took a specific action.
  3. Confidentiality: Encryption ensures that only users who have the correct key are able to decrypt and read encrypted data.
  4. Integrity: Encryption supports the creation of checksums that are used to prove the data has not been altered.

To be fully secure, data must be protected when it is at rest, stored in files; when it is in motion, while being transmitted across networks; and when it is in use, brought into RAM when needed for processing.

How Encryption Works

Encryption works by substituting part of the plaintext with new values, such as replacing every instance of the letter L with the letter Q; encryption algorithms can also apply other transformations like transposition. The inverse of the encryption procedure allows a receiver to decrypt the message. The sender and receiver must use the same algorithm for encryption and decryption.

Modern encryption is far more complex than the simple substitution method used in children's puzzles, but somewhat surprisingly, keeping the encryption algorithm secret isn't what's important for protecting a secret message. Instead, it's the secrecy of the key that provides the most protection. In fact, it's recommended that encryption should always use ciphers which have been published and verified for effectiveness. Full security for encrypted methods requires secure infrastructure to prevent access to the data before it's encrypted a key that is long enough to prevent brute force attempts at decryption (which systematically try all possible keys) from succeeding, and an algorithm that has been published and widely reviewed for flaws.

Symmetric Cryptography

Symmetric cryptography, also called private key cryptography, encrypts and decrypts data using a single shared secret key known to both the sender and the recipient. These algorithms are relatively fast, but rely on both parties having securely exchanged the encryption key, which is a challenging process on its own. Additionally, every possible pair of users needs to use a unique key, so managing keys can become complex.

The algorithms can work as a stream cipher, applying encryption to individual bits as the data is transmitted, or as a block cipher, applying encryption to chunks of data. Stream ciphers protect data from being read, but don't offer protection against tampering. A third party who is able to intercept the message can alter its values, as long as this third party knows the structure of the message; knowing the key isn't necessary.

Most modern symmetric cryptography methods apply block ciphers combined with modes that make decryption of each block of data dependent on successfully decrypting the previous block of data. As a result, tampering makes the message unreadable.

Asymmetric Cryptography

Asymmetric cryptography, also called public key cryptography, uses two different keys, one to encrypt the data and one to decrypt it. One of the keys, called the public key, is freely distributed. The other key, called the private key, is never shared. Through these two keys, a message can be securely sent to any party by encrypting it with the recipient's public key. Only the intended recipient has the corresponding private key needed to decrypt the message. The paired keys also allow a sender to prove he sent a message encrypted by his private key, as only his public key can decrypt it.

Because public keys are not unique to each pair of users, managing keys is easier than in symmetric cryptography. However, public key cryptography can be slower than symmetric cryptography algorithms.

Public key cryptography relies on public keys being distributed securely. The methods are vulnerable to "man in the middle" attacks where third parties intercept the exchange of public keys and substitute their own values, which enables them to read the transmitted messages.

Hashes

Hashes are another form of encryption. They take arbitrary data and transform it to a fixed-length hash. Calculating the hash is easy, but the original value is difficult to recover from the hash. While this isn't appropriate for passing messages that need to be decrypted and read by the recipient, this approach is typically used to securely store passwords. Rather than store and match the plaintext password, the system compares the hashed value of the text submitted from a login screen to the stored hashed value. Because the hashed value uniquely corresponds to the original value, this guarantees that the correct password was entered and protects the password from being stolen.

Steganography

Steganography is not encryption, but it's another method of concealing a message from unauthorized views. In steganography, a message is hidden within another file or message. A simple form of steganography would be a shopping list where the first letters of each item on the list spell out another item. In digital usage, steganography often uses media files because they are commonly very large. Minor changes to individual pixels don't degrade the image but can transmit another message. Because the files containing the hidden message appear innocent, they don't need to be kept secret; they can be publicly posted and only the intended recipient understands the hidden meaning. This means there is no need for the sender and recipient to ever communicate directly.

Honey Encryption

Honey encryption is a kind of encryption designed to make it difficult for attackers using brute force to know when they have correctly identified the key. With most encryption algorithms, all keys except the correct one result in gibberish being returned. When a non-gibberish answer is created, it's obvious that this is the correct key and that's the meaning of the message.

Honey encryption makes brute force attacks less effective by returning plausible-looking plaintext even when the correct key isn't used. While honey encryption has limitations -- it can't be used when the format of the plaintext isn't known -- it can be used to protect data that follows patterns, such as credit card or social security numbers.

Major Encryption Algorithms

DES (Data Encryption Standard), triple DES 3DES and AES (Advanced Encryption Standard) are all symmetric algorithms that have been widely used. The primary weakness in DES is its fairly short key, 56 bits, which allows today's computers to break the encryption using brute force attacks that simply try all possible keys. Triple DES applies the DES algorithm three times, using different keys each time. The effective key length is thus 168 bits, although some claim the security provided is equivalent only to 112 bits.

AES is currently the most secure encryption method and is the U.S. government standard. AES uses a stronger algorithm than DES and 3DES, and allows keys up to 256 bits. The algorithm applies 10 to 14 rounds of transformations to the plaintext, depending on the key length, with each round involving multiple substitution, transposition and mixing steps. Unlike DES, which was intended to be implemented in hardware, AES was also designed to run efficiently in software.

Blowfish and Twofish are widely used public domain symmetric encryption algorithms.

Diffie-Hellman and RSA (Rivest-Shamir-Adleman) are other common public key algorithms. Diffie-Hellman is used for generating and sharing private keys. RSA is widely used to encrypt data sent over the Internet.

Common algorithms for hashing include MD5 and SHA-1, though neither is considered secure today. SHA-256 is a stronger modern alternative.

Encryption in Business

Although encryption is often thought of as a military or national security need, it has wide applications in business and consumer life as well as in government. Companies risk fines and loss of business when their sensitive information is exposed, so businesses use encryption to protect their customers' information, to conform to regulatory requirements, and to protect their intellectual property against industrial espionage.

Protecting Data at Rest

While companies commonly rely on firewalls, intrusion protection software, antivirus software and data loss prevention software to protect against unintended exposure of information, encryption ensures that if those defenses fail, the attackers are unable to read the data they access. The laws that require notifying consumers if their information is stolen in a breach often don't apply to encrypted data, meaning companies protect their reputations as well as the information.

Companies can protect their data by encrypting the file systems where it is stored. Encryption is built into many modern operating systems and storage devices, enabling files to be encrypted automatically. Full disk encryption guarantees that all data is protected, not just specific files. This is especially important on laptops, tablets and other mobile devices, which are vulnerable to being lost or stolen. Files should also be encrypted on removable media, such as USB sticks and other external drives. It's also important that files be encrypted on backup media.

Protecting Data in Motion

Along with using encryption to protect stored data, businesses commonly use encryption to provide employees with secure remote access to corporate applications over a virtual private network (VPN). A VPN creates an encrypted tunnel that makes data transmission secure even over an unsecured Wi-Fi network.

Businesses also can use encryption to protect their information when it moves out of the corporate network via email or to cloud storage. Cloud providers often offer encryption services, but they also hold the keys, meaning they have access to the data. Companies that want to control their own keys can encrypt files before uploading to the cloud and use a cloud access security broker to provide other security controls.

Protecting Data in Use

The risks of exposing data in RAM to memory-scraping malware are the least addressed of the vulnerabilities. When data was only used on computers within the company's own data centers, the risk was seen as minimal, but the growth of cloud computing means that corporate data is often loaded on shared machines outside the company's control. New methods that enable searching on encrypted data are helping to reduce the risk.

Encryption as a Way of Doing Business

In addition to using encryption to protect their data, businesses often incorporate encryption into their operational processes. Digital signatures allow employees and business contacts to electronically sign documents, eliminating the need to pass around paper copies. Time stamping uses encryption to generate proof that an electronic document was delivered at a particular time.

Encryption in Government

In the United States, the National Security Agency is responsible for all government encryption systems, but use of encryption isn't limited to military and national security agencies. All government agencies hold large amounts of sensitive personal information, such as employment records and taxpayer data, in addition to the data related to the work of the agency. Agencies are required to comply with numerous guidelines and standards.

Within the United States, Federal Information Processing Standard (FIPS) 140-2, issued by the National Institute of Standards and Technology, defines the standards that apply to cryptographic modules used by the government. There are four levels, which increase in the security they provide and the controls they require. Level 1 requires only basic cryptography without additional physical security measures; the controls increase through Level 4, which provides a high probability of detecting attacks on the cryptographic module. The Federal Information Security Management Act (FISMA) provides a framework and compliance guidelines for data security in government agencies. The standards also apply to companies that act as contractors providing services to the government.

Other relevant federal standards include the Office of Management and Budget (OMB) M06-16 mandate, which requires sensitive information to be encrypted on mobile devices, the Federal Desktop Core Configuration (FDCC) standard, and the Director of Central Intelligence Directive (DCID) 6/3, which specifies how intelligence information should be protected in computer systems.

Although the term "military-grade encryption" is commonly used, the U.S. military uses standard AES encryption. The key length varies, with 128 bits allowed for Secret data and 256 bits required for Top Secret data. Additional protection for sensitive data is provided by the controls on the physical security of the premises where the data is stored.

Along with using encryption to secure data, the government has an interest in developing code-breaking methods to allow it to decrypt data belonging to other governments or to criminal enterprises.

Encryption in Everyday Consumer Life

Encryption is part of the routine of everyday consumers, even though they aren't always aware of it, and unlike businesses, consumers rarely explicitly manage encryption keys. Encryption underlies the functioning of devices commonly used by consumers, including mobile phones and ATMs.

Much like businesses, consumers can choose to encrypt the files on their home computers and mobile devices to safeguard files containing personal information, such as their tax returns and other documents holding social security numbers.

Consumers routinely use encryption when interacting with businesses over the Internet. Much of the data users send over the Internet while browsing is encrypted, with SSL (HTTPS) connections used to transmit login information and financial transactions. SSL uses public key asymmetric cryptography to securely transmit a secret key, and then uses the secret key for symmetric cryptography throughout the rest of the connection. The symmetric encryption algorithms are faster and enable more efficient communication.

Many mobile phones and tablets now enable encryption by default. Some consumers choose to download tools that support encrypted messaging and video calls; both participants in the conversation need to use the same tools for the encryption to work. Studies find that few users select applications based on security and privacy protections; most users simply choose the common tools used by the friends they need to communicate with.

Encryption's Dangers

While encryption provides necessary privacy and security functions to legitimate users, encryption is also exploited by malicious users and criminals. Criminals and terrorists often use encryption to protect their communications from be intercepted and read by law enforcement. Terrorists are known to communicate using steganography, as it enables cells to communicate without members' revealing their identities.

In addition to encrypting their own files, criminals are able to exploit companies' usage of secure channels such as SSL/TLS. Because those communication channels are encrypted, the antivirus software and intrusion prevention systems companies rely on to defend their networks aren't able to detect malware downloaded over these channels.

Encryption can also be used directly by criminals to attack other systems. One form of malware, ransomware, works by encrypting the victim's filesystem. The victim must pay a ransom to receive the key needed to decrypt the files. There is often a threatened deadline at which the key will be destroyed and the files become unrecoverable, and payment is often required in untraceable bitcoins.

Debates Over Encryption

Because encryption can be used for illegal purposes, governments around the world have attempted to restrict and control encryption in various ways. The modern world is too dependent on encryption to ban it, but countries like the United States have placed limitations on exporting strong encryption to other countries.

In the United States, accused criminals have been imprisoned for refusing to provide the passwords to decrypt their drives; defendants have claimed this violated their Fifth Amendment rights against self-incrimination. There have been proposals to build "backdoors" into commercial encryption, which would require companies to provide the government with the keys needed to decrypt data. These methods have not yet been implemented, though it is widely rumored that the U.S. National Security Agency weakens the publicly available encryption algorithms to ensure it is able to decrypt data.

Other countries including China and Russia have restrictions that require encryption users to register and provide their keys to the government.

Integrating Effective Encryption Practices Into Your Organization

Implementing an effective encryption strategy, whether in a company, government agency or at home, isn't as simple as installing and enabling an algorithm. It requires a strategy developed with the overall objective of achieving information security, rather a simple goal to encrypt certain kinds of data; it requires collaboration between business users, management and the technology team.

The first step is to decide what needs to be encrypted. This requires reviewing the data, to classify it according to its risk and importance; identifying the laws and regulations which apply to that kind of data; and discovering where the data is used.

Once the data is evaluated, you can select which data needs to be encrypted. You also need to select the data encryption method and implement a process to manage encryption keys securely. Both keys and certificates should be inventoried and access to them controlled. If you have a large number of keys, key management software makes managing and distributing keys simpler.

Along with implementing encryption, you also need to establish access controls which ensure that only authorized users are given the keys that enable them to access data. Policies prohibiting sharing access credentials need to be enforced. There should also be a policy and procedure to address deprovisioning hardware and retiring obsolete applications that makes sure data isn't accidentally exposed.

In addition, any organization should view encryption as the last line of defense rather than the first; encrypting data doesn't eliminate the need for firewalls, intrusion prevention systems, data loss prevention software, antivirus software and other security measures. It is the combination of encryption with other meaningful policies and comprehensive security controls that truly protects confidential data.