Stop Compromising Your TLS / SSL Private Key’s Security!

Executive Summary

TLS is a critical component ensuring the security of data coming into and out of the enterprise. This security can be instantly negated by mishandling of the core component, the private key. The private key must remain an absolute secret with a strictly minimal access control policy surrounding it. In enterprises that lack a comprehensive security policy, these keys can be leaked by resources who are unfamiliar with the technology. Once leaked, the key should be treated as compromised and a replacement key and certificate ordered and installed immediately.

TLS 101

I found a fantastic plain-english explanation of why TLS is needed and how it works.  This link provides an example with Bob and Alice attempting to exchange private information where a would-be attacker Mallory is present and trying to actively intercept and disrupt communication. In the example, the term ‘public key’ is interchangeable with ‘certificate’ as a certificate is simply a special type of public key created, signed and maintained by a certificate authority like VeriSign.

After reading this article, you should have a solid understanding of how TLS solves security challenges and provides a secure channel along with identity assertion of both parties. This is only possible due to the privacy of the private key. If Mallory ever gets the private key then all the security and privacy disappears instantly.

How Private Keys Leak

Private keys leak when the enterprise strategy for secure key generation, storage and management is weak. Instead of a detailed documented process, the onus is placed solely on the individual product experts to ensure they are following best practices. The assumption is that the subject matter expert (SME) is deeply skilled not only on the product but also every peripheral technology used. In my experience, this is isn’t true. SME experiences vary greatly with external technologies. They may have never performed a TLS configuration task so they may not understand the implications of their request for keys to be sent via email or copying a key to a USB drive. The lack of an enterprise process for TLS, a technology that touches every single technology, greatly increases key leaks as each team does it their own way with little oversight.

There are three common points in the private key life-cycle when they can leak:

• At the point of generation
• At the point of installation
• At the point of systems integration

Key Generation

The point of generation is when the private key is created. This will most likely be on a laptop, a shared network drive, a USB stick, or directly on the server that uses the key. The key may be copied many times between all of those locations. The private key is also needed to generate a Certificate Signing Request (CSR). A CSR is created when a certificate is requested from a certificate authority. This implies that whomever is performing these tasks must be completely trusted. They can use this opportunity to make uncontrolled copies at will. If the process they are following is sloppy and leaves multiple copies of the key lying around, then each of these copies is a leak.

At this point, the key is unused. In the next step, the key will be installed into the appropriate system to secure communications.

Key Installation

The point of installation is when the private key needs to be installed onto the server that will create the secured TLS connection. Sometimes, the person who performed the generation task is not the person who installs it. This leaves a gap on how the key is transferred between the two parties and this becomes another opportunity for leaks. Another chance for a leak is when the key is taken from the generation location placed onto the actual server. The location that a file can be uploaded to a server is usually not the final filesystem location for the server to reference it. It needs to be copied. If all these copies are not immediately cleaned up, then the key can be leaked.

This really should be the final opportunity to leak the private key, but in my experience there’s a third chance for leaks, integration.

TLS Integration

This point of integration is when two systems need to be configured in order to communicate over a TLS connection. This is where poor assumptions are made as to the requirements for successful TLS configuration. Too many times when configuring a service consumer such as DataPower, I was sent both the public and private keys of the service provider. This person believes that all parties need the public and private keys. This is wrong. Remember back to the example above. A remote party only needs a public key (or certificate) in order to establish secure communications. By sending the private key the secret has been compromised.

I’ve found that this type of leak can be attributable to a combination of the amount of stress surrounding a project and the experience of the assigned resource. In a frantic attempt to keep projects moving, people who are unfamiliar with TLS tend default to sending everything in the hope that things work. This leads to emailing the entire key store used by the server. This key store always contains both the public and private key as it’s needed by the server to encrypt and decrypt messages. To make matters worse, the leak occurs over internet email which is as secure as broadcasting your private key during the Super Bowl.

It is true that key stores are password protected, but relying on this password is infinitely less secure than never giving it out in the first place. The password is almost always shared between every store in the enterprise (because who can ever remember all those passwords!) and rarely more than 6 characters. I could probably open 50% of leaked stores with ‘abc123’ or ‘password’ as the password.

The Cost of a Leak

This should be the longest section of the article, where I delve into the immense value of the confidentiality of information that is flowing between your endpoints. I should talk about what the business costs will be when an attacker steals customer data, or when an attacker impersonates a trusted third-party and invokes your services at will. But I won’t. The cost of a leaks can be summarized as this: You should now act as if you have no security between your endpoints. What’s the cost of that? The answer is always ‘high’. You need to act appropriately to resolve this situation. This means solving your immediate problems and also addressing the root cause of the leak: a lack of standardized process and oversight.

Unacceptable Leak Excuses

When I point out that a private key has been leaked and requires regeneration, I usually get one of the following excuses as to why it’s not a problem:

• Everyone already has production access anyway: Well that’s an even bigger problem than leaking this single key. Access to the private key should be locked down to only a very small subset of a team that needs it. If everyone has access, how is private key confidentiality guaranteed?
• No one noticed: Maybe, but it’s sitting in everyone’s inbox for up to 7+ years. That’s a decade of risk of attitudes and motivations to change. It’s entirely possible that one of your colleagues is quietly collecting private data to use for nefarious purposes. They won’t point out that the leak. You can’t assume that because no one identified the problem publicly, it wasn’t privately noticed and catalogued.
• Fix it later: This might be the original lie in I.T. from which all other lies were born. Proposing it do it later is the political way of proposing to never fix it. Think about it. There is a security breach of the system, but it’s not a priority to immediately resolve? The only way that fixing it later is a valid answer is if your system has no value. If it has no value why are you even working on it?
• They wouldn’t even know what to do with it: Every day, tools are being released that make compromising security easier for the less technically inclined. You can download an app, tap a button and within a few minutes and you are given a WEP wireless password and can use it to obtain free WiFi.
• Its password protected: If I have the physical key file, I can run brute force dictionary attacks against it. I can go to a cloud provider and spin up thousands of machine instances to crack that password for the price of a cheeseburger. Millions upon millions of cracking attempts every second. Do you think your password will hold up to that test? Was the password created with this scenario in mind?

Avoiding Leaks

The only way to ensure that your private keys are not leaking is to invest in an enterprise process with resources who work with keys and certificates for all technologies. They can generate, manage and install keys securely as needed for the technology. Resources outside this group should never have enough access to have the opportunity to leak private keys.

Ideally, this process should be highly automated such that keys can be generated, backed up and installed into file systems without user interaction. This would eliminate most of the common causes of leaks. When the keys reside on a server, they should be tightly access controlled such that only the automated user accounts have access and never interactive user accounts. The book of record for keys should reside in an extremely secure location and mimic the security of nuclear launch codes. The security of your business and your customer information depend on it.

If you can’t go as far as a security team and lack automated scripts then a quick tip is to know what you attach in your emails when it comes to TLS. If you aren’t sure exactly what the contents are of a .zip, .jks, .cer or .der are, don’t press send. Reach out to someone who can clarify the situation. If you are asked to send a private key, challenge them as to the reason. When it comes to security, it must be acceptable to be meticulous as the ramifications of being wrong are dire and costly.

Conclusion

Private keys need to be managed with the same care you would give someone who asked you to hold a Fabergé egg while they tied their shoes. Too often keys are not treated with respect and leaks occur due to a mismanaged (or non-existent) security process where too many people are exposed to a critical security component of the enterprise. Security is everyone’s responsibility. When a leak occurs, it needs to be confessed and addressed and plans put in place to ensure it can’t happen again.