Machine Identities and Secrets

Machine Identity - Who’s Who in the Cloud?

Tom McNamara

July 8, 2023

The 1942 Loony Tunes film “ Who's who in the zoo?“ was a short comic sketch of various cartoon animal characters. Many characters were easy for audiences to identify with. It’s now become a colloquial metaphor for identity and trust in team relationships. In 1942 no one could’ve imagined the importance of identity in the 21st century. The global economy has transformed from brick and mortar businesses with fixed assets and on-premises data centers to a digital economy with virtual properties, highly variable assets, and cloud data centers. As the digital transformation continues, the cloud is becoming the ubiquitous global “zoo.”  And knowing who’s who is just as important today as it was in 1942. But in the cloud the identities are not animal cartoons or characters, they are “machines” - a label that describes a broad range of hardware and software identities. These machines perform critical operations, and they include virtual machines, containers, applications, services, and microservices. When operating, they’re known as “workloads'' because they produce and use data, perform processing, and execute business logic within the cloud. 

In this article we’ll answer some questions about who these machines are. Why does their identity matter? And most importantly, how are businesses to trust machines they no longer own and control (the cloud)?

Certificates (Keys) Establish Machine Identity

Just like identity is important for humans, it is important to have trust in, and know the identity of, machines. The proof of identity for most machines and workloads is cryptographic key material, often referred to as asymmetric encryption or public key cryptography. It involves assignment of 2 keys to a machine: one private and one public. At a high level, the private key material is used to sign a “certificate” (the public key) that vouches for the identity of the machine. Asymmetric encryption is also used to “sign” code (applications) to certify its authenticity and confirm it has not been tampered with. Certificates are issued by “authorities” (CAs) who vouch for the identity of a machine or application. The most visible use of certificates occurs with web domains and the HTTPS protocol. Public certificate authorities vet web domains to validate their identity and to provide the key material for their certificate to the web host for the domain. There are three types of domain validation available: Domain Validation (DV), Organization Validation (OV), and Extended Validation (EV), and validation can take from a few days to a week.

One characteristic of certificates is their expiration date. The trend has been to progressively reduce the lifetime of domain certificates. Recently the governing body for X.509 certificates (X.509 is the standard for certificates) shortened the allowable lifetime of domain certificates from 24 months to 13 months. The decrease is due to the need to improve website security. A shorter lifetime reduces the period in which compromised or bogus certificates can be exploited to mount phishing and malware attacks. This domain lifetime constraint isn’t an issue for machine certificates, but it does suggest a general vulnerability of certificates.

The certificates that establish the identity of machines in the cloud have lifetimes that are much shorter than their domain certificate counterparts. They are issued and revoked very easily and very quickly. This makes them easy to produce, and there are now many CAs for cloud machines. It’s a little like qualifying as a wedding official today, you don’t have to do much to be authorized by a US State to officiate a marriage ceremony and sign a marriage certificate. 

Certificate Authorities

Third party CAs, such as Venafi and HashiCorp have emerged to manage machine identities and issue ephemeral CAs for cloud workloads. Cloud data centers bring machines online and take them off-line rapidly and  frequently. It is impractical for them to depend on a weeklong vetting process for every machine identity that is dynamically created. So they did the next best thing. They create private Certificates for the machines that can be issued very quickly and require very little authoritative vetting for the purposes of running workloads. (HashiCorp Vault is a CA that provides certificates but they are completely independent of the root domain of the businesses that use their certificates). There are even free CAs that have issued over a billion certificates since 2015.

Most machines in the cloud don’t get public domain certificates. Instead, they are provided with private certificates from a CA that is trusted by the owner/holder of the initial or ‘root' CA. There may often be several layers of CAs that form a chain of trust. For example, a business might issue private certificates to their data centers and the data centers might issue other private certificates to machines in a data center so they can be easily and quickly used by network admins. In commercial clouds, a certificate management service often exists to issue certificates to the cloud’s machines on demand as they are needed and they may be authorized for short  lifetimes. 

Machine Identity Challenges

But there are security management challenges with certificates. As cryptographic key material, the certificates (the public key part of the asymmetric encryption) also have their corresponding private key that must be kept secret. As the number of machine certificates and applications grows, so do the number of private keys. Security teams are often challenged with tracking all of these keys (public and private) and their CAs and lifetimes. It's important to note that the chain of trust is with the CAs and not necessarily with the certificates or the machines themselves. 

The security of certificates has received a lot of attention in 2021. The SolarWinds SUNBURST discovered by FireEye in December 2020 (it was discovered then, but likely began long before December) revealed that attackers modified SolarWinds’ ORION source code to include a malicious backdoor, which was then compiled, signed, and delivered by SolarWinds to nearly 18,000 customers via their customary update and code-signing systems. The tampered software was signed by SolarWinds, making the code appear trustworthy to end user systems. 

With all of the machines running in the cloud and all of the various authorities that can produce  certificates and key material, can a machine’s certificate be trusted? It may look legitimate but still not represent a true or trustworthy identity. Security researchers report rapid growth in the trend for theft and misuse of keys and digital certificates. For machines that operate within an on-premises business data center, the certificates issued by an internal CA or CA service are as trustworthy as the systems themselves (The SUNBURST attack was on SolarWinds internal systems that used the certificate and not on the CA that issued it). And the same would be likely for the large public cloud vendors and their CAs. Within their clouds, their certificates should be trustworthy. Another dimension of trust is to consider the lifetime of the machine. For example if a Kubernetes container only runs for a few minutes then its ephemeral certificate may be trustworthy for the short time it runs.

But most of the security practices around certificates and asymmetric cryptography for machine identities were established at a time when security perimeters were effective and there was little worry about whether or not the machines in the data center had trustworthy identities. In today’s environment, we know this is no longer a good assumption. Machine identities based solely on certificates may not be sufficient to meet Zero Trust principles. Machine identity is an important credential to authentication, and zero trust requires authentication at every machine interaction.

Extending Identity for Zero Trust

To accommodate the change in the threat environment, we believe the longstanding PKI/certificate model needs extending. Machine identities in a zero trust environment should follow five principles:

  1. Rigorous identity vetting at first meeting (registration)
  2. Trial the trust - Zero Trust doesn’t assume. It tests.
  3. Extend the identity and trust at each machine session
  4. Get to know the identity (behavior analytics)
  5. Build the chain of trust at the machine level (not just at the CA level)

These principles are embedded in the architecture and protocols of our patent pending technology and produce a dynamic identity component that is more secure, increases the speed of DevOps and assures trustworthiness of cloud machine identities.