Identifying Abnormal AWS Sessions Originating from Temporary Credentials (1/2)

As organizations move their application workloads to the public cloud, it is more vital than ever for customers to protect the credentials used in their public cloud environments.

Compromised or mismanaged credentials are increasingly at the root of adverse cyber events. Research firm Gartner estimates that "by 2023, 75% of security failures will result from inadequate management of identities, access and privileges, up from 50% in 2020."

At Uptycs, we feel Identity Threat Detection and Response (ITDR) capability for the cloud will be crucial for your company to avoid falling into this statistic.

This is the first of a two-part series. In this first blog post, we will discuss short-term access keys (unofficially also known as temporary tokens or temporary credentials) in AWS, what they are, and how they can be compromised. In part two, we discuss how the Uptycs’ ITDR offering for the cloud will detect and provide a means to remediate the likely compromised temporary credentials.

Access Keys

What Are Access Keys?

When you use AWS programmatically, AWS verifies your identity in these programmatic calls by the access keys (credentials) you provide AWS. In other words, AWS will verify you are who you say you are by the credentials (access keys) you provide them.

There are two types of access keys: long-term and short-term (temporary). Let’s dive into what the difference is between these two credentials:

Long-term Access Key

Consists of an access key ID and a secret access key. Think of it as a username (access key ID) and password (secret access key). They are used for an IAM user or a root account user. As the name implies, long-term access keys are just that—long-term credentials and never expire.

They may be more convenient to use, but can pose a greater security risk as they give the threat actor a much better chance at gaining and maintaining access if they are stolen.

Short-term (temporary) Access Key

Consists of an access key ID, a secret access key, and a security token. The security token indicates when these temporary credentials expire. They normally last for anywhere between a few minutes to a few hours (typically 15 minutes to 12 hours).

After the temporary credentials expire, they will become invalid which means the same temporary credentials cannot be used again for any operation in AWS. Temporary credentials are not stored with the user but are generated dynamically.

For these reasons, temporary credentials are recommended to reduce the risk of unauthorized access and persistence within the environment should a threat actor compromise the temporary credentials. They are used for several reasons, some of which we’ll go into more detail later in this blog.

Identifying Long-term & Short-term Credentials in CloudTrail Logs

Long-term Credentials

Long-term credentials consist of two parts: Access key ID and secret access key. The access key ID will always start with the prefix ‘AKIA’, whereas the secret access key is a string of randomly generated alphanumeric characters (which will never appear in CloudTrail). In the following CloudTrail log, the access key ID is shown in the field “accessKeyId”.

Figure 1 - AKIA key shown in a CloudTrail log

Short-term Credentials

Short-term credentials consist of three parts: Access key ID, secret access key, and a security token. The access key ID for short-term credentials will always start with the prefix ‘ASIA’, whereas the secret access key and security token are individual strings of randomly generated alphanumeric characters. In the following CloudTrail log, the access key ID and session token are shown in the fields “accessKeyId” and “sessionToken”, respectively.

Figure 2 - ASIA key shown in a CloudTrail log

Why Are Temporary Credentials Needed?

There are many legitimate scenarios in AWS where temporary credentials are needed to perform an action.

For example, a user/group might be required to be authenticated with multi-factor authentication (MFA) before being allowed to make certain API operations (ex. StopInstance; StartInstance; GetObject; DeleteBucket, etc.) via CLI or SDK.

Another example is when users use a public web-based identity provider (Login with Amazon, Facebook, Google, etc.). A temporary set of credentials will be provided to federated users who are authenticated through a public identity provider for a temporary period of time.

Using long-term credentials in such scenarios is not a good security best practice, hence AWS provides temporary credentials to cater to such scenarios.

Temporary Credentials Generated?

Temporary credentials are generated dynamically by a service called AWS STS (Security Token Service). Depending on the use-case, there are five ways to acquire temporary credentials from the STS API:

1. GetSessionToken
2. AssumeRole
3. AssumeRoleWithWebIdentity
4. AssumeRoleWithSAML
5. GetFederationToken
   
   

GetSessionToken

The GetSessionToken API call returns a set of temporary credentials based on the permissions that an IAM user already has, or in the case of AWS account root user calling GetSessionToken, the set of temporary credentials returned will also have root user permissions.

This means that the temporary credentials generated via the GetSessionsToken API will never have more permissions than the IAM user making the call. Only IAM users or AWS account root users can call the GetSessionToken API using the long-term credentials (AKIA access key).

Typically, GetSessionToken is called if you want to use MFA to protect programmatic calls to specific AWS API operations (ex. StopInstance; StartInstance; GetObject; DeleteBucket, etc.). MFA-enabled IAM users would need to call GetSessionToken and submit an MFA code that is associated with their MFA device. Using the temporary credentials that are returned from the call, IAM users can then make programmatic calls to API operations that require MFA authentication.

It’s important to note no permissions are required for calling GetSessionToken and acquiring temporary credentials. Meaning, no IAM policy can prevent a user from calling the GetSessionToken API. Additionally, the temporary security credentials created by the GetSessionToken API call can only call two STS APIs: AssumeRole or GetCallerIdentity.

GetSessionToken Use Case:

If you have a requirement for versioning to be enabled on all your objects in one S3 bucket. You can provide an additional security layer to your S3 bucket by configuring MFA delete so that, for example, only Mary or Joe can change the versioning state of that S3 bucket.

Both Mary and Joe would have to submit an MFA code when calling GetSessionToken to acquire the temporary credentials, which would then be used to call the PutBucketVersioning API to change the versioning state of your S3 bucket.

AssumeRole

The AssumeRole API call returns a set of temporary credentials based on the permissions that IAM role has. A role is an identity you can create in your account that has specific permissions that determines what the entity assuming it can and cannot do in AWS.

The entity gives up their original permissions and takes on the permissions assigned to the role. When the users exit the role, their original permissions are restored. Instead of being uniquely associated with one person, a role is intended to be assumable (i.e. AssumeRole) by anyone who needs it and has the permissions to be able to assume it.

Typically, you use roles to delegate access to users, applications, or services that don't normally have access to your AWS resources. It’s important to note that an IAM user or IAM role with existing temporary security credentials can call the AssumeRole API. Additionally, the temporary security credentials created by the AssumeRole API call cannot call GetFederationToken or GetSessionToken APIs.

AssumeRole Use Case:

If you have an EC2 instance that needs to send objects (log files; documents, etc.) to a secure S3 bucket, you need to explicitly give that EC2 instance permission to be able to add these objects to the S3 bucket. Instead of configuring long-term credentials on the EC2 instance to do this, you would add an IAM role to that instance (via an Instance Profile).

The IAM role will have two policies: 1) a trust policy that specifies who can assume it (in our example it is EC2), and 2) a permission policy that specifies what the trusted entity can do once they’ve assumed it (in our example it is to add objects to S3). So whenever your EC2 instance needs to access S3, your EC2 instance will assume the role, receive temporary credentials which allow it to add objects to the S3 bucket.

Scenarios in Which Temporary Credentials Can Be Compromised & Exploited:

While there can be a variety of scenarios in which AWS temporary credentials can be compromised, we will focus on two common scenarios.

Scenario 1: Abusing Short-term Credentials Using Long-Term Credentials

Assume that a threat actor has gotten hold of long-term credentials (let’s call them AKIAX) of a user (this can happen through various means, such as accidentally posting them on GitHub or their laptop is compromised which had the long-term credentials saved locally). They then use these credentials to assume a role which generates short-term credentials (let’s call these credentials ASIA1).

Now, one might think that because temporary credentials are short-lived the threat actor can only have access to the AWS environment for as long as credential ASIA1 is valid for. But during this period, the threat actor then uses credentials ASIA1 to either assume the same role they first assumed, or assume a different role (also known as role chaining) which generates a new set of temporary credentials (let’s call these credentials ASIA2).

So even if temporary credentials ASIA1 expires, the threat actor still has access to the AWS environment through temporary credentials ASIA2. It’s important to note that role chaining limits your AWS CLI or AWS API role session to a maximum of one hour.

Using credentials ASIA2 they can keep assuming different roles until they find a role that allows them to achieve their desired outcome. In our diagram below, that outcome is creating the user Bob and new long-term credentials (AKIAY) using temporary credentials ASIA3.

Figure 3 - Attackers can use role chaining to eventually create new long-term credentials (AKIAY)

Now after some time, even if the security or DevOps team realizes that the original long-term credentials (AKIAX) were stolen and deletes or disables them, it will not prevent the threat actor from persisting in the AWS environment as they are using a tree of temporary credentials and new long-term credentials to maintain persistence access to the AWS environment.

Scenario 2: Compromising Short-term Credentials From an EC2 Instance

In the above scenario, we discussed how temporary credentials can be abused by assuming roles when starting from long-term credentials. Now, let us look at the scenario of how temporary credentials can get abused by compromising an EC2 instance that has a role attached to it.

Let’s pretend a threat actor gets access to an EC2 instance (through Server-Side Request Forgery; a vulnerable Jenkins server, etc.) which has an overly permissive role attached to it. They are able to retrieve the temporary credentials provided by the role from the instance metadata by running the following command on the instance:

curl http://169.254.169.254/latest/meta-data/iam/security-credentials/MyRole-456

Let’s call these credentials ASIA1. Using ASIA1, they’re now able to follow the same method outlined in scenario A and assume a different role or keep running curl on the instance to retrieve temporary credentials provided by the role for that EC2 instance.

It’s important to note AWS does not treat using roles to grant permissions to applications that run on EC2 instances as role chaining, which means they are not subject to the one hour limitation role chaining has, but rather can be set up to a maximum of 12 hours.

Figure 4 - Attackers can use server-side request forgery to retrieve temporary credentials

Scenario 3: Compromising Short-term Credentials From an EC2 Instance … Then Using Them Outside of That EC2 Instance

In the previous scenario, we highlight how a threat actor is able to retrieve the temporary credentials provided by the role from the instance metadata by running the curl command.

Once the threat actor has the combination of temporary credentials—access key; secret access key; security token—they’re able to use them on any machine that utilizes the AWS CLI tool by applying them in their local AWS CLI credentials and configuration settings. From here, they’re able to make programmatic calls to the victim AWS account.

Figure 5 - Attacker using server-side request forgery to steal temporary credentials and then execute commands on a new system

Conclusion

In all three scenarios, trying to trace the beginning and end of this attack is no easy task. Unless you’re actively paying for GuardDuty, detecting when temporary credentials are being used  outside of the EC2 instance they’re created for is also difficult.

Because of the nature of temporary credentials, they tend to “fly under the radar”, so to speak, with most customers unable to clearly visualize if any anomalous behavior is taking place.

So what do you do then? In part two of this blog series, we’ll dig into how the new Uptycs Identity Threat Detection and Response (ITDR) solution for the cloud will detect and provide guidance on how to remediate the likely compromised temporary credentials.