AWS EBS snapshots are static backups of AWS EBS volumes. In other words, they are copies of the disks attached to an EC2 Instance at a specific point in time. EBS snapshots can be copied across regions and accounts, or even downloaded and run locally.

The EBS Direct API’s are interesting because they allow low-level access to the underlying data in a snapshot. You can download, upload or view changed blocks between snapshots (blocks representing the smallest addressable slice of raw data from the original EBS volume independent of any filesystem that is formatted on top of it). Since these are considered a part of the Data Plane API, similar to S3 Get* and Put*, there is no logging enabled by default. 

However, unlike S3, there is currently no way to enable logging of these Data Plane actions, and the API calls that are logged (like DescribeSnapshots or StartSnapshot) seem fairly innocuous at first glance. For a malicious actor, this makes for an interesting method to exfiltrate data without being detected.

EBS snapshots are often copies of active AWS Elastic Compute Cloud (EC2) instance volumes and can contain an abundance of interesting information or secrets. These snapshots tend to make for interesting targets during a cloud penetration test. 

In Pacu (Rhino’s open-source AWS exploitation framework), we have the ebs__explore_snapshots for exploring snapshots. However, this can be difficult to use as it requires an already running EC2 instance in the target account. 

In this blog, we will download and search EBS snapshots locally, introduce a new Python library called dsnap, and review the new ebs__download_snapshots Pacu module. Both dsnap and the module were inspired by the AWS tool, coldsnap.

The Python Command Line Interface (CLI) tool and library dsnap, and the associated Pacu module, ebs__download_snapshots, aim to simplify the process of downloading and exploring snapshots locally. Along with enabling the ability to quietly exfiltrate data, these tools can act as an alternative solution when the permissions needed for traditional methods of searching snapshots are unavailable.

Dsnap is easy to install and use, and the integration with Pacu allows you to reuse already discovered snapshot data. This is of interest when your goal is to avoid repetitive API calls to reduce the chances of being detected.

We’ll be downloading the snapshot with dsnap, converting it with VirtualBox, and then exploring it with Vagrant. For more information about the various ways of using dsnap either as a library or as a CLI see the dsnap readme.

You’ll need python 3.6 or later installed and PIP updated with `pip3 install -U pip`. Then, install dsnap with `pip3 install ‘dsnap[cli]’`.

In this blog we’ll be using Vagrant and VirtualBox for converting the downloaded snapshots. Listed below are various ways of installing these tools.

• Vagrant
  • MacOS: `brew install vagrant`
  • Windows w/ Chocolatey: `choco install vagrant`
  • For any platform: https://www.vagrantup.com/downloads
• VirtualBox
  • MacOS: `brew install virtualbox
  • Windows w/ Chocolatey: `choco install virtualbox`
  • For any platform: https://www.virtualbox.org/wiki/Downloads

The Vagrantfile we’ll be using later takes advantage of the `VBoxManage convertdd` command for converting the downloaded snapshot to a VDI file. Vagrant is then used to bring up a local virtual machine to mount and explore the images.

For this walkthrough, we’ll be writing a secret to the disk of a newly created EC2 instance and showing how we can use dsnap to take a snapshot, download, and then mount it offline to recover this same secret.

We’ll skip creating the instance, assume the following instance was recently spun up. 

First, we write a secret to the root of the disk with the following command. We’ll be coming back to this later after we are able to explore a snapshot of this instance locally.

Since this is a new EC2 instance, there aren’t any snapshots of it yet. However, if you go ahead and pass dsnap an instance, it will create a temporary snapshot that will be removed on exit.

But before we pass dsnap an instance we’ll want to run `dsnap init`. This will write out the Vagrantfile we’ll be using to mount our downloaded images.

Next, we run `dsnap get <instance-id>` using the ID of the EC2 instance that we created.

The temporary snapshot is downloaded to i-0d706e33814c1ef9a.img which can be converted to a VDI image for use with VirtualBox. This conversion happens automatically when Vagrant is running – no need to run it manually.

To kick off the VM with this image, we’ll prefix Vagrant with the filename of the image we want to mount. So if the image name is i-0d706e33814c1ef9a.img in our current directory, we will run:

After doing this we can see that the VM booted up and ran our provisioner which attached the disk we specified with IMAGE to /mnt/snapshot. We can now explore this disk by logging in with `vagrant ssh`. Once logged in, we’ll automatically get dropped into the directory where the snapshot was mounted.

In the above screenshot you can see we found our flag in the root directory of the snapshot created with dsnap. After we’re done looking through the snapshot, we can exit the Vagrant shell and destroy our VM. This will also remove the VDI image we created before, but we can reconvert from the original image if we need to explore the same snapshot again.

In this example of using dsnap, a few extra steps were done which resulted in more API calls logged in CloudTrail than necessary. If a snapshot had already existed and we passed dsnap the snapshot id instead of the instance name, then it wouldn’t have been possible to detect this activity.

Similarly, if we already have info gathered via Pacu (i.e we’ve ran ebs__enum_volumes_snapshots sometime in the past) the Pacu module ebs__download_snapshots won’t make any requests that can be logged.

Since in most cases using this technique will generate a few DescribeInstances or DescribeSnapshot logs in CloudTrail, it would be difficult to effectively roll out monitoring for it across an organization. Instead we’ll want to make sure that no access to the EBS direct API’s are granted to any User or Role that doesn’t need them.

The following policy will deny access to the API’s used to download snapshots while still allowing the ability to create them. The action ec2:ModifySnapshotAttribute is also included here to prevent sharing the snapshot with another account and running the download from there.

After attaching this policy, we get an error similar to the following when trying to run dsnap.

Using the EBS Direct APIs can be a simple and effective way to exfiltrate data and discover secrets in an AWS network without alerting anyone. Instead of attempting to monitor for this behavior, it is best to make sure any overprivileged roles are locked down to prevent the use of these APIs.

As always, feel free to open issues and pull requests on GitHub if you’d like and we will try to get to them to support future usage of these tools. Follow us on Twitter for more blog posts: @RhinoSecurity, @Ryan_Jarv