Skip to content

Lesson 3 of 5

Add object storage with S3

Create an S3 bucket (object storage), upload a file from the management console, and access that file from your EC2 instance using the AWS CLI; this bucket stores files for your app and maps to Azure Blob Storage and Google Cloud Storage.

8 minCloud

After this lesson, you will be able to create an S3 bucket, upload a file to it from the AWS Management Console, and access that file from your EC2 instance using the AWS CLI. You will also know how S3 compares to object storage on Azure and Google Cloud.

What is S3?

Amazon S3 (Simple Storage Service) stores files as objects in buckets. A bucket is a container for objects, and each object has a key, which is its full path inside the bucket. Unlike a file system on your computer, S3 has no folders, but you can use slashes in keys to organise objects. S3 is designed for durability and can store any amount of data.

Azure's equivalent is Blob Storage, where objects live in containers inside a storage account. Google Cloud's equivalent is Cloud Storage, where objects live in buckets. The ideas are the same: you create a place to store files, upload them, and access them over the internet with permissions.

Create an S3 bucket

Open the AWS Management Console and search for S3. Choose Create bucket. Bucket names must be unique across all AWS accounts, so pick a name like my-web-app-bucket-12345 with your own numbers. Choose the AWS Region where your EC2 instance runs. Leave all other settings as default and choose Create bucket.

You now have a bucket. In the console, you can see it in the list of buckets. Click on its name to open it.

Upload a file from the console

Create a small text file on your computer. You can use any text editor. Save it as hello.txt with the content Hello from S3.

In the S3 console, with your bucket open, choose Upload. Choose Add files and select hello.txt. Then choose Upload. After a moment, the file appears in the bucket's object list.

Now the file is stored in S3. You can view it by clicking on the object name, but by default it is private, so you cannot open it in a browser without extra permissions.

Access the file from your EC2 instance

You need the AWS CLI on your EC2 instance. Connect to your instance with SSH as you did in the previous lesson. The AWS CLI is already installed on Amazon Linux, so you can use it right away.

First, check that the CLI works and which identity it uses:

aws sts get-caller-identity

This command prints the account ID, user ID, and ARN of the identity making the request. Since your instance has no IAM role attached yet, the CLI may not be able to find credentials. If you see an error about credentials, that is expected. In the next lesson, you will attach a role to the instance so it can access S3 without storing passwords. For now, you will use an access key from your IAM user.

On your local computer, open the IAM console and find your IAM user. Choose Security credentials and then Create access key. Download the CSV file with the access key ID and secret access key. These are long-term credentials for your user, so keep them safe.

Back on your EC2 instance, configure the CLI with these credentials:

aws configure

When prompted, enter the access key ID, secret access key, default region name (the same region as your bucket, for example eu-west-2), and output format json.

Now test access to S3 by listing your buckets:

aws s3 ls

This command lists all buckets in your account. You should see your bucket name in the output. To list the objects inside your bucket, use the bucket name with a trailing slash:

aws s3 ls s3://my-web-app-bucket-12345/

Replace my-web-app-bucket-12345 with your bucket name. The output shows the objects in the bucket, including hello.txt.

If you get an access denied error, first check that you entered the access key and secret correctly during aws configure. If the console allowed you to create and upload to the bucket, your IAM user already has S3 permissions, so the problem is usually credentials, not permissions. If you could not create the bucket earlier, attach the AmazonS3FullAccess policy to your user: in the IAM console, select the user, choose Add permissions, Attach policies directly, search for AmazonS3FullAccess, select it, and choose Next: Review, then Add permissions.

To download the file from S3 to your instance, use the cp command:

aws s3 cp s3://my-web-app-bucket-12345/hello.txt .

Replace my-web-app-bucket-12345 with your bucket name. This copies the file to the current directory on your instance. You can view its contents with:

cat hello.txt

You should see Hello from S3.

How this maps to Azure and Google Cloud

On Azure, you create a storage account, then a container inside it, and upload blobs. You can access blobs with the Azure CLI using az storage blob download.

On Google Cloud, you create a bucket and upload objects with the gsutil command or the Cloud Console. The Google Cloud CLI uses gsutil cp to copy files.

All three services store objects with a key, support large files, and control access with identity and network policies. The commands differ, but the workflow is the same.

Clean up

The file hello.txt in your bucket is fine to keep for the next lesson. The access key you created for your IAM user is needed only for this lesson's CLI commands. In the next lesson, you will attach an IAM role to the instance and no longer need these long-term credentials. You can delete the access key in the IAM console now to keep your account secure. To delete it, go to your IAM user's Security credentials, find the access key, and choose Deactivate and then Delete.

Key takeaways

  • S3 stores files as objects in buckets, with a key for each object.
  • You can upload files from the AWS Management Console and access them with the AWS CLI.
  • The AWS CLI on EC2 needs credentials; for now you used an IAM user's access key, but later you will use an IAM role.
  • To list objects in a bucket, use aws s3 ls s3://bucket-name/.
  • Azure Blob Storage and Google Cloud Storage work similarly with different command names.

Try it

Create a new text file on your computer called notes.txt with any content. Upload it to your S3 bucket using the console. Then, from your EC2 instance, use the AWS CLI to list the objects in the bucket (using aws s3 ls s3://your-bucket-name/) and download notes.txt with aws s3 cp. Check its contents with cat. A good result is that you see both hello.txt and notes.txt in the bucket listing, and the downloaded file matches what you uploaded.

Official documentation

Ready to use this in a job?

Rolebound applies to matching roles for you.

Start now

Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.

    Cookies on Rolebound

    We use necessary cookies to sign you in and keep the site working — those cannot be turned off. We would also like to set analytics and advertising cookies so we can see which pages help people get interviews and which adverts brought them here. Nothing optional is set unless you say yes, and you can change your mind at any time from “Cookie settings” in the footer. Read our Privacy Policy.

    Add object storage with S3 | Cloud Basics: Deploy a Secure Web App on AWS | Rolebound