Lesson 3 of 5
Add object storage with S3
Create an S3 bucket (object storage), upload a file from the management console, and access that file from your EC2 instance using the AWS CLI; this bucket stores files for your app and maps to Azure Blob Storage and Google Cloud Storage.
8 minCloud
After this lesson, you will be able to create an S3 bucket, upload a file to it from the AWS Management Console, and access that file from your EC2 instance using the AWS CLI. You will also know how S3 compares to object storage on Azure and Google Cloud.
What is S3?
Amazon S3 (Simple Storage Service) stores files as objects in buckets. A bucket is a container for objects, and each object has a key, which is its full path inside the bucket. Unlike a file system on your computer, S3 has no folders, but you can use slashes in keys to organise objects. S3 is designed for durability and can store any amount of data.
Azure's equivalent is Blob Storage, where objects live in containers inside a storage account. Google Cloud's equivalent is Cloud Storage, where objects live in buckets. The ideas are the same: you create a place to store files, upload them, and access them over the internet with permissions.
Create an S3 bucket
Open the AWS Management Console and search for S3. Choose Create bucket. Bucket names must be unique across all AWS accounts, so pick a name like my-web-app-bucket-12345 with your own numbers. Choose the AWS Region where your EC2 instance runs. Leave all other settings as default and choose Create bucket.
You now have a bucket. In the console, you can see it in the list of buckets. Click on its name to open it.
Upload a file from the console
Create a small text file on your computer. You can use any text editor. Save it as hello.txt with the content Hello from S3.
In the S3 console, with your bucket open, choose Upload. Choose Add files and select hello.txt. Then choose Upload. After a moment, the file appears in the bucket's object list.
Now the file is stored in S3. You can view it by clicking on the object name, but by default it is private, so you cannot open it in a browser without extra permissions.
Access the file from your EC2 instance
You need the AWS CLI on your EC2 instance. Connect to your instance with SSH as you did in the previous lesson. The AWS CLI is already installed on Amazon Linux, so you can use it right away.
First, check that the CLI works and which identity it uses:
aws sts get-caller-identityThis command prints the account ID, user ID, and ARN of the identity making the request. Since your instance has no IAM role attached yet, the CLI may not be able to find credentials. If you see an error about credentials, that is expected. In the next lesson, you will attach a role to the instance so it can access S3 without storing passwords. For now, you will use an access key from your IAM user.
On your local computer, open the IAM console and find your IAM user. Choose Security credentials and then Create access key. Download the CSV file with the access key ID and secret access key. These are long-term credentials for your user, so keep them safe.
Back on your EC2 instance, configure the CLI with these credentials:
aws configureWhen prompted, enter the access key ID, secret access key, default region name (the same region as your bucket, for example eu-west-2), and output format json.
Now test access to S3 by listing your buckets:
aws s3 lsThis command lists all buckets in your account. You should see your bucket name in the output. To list the objects inside your bucket, use the bucket name with a trailing slash:
aws s3 ls s3://my-web-app-bucket-12345/Replace my-web-app-bucket-12345 with your bucket name. The output shows the objects in the bucket, including hello.txt.
If you get an access denied error, first check that you entered the access key and secret correctly during aws configure. If the console allowed you to create and upload to the bucket, your IAM user already has S3 permissions, so the problem is usually credentials, not permissions. If you could not create the bucket earlier, attach the AmazonS3FullAccess policy to your user: in the IAM console, select the user, choose Add permissions, Attach policies directly, search for AmazonS3FullAccess, select it, and choose Next: Review, then Add permissions.
To download the file from S3 to your instance, use the cp command:
aws s3 cp s3://my-web-app-bucket-12345/hello.txt .Replace my-web-app-bucket-12345 with your bucket name. This copies the file to the current directory on your instance. You can view its contents with:
cat hello.txtYou should see Hello from S3.
How this maps to Azure and Google Cloud
On Azure, you create a storage account, then a container inside it, and upload blobs. You can access blobs with the Azure CLI using az storage blob download.
On Google Cloud, you create a bucket and upload objects with the gsutil command or the Cloud Console. The Google Cloud CLI uses gsutil cp to copy files.
All three services store objects with a key, support large files, and control access with identity and network policies. The commands differ, but the workflow is the same.
Clean up
The file hello.txt in your bucket is fine to keep for the next lesson. The access key you created for your IAM user is needed only for this lesson's CLI commands. In the next lesson, you will attach an IAM role to the instance and no longer need these long-term credentials. You can delete the access key in the IAM console now to keep your account secure. To delete it, go to your IAM user's Security credentials, find the access key, and choose Deactivate and then Delete.
Key takeaways
- S3 stores files as objects in buckets, with a key for each object.
- You can upload files from the AWS Management Console and access them with the AWS CLI.
- The AWS CLI on EC2 needs credentials; for now you used an IAM user's access key, but later you will use an IAM role.
- To list objects in a bucket, use
aws s3 ls s3://bucket-name/. - Azure Blob Storage and Google Cloud Storage work similarly with different command names.
Try it
Create a new text file on your computer called notes.txt with any content. Upload it to your S3 bucket using the console. Then, from your EC2 instance, use the AWS CLI to list the objects in the bucket (using aws s3 ls s3://your-bucket-name/) and download notes.txt with aws s3 cp. Check its contents with cat. A good result is that you see both hello.txt and notes.txt in the bucket listing, and the downloaded file matches what you uploaded.
Official documentation
Written with AI and checked automatically. Tools change, so a command may behave differently in the version you install. Spotted a mistake? Tell us.