Working with the AWS S3 CLI in Linux

AWS S3 (Simple Storage Service) is a reliable and scalable object storage service commonly used for storing and retrieving data across various industries. The AWS S3 CLI (Command Line Interface) provides developers and system administrators with powerful command-line tools to interact with S3 buckets and objects efficiently. This article explores essential AWS S3 CLI commands for managing buckets, uploading and downloading files, and performing other common operations in Linux environments.

Setting up the AWS CLI

Before using the AWS S3 CLI, you need to install and configure the AWS CLI on your Linux system.

Installation

Install the AWS CLI using your distribution's package manager

sudo apt-get install awscli

Configuration

Configure the AWS CLI by running the following command and entering your credentials

aws configure

This will prompt you to enter

• AWS Access Key ID Your programmatic access key

• AWS Secret Access Key Your secret key

• Default region name e.g., us-east-1

• Default output format json, text, or table

You can obtain the access key ID and secret access key from the AWS Management Console under IAM (Identity and Access Management).

Creating an S3 Bucket

Create a new S3 bucket using the mb (make bucket) command

aws s3 mb s3://bucket-name

Replace bucket-name with your desired bucket name. Bucket names must be globally unique across all AWS accounts.

Listing S3 Buckets and Contents

List All Buckets

Display all S3 buckets in your AWS account

aws s3 ls

List Bucket Contents

List the contents of a specific bucket

aws s3 ls s3://bucket-name

Detailed Listing with Recursive Search

For detailed information including subdirectories and human-readable file sizes

aws s3 ls s3://bucket-name --recursive --human-readable

Add the --summarize flag to display total size and object count

aws s3 ls s3://bucket-name --recursive --human-readable --summarize

Copying Files to S3

Upload Single File

Copy a file from your local system to an S3 bucket

aws s3 cp /path/to/local/file s3://bucket-name/path/to/s3/object

Example: Upload a file named example.txt from the home directory

aws s3 cp /home/user/example.txt s3://my-bucket/my-folder/example.txt

Upload Directory Recursively

Copy an entire directory with all its contents

aws s3 cp /home/user/my-folder s3://my-bucket/my-folder --recursive

Downloading Files from S3

Download files from S3 to your local system using the cp command

aws s3 cp s3://bucket-name/file-path /local/destination/path

Example: Download example.txt from S3 to local folder

aws s3 cp s3://my-bucket/example.txt /path/to/local/folder/

Deleting S3 Resources

Delete Files

Remove a specific file from an S3 bucket

aws s3 rm s3://bucket-name/path/to/s3/object

Delete Buckets

Remove an empty S3 bucket

aws s3 rb s3://bucket-name

To delete a bucket and all its contents, use the --force flag

aws s3 rb s3://bucket-name --force

Common S3 CLI Commands Summary

Conclusion

The AWS S3 CLI provides efficient command-line tools for managing S3 buckets and objects in Linux environments. By mastering these essential commands, developers and system administrators can streamline their workflows, automate repetitive tasks, and integrate S3 storage seamlessly into their applications and scripts.