Storage commands (gsutil) in GCP - Commands & Configuration

Introduction

Sometimes you need to save files or share data in the cloud. Google Cloud Storage lets you do this easily. The gsutil tool helps you move files to and from this cloud storage using simple commands.

When you want to upload photos or documents from your computer to the cloud for backup.

When you need to download files from cloud storage to your local machine to work on them.

When you want to list all files stored in a specific cloud storage bucket to check what is saved.

When you want to copy files between two cloud storage buckets without downloading them first.

When you want to delete old files from cloud storage to free up space.

Commands

This command creates a new storage bucket named 'example-bucket-storage' where you can store your files.

Terminal

gsutil mb gs://example-bucket-storage

Expected Output

Creating gs://example-bucket-storage/...

This command uploads the file 'sample-file.txt' from your computer to the bucket you just created.

Terminal

gsutil cp sample-file.txt gs://example-bucket-storage/

Expected Output

Copying file://sample-file.txt [Content-Type=text/plain]...

/ [1 files][    12.0 B/    12.0 B] 100% Done

Operation completed over 1 objects.

This command lists all files currently stored in the 'example-bucket-storage' bucket so you can see what is saved.

Terminal

gsutil ls gs://example-bucket-storage/

Expected Output

gs://example-bucket-storage/sample-file.txt

This command downloads the file 'sample-file.txt' from the cloud bucket to your local machine and saves it as 'downloaded-file.txt'.

Terminal

gsutil cp gs://example-bucket-storage/sample-file.txt downloaded-file.txt

Expected Output

Copying gs://example-bucket-storage/sample-file.txt [Content-Type=text/plain]...

/ [1 files][    12.0 B/    12.0 B] 100% Done

Operation completed over 1 objects.

This command deletes the file 'sample-file.txt' from the cloud bucket to free up space or remove unwanted files.

Terminal

gsutil rm gs://example-bucket-storage/sample-file.txt

Expected Output

Removing gs://example-bucket-storage/sample-file.txt...

Key Concept

If you remember nothing else from this pattern, remember: gsutil lets you easily move files to, from, and inside Google Cloud Storage using simple commands.

Common Mistakes

Trying to upload a file to a bucket that does not exist.

The command fails because the storage location is missing and gsutil cannot create files without a bucket.

Always create the bucket first using 'gsutil mb' before uploading files.

Using incorrect bucket names with typos or missing 'gs://' prefix.

Commands fail because gsutil cannot find the bucket or interprets the name as a local path.

Use the full bucket path starting with 'gs://' and double-check spelling.

Deleting files without confirming the file name or path.

You might remove important files accidentally, causing data loss.

List files first with 'gsutil ls' and confirm the exact file name before deleting.

Summary

Create a storage bucket with 'gsutil mb' to hold your files.

Upload files to the bucket using 'gsutil cp'.

List files in the bucket with 'gsutil ls' to see what is stored.

Download files from the bucket back to your computer using 'gsutil cp'.

Remove files from the bucket with 'gsutil rm' when no longer needed.

Practice

1. What does the gsutil ls gs://my-bucket command do?

easy

A. Lists all files and folders inside the bucket named 'my-bucket'.

B. Deletes the bucket named 'my-bucket'.

C. Creates a new bucket named 'my-bucket'.

D. Copies files from 'my-bucket' to your local machine.

Solution

1. Step 1: Understand the 'ls' command in gsutil

   The 'ls' command lists contents of a bucket or folder in Google Cloud Storage.

2. Step 2: Apply the command to the bucket 'my-bucket'

   Running gsutil ls gs://my-bucket shows all files and folders inside that bucket.

3. Final Answer:

   Lists all files and folders inside the bucket named 'my-bucket'. -> Option A

4. Quick Check:

   gsutil ls lists bucket contents = C [OK]

Hint: Remember: 'ls' means list contents of bucket or folder [OK]

Common Mistakes:

• Confusing 'ls' with 'rm' which deletes files
• Thinking 'ls' creates buckets
• Assuming 'ls' copies files

2. Which of the following is the correct syntax to copy a local file named photo.jpg to a bucket gs://images-bucket using gsutil?

easy

A. gsutil copy photo.jpg gs://images-bucket

B. gsutil cp gs://images-bucket photo.jpg

C. gsutil cp photo.jpg gs://images-bucket

D. gsutil upload photo.jpg gs://images-bucket

Solution

1. Step 1: Identify the correct gsutil command for copying

   The command to copy files is gsutil cp, followed by source then destination.

2. Step 2: Place source and destination correctly

   Local file is source (photo.jpg), bucket is destination (gs://images-bucket), so syntax is gsutil cp photo.jpg gs://images-bucket.

3. Final Answer:

   gsutil cp photo.jpg gs://images-bucket -> Option C

4. Quick Check:

   cp source destination = A [OK]

Hint: Copy command is 'cp' with source first, then destination [OK]

Common Mistakes:

• Swapping source and destination order
• Using 'copy' instead of 'cp'
• Using 'upload' which is not a gsutil command

3. What will be the output of the command gsutil ls gs://my-bucket/folder/ if the folder contains files file1.txt and file2.txt?

medium

A. gs://my-bucket/folder/file1.txt\ngs://my-bucket/folder/file2.txt

B. file1.txt\nfile2.txt

C. gs://my-bucket/file1.txt\ngs://my-bucket/file2.txt

D. No output, command will fail

Solution

1. Step 1: Understand gsutil ls output format

   The 'gsutil ls' command lists full paths of files including bucket and folder names.

2. Step 2: Apply to files inside 'folder/'

   Files inside 'folder/' will be listed with full path prefix: gs://my-bucket/folder/file1.txt and gs://my-bucket/folder/file2.txt.

3. Final Answer:

   gs://my-bucket/folder/file1.txt gs://my-bucket/folder/file2.txt -> Option A

4. Quick Check:

   gsutil ls shows full gs:// paths = B [OK]

Hint: gsutil ls shows full gs:// path, not just file names [OK]

Common Mistakes:

• Expecting only file names without bucket path
• Confusing folder path with bucket root
• Assuming command fails if folder exists

4. You run gsutil rm gs://my-bucket/data.csv but get an error saying the file does not exist. What is the most likely cause?

medium

A. You need to add '-r' flag to remove a single file.

B. You used 'rm' instead of 'delete' command.

C. The bucket 'my-bucket' does not exist.

D. The file 'data.csv' is not in the bucket 'my-bucket'.

Solution

1. Step 1: Understand the error message about file not existing

   The error means the specified file path is not found in the bucket.

2. Step 2: Check command and flags correctness

   'rm' is the correct command to remove files; '-r' is only for recursive folder deletion, not needed for single files.

3. Final Answer:

   The file 'data.csv' is not in the bucket 'my-bucket'. -> Option D

4. Quick Check:

   File missing causes error, not command or flags [OK]

Hint: Check file path exists before removing [OK]

Common Mistakes:

• Thinking 'rm' is wrong command
• Adding '-r' unnecessarily for single files
• Assuming bucket missing causes file not found error

5. You want to delete an empty bucket named gs://temp-bucket. Which command should you use to safely remove it?

hard

A. gsutil cp gs://temp-bucket ./

B. gsutil rb gs://temp-bucket

C. gsutil delete gs://temp-bucket

D. gsutil rm gs://temp-bucket

Solution

1. Step 1: Identify the command to remove buckets

   The command to remove a bucket is gsutil rb (remove bucket).

2. Step 2: Confirm bucket is empty before removal

   Bucket must be empty to remove; if not, remove files first. Then run gsutil rb gs://temp-bucket.

3. Final Answer:

   gsutil rb gs://temp-bucket -> Option B

4. Quick Check:

   Remove bucket = rb command [OK]

Hint: Use 'rb' to remove empty buckets safely [OK]

Common Mistakes:

• Using 'rm' which deletes files, not buckets
• Using 'delete' which is not a gsutil command
• Trying to copy bucket instead of removing