Storage transfer service in GCP - Step-by-Step Execution

Process Flow - Storage transfer service

Define Transfer Job

↓

Set Source and Destination

↓

Configure Schedule and Options

↓

Start Transfer Job

↓

Service Copies Data

↓

Monitor Transfer Status

↓

Transfer Complete or Error

The flow shows how you create a transfer job, set source and destination, start it, and monitor until completion.

Execution Sample

GCP

gcloud transfer jobs create \
  --source-bucket=source-bucket-name \
  --sink-bucket=destination-bucket-name \
  --schedule-start-date=2024-06-01 \
  --schedule-end-date=2024-06-01 \
  --description="Daily backup"

This command creates a transfer job to copy data from one bucket to another on a specific date.

Process Table

Step	Action	Input/Config	Service Behavior	Result
1	Define Transfer Job	Job name, description	Registers job in service	Job created with ID
2	Set Source and Destination	Source: source-bucket-name Destination: destination-bucket-name	Validates buckets exist	Buckets linked to job
3	Configure Schedule	Start date: 2024-06-01 End date: 2024-06-01	Schedules job for that day	Job scheduled
4	Start Transfer Job	Trigger job start	Service begins copying data	Transfer in progress
5	Monitor Transfer Status	Check job status	Reports progress, errors	Status: Running or Completed
6	Transfer Complete	Job finishes copying	Marks job as done	Data copied successfully
7	Exit	No more scheduled runs	Job ends	Transfer job completed

💡 Transfer job completes after copying all data on scheduled date.

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 6	Final
Job ID	None	Job123	Job123	Job123	Job123	Job123
Source Bucket	None	source-bucket-name	source-bucket-name	source-bucket-name	source-bucket-name	source-bucket-name
Destination Bucket	None	destination-bucket-name	destination-bucket-name	destination-bucket-name	destination-bucket-name	destination-bucket-name
Schedule	None	None	2024-06-01 to 2024-06-01	2024-06-01 to 2024-06-01	2024-06-01 to 2024-06-01	2024-06-01 to 2024-06-01
Status	None	Pending	Pending	Running	Completed	Completed

Key Moments - 3 Insights

Why does the transfer job have a status of 'Pending' before it starts copying data?

Because after setting the schedule, the job waits for the scheduled time to start. This is shown in execution_table row 3 where the job is scheduled but not yet running.

What happens if the source or destination bucket does not exist when creating the job?

The service validates buckets at step 2 and will not create the job if buckets are invalid. This prevents errors during transfer.

How do you know when the transfer job is fully complete?

At step 6 in the execution_table, the status changes to 'Completed' indicating all data has been copied and the job is done.

Visual Quiz - 3 Questions

Look at the execution_table, what is the job status after step 4?

A Pending

B Completed

C Running

D Failed

💡 Hint

Check the 'Status' variable in variable_tracker after Step 4.

At which step does the service start copying data?

A Step 3

B Step 4

C Step 2

D Step 6

💡 Hint

Look at the 'Service Behavior' column in execution_table for when copying begins.

If the schedule end date is before the start date, what would happen?

A Job creation fails or no runs scheduled

B Job runs once on start date

C Job runs multiple times

D Job runs indefinitely

💡 Hint

Consider the scheduling logic in execution_table step 3 and how invalid dates affect scheduling.

Concept Snapshot

Storage Transfer Service lets you copy data between buckets or from other sources.
You create a transfer job, set source and destination buckets, and schedule when to run.
The service copies data on schedule and reports status.
You monitor progress and know when transfer completes.
It helps automate data movement safely and reliably.

Full Transcript

Storage Transfer Service in Google Cloud helps move data from one storage location to another automatically. First, you create a transfer job by defining its name and description. Then, you specify the source bucket where data currently lives and the destination bucket where data should go. Next, you set a schedule for when the transfer should happen, such as a specific date or recurring times. When the scheduled time arrives, the service starts copying data from source to destination. You can check the job status anytime to see if it is running, completed, or if there were errors. Once all data is copied, the job marks itself complete. This process helps keep data backed up or moved without manual copying. The execution table shows each step from job creation to completion, and the variable tracker shows how job status and settings change over time. Common confusions include understanding the 'Pending' status before start, bucket validation, and knowing when the job is done. The visual quiz tests understanding of these steps and statuses. Overall, Storage Transfer Service automates and simplifies data transfers in the cloud.