File download automation in Bash Scripting - Cheat Sheet & Quick Revision

Recall & Review

beginner

What command is commonly used in Bash to download files from the internet?

The curl command is commonly used to download files from the internet in Bash. Another popular command is wget.

beginner

How do you save a file with a specific name using curl?

Use the -o option followed by the desired filename. Example: curl -o myfile.txt https://example.com/file.txt saves the file as myfile.txt.

intermediate

What does the -L option do in curl?

The -L option tells curl to follow redirects. This is useful when the URL redirects to another location before the file is downloaded.

intermediate

How can you automate downloading multiple files in Bash?

You can use a loop to iterate over a list of URLs and download each file using curl or wget. For example, a for loop with URLs stored in an array.

beginner

Why is it helpful to check if a file already exists before downloading it in a script?

Checking if a file exists prevents unnecessary downloads, saves bandwidth, and avoids overwriting files. This makes the script more efficient and safer.

Which command option saves a downloaded file with a specific name using curl?

A -x

B -d

C -o

D -s

The -o option in curl specifies the output filename.

What does the -L option do in curl?

A Limits download speed

B Follows redirects

C Downloads in background

D Shows download progress

The -L option makes curl follow HTTP redirects automatically.

Which Bash structure is best for downloading multiple files automatically?

A case statement

B while loop

C if statement

D for loop

A for loop is commonly used to iterate over a list of URLs to download multiple files.

Why check if a file exists before downloading in a script?

A To avoid overwriting and save bandwidth

B To change file permissions

C To delete old files

D To speed up the download

Checking file existence avoids unnecessary downloads and prevents overwriting existing files.

Which command can also be used to download files besides curl?

A wget

B ls

C cat

D grep

wget is another popular command-line tool for downloading files.

Explain how you would write a Bash script to download a list of files from URLs stored in an array.

Think about looping through each URL and using curl with -o option.

Describe why following redirects is important when automating file downloads and how to enable it in curl.

Consider what happens if the file is moved to a new address.

Practice

1. What is the main purpose of automating file downloads in bash scripting?

easy

A. To slow down the internet speed

B. To make files larger

C. To save time and reduce manual errors

D. To delete files automatically

Solution

1. Step 1: Understand automation benefits

   Automating repetitive tasks like file downloads saves time and reduces mistakes.

2. Step 2: Identify the correct purpose

   The main goal is efficiency and accuracy, not file size or deletion.

3. Final Answer:

   To save time and reduce manual errors -> Option C

4. Quick Check:

   Automation = Save time & reduce errors [OK]

Hint: Automation means doing tasks faster and safer [OK]

Common Mistakes:

• Thinking automation changes file size
• Confusing automation with file deletion
• Assuming automation slows internet

2. Which of the following is the correct syntax to download a file using wget in bash?

easy

A. curl -download http://example.com/file.txt

B. download http://example.com/file.txt

C. fetch http://example.com/file.txt

D. wget http://example.com/file.txt

Solution

1. Step 1: Recall wget command syntax

   The basic wget command to download a file is 'wget URL'.

2. Step 2: Compare options

   Only wget http://example.com/file.txt uses the correct wget syntax; others use invalid commands.

3. Final Answer:

   wget http://example.com/file.txt -> Option D

4. Quick Check:

   wget URL = correct syntax [OK]

Hint: Use 'wget URL' to download files simply [OK]

Common Mistakes:

• Using non-existent commands like 'download' or 'fetch'
• Mixing curl options with wget
• Forgetting to specify the URL

3. What will be the output of this bash command?

curl -O https://example.com/sample.txt
ls | grep sample.txt

medium

A. sample.txt

B. No output

C. curl: command not found

D. Error: file not found

Solution

1. Step 1: Understand curl -O behavior

   The command downloads 'sample.txt' to the current directory.

2. Step 2: Check the ls and grep output

   Listing files and filtering for 'sample.txt' will show the filename if download succeeded.

3. Final Answer:

   sample.txt -> Option A

4. Quick Check:

   curl -O downloads file, ls | grep shows filename [OK]

Hint: curl -O saves file; grep finds it in list [OK]

Common Mistakes:

• Assuming curl prints file content
• Expecting no output after download
• Confusing curl errors with normal output

4. Identify the error in this script snippet for downloading a file:

curl -o https://example.com/data.csv

medium

A. URL is incorrect

B. Missing filename after -o option

C. curl cannot download CSV files

D. No error, script is correct

Solution

1. Step 1: Check curl -o usage

   The '-o' option requires a filename to save the download.

2. Step 2: Analyze the command

   The command provides URL but no filename after '-o', causing error.

3. Final Answer:

   Missing filename after -o option -> Option B

4. Quick Check:

   curl -o needs filename [OK]

Hint: Always give filename after curl -o [OK]

Common Mistakes:

• Forgetting filename after -o
• Thinking URL is wrong format
• Believing curl can't download certain file types

5. You want to automate downloading multiple files listed in a text file named urls.txt. Which bash script snippet correctly downloads each file using wget?

hard

A. while read url; do wget "$url"; done < urls.txt

B. for url in urls.txt; do wget $url; done

C. wget -i urls.txt -O output.log

D. cat urls.txt | wget

Solution

1. Step 1: Understand reading lines from file

   The 'while read url; do ... done < urls.txt' reads each line as a URL.

2. Step 2: Check wget usage inside loop

   Using wget "$url" downloads each URL correctly.

3. Step 3: Analyze other options

   'for url in urls.txt; do wget $url; done' treats filename as URL, 'wget -i urls.txt -O output.log' misuses -O option causing all files to overwrite as 'output.log', 'cat urls.txt | wget' pipes incorrectly.

4. Final Answer:

   while read url; do wget "$url"; done < urls.txt -> Option A

5. Quick Check:

   Loop reading lines + wget = correct [OK]

Hint: Use while read loop to process URLs line by line [OK]

Common Mistakes:

• Using for loop directly on filename
• Misusing wget options
• Piping file content directly to wget