HTTP requests with curl in scripts in Bash Scripting - Step-by-Step Execution

Concept Flow - HTTP requests with curl in scripts

Start script

↓

Prepare curl command

↓

Execute curl command

↓

Receive HTTP response

↓

Process or display response

↓

End script

The script starts, prepares a curl command to make an HTTP request, executes it, receives the response, then processes or shows the response before ending.

Execution Sample

Bash Scripting

response=$(curl -s https://api.github.com)
echo "$response"

This script sends a silent HTTP GET request to GitHub API and prints the response.

Execution Table

Step	Action	Command/Expression	Result/Output
1	Prepare curl command	curl -s https://api.github.com	Command ready to run
2	Execute curl command	curl -s https://api.github.com	HTTP response JSON received
3	Store response	response=$(curl -s https://api.github.com)	Variable 'response' holds JSON text
4	Print response	echo "$response"	JSON text printed to terminal
5	End script	N/A	Script finishes execution

💡 Script ends after printing the HTTP response stored in 'response' variable.

Variable Tracker

Variable	Start	After Step 3	Final
response	empty	JSON text from GitHub API	JSON text from GitHub API

Key Moments - 3 Insights

Why do we use the -s option with curl in the script?

The -s option makes curl silent, so it does not show progress or errors, only the response. This keeps the output clean as shown in execution_table step 2.

How does the script store the HTTP response in a variable?

Using command substitution $(...), the output of curl is captured and assigned to the variable 'response' as shown in execution_table step 3.

What happens if we omit the quotes around "$response" in echo?

Without quotes, the JSON response might be split or misformatted when printed. Quoting preserves the exact response text, as shown in step 4.

Visual Quiz - 3 Questions

Look at the execution table, what does the variable 'response' contain after step 3?

A The HTTP response JSON text

B The curl command itself

C An empty string

D An error message

💡 Hint

Check the 'Result/Output' column in step 3 of the execution_table.

At which step does the script print the HTTP response to the terminal?

A Step 2

B Step 4

C Step 3

D Step 5

💡 Hint

Look for the 'Print response' action in the execution_table.

If we remove the -s option from curl, what changes in the output?

A The response will be empty

B Curl will not send the HTTP request

C Curl shows progress and error messages along with the response

D The script will fail to run

💡 Hint

Refer to the key moment about the -s option and step 2 in the execution_table.

Concept Snapshot

curl command in scripts:
- Use curl to send HTTP requests
- -s option makes curl silent (no progress shown)
- Capture output with response=$(curl ...)
- Use quotes when printing variables to keep formatting
- Useful for automation and API calls

Full Transcript

This script uses curl to send an HTTP GET request silently to a URL. The response is captured into a variable using command substitution. Then the script prints the response exactly as received. The -s option keeps the output clean by hiding progress info. Quoting the variable when printing preserves the response format. The script ends after printing the response.

Practice

1. What does the curl command do in a bash script?

easy

A. It creates a new directory on the system.

B. It sends HTTP requests to websites or APIs.

C. It compiles the bash script into an executable.

D. It displays the current date and time.

Solution

1. Step 1: Understand the purpose of curl

   The curl command is designed to transfer data from or to a server using various protocols, mainly HTTP.

2. Step 2: Match curl's function to the options

   Among the options, only sending HTTP requests matches curl's purpose.

3. Final Answer:

   It sends HTTP requests to websites or APIs. -> Option B

4. Quick Check:

   curl = HTTP requests [OK]

Hint: Remember curl talks to websites using HTTP [OK]

Common Mistakes:

• Confusing curl with system commands like mkdir or date
• Thinking curl compiles scripts
• Assuming curl only downloads files

2. Which of the following is the correct syntax to send a GET request with curl in a bash script?

easy

A. curl -send https://example.com

B. curl --post https://example.com

C. curl -X GET https://example.com

D. curl get https://example.com

Solution

1. Step 1: Identify the correct curl option for GET

   The -X GET option explicitly tells curl to use the GET method, which is the default.

2. Step 2: Check other options for correctness

   Options B, C, and D use invalid or incorrect flags for curl.

3. Final Answer:

   curl -X GET https://example.com -> Option C

4. Quick Check:

   GET request syntax = curl -X GET [OK]

Hint: Use -X GET to specify GET method explicitly [OK]

Common Mistakes:

• Using --post instead of -X POST for POST requests
• Typing invalid flags like -send or get
• Omitting the URL

3. What will the following script output?

response=$(curl -s -o /dev/null -w "%{http_code}" https://httpbin.org/status/200)
echo $response

medium

A. 200

B. OK

C. 404

D. curl: command not found

Solution

1. Step 1: Understand curl options used

   The -s option silences progress, -o /dev/null discards output, and -w "%{http_code}" prints the HTTP status code.

2. Step 2: Analyze the URL and expected status

   The URL https://httpbin.org/status/200 returns HTTP status 200, so the variable response will hold "200".

3. Final Answer:

   200 -> Option A

4. Quick Check:

   HTTP status code = 200 [OK]

Hint: Use -w "%{http_code}" to get HTTP status code [OK]

Common Mistakes:

• Expecting full page content instead of status code
• Missing quotes around %{http_code}
• Not using -s to silence output

4. Identify the error in this script snippet:

curl -X POST -d 'name=John' https://example.com/api
if [ $? -eq 0 ]; then
  echo "Success"
else
  echo "Failed"
fi

medium

A. No error; script works correctly

B. Missing quotes around URL

C. Single quotes prevent variable expansion in -d

D. Using -X POST with -d is invalid

Solution

1. Step 1: Check curl command syntax

   The command uses -X POST with -d 'name=John' correctly to send POST data.

2. Step 2: Verify script logic for success check

   The script checks the exit status $? properly and echoes success or failure accordingly.

3. Final Answer:

   No error; script works correctly -> Option A

4. Quick Check:

   Script syntax and logic = correct [OK]

Hint: Check exit status $? after curl for success [OK]

Common Mistakes:

• Thinking single quotes block variable expansion here (no variables used)
• Assuming -X POST and -d can't be combined
• Forgetting to check curl exit status

5. You want to write a bash script that sends a JSON POST request with curl to https://api.example.com/data and saves the response to response.json. Which script snippet correctly does this?

hard

A. curl -X POST -H 'Content-Type: application/json' -d '{key:value}' https://api.example.com/data >> response.json

B. curl -POST -H 'Content-Type: application/json' -d '{key:value}' https://api.example.com/data > response.json

C. curl -X POST -H Content-Type application/json -d '{"key":"value"}' https://api.example.com/data > response.json

D. curl -X POST -H 'Content-Type: application/json' -d '{"key":"value"}' https://api.example.com/data > response.json

Solution

1. Step 1: Check HTTP method and headers

   curl -X POST -H 'Content-Type: application/json' -d '{"key":"value"}' https://api.example.com/data > response.json correctly uses -X POST and sets the header -H 'Content-Type: application/json' with quotes.

2. Step 2: Validate JSON data and output redirection

   curl -X POST -H 'Content-Type: application/json' -d '{"key":"value"}' https://api.example.com/data > response.json uses valid JSON with escaped quotes and redirects output with > to overwrite response.json.

3. Step 3: Identify errors in other options

   curl -POST -H 'Content-Type: application/json' -d '{key:value}' https://api.example.com/data > response.json uses invalid -POST flag and unquoted JSON keys. curl -X POST -H Content-Type application/json -d '{"key":"value"}' https://api.example.com/data > response.json misses quotes around header value. curl -X POST -H 'Content-Type: application/json' -d '{key:value}' https://api.example.com/data >> response.json appends with >> and has invalid JSON keys.

4. Final Answer:

   curl -X POST -H 'Content-Type: application/json' -d '{"key":"value"}' https://api.example.com/data > response.json -> Option D

5. Quick Check:

   Correct POST JSON syntax and output redirection = curl -X POST -H 'Content-Type: application/json' -d '{"key":"value"}' https://api.example.com/data > response.json [OK]

Hint: Use -H for headers, -d with valid JSON, > to save output [OK]

Common Mistakes:

• Forgetting quotes around JSON keys and values
• Using invalid curl flags like -POST
• Appending instead of overwriting output file