Google Dataproc overview in Apache Spark - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to create a Dataproc cluster using the Google Cloud SDK.

Apache Spark

gcloud dataproc clusters create [1] --region=us-central1 --single-node

Drag options to blanks, or click blank then click option'

Amy-cluster

Bcreate-cluster

Cdataproc-cluster

Dcluster-1

Attempts:

3 left

💡 Hint

Common Mistakes

Using a command keyword instead of a cluster name.

Including spaces in the cluster name.

The command requires the cluster name after 'create'. 'my-cluster' is a valid example name.

2fill in blank

medium

Complete the code to submit a PySpark job to the Dataproc cluster.

Apache Spark

gcloud dataproc jobs submit pyspark [1] --cluster=my-cluster --region=us-central1

Drag options to blanks, or click blank then click option'

Amain.py

Bscript.sh

Cjob.py

Djob.jar

Attempts:

3 left

💡 Hint

Common Mistakes

Using a shell script or Java jar file instead of a Python script.

Incorrect file extension.

The PySpark job file must be a Python script, so 'job.py' is correct.

3fill in blank

hard

Fix the error in the code to list all Dataproc clusters in a region.

Apache Spark

gcloud dataproc clusters list --region=[1]

Drag options to blanks, or click blank then click option'

Aus-central1

Bcentral1

Cus-central

Dus_central1

Attempts:

3 left

💡 Hint

Common Mistakes

Using underscores instead of hyphens.

Omitting part of the region name.

The correct region format is 'us-central1' with a hyphen.

4fill in blank

hard

Fill both blanks to create a Dataproc cluster with 3 worker nodes and specify the machine type.

Apache Spark

gcloud dataproc clusters create my-cluster --region=us-central1 --num-workers=[1] --worker-machine-type=[2]

Drag options to blanks, or click blank then click option'

A3

Bn1-standard-4

Cn1-standard-1

D5

Attempts:

3 left

💡 Hint

Common Mistakes

Using too few or too many workers.

Choosing an invalid machine type.

To create 3 workers, use '3'. For a common machine type, 'n1-standard-4' is suitable.

5fill in blank

hard

Fill all three blanks to create a cluster with autoscaling enabled and specify the autoscaling policy.

Apache Spark

gcloud dataproc clusters create my-cluster --region=us-central1 --autoscaling-policy=[1] --num-preemptible-workers=[2] --worker-machine-type=[3]

Drag options to blanks, or click blank then click option'

Amy-autoscale-policy

B2

Cn1-standard-2

Ddefault-policy

Attempts:

3 left

💡 Hint

Common Mistakes

Using an incorrect policy name.

Setting preemptible workers to zero or too high.

Choosing an unsupported machine type.

Use your autoscaling policy name, set 2 preemptible workers, and choose a machine type like 'n1-standard-2'.