A/B testing models in ML Python - Interactive Code Practice

Practice - 5 Tasks

1fill in blank

easy

Complete the code to split data into two groups for A/B testing.

ML Python

group_a = data.sample(frac=[1], random_state=42)
group_b = data.drop(group_a.index)

Drag options to blanks, or click blank then click option'

A1.0

B0.5

C0.25

D0.75

Attempts:

3 left

💡 Hint

Common Mistakes

Using 1.0 selects all data, leaving no data for the other group.

Using fractions other than 0.5 causes uneven group sizes.

Using frac=0.5 splits the data evenly into two groups for A/B testing.

2fill in blank

medium

Complete the code to calculate the conversion rate for group A.

ML Python

conversion_rate_a = group_a['converted'].[1]()

Drag options to blanks, or click blank then click option'

Acount

Bsum

Cmean

Dmax

Attempts:

3 left

💡 Hint

Common Mistakes

Using sum() gives total conversions, not rate.

Using count() gives total rows, not conversion rate.

The mean() function calculates the average conversion rate (proportion of converted users).

3fill in blank

hard

Fix the error in the code to perform a t-test between groups A and B.

ML Python

from scipy.stats import ttest_ind

stat, p_value = ttest_ind(group_a['converted'], [1]['converted'])

Drag options to blanks, or click blank then click option'

Agroup_a

Bconverted

Cdata

Dgroup_b

Attempts:

3 left

💡 Hint

Common Mistakes

Using group_a twice compares the group to itself.

Using the whole data instead of group B mixes groups.

The t-test compares the 'converted' values between group A and group B, so the second argument must be group_b['converted'].

4fill in blank

hard

Fill both blanks to calculate the p-value and check if the result is significant at 5%.

ML Python

stat, p_value = ttest_ind(group_a['converted'], group_b['converted'])
is_significant = p_value [1] 0.05

Drag options to blanks, or click blank then click option'

A<

B>

C==

D!=

Attempts:

3 left

💡 Hint

Common Mistakes

Using > or == will not correctly check significance.

Using != is not meaningful for p-value comparison.

We check if the p-value is less than 0.05 to decide if the difference is statistically significant.

5fill in blank

hard

Fill all three blanks to create a summary dictionary with group names, conversion rates, and significance result.

ML Python

summary = {
    '[1]': conversion_rate_a,
    '[2]': conversion_rate_b,
    'significant_difference': [3]
}

Drag options to blanks, or click blank then click option'

AGroup A

BGroup B

Cis_significant

Dsignificant

Attempts:

3 left

💡 Hint

Common Mistakes

Using variable names as keys instead of strings.

Using incorrect keys that don't describe the data.

The dictionary keys should be descriptive group names and the boolean variable for significance.