In [6]:
# CODE HERE
plt.figure(figsize=(12,7),dpi=150)
sns.scatterplot(data=fandango,x='RATING',y='VOTES')
Out[6]:
<AxesSubplot:xlabel='RATING', ylabel='VOTES'>
If you are planning on going out to see a movie, how well can you trust online reviews and ratings? Especially if the same company showing the rating also makes money by selling movie tickets. Do they have a bias towards rating movies higher than they should be rated?
Your goal is to complete the tasks below based off the 538 article and see if you reach a similar conclusion. You will need to use your pandas and visualization skills to determine if Fandango's ratings in 2015 had a bias towards rating movies better to sell more tickets.
Complete the tasks written in bold.
TASK: Read this article: Be Suspicious Of Online Movie Ratings, Especially Fandango’s
TASK: After reading the article, read these two tables giving an overview of the two .csv files we will be working with:
This is the data behind the story Be Suspicious Of Online Movie Ratings, Especially Fandango’s openly available on 538's github: https://github.com/fivethirtyeight/data. There are two csv files, one with Fandango Stars and Displayed Ratings, and the other with aggregate data for movie ratings from other sites, like Metacritic,IMDB, and Rotten Tomatoes.
all_sites_scores.csv contains every film that has a Rotten Tomatoes rating, a RT User rating, a Metacritic score, a Metacritic User score, and IMDb score, and at least 30 fan reviews on Fandango. The data from Fandango was pulled on Aug. 24, 2015.
| Column | Definition |
|---|---|
| FILM | The film in question |
| RottenTomatoes | The Rotten Tomatoes Tomatometer score for the film |
| RottenTomatoes_User | The Rotten Tomatoes user score for the film |
| Metacritic | The Metacritic critic score for the film |
| Metacritic_User | The Metacritic user score for the film |
| IMDB | The IMDb user score for the film |
| Metacritic_user_vote_count | The number of user votes the film had on Metacritic |
| IMDB_user_vote_count | The number of user votes the film had on IMDb |
fandango_scrape.csv contains every film 538 pulled from Fandango.
| Column | Definiton |
|---|---|
| FILM | The movie |
| STARS | Number of stars presented on Fandango.com |
| RATING | The Fandango ratingValue for the film, as pulled from the HTML of each page. This is the actual average score the movie obtained. |
| VOTES | number of people who had reviewed the film at the time we pulled it. |
TASK: Import any libraries you think you will use:
# IMPORT HERE!
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
Let's first explore the Fandango ratings to see if our analysis agrees with the article's conclusion.
TASK: Run the cell below to read in the fandango_scrape.csv file
fandango = pd.read_csv("fandango_scrape.csv")
TASK: Explore the DataFrame Properties and Head.
fandango.head()
| FILM | STARS | RATING | VOTES | |
|---|---|---|---|---|
| 0 | Fifty Shades of Grey (2015) | 4.0 | 3.9 | 34846 |
| 1 | Jurassic World (2015) | 4.5 | 4.5 | 34390 |
| 2 | American Sniper (2015) | 5.0 | 4.8 | 34085 |
| 3 | Furious 7 (2015) | 5.0 | 4.8 | 33538 |
| 4 | Inside Out (2015) | 4.5 | 4.5 | 15749 |
fandango.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 504 entries, 0 to 503 Data columns (total 4 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 FILM 504 non-null object 1 STARS 504 non-null float64 2 RATING 504 non-null float64 3 VOTES 504 non-null int64 dtypes: float64(2), int64(1), object(1) memory usage: 15.9+ KB
fandango.describe()
| STARS | RATING | VOTES | |
|---|---|---|---|
| count | 504.000000 | 504.000000 | 504.000000 |
| mean | 3.558532 | 3.375794 | 1147.863095 |
| std | 1.563133 | 1.491223 | 3830.583136 |
| min | 0.000000 | 0.000000 | 0.000000 |
| 25% | 3.500000 | 3.100000 | 3.000000 |
| 50% | 4.000000 | 3.800000 | 18.500000 |
| 75% | 4.500000 | 4.300000 | 189.750000 |
| max | 5.000000 | 5.000000 | 34846.000000 |
TASK: Let's explore the relationship between popularity of a film and its rating. Create a scatterplot showing the relationship between rating and votes. Feel free to edit visual styling to your preference.
# CODE HERE
plt.figure(figsize=(12,7),dpi=150)
sns.scatterplot(data=fandango,x='RATING',y='VOTES')
<AxesSubplot:xlabel='RATING', ylabel='VOTES'>
TASK: Calculate the correlation between the columns:
# CODE HERE
fandango.corr()
| STARS | RATING | VOTES | |
|---|---|---|---|
| STARS | 1.000000 | 0.994696 | 0.164218 |
| RATING | 0.994696 | 1.000000 | 0.163764 |
| VOTES | 0.164218 | 0.163764 | 1.000000 |
TASK: Assuming that every row in the FILM title column has the same format:
Film Title Name (Year)
Create a new column that is able to strip the year from the title strings and set this new column as YEAR
# CODE HERE
fandango['YEAR']=fandango['FILM'].str.split('( )').str[-1]
fandango['YEAR']=fandango['YEAR'].str.replace('(','')
fandango['YEAR']=fandango['YEAR'].str.replace(')','')
fandango.head()
| FILM | STARS | RATING | VOTES | YEAR | |
|---|---|---|---|---|---|
| 0 | Fifty Shades of Grey (2015) | 4.0 | 3.9 | 34846 | 2015 |
| 1 | Jurassic World (2015) | 4.5 | 4.5 | 34390 | 2015 |
| 2 | American Sniper (2015) | 5.0 | 4.8 | 34085 | 2015 |
| 3 | Furious 7 (2015) | 5.0 | 4.8 | 33538 | 2015 |
| 4 | Inside Out (2015) | 4.5 | 4.5 | 15749 | 2015 |
TASK: How many movies are in the Fandango DataFrame per year?
#CODE HERE
fandango['YEAR'].value_counts()
2015 478 2014 23 2012 1 1964 1 2016 1 Name: YEAR, dtype: int64
TASK: Visualize the count of movies per year with a plot:
#CODE HERE
plt.figure(figsize=(7,5))
sns.countplot(x='YEAR',data=fandango)
<AxesSubplot:xlabel='YEAR', ylabel='count'>
TASK: What are the 10 movies with the highest number of votes?
#CODE HERE
fandango['VOTES'].nlargest(10).index
fandango.iloc[0:10]
| FILM | STARS | RATING | VOTES | YEAR | |
|---|---|---|---|---|---|
| 0 | Fifty Shades of Grey (2015) | 4.0 | 3.9 | 34846 | 2015 |
| 1 | Jurassic World (2015) | 4.5 | 4.5 | 34390 | 2015 |
| 2 | American Sniper (2015) | 5.0 | 4.8 | 34085 | 2015 |
| 3 | Furious 7 (2015) | 5.0 | 4.8 | 33538 | 2015 |
| 4 | Inside Out (2015) | 4.5 | 4.5 | 15749 | 2015 |
| 5 | The Hobbit: The Battle of the Five Armies (2014) | 4.5 | 4.3 | 15337 | 2014 |
| 6 | Kingsman: The Secret Service (2015) | 4.5 | 4.2 | 15205 | 2015 |
| 7 | Minions (2015) | 4.0 | 4.0 | 14998 | 2015 |
| 8 | Avengers: Age of Ultron (2015) | 5.0 | 4.5 | 14846 | 2015 |
| 9 | Into the Woods (2014) | 3.5 | 3.4 | 13055 | 2014 |
TASK: How many movies have zero votes?
#CODE HERE
fandango[fandango['VOTES']==0].count()['FILM']
69
TASK: Create DataFrame of only reviewed films by removing any films that have zero votes.
#CODE HERE
df=fandango
df['zero_votes']=df['VOTES']==0
df=df.set_index('zero_votes')
df=df.drop(True)
df=df.reset_index()
df=df.drop('zero_votes',axis=1)
df['VOTES'].min()
1
As noted in the article, due to HTML and star rating displays, the true user rating may be slightly different than the rating shown to a user. Let's visualize this difference in distributions.
TASK: Create a KDE plot (or multiple kdeplots) that displays the distribution of ratings that are displayed (STARS) versus what the true rating was from votes (RATING). Clip the KDEs to 0-5.
#CODE HERE
plt.figure(figsize=(12,5))
sns.kdeplot(x='RATING',data=fandango,clip=[0,5],label='True Rating',bw_adjust=0.5,fill=True)
sns.kdeplot(x='STARS',data=fandango,clip=[0,5],label='STARS Displayed',bw_adjust=0.5,fill=True)
plt.xlim(0.9,5.9)
plt.ylim(0,0.6)
plt.legend(bbox_to_anchor=(1.3,0.5))
<matplotlib.legend.Legend at 0x23e74978948>
TASK: Let's now actually quantify this discrepancy. Create a new column of the different between STARS displayed versus true RATING. Calculate this difference with STARS-RATING and round these differences to the nearest decimal point.
#CODE HERE
fandango['STARS_DIFF']=np.round(fandango['STARS']-fandango['RATING'],1)
fandango=fandango.drop('zero_votes',axis=1)
fandango
| FILM | STARS | RATING | VOTES | YEAR | STARS_DIFF | |
|---|---|---|---|---|---|---|
| 0 | Fifty Shades of Grey (2015) | 4.0 | 3.9 | 34846 | 2015 | 0.1 |
| 1 | Jurassic World (2015) | 4.5 | 4.5 | 34390 | 2015 | 0.0 |
| 2 | American Sniper (2015) | 5.0 | 4.8 | 34085 | 2015 | 0.2 |
| 3 | Furious 7 (2015) | 5.0 | 4.8 | 33538 | 2015 | 0.2 |
| 4 | Inside Out (2015) | 4.5 | 4.5 | 15749 | 2015 | 0.0 |
| ... | ... | ... | ... | ... | ... | ... |
| 499 | Valiyavan (2015) | 0.0 | 0.0 | 0 | 2015 | 0.0 |
| 500 | WWE SummerSlam 2015 (2015) | 0.0 | 0.0 | 0 | 2015 | 0.0 |
| 501 | Yagavarayinum Naa Kaakka (2015) | 0.0 | 0.0 | 0 | 2015 | 0.0 |
| 502 | Yesterday, Today and Tomorrow (1964) | 0.0 | 0.0 | 0 | 1964 | 0.0 |
| 503 | Zarafa (2012) | 0.0 | 0.0 | 0 | 2012 | 0.0 |
504 rows × 6 columns
TASK: Create a count plot to display the number of times a certain difference occurs:
#CODE HERE
plt.figure(figsize=(12,5),dpi=200)
sns.countplot(x='STARS_DIFF',data=fandango,palette='magma')
<AxesSubplot:xlabel='STARS_DIFF', ylabel='count'>
TASK: We can see from the plot that one movie was displaying over a 1 star difference than its true rating! What movie had this close to 1 star differential?
#CODE HERE
fandango[fandango['STARS_DIFF']==1]
| FILM | STARS | RATING | VOTES | YEAR | STARS_DIFF | |
|---|---|---|---|---|---|---|
| 381 | Turbo Kid (2015) | 5.0 | 4.0 | 2 | 2015 | 1.0 |
Let's now compare the scores from Fandango to other movies sites and see how they compare.
TASK: Read in the "all_sites_scores.csv" file by running the cell below
all_sites = pd.read_csv("all_sites_scores.csv")
TASK: Explore the DataFrame columns, info, description.
all_sites.head()
| FILM | RottenTomatoes | RottenTomatoes_User | Metacritic | Metacritic_User | IMDB | Metacritic_user_vote_count | IMDB_user_vote_count | |
|---|---|---|---|---|---|---|---|---|
| 0 | Avengers: Age of Ultron (2015) | 74 | 86 | 66 | 7.1 | 7.8 | 1330 | 271107 |
| 1 | Cinderella (2015) | 85 | 80 | 67 | 7.5 | 7.1 | 249 | 65709 |
| 2 | Ant-Man (2015) | 80 | 90 | 64 | 8.1 | 7.8 | 627 | 103660 |
| 3 | Do You Believe? (2015) | 18 | 84 | 22 | 4.7 | 5.4 | 31 | 3136 |
| 4 | Hot Tub Time Machine 2 (2015) | 14 | 28 | 29 | 3.4 | 5.1 | 88 | 19560 |
all_sites.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 146 entries, 0 to 145 Data columns (total 8 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 FILM 146 non-null object 1 RottenTomatoes 146 non-null int64 2 RottenTomatoes_User 146 non-null int64 3 Metacritic 146 non-null int64 4 Metacritic_User 146 non-null float64 5 IMDB 146 non-null float64 6 Metacritic_user_vote_count 146 non-null int64 7 IMDB_user_vote_count 146 non-null int64 dtypes: float64(2), int64(5), object(1) memory usage: 9.2+ KB
all_sites.describe()
| RottenTomatoes | RottenTomatoes_User | Metacritic | Metacritic_User | IMDB | Metacritic_user_vote_count | IMDB_user_vote_count | |
|---|---|---|---|---|---|---|---|
| count | 146.000000 | 146.000000 | 146.000000 | 146.000000 | 146.000000 | 146.000000 | 146.000000 |
| mean | 60.849315 | 63.876712 | 58.808219 | 6.519178 | 6.736986 | 185.705479 | 42846.205479 |
| std | 30.168799 | 20.024430 | 19.517389 | 1.510712 | 0.958736 | 316.606515 | 67406.509171 |
| min | 5.000000 | 20.000000 | 13.000000 | 2.400000 | 4.000000 | 4.000000 | 243.000000 |
| 25% | 31.250000 | 50.000000 | 43.500000 | 5.700000 | 6.300000 | 33.250000 | 5627.000000 |
| 50% | 63.500000 | 66.500000 | 59.000000 | 6.850000 | 6.900000 | 72.500000 | 19103.000000 |
| 75% | 89.000000 | 81.000000 | 75.000000 | 7.500000 | 7.400000 | 168.500000 | 45185.750000 |
| max | 100.000000 | 94.000000 | 94.000000 | 9.600000 | 8.600000 | 2375.000000 | 334164.000000 |
Let's first take a look at Rotten Tomatoes. RT has two sets of reviews, their critics reviews (ratings published by official critics) and user reviews.
TASK: Create a scatterplot exploring the relationship between RT Critic reviews and RT User reviews.
# CODE HERE
plt.figure(figsize=(12,5),dpi=200)
plt.xlim(0,100)
plt.ylim(0,100)
sns.scatterplot(x='RottenTomatoes',y='RottenTomatoes_User',data=all_sites)
<AxesSubplot:xlabel='RottenTomatoes', ylabel='RottenTomatoes_User'>
Let's quantify this difference by comparing the critics ratings and the RT User ratings. We will calculate this with RottenTomatoes-RottenTomatoes_User. Note: Rotten_Diff here is Critics - User Score. So values closer to 0 means aggrement between Critics and Users. Larger positive values means critics rated much higher than users. Larger negative values means users rated much higher than critics.
TASK: Create a new column based off the difference between critics ratings and users ratings for Rotten Tomatoes. Calculate this with RottenTomatoes-RottenTomatoes_User
#CODE HERE
all_sites['Rotten_Diff']=all_sites['RottenTomatoes']-all_sites['RottenTomatoes_User']
Let's now compare the overall mean difference. Since we're dealing with differences that could be negative or positive, first take the absolute value of all the differences, then take the mean. This would report back on average to absolute difference between the critics rating versus the user rating.
TASK: Calculate the Mean Absolute Difference between RT scores and RT User scores as described above.
# CODE HERE
np.abs(all_sites['Rotten_Diff']).mean()
15.095890410958905
TASK: Plot the distribution of the differences between RT Critics Score and RT User Score. There should be negative values in this distribution plot. Feel free to use KDE or Histograms to display this distribution.
#CODE HERE
plt.figure(figsize=(17,10),dpi=500)
sns.displot(x='Rotten_Diff',data=all_sites,kde=True)
<seaborn.axisgrid.FacetGrid at 0x23e74be4948>
<Figure size 8500x5000 with 0 Axes>
TASK: Now create a distribution showing the absolute value difference between Critics and Users on Rotten Tomatoes.
#CODE HERE
plt.figure(figsize=(20,7),dpi=200)
val=all_sites['Rotten_Diff'].abs()
sns.displot(x=val,data=all_sites,bins=20,kde=True)
<seaborn.axisgrid.FacetGrid at 0x23e74bd1e88>
<Figure size 4000x1400 with 0 Axes>
Let's find out which movies are causing the largest differences. First, show the top 5 movies with the largest negative difference between Users and RT critics. Since we calculated the difference as Critics Rating - Users Rating, then large negative values imply the users rated the movie much higher on average than the critics did.
TASK: What are the top 5 movies users rated higher than critics on average:
# CODE HERE
f=all_sites.nsmallest(n=5,columns='Rotten_Diff')
f[['FILM','Rotten_Diff']]
| FILM | Rotten_Diff | |
|---|---|---|
| 3 | Do You Believe? (2015) | -66 |
| 85 | Little Boy (2015) | -61 |
| 105 | Hitman: Agent 47 (2015) | -42 |
| 134 | The Longest Ride (2015) | -42 |
| 125 | The Wedding Ringer (2015) | -39 |
TASK: Now show the top 5 movies critics scores higher than users on average.
# CODE HERE
f=all_sites.nlargest(n=5,columns='Rotten_Diff')
f[['FILM','Rotten_Diff']]
| FILM | Rotten_Diff | |
|---|---|---|
| 69 | Mr. Turner (2014) | 42 |
| 112 | It Follows (2015) | 31 |
| 115 | While We're Young (2015) | 31 |
| 37 | Welcome to Me (2015) | 24 |
| 40 | I'll See You In My Dreams (2015) | 24 |
Now let's take a quick look at the ratings from MetaCritic. Metacritic also shows an average user rating versus their official displayed rating.
TASK: Display a scatterplot of the Metacritic Rating versus the Metacritic User rating.
# CODE HERE
plt.figure(figsize=(12,5),dpi=200)
sns.scatterplot(x='Metacritic',y='Metacritic_User',data=all_sites)
plt.xlim(0,100)
plt.ylim(0,10)
(0.0, 10.0)
Finally let's explore IMDB. Notice that both Metacritic and IMDB report back vote counts. Let's analyze the most popular movies.
TASK: Create a scatterplot for the relationship between vote counts on MetaCritic versus vote counts on IMDB.
#CODE HERE
plt.figure(figsize=(12,5),dpi=200)
sns.scatterplot(x='Metacritic_user_vote_count',y='IMDB_user_vote_count',data=all_sites)
plt.xlim(0,2000)
plt.ylim(0,350000)
(0.0, 350000.0)
Notice there are two outliers here. The movie with the highest vote count on IMDB only has about 500 Metacritic ratings. What is this movie?
TASK: What movie has the highest IMDB user vote count?
#CODE HERE
all_sites.nlargest(n=1,columns='IMDB_user_vote_count')
| FILM | RottenTomatoes | RottenTomatoes_User | Metacritic | Metacritic_User | IMDB | Metacritic_user_vote_count | IMDB_user_vote_count | Rotten_Diff | |
|---|---|---|---|---|---|---|---|---|---|
| 14 | The Imitation Game (2014) | 90 | 92 | 73 | 8.2 | 8.1 | 566 | 334164 | -2 |
TASK: What movie has the highest Metacritic User Vote count?
#CODE HERE
all_sites.nlargest(n=1,columns='Metacritic_user_vote_count')
| FILM | RottenTomatoes | RottenTomatoes_User | Metacritic | Metacritic_User | IMDB | Metacritic_user_vote_count | IMDB_user_vote_count | Rotten_Diff | |
|---|---|---|---|---|---|---|---|---|---|
| 88 | Mad Max: Fury Road (2015) | 97 | 88 | 89 | 8.7 | 8.3 | 2375 | 292023 | 9 |
Finally let's begin to explore whether or not Fandango artificially displays higher ratings than warranted to boost ticket sales.
TASK: Combine the Fandango Table with the All Sites table. Not every movie in the Fandango table is in the All Sites table, since some Fandango movies have very little or no reviews. We only want to compare movies that are in both DataFrames, so do an inner merge to merge together both DataFrames based on the FILM columns.
#CODE HERE
pd.merge(all_sites,fandango,how='inner',on='FILM').info()
<class 'pandas.core.frame.DataFrame'> Int64Index: 145 entries, 0 to 144 Data columns (total 14 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 FILM 145 non-null object 1 RottenTomatoes 145 non-null int64 2 RottenTomatoes_User 145 non-null int64 3 Metacritic 145 non-null int64 4 Metacritic_User 145 non-null float64 5 IMDB 145 non-null float64 6 Metacritic_user_vote_count 145 non-null int64 7 IMDB_user_vote_count 145 non-null int64 8 Rotten_Diff 145 non-null int64 9 STARS 145 non-null float64 10 RATING 145 non-null float64 11 VOTES 145 non-null int64 12 YEAR 145 non-null object 13 STARS_DIFF 145 non-null float64 dtypes: float64(5), int64(7), object(2) memory usage: 17.0+ KB
all_tables=pd.merge(fandango,all_sites,how='inner',on='FILM')
all_tables
| FILM | STARS | RATING | VOTES | YEAR | STARS_DIFF | RottenTomatoes | RottenTomatoes_User | Metacritic | Metacritic_User | IMDB | Metacritic_user_vote_count | IMDB_user_vote_count | Rotten_Diff | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Fifty Shades of Grey (2015) | 4.0 | 3.9 | 34846 | 2015 | 0.1 | 25 | 42 | 46 | 3.2 | 4.2 | 778 | 179506 | -17 |
| 1 | Jurassic World (2015) | 4.5 | 4.5 | 34390 | 2015 | 0.0 | 71 | 81 | 59 | 7.0 | 7.3 | 1281 | 241807 | -10 |
| 2 | American Sniper (2015) | 5.0 | 4.8 | 34085 | 2015 | 0.2 | 72 | 85 | 72 | 6.6 | 7.4 | 850 | 251856 | -13 |
| 3 | Furious 7 (2015) | 5.0 | 4.8 | 33538 | 2015 | 0.2 | 81 | 84 | 67 | 6.8 | 7.4 | 764 | 207211 | -3 |
| 4 | Inside Out (2015) | 4.5 | 4.5 | 15749 | 2015 | 0.0 | 98 | 90 | 94 | 8.9 | 8.6 | 807 | 96252 | 8 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 140 | Kumiko, The Treasure Hunter (2015) | 3.5 | 3.5 | 41 | 2015 | 0.0 | 87 | 63 | 68 | 6.4 | 6.7 | 19 | 5289 | 24 |
| 141 | The Diary of a Teenage Girl (2015) | 4.0 | 3.6 | 38 | 2015 | 0.4 | 95 | 81 | 87 | 6.3 | 7.0 | 18 | 1107 | 14 |
| 142 | The Wrecking Crew (2015) | 4.5 | 4.2 | 38 | 2015 | 0.3 | 93 | 84 | 67 | 7.0 | 7.8 | 4 | 732 | 9 |
| 143 | Tangerine (2015) | 4.0 | 3.9 | 36 | 2015 | 0.1 | 95 | 86 | 86 | 7.3 | 7.4 | 14 | 696 | 9 |
| 144 | Maps to the Stars (2015) | 3.5 | 3.1 | 35 | 2015 | 0.4 | 60 | 46 | 67 | 5.8 | 6.3 | 46 | 22440 | 14 |
145 rows × 14 columns
Notice that RT,Metacritic, and IMDB don't use a score between 0-5 stars like Fandango does. In order to do a fair comparison, we need to normalize these values so they all fall between 0-5 stars and the relationship between reviews stays the same.
TASK: Create new normalized columns for all ratings so they match up within the 0-5 star range shown on Fandango. There are many ways to do this.
Hint link: https://stackoverflow.com/questions/26414913/normalize-columns-of-pandas-data-frame
Easier Hint:
Keep in mind, a simple way to convert ratings:
# CODE HERE
all_tables['RT_Norm']=np.round(all_tables['RottenTomatoes']/20,1)
all_tables['RTU_Norm']=np.round(all_tables['RottenTomatoes_User']/20,1)
all_tables['Meta_Norm']=np.round(all_tables['Metacritic']/20,1)
all_tables['Meta_U_Norm']=np.round(all_tables['Metacritic_User']/2,1)
all_tables['IMDB_Norm']=np.round(all_tables['IMDB']/2,1)
all_tables.head()
| FILM | STARS | RATING | VOTES | YEAR | STARS_DIFF | RottenTomatoes | RottenTomatoes_User | Metacritic | Metacritic_User | IMDB | Metacritic_user_vote_count | IMDB_user_vote_count | Rotten_Diff | RT_Norm | RTU_Norm | Meta_Norm | Meta_U_Norm | IMDB_Norm | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Fifty Shades of Grey (2015) | 4.0 | 3.9 | 34846 | 2015 | 0.1 | 25 | 42 | 46 | 3.2 | 4.2 | 778 | 179506 | -17 | 1.2 | 2.1 | 2.3 | 1.6 | 2.1 |
| 1 | Jurassic World (2015) | 4.5 | 4.5 | 34390 | 2015 | 0.0 | 71 | 81 | 59 | 7.0 | 7.3 | 1281 | 241807 | -10 | 3.6 | 4.0 | 3.0 | 3.5 | 3.6 |
| 2 | American Sniper (2015) | 5.0 | 4.8 | 34085 | 2015 | 0.2 | 72 | 85 | 72 | 6.6 | 7.4 | 850 | 251856 | -13 | 3.6 | 4.2 | 3.6 | 3.3 | 3.7 |
| 3 | Furious 7 (2015) | 5.0 | 4.8 | 33538 | 2015 | 0.2 | 81 | 84 | 67 | 6.8 | 7.4 | 764 | 207211 | -3 | 4.0 | 4.2 | 3.4 | 3.4 | 3.7 |
| 4 | Inside Out (2015) | 4.5 | 4.5 | 15749 | 2015 | 0.0 | 98 | 90 | 94 | 8.9 | 8.6 | 807 | 96252 | 8 | 4.9 | 4.5 | 4.7 | 4.4 | 4.3 |
TASK: Now create a norm_scores DataFrame that only contains the normalizes ratings. Include both STARS and RATING from the original Fandango table.
#CODE HERE
df1=fandango[['STARS','RATING']]
df2=all_tables[['RT_Norm','RTU_Norm','Meta_Norm','Meta_U_Norm','IMDB_Norm']]
norm_scores=pd.concat([df1,df2],1)
norm_scores.head()
| STARS | RATING | RT_Norm | RTU_Norm | Meta_Norm | Meta_U_Norm | IMDB_Norm | |
|---|---|---|---|---|---|---|---|
| 0 | 4.0 | 3.9 | 1.2 | 2.1 | 2.3 | 1.6 | 2.1 |
| 1 | 4.5 | 4.5 | 3.6 | 4.0 | 3.0 | 3.5 | 3.6 |
| 2 | 5.0 | 4.8 | 3.6 | 4.2 | 3.6 | 3.3 | 3.7 |
| 3 | 5.0 | 4.8 | 4.0 | 4.2 | 3.4 | 3.4 | 3.7 |
| 4 | 4.5 | 4.5 | 4.9 | 4.5 | 4.7 | 4.4 | 4.3 |
Now the moment of truth! Does Fandango display abnormally high ratings? We already know it pushs displayed RATING higher than STARS, but are the ratings themselves higher than average?
TASK: Create a plot comparing the distributions of normalized ratings across all sites. There are many ways to do this, but explore the Seaborn KDEplot docs for some simple ways to quickly show this. Don't worry if your plot format does not look exactly the same as ours, as long as the differences in distribution are clear.
Quick Note if you have issues moving the legend for a seaborn kdeplot: https://github.com/mwaskom/seaborn/issues/2280
#CODE HERE
norm_scores
| STARS | RATING | RT_Norm | RTU_Norm | Meta_Norm | Meta_U_Norm | IMDB_Norm | |
|---|---|---|---|---|---|---|---|
| 0 | 4.0 | 3.9 | 1.2 | 2.1 | 2.3 | 1.6 | 2.1 |
| 1 | 4.5 | 4.5 | 3.6 | 4.0 | 3.0 | 3.5 | 3.6 |
| 2 | 5.0 | 4.8 | 3.6 | 4.2 | 3.6 | 3.3 | 3.7 |
| 3 | 5.0 | 4.8 | 4.0 | 4.2 | 3.4 | 3.4 | 3.7 |
| 4 | 4.5 | 4.5 | 4.9 | 4.5 | 4.7 | 4.4 | 4.3 |
| ... | ... | ... | ... | ... | ... | ... | ... |
| 499 | 0.0 | 0.0 | NaN | NaN | NaN | NaN | NaN |
| 500 | 0.0 | 0.0 | NaN | NaN | NaN | NaN | NaN |
| 501 | 0.0 | 0.0 | NaN | NaN | NaN | NaN | NaN |
| 502 | 0.0 | 0.0 | NaN | NaN | NaN | NaN | NaN |
| 503 | 0.0 | 0.0 | NaN | NaN | NaN | NaN | NaN |
504 rows × 7 columns
def move_legend(ax, new_loc, **kws):
old_legend = ax.legend_
handles = old_legend.legendHandles
labels = [t.get_text() for t in old_legend.get_texts()]
title = old_legend.get_title().get_text()
ax.legend(handles, labels, loc=new_loc, title=title, **kws)
#plt.figure(figsize=(13,11),dpi=200)
#sns.kdeplot(data=norm_scores,x='STARS',fill=True,clip=[0,5],label='STARS')
#sns.kdeplot(data=norm_scores,x='RATING',fill=True,clip=[0,5],label='RATING')
#sns.kdeplot(data=norm_scores,x='RT_Norm',fill=True,clip=[0,5],label='RT_Norm')
#sns.kdeplot(data=norm_scores,x='RTU_Norm',fill=True,clip=[0,5],label='RTU_Norm')
#sns.kdeplot(data=norm_scores,x='Meta_Norm',fill=True,clip=[0,5],label='Meta_Norm')
#sns.kdeplot(data=norm_scores,x='Meta_U_Norm',fill=True,clip=[0,5],label='Meta_U_Norm')
#sns.kdeplot(data=norm_scores,x='IMDB_Norm',fill=True,clip=[0,5],label='IMDB_Norm')
fig,ax=plt.subplots(figsize=(15,6),dpi=100)
sns.kdeplot(data=norm_scores,fill=True,clip=[0,5],ax=ax)
move_legend(ax, "upper left")
Clearly Fandango has an uneven distribution. We can also see that RT critics have the most uniform distribution. Let's directly compare these two.
TASK: Create a KDE plot that compare the distribution of RT critic ratings against the STARS displayed by Fandango.
#CODE HERE
fig, ax = plt.subplots(figsize=(15,6),dpi=150)
sns.kdeplot(data=norm_scores[['RT_Norm','STARS']],clip=[0,5],fill=True,palette='Set1',ax=ax)
<AxesSubplot:ylabel='Density'>
OPTIONAL TASK: Create a histplot comparing all normalized scores.
#CODE HERE
plt.figure(figsize=(15,6),dpi=100)
sns.histplot(data=norm_scores)
move_legend(ax, "upper left")
TASK: Clearly Fandango is rating movies much higher than other sites, especially considering that it is then displaying a rounded up version of the rating. Let's examine the top 10 worst movies. Based off the Rotten Tomatoes Critic Ratings, what are the top 10 lowest rated movies? What are the normalized scores across all platforms for these movies? You may need to add the FILM column back in to your DataFrame of normalized scores to see the results.
# CODE HERE
norm_scores['FILM']=all_tables['FILM']
d=norm_scores.nsmallest(n=10,columns='RT_Norm')
d
| STARS | RATING | RT_Norm | RTU_Norm | Meta_Norm | Meta_U_Norm | IMDB_Norm | FILM | |
|---|---|---|---|---|---|---|---|---|
| 49 | 4.5 | 4.1 | 0.2 | 1.8 | 0.6 | 1.2 | 2.2 | Paul Blart: Mall Cop 2 (2015) |
| 25 | 4.5 | 4.2 | 0.4 | 2.3 | 1.3 | 2.3 | 3.0 | Taken 3 (2015) |
| 28 | 4.5 | 4.1 | 0.4 | 1.0 | 1.4 | 1.2 | 2.0 | Fantastic Four (2015) |
| 54 | 4.5 | 4.1 | 0.4 | 1.8 | 1.6 | 1.8 | 2.4 | Hot Pursuit (2015) |
| 84 | 5.0 | 4.6 | 0.4 | 2.4 | 1.4 | 1.6 | 3.0 | Hitman: Agent 47 (2015) |
| 50 | 4.0 | 3.5 | 0.5 | 1.8 | 1.5 | 2.8 | 2.3 | The Boy Next Door (2015) |
| 77 | 4.0 | 4.0 | 0.6 | 1.8 | 1.5 | 2.0 | 2.8 | Seventh Son (2015) |
| 78 | 3.0 | 3.0 | 0.6 | 1.5 | 1.4 | 1.6 | 2.8 | Mortdecai (2015) |
| 83 | 3.0 | 2.9 | 0.6 | 1.7 | 1.6 | 2.5 | 2.8 | Sinister 2 (2015) |
| 87 | 3.5 | 3.3 | 0.6 | 1.4 | 1.6 | 1.9 | 2.7 | Unfinished Business (2015) |
FINAL TASK: Visualize the distribution of ratings across all sites for the top 10 worst movies.
# CODE HERE
fig,ax=plt.subplots(figsize=(15,6),dpi=150)
sns.kdeplot(data=d,clip=[0,5],ax=ax,fill=True)
<AxesSubplot:ylabel='Density'>
Final thoughts: Wow! Fandango is showing around 3-4 star ratings for films that are clearly bad! Notice the biggest offender, Taken 3!. Fandango is displaying 4.5 stars on their site for a film with an average rating of 1.86 across the other platforms!
norm_scores.iloc[25]
STARS 4.5 RATING 4.2 RT_Norm 0.4 RTU_Norm 2.3 Meta_Norm 1.3 Meta_U_Norm 2.3 IMDB_Norm 3 FILM Taken 3 (2015) Name: 25, dtype: object
0.4+2.3+1.3+2.3+3
9.3
9.3/5
1.86