How do animated movie studios hire top notch executive producers? And what types of experiences are considered valuable within this industry? I analyzed movie credits to answer these questions and understand the career paths of first-time executive producers.

For animated movies released since 1980, I determined who the executive producers were by scraping data from IMDb. For each of the producers, I created a map of their career to identify when they became an "executive producer," and what that person worked on prior to executive-producing. Not surprisingly, a lot of executive producers had experience writing movie scripts:

The wide spread in IMDb rating for producers with no writing experience suggests that these people might be riskier bets as executive producers, if you are hoping to make a high rated movie. In contrast, IMDb ratings are above 4.0 for movies whose executive producer had at least one writing credit in the past. The plot above shows that there are less and less data available describing first-time executive producers with increasing amounts of writing experience. What if you want to use this dataset to compare a person who has helped write 1 movie script versus another person who has helped write 10 movies? The dataset is small, and so it becomes increasingly difficult to glean information regarding IMDb rating vs writing experience for producers with greater than 1 writing credit. Assigning the data to 2 distributions based on a distinct cutoff value for "writing experience" can help one discern the impact of writing experience on IMDb rating. Shown below are the same dataset separated into 2 distributions. The plot on the left shows the distributions based on a cutoff value of 1 writing credit, while the plot on the right shows the distributions based on a cutoff value of 9 writing credits.

In both of the plots above, the green distributions are shifted slightly to the right, indicating that writing experience is correlated with higher IMDb rating. The distributions overlap each other to a greater degree in the left plot versus the right plot. In other words, people with at least 9 writing credits have been hired on as executive producer to help make movies that are noticeably higher quality than those people with less than 9 writing credits. The average IMDb rating (x̄) for each distribution can be calculated to compare the distributions more precisely, and statistics can be used to support the significance of these findings. In the plot below, x̄₂ is the average IMDb rating of the green distribution and x̄₁ is the average IMDb rating of the blue distribution. Their difference (x̄₂ - x̄₁) is shown in black on the left axis, while the statistical power of the analysis is shown in red on the right axis. (Although I used power analysis to derive insights from this data, please read my "caveats" below regarding why this approach might not be entirely correct). Statistical power was determined using a significance level of 0.05, and the target difference between x̄₂ and x̄₁ was 1 IMDb rating. In other words, I want to be 95% confident in my findings and also want to see a difference of at least 1 IMDb rating when comparing the distributions (1 IMDb rating is also comparable to the pooled standard deviation for the distributions).

As the number of writing credits increases (x-axis value), x̄₂ - x̄₁ increases, however the statistical power decreases. This result is expected because there is less and less data available as number of writing credits increases. If we consider any findings below a statistical power of 0.8 to be below our tolerance threshold, then we can conclude that having more writing experience (up to 9 movie credits) has historically described the career paths of more successful first-time executive producers. I hesitate to make a strong recommendation for a potential executive producer with 9 movie credits compared to someone with only 8 movie credits. My intention for running a power analysis was not to make such strong cutoffs, but rather to pick out relevant features in a person's career path.

The analysis described above was applied to animation experience, production experience (people who worked on the production team but were not yet "executive producer"), director experience, and visual effects experience:

The plots above show that people with animation experience (up to 6 animation credits) have went on to become successful executive producers. The impact on IMDb rating is greater for people with writing experience than for people with animation experience, although both contribute to success. All other areas which were analyzed (production, direction, and visual effects) do not appear to be valuable experiences for a successful executive producer to have. Importantly, the difference in average IMDb ratings for the distributions were quite small and not dependent on the x-axis (where x-axis represents the number of movies where that particular skill was gained).

The accompanying distributions for the plots above are shown below. Note that it's difficult to glean any insights from the distributions below without looking at the results of the power analysis.

My primary motivation for doing this analysis is simply an interest in people analytics. Most people stop watching movies once the credits start rolling, but I see movie credits as the perfect opportunity to quantify talent within the movie industry! I wanted to clarify how people move up the ladder within the animated movie industry, and to understand what makes executive producers so special. When I did a general online search for "executive producer," I quickly discovered that it's not a very well-defined role, and nobody offers up suggestions for how to become one. Sometimes executive producers manage budgets or timelines, and sometimes they are granted decision-making power for a movie. While it sounds like executive producers must posess business savvy, the analysis showed that executive producers actually have backgrounds in art, writing, and animation (not management or business).

There are a lot of limitations and caveats to this analysis. One obvious problem is that movies with higher budgets can afford to hire more experienced people, and these high budget movies might have higher IMDb ratings. If this were true, it may not be appropriate to apply a power analysis to draw any conclusions. In fact, scientists should always be critical of applying power analsis on retrospective data, rather than on a well-controlled experiment lacking confounding underlying features. For the most part, I used the power analysis to pick out valuable work experiences and to negate other roles (director, producer, visual effects).

Another problem with the analysis is that industries are always changing. There may have been some writers who became successful movie producers in the past, but that doesn't mean someone can move into production simply because they are a writer. I did not quantify how many writers exist within the animated movie industry and what percentage of them move on to become producers. I recognize all these caveats, yet still chose to publish my analysis because I think that similar analyses can be performed within different industries, and that this project serves as a good example of how to glean insights from small datasets when predictive modeling is not possible.