First off, a huge thank you to Shawn Anthony (@desolutionist ) for watching replays after the tournament so that this analysis was possible. Neither Matt nor I played in the tournament and this year Wizards restricted watching replays to partipants in the tournament.

52 players entered in the Monthly Vintage P9 Challenge, which was timeshifted to the July 2016 cardpool. Congratulations to cageykg for taking down the tournament!

Top 16 Decklists:
http://magic.wizards.com/en/articles/archive/mtgo-standings/vintage-premier-2016-09-25

Top 8:
cageykg - Artifact Heavy Thoughtcast Mentor
jcardshark - Jeskai Mentor
BalloonBrigade - UR Delver
ecobaronen - Doomsday
Boin - White Eldrazi
Montolio - Nahiri Odd Oath
ziguyan - 5C Dredge
stsung - Grixis Therapy

Remaining 4-2s:
JdPhoenix - Blue Moon
ravager101 - DPS
MuseofAnger - 5C Dredge
Thiim - Red Shops
oddseidank - Pitch Dredge with DD Transform
UnrestrictBrainstorm - TKS/Smasher Shops
RegB - DPS
desolutionist - TNT Shops
Preordain - Jeskai Nahiri Gush

Metagame Breakdown:

To try to fit everything into one narrative, it looks as though the decks that did best this P9 were those which players were less prepared for. The narrative for months has been how Ravager TKS Shops and Jeskai Mentor/Grixis Therapy are the two best decks. Here we see Oath, Big Blue, Eldrazi, and non-Ravager shops decks perform well. I suspect that most people became too narrowed in on the "best decks" and gave up enough general percentage points. If anyone has other thoughts, please share.

Link to googledoc:
https://docs.google.com/spreadsheets/d/1ZdqPwVGPKwVRp-XXMcyWzzhrrPmGyi-x2hepryBxjno/edit?usp=sharing

Let me know if anyone spots errors, either in the calculations or deck classifications.

Congrats to Cageykg for taking it down. Well done.
As always huge ty to Ryan for organizing the data and to Shawn for watching a ton of replays.
p.s Shawn your deck was just super sweet. Survival is the bee's knee's.

Gush decks were half the top 8, 3/4 the top 4, and but no worries because the archetype had sub-50% win rate...

Thanks to all involved for doing this breakdown again - it always makes for fascinating reading. Sadly, having missed the last one due to a holiday, I couldn't really be arsed with this one - and it seems other Vintage players are voting with their feet, too. I'm not in love with the format right now and the inability to actually play with a relevant card pool pretty much sealed my disinterest in the P9 Challenge. Hopefully, mtgo will be up to speed for the next one - but I worry the event/format may have permanently shed players by then, bored by the choice of playing either Gush or a deck that feels hideously inconsistent in comparison. Hopefully, I'm wrong and EW will pique mtgo players' interest and we'll clamber back up to 100 players a month for these events.

In the meantime, congrats to all the top 8, really look forward to seeing some of the more unusual lists. And thanks to Ryan, Matt and Shawn for their heroic data mining. If only WOTC were such good custodians of the format.

@ChubbyRain Other than the Jeskai Mentor deck, most of the best performing gush decks were a bit off the beaten path. To be clear, I wasn't stating that Gush and Shops aren't good, but rather that the better performing lists were non-standard.

Also, implying the winrate is nearly useless seems a bit silly to me. It really depends on how you want to use the data. If you want to know what you're likely to face at the top tables, obviously using "gush has a sub 50% winrate" to conclude "I don't need to worry about it after the second round" or "gush isn't warping the metagame" is not good logic.

@diophan Comment wasn't directed at you, Ryan. It's become a familiar refrain in these types of analyses, so I thought I'd beat others to the punch.

Also, I wouldn't say winrate is nearly useless. I would say that winrate (for a single event) is completely useless. The sample size is not large enough to make up for deck heterogeneity, player skill, and the variance inherent to Magic.

@ChubbyRain Its not even just that its from a single event. Its that a focus on win % ignores the metagame % entirely. The 3 most popular decks at this event were also the 3 worst according to win % (not counting other).

Lists up here: http://magic.wizards.com/en/articles/archive/mtgo-standings/vintage-premier-2016-09-25

thanks for your time and hard work. it is nice to see what decks people played. I was wondering how many people were on storm or doomsday. it seems that more than I though.

Win percentage against the field aside, Gush was disproportionately represented in the top 8 anyway if you consider that it was (only?) 32.7% of the field while making up 50% of the top8. When people were gunning for it that is.

I mean the takeaway from the win% + metagame representation seems pretty clear to me. If you're not going to pilot a gush deck significantly better than the average player you should play a different archetype. How is that a useless takeaway? I really disagree with the sentiment that people can draw the wrong conclusions (or conclusions one disagrees with) from derived information so we should disparage that derived information. The sample for each of these tournaments is small, but look at all the online and paper results from the last 3 to 6 months and the story for gush is amazingly constant.

@diophan If we are just trying to describe what occurred during the tournament the win% is fine. When making a decision like what to play based on empirical data, you'd want the sample size to be sufficient. Decision rules based on numbers always require a high sample size to reduce your type I and type II error rates.

In June and March, Gush's win% was well above 50% and in all of the other months its been hovering around 50%, except August. I'd gather that aggregating them together, it would be at least 50%.

The Big Blue decks performed well today, biased by a sample size of 3 and one of them being the winner, whereas it was atrocious in May and June, sub 30%.

@vaughnbros Obviously I agree that concluding that Big Blue is amazing from this is nonsense. I agree that gush's aggregate winrate is around 50%, not sure if it's above or below without actually doing it. I do stand by my conclusion if you look at the last 3-6 months of tournament data that it suggests if you are a potential middle-of-the-road gush player you are better served by attacking the metagame than by playing a highly targeted deck.

And yes, the general idea is that for a given tournament the win % only describes what happened. Without doing this for the individual tournaments there's nothing to aggregate though. I'm mostly objecting to throwing around the term "useless".

Good job on putting this together again always great information and interesting read

Is the agreement to group Doomsday with the Gush archetype instead of Combo?

When I think Gush decks it more Mentor/Pyromancer/Grow strategies. And doomsday feel more like a pure combo deck to me, but I could be alone in this thinking..

@diophan I agree that Matt calling it "completely useless" is incorrect. A common thing that my advisor likes to say is: "A poor estimate based on empirical data is better than no estimate at all."

@TurboK Matt posted about the issue of classifying Doomsday months ago when we started this, in a thread that completely derailed. In our original metagame analysis we put Doomsday with gush, and we've/I've kept it that way for consistency's sake. I don't have a strong opinion either way.

I ran into that when I was listing the deck archetypes for each player. And at first I wanted to list it as Gush because it plays Gush and operates with the Gush "rules" and that's the easiest way to classify it quickly. But I agree the mindset of playing the deck is more like Combo. But I believe the main archetype and the sub archetype are sort of interchangeable for this deck. You could call it Gush Combo or call it Combo Gush. The metagame breakdown takes into account the first word.