True i guess youd also have to test for homogeneity of variance and what not as well. For a bunch of people who claim to not be very experienced in stats, you guys know a lot more than you let on 
i dont have a ton of practical experience with applying statistics, still learning! Ive been told to assume an alpha of 0.05 in most situations, so maybe that could work? The only chi tests i have experience with are goodness of fit and tests of independence. I figured you could find the average by looking at how many packs opened vs how many of a card or rarity pulled (which i assume is how they got their numbers in the first place), but i guess youd technically need like 30 of each card pulled at least to conduct the test. And even then, that does go out the window when u consider that the different groups do impact each other; you cant pull two gold cards from one pack, each pack and each card slot doesnt not have an equal chance at containing the card of interest. Interesting stuff, thanks for the responses guys!
I always hate these graphs. And people on reddit really seem to think any one alt art does actually have a significantly different pull rate than the other.
so what card do i need to buy because its gonna stonk hard
I have been looking into the topic of pull rates since 2020, when I watched and noted down the information from a ton of youtube opening videos for want of something better to do during covid lockdowns, and have continued to do so for the sets released since. Still working on bringing all the data into a useful form for making public, but I have posted some tidbits on my blog (Can we estimate pull rates for modern ultra-rares? – The Pokémon Collation Project) - shameless self-advertising I know, but it fits the topic here. I agree with the OP that most of these extremely detailed pull rates given are bullshit. However, there are some things I have learned:
- Pull rates for categories of cards (e.g. holo, regular art, full art etc.) can be estimated quite well if you look it booster boxes and not individual packs, because the variance is smaller than in a random binomial distribution where you either get this category of card in a pack or not. For example, if it were a purely random binomial distribution whether you get a secret rare in a pack, you would see multiple boxes with 3, 4 or even 5 secret rares in a large sample, while in fact it is usually 0 or 1 and very occasionally 2 for most sets.
- You need keep in mind Pokémon cards are printed on uncut sheets and often more than one category of cards will go on a sheet (e.g. VMAX, VSTAR, Full Art, Alt Art on a 10x10 sheet for ultra rates). This leaves little room for fine, arbitrary variations between individual card rarities for large modern sets, but at the same time sometimrs automatically introduces differences in rarities - look at Skyridge for example, the 73 commons will appear on a 11x11 sheet either once or twice, yielding pull rates of 1:13 or 1:26 - a very significant difference. For finding out what may have been printed on the same sheet, miscuts can be incredibly helpful.
- There are definitely some cases with arbitrarily introduced rarity differences of cards of the same category, and some cases where it is hard to tell. If the difference can be explained logically (legendaries may be rarer than non-legendaries, full arts which have a corresponding alt art may be rarer than those that don’t), or is very large with a large sample, it is probably real, if it is bizarre (random card pulled less often) and the sample is small, it more likely is not.
Ah statistical analysis, takes me back to my MMORPG days. Thank you for doing this @pfm . Very insightful and well laid out. The yellow border era is going out with a bang, that’s to be sure.
…or you could be a superstar prof at UoNT.
Thanks for doing the legwork. And because of the results, I think I’ll leave the hunting work to others.
@pfm I didn’t see where you calculated my lucky pull ritual (three double taps, spin left, spin right…and I won’t share the last step because it will break the luck)!