Rarest sets in general, of which (almost) the entire set is available (so skipping unlimited Japanese E1 and L1 cards), is hard to answer. Some that come to mind in chronological order:
- unlimited edition Korean Base Set
- 1st edition Portuguese Jungle
- unlimited edition ©1999-2000 English Fossil
- unlimited edition Dutch Fossil
- 1st edition Spanish Neo Genesis
- unlimited edition Japanese World Championship Pack
- unlimited edition Japanese L2 & L3
- probably a bunch of unlimited edition XY era Japanese sets, although it slipped my mind which are the rarest of the 11 available
If we’re only speaking of booster boxes here, I’ve personally never seen a picture of a booster box for the unlimited edition Korean Base Set, nor the unlimited edition Japanese L2/L3 sets. The other sets mentioned above I have seen a booster box of at least once each.
For the unlimited edition Japanese L2/L3 set I’m not even sure if a booster box was ever created. It wouldn’t surprise me if all unlimited edition L2/L3 booster packs came from blister packs and such. 
But I could be mistaken, I haven’t looked into the products of those sets too often tbh.
So although there are a lot of rare sets, if there is one that’s the rarest set across all languages and eras, I’d say it’s the Korean unlimited Base Set. Cards from the set itself (excluding those from the theme decks) are also close to impossible to find.
Greetz,
Quuador
Yeah, good point. But indeed, the idea was more only main-line English sets.