How is it possible that bip39 compliant seed phrases don't overlap across all the different blockchains?

Like for instance when you first use a Ledger you make a 24 seed phrase that works across every blockchain. What if for example someone on Cardano made a wallet years ago and someone else creates a wallet years later for Ethereum on a Ledger that has no intent of using Cardano. Couldn't they match up since each phrase isn't being used on the same chain? Are the odds of that happening really that low that it doesn't even matter? Couldn't someone start a Cardano wallet by importing an Ethereum seed phrase?