I could imagine that we mixed up a lot of times- mixed up without loosing our same ancestors- just added a few more. Its just a thought, but a great problem until the neolithicum was the lack of manpower, if the klimate dont take care for a rich food offer.
If you running out of manpower and you meet another group there are different options, but one of them is: Kill the males, get the females.
Maybe, for example, the foundergroup, which arrived china did so... and i do think homo sapiens did so when they meet neandertalensis in europe.
As far as i know, there are two gene tests about the mixing between sapiens and neandertalensis. The results:
The first: It shows that we mixed up.
The second: It shows thats we mixed not up.

Btw, thats why im very sceptical about gene stuff... but, as i said, im not up to date with genetic. Maybe there are new result of new tests.
Fictitious the foundergroups mixed up as i think, this could explain why we all homo sapiens and still have some varieties.
But yes, its just a guess, too.