...just one thought. The Buddha Shakyamuny (the historical Buddha) said while going back through all his lifetimes he couldn't find a beginning of "all that is" and assumed that the wheel of Samsara is ever turning, until we come across a teaching like Buddhism that shows us the way out.
One thought: What if there are realms that are just not accessible with the mind? I personally find it hard to believe that we don't have a choice between live times to choose what the next one is going to be like. The Buddha assumed that according to the level of distortion in our mind our rebirth will be determined. So it is pretty much a coincidence thing as what we will be reborn. We die and pop out the other end as a new born being - it could be anything. ...hmm... What happens in between?
Just a thought, maybe someone has an explanation for this

All the best,
Andreas