My Thoughts. . .
When the Jews began their return to Palestine from Babylonian captivity, only 50,000 of the displaced wanted to go. During the seventy years of captivity, they had built houses, established businesses, were allowed a good measure of freedom, and had members who enjoyed interaction with the rulers. Life in Babylon was a lot better than what they would find by returning.
The trip back was about 618+ miles. It would not be pleasant. They did not have Interstate nor State highways to travel on. Restaurants, Rest Stops, Motels, and such were unknown. Walking was their transportation and camping out was not a recreational vacation. They did not have refrigeration to keep items from spoiling. Someone would need to hunt, kill, return with the kill, skin it, cut it up, and cook it. Older travelers would slow them down. Finding water would be a problem. Bathing would be a luxury. Protection from the locals would require constant lookouts and a well-regulated defense to remain safe.
When the time arrived for that large group to leave Babylon, did those who stayed have different feelings about those who were leaving? Did they think they were crazy for wanting to return to what had been destroyed? Did those who were leaving think those who remained were void of trust in God? The ones who were leaving would not find a welcoming party waiting to greet them. Jerusalem without walls would open them up to marauding groups who would wish to rob or kill them. They were leaving good houses behind with no guarantee of housing where they were going. Their travel time would rule out planting and harvesting food. The trip would be long. Danger would be their traveling companion. Would older folks making the trip make it? Starvation could happen. Rebuilding might end in failure due to those who now occupied the land.
What would you have done? You have a nice house to live in. You are surrounded by friends and relatives that are not going. You have a great job and are making a good living. You have no problem with the Babylonians, and they are not making life miserable for you. Your children are educated by those who believe as you do. Your older children have married and have your grandchildren and they are established here and do not plan to leave. Why go on a trip that will be unpleasant and present unknown dangers? Once resettled, life will continue to be hard due to those foreigners living there. They will attempt to remove you by force.
Sometimes, Scripture does not offer details nor the conflicting decisions that had to be made. These are just my thoughts. What are yours?