Pretty much the only time we need passports is if we travel outside the U.S. and territories. Those that take cruises or cross borders to other countries would, but generally speaking a majority of Americans don’t have passports.