It's worth clarifying a key distinction here: federal prisoners are housed based on where the Bureau of Prisons (BOP) assigns them, not necessarily their home state. So the question can be looked at two ways — which states have the most federal prison facilities (and thus physically hold the most federal inmates), or which states send the most residents to federal prison.

States with the most federal prison infrastructure/inmates physically housed:
The largest federal prison complexes are not randomly distributed. Texas, Florida, Mississippi, California, and North Carolina all appear near the top of the ranking. Many of these sites are located in rural counties where land availability supports expansion and where federal facilities often provide a major source of employment.
The Coleman Complex in Florida remains the largest federal prison site in the country, with more than six thousand inmates housed within four institutions. Beaumont in Texas and Yazoo City in Mississippi follow, each holding well above four thousand individuals.
West Virginia appears several times with Hazelton, Beckley, and Gilmer.
States that send the most residents to federal prison (by home state of legal residence) closely mirror overall population size. The largest states — Texas, California, and Florida — consistently top that list as well, simply due to their sheer population and the volume of federal cases prosecuted there.
A few key context points:
So, in short: Texas, Florida, California, West Virginia, and Mississippi stand out as states with the heaviest federal prison presence, whether measured by facilities or inmate populations.