The Federation also said the system needed to be changed so that it was easier for working people to benefit from social housing.
The poll, commissioned by the Federation to highlight the recommendations in its housing election manifesto, also revealed that 62% of people with a household income of £30,000 or less, thought workers had less chance of being allocated an affordable home than their parents did in previous decades.
Over the last seven years, demand for social housing has increased by 70%, and the number of people on waiting lists is expected to hit a new record high of five million next year, as the recession and rising unemployment further fuels the crisis.
The chronic shortage of affordable housing means homes are increasingly allocated to people in acute need, such as those classed as officially homeless, over other groups in desperate need of housing.
Meanwhile, those in work on lower incomes often end up further down the pecking order and face little chance of being housed under the criteria currently used to allocate the limited number of homes available.
NHF Chief executive David Orr said: "The chronic shortage of social homes in this country has created an allocations system whereby only the most desperate and vulnerable have a realistic chance of getting a home.
"This creates neighbourhoods where the most vulnerable and marginalised are housed together, resulting in areas of high economic inactivity, poverty and disadvantage."
Have your say on this story using the comment section below