The mayors of Auckland and Christchurch say their councils are not to blame for rising inequality in this country.
At yesterday's release of the Government's books for the last financial year, Mr English said inequality would have decreased if not for councils' poor planning.
He said the cost and complexity of getting permission to build a house is the biggest single contributor.
"The growth in housing costs over time, to the point where you're seeing families spending 50 or 60 percent of their income on housing - that's pretty devastating at the low end.
"So councils need to understand that when they run these policies that restrict the availability of land and the opportunity for lower value housing they are causing poverty."
Acting Auckland Mayor Penny Hulse said despite many meetings with Mr English, it was the first time she had ever heard his concerns.
Listen to more
She said the council had cut consent times for special housing areas but made no apologies for rules aimed at avoiding another leaky building disaster.
Ms Hulse said her council had worked hard to improve housing in the city, including reducing the time it took to get consents in special housing areas to a matter a months.
"We've got land now ready for building and housing. The reality is, and the minister has said it himself, that the housing crisis has many causes - there is no silver bullet."
Christchurch Mayor Lianne Dalziel said housing was at the top of her council's agenda, and it had also improved its consenting process and created new social housing projects.
Local Government New Zealand president Lawrence Yule said Mr English was drawing a "very long bow" in saying local government was responsible.
Listen to Lawrence Yule
"I think he has a point in parts of New Zealand ... there are some land supply and housing issues particularly in Auckland, and in Christchurch because of the earthquake but they're not universal.
Housing was not the sole driver of poverty, Mr Yule told Morning Report.
"In many parts of New Zealand, you'll go and you'll find empty shops everywhere, you'll find houses you can buy for under $100,000."
The Government needed to be proactive and, for example, relocate departments to the regions, he said.
"In many parts of New Zealand there's lots of land, there's lots of houses actually in most communities ... they'd like people, they'd like jobs and they'd like growth."
Opposition critical
Green Party co-leader Metiria Turei said those comments were a new low for the Minister.
She said it was bizarre that after spending the past six years denying that poverty exists, Bill English was now admitting there was a problem and that it was because of urban planning processes.
"Nowhere in any report from any non-government organisation or Government department has urban planning been blamed for child poverty.
"What I think is happening is Bill English is trying to divert attention from the fact that the solutions are obvious and within the power of the Government to implement, but they don't want to."
Listen to Phil Twyford
Labour's housing spokesperson Phil Twyford said it was farcical for the Government to blame local councils and the Resource Management Act for what he said was a housing crisis.
Mr Twyford told Morning Report it was dishonest for the Government to claim the RMA and councils were responsible, when it refused to build more houses itself.
ANZ chief economist Cameron Bagrie said restrictions around the availability of land had affected housing affordability but it wasn't the only factor to blame for poverty.
He said there were a lot of other challenges behind the scenes, and there was no one-size-fits-all solution to make houses more affordable.
Mr Bagrie said housing unaffordability was possibly due to wages being too low.
Prime Minister John Key said proposed changes to the Resource Management Act would be core to improving housing affordability by allowing councils to free up land more quickly.