Advocates and opposition parties have accused the coalition of treating disabled people with callous disregard in scaling back their dedicated agency.
The government on Thursday announced plans to strip Whaikaha of its responsibility for delivering support services, shifting it to the Social Development Ministry.
Whaikaha would be left as a smaller, standalone department with a focus on advocacy and policy advice.
It comes after a scathing review - led by top public servant Sir Maarten Wevers - found major failings around Whaikaha's finances, warning it was on track for a budget blowout due to ongoing overruns and inadequate controls.
Minister for Disability Issues Louise Upston has sheeted responsibility back to Labour for rushing the set up of Whaikaha in 2021.
In response, Labour MP Carmel Sepuloni told RNZ it was clear that Whaikaha needed more help, but that instead of stepping up, the government had given up.
"Rather than provide that support, the government has said, well, you're not doing a good enough job, so we're going to take it off you and give it to a bigger government agency.
"I tell you what - it didn't work last time and it won't work now."
Sepuloni said the changes dealt the coalition's harshest blow yet to disabled people.
"Disabled people have not been part of the review, in fact, didn't even receive it... people are feeling completely ignored."
Green Party disability spokesperson Kahurangui Carter said the government's plan was "a major setback" for disabled communities.
"Weakening this ministry does not empower our disabled people," she said. "It erodes their agency... the ministry is still in its infancy, and this review should be about improving it, not gutting it."
New Zealand Disability Support Network chief executive Peter Reynolds said the sector had fought long and hard to get its own ministry, giving them control over how they accessed support.
"That has now been taken away. So you'll be getting a one-size-fits-all limited package of support. You're likely to see some trimming back of the breadth of that support, any flexibility that may have been there is likely to be removed."
Reynolds said the sector had been starved of funding over many years: "You just need to give it some more money".
Speaking to RNZ, disability rights lawyer Huhana Hickey echoed the concerns about funding.
"They're cutting our funding. They're attacking us on benefits. They're attacking the health system which is a big impact on the disabled.
"They don't know what they're doing."
Speaking earlier on Thursday, Upston rejected that.
"Funding isn't the issue here. We provided a record $1.1 billion dollar funding boost to support services in Budget 24.
"Our government will build a fairer, sustainable support system that delivers better outcomes for disabled people based on need."
In a statement, the Public Service Association (PSA) described the move as a reckless decision and a "slap in the face" to disabled people.
"Whaikaha was never given a chance to succeed. Now the disabled community will lose its strong voice to ensure support services are well funded and a joined-up disability support system so thousands of people can lead ordinary lives," PSA national secretary Kerry Davies said.
"Turning Whaikaha into a policy shop will water down the voice of the disabled community in government."