The National Party leader supports the start of discussions about a possible exit plan from level-4 restrictions, as long as the government remains focused on enforcing an effective lockdown.
The rate of new Covid-19 cases is giving the government some reassurance the level-4 lockdown is having some success.
At Sunday's media briefing Director-General of Health Ashley Bloomfield said there were 89 new cases which included 48 confirmed and 41 probable cases, taking the total to 1039.
Dr Bloomfield said there were now 12 clusters, which are defined as 10 or more infections from the same place, an increase of two from Saturday. The Marist College cluster had 66 cases and was the largest cluster in New Zealand.
Prime Minister Jacinda Ardern said on Sunday the lockdown was making a difference and cited modelling by economist Rodney Jones which forecasted that New Zealand could have had 4000 cases to date if not for the strict isolation measures.
"Those 3000 fewer cases shows the difference that cumulative action can make. Three thousand fewer people sick with Covid-19, 3000 fewer people passing the virus onto others and into others," she said.
"We can and we must continue to break the chain of transmission."
Cabinet ministers and senior officials will now start talking about a possible exit plan from alert level 4, but the level of community transmission would be a key factor on any decision.
Bridges said while it was too early to draw any firm conclusions, the relatively low rate of increase was encouraging.
However, New Zealand had to "guard against counting its chickens", he said.
"Where we say 'oh well that's fine, we've only got the one fatality, and I don't mean to be light about that, but that means we can all sort of move on'..."
Other countries had experienced "rebounds" so New Zealand just had to "take things gently and be agile", he said.
New Zealand should be trying to get out of lockdown as soon as it could, but it didn't have to be "all or nothing".
There was a lot of discussion among businesses about how they could work; "in a safe way, in a contactless way, mitigating their risk", Bridges said.
He wanted the government to stay open to ways to limit economic damage, such as allowing businesses that could find ways to operate online or with enough social distancing, to open.
It was "fine" to be talking about New Zealand getting out of lockdown, he said, because health and the economy went together.
"We want them both in the strongest possible situation."
There had been strong measures taken by other countries but not the "strong shutdown" New Zealand was undergoing, he said.
He believed there needed to be yet more testing - "many thousands a day" - in order to get real confidence about community infection rates.
National was also still pushing for more contact tracing, tougher quarantine rules at the border and a guaranteed supply of personal protective equipment to frontline workers.