Statues of royalty, politicians, military leaders and explorers have been the most regularly vandalised in recent decades, research has found.
Conversely, statues of artists and sportspeople have been left largely untouched.
The findings of an Otago University study comes as some question what we should do with depictions of historical figures with unsavoury pasts.
The study of 123 named statues on public land around the country found one in every four had been vandalised.
Some more than once, such as the representation of King George V in Matakana, which has been decapitated five times.
The lead researcher, Otago University professor Nick Wilson, said the common theme running through all was colonialism.
"These attacks are quite clearly driven by issues around colonisation, and also militarism. People are attacking these statues because of past injustices," Wilson said.
He said there had been an uptick in physical displays of protest since the 1990s.
"We were seeing attacks on statues back in the 1930s for example, after World War I, so this is not a new phenomena," Wilson said. "But it does seem attacks are increasing as people are thinking more about these issues of colonisation and militarism."
In 2018, Waikato-Tainui kaumātua Taitimu Maipi sprayed red paint on a statue of British army Captain John Hamilton. Last week he told the council that he was going to go a step further and rip it down himself - but before he could, the council took it away.
Maipi said the time was up for statues like this, with the same fate due for all other colonial-era statues in the country.
"Why are we celebrating a monster like Hamilton who wanted to create havoc amongst our people?" Maipi said.
"We're not even mentioned at all in any of the history of those statues. As a matter of fact, those statues are glorified by the Crown, over many, many years, because they came to kill our people. What's [there to] glorify about that?"
History academic and PhD candidate, Hayden Thorne, said he would prefer that statues were generally left up.
There were alternatives to pulling them down that might be appropriate, he said.
"Whether that's in the form of additional plaques or information boards, or moving them to more appropriate locations where you can provide that extra information; I know in some places overseas they've moved statues to museums to allow a more full discussion about them," Thorne said.
"Or to put up other statues around them that better represent a wider group of people."
Historian Vincent O'Malley said the statues around the country systematically ignore Māori views, and mythologise European ones.
He said very few educate any visitors with much depth of the history, and those that do with little balance.
Whether statues stay or go, Dr O'Malley said at the very least the truth should be told with contextual information on a board or even an app.
"The reality is that these were bloody and brutal conflicts, and there were a number of atrocities committed - including at Ōrākau where female prisoners were bayoneted in cold blood," Dr O'Malley said.
"The reality of what took place doesn't fit with this myth-making around the wars. The monuments often commemorate the mythical version of history rather than what actually took place."
Dr O'Malley said iwi and hapū should be at the forefront of these discussions, with local councils.