Shoppers were stung with $187 million last year, as businesses raised their prices to cover the cost of accepting credit card payments.
Retailers are asking for regulation of the system to help bring down costs, but the banks say there is no need.
Last year the Ministry of Business, Innovation and Employment (MBIE) called for submissions on the retail payments system after concerns about credit card acceptance fees - called interchange pricing - were raised by retailers.
The fee amount is dependent on the card type and transaction size but Retail NZ said the average fee was 1.7 percent - more than double that of the United Kingdom and Australia.
Vijay Parbhu runs a shoe repair shop in Wellington.
He said his bank charged him $300-$400 dollars a month for accepting credit cards.
"The merchant fee adds up because a lot of people want to pay on credit because then they can spread their payment to the following month, so even for the smallest amount people will put it on credit and you're sort of vulnerable to accept it because if you don't they'll just go somewhere else."
He said small businesses in particular were vulnerable to the system, and had to pay whatever they were asked.
"Because you're small your leverage is less your prices are high and your pushing power is very limited as well."
Melina Martin also runs a small business - a clothing store in central Wellington.
She said one way around it was to add a surcharge for people who chose to pay using their credit card.
"I think that would be a bit of a deterrent though, it sort of puts people off."
Consumer NZ chief executive Sue Chetwin said it was the customer who ultimately ended up wearing the cost of this fee, either through price hikes or surcharges at the terminal.
She said rewards schemes offered by credit card providers gave the consumer the impression they would eventually make their money back, but this was misguided.
"We've looked at these schemes, they're not that good and in many respects you can only earn from them if you're a really high spender. If you're an ordinary consumer you don't really benefit from them that much anyway so we think there's plenty of room for regulation there."
The New Zealand Bankers Association said any changes should be dealt with by the banks and card providers, such as Visa and Mastercard, and regulation was not necessary.
They said there was no clear evidence of a market failure and were not convinced by the $187 million figure from MBIE. The association said reducing interchange pricing would decrease the revenues earned by banks and they would have to respond by either increasing cardholder fees or reducing the benefits that cardholders were eligible for via rewards schemes.
Under instruction from the former government, the industry was asked to be more open about how their fees work, and to make it easier for merchants to negotiate their fee.
Payments NZ has been tasked with reporting back on this matter to the new government in April. Chief executive Steve Wiggins said the review was a chance for banks to be transparent about fees, and to avoid government intervention.
"The view is, say from the retailers, that with greater transparency, that will provide a greater opportunity for merchants to negotiate, so I think we need to see how that plays out and that's what we'll be reporting back to the minister in April on."
The new consumer affairs minister Kris Faafoi said until the report comes back in April it was too early to step in, but said consumers getting a fair go would be his top priority.