Prime Minister Yoshihide Suga said his government would decide on new measures to stop the spread of the virus.
Those measures are expected to determine whether spectators can attend Olympic events.
Medical experts have said for weeks that having no spectators at the Olympics would be the least risky option amid widespread public concern that the Games will fuel new surges of coronavirus infections.
Organisers have already banned overseas spectators and set a cap on domestic spectators at 50% of capacity, up to 10,000 people, to contain the coronavirus outbreak.
Japan's National Stadium could yet remain empty of spectators during Games. Photo: AFP
Officials have been wrestling with the question for months but a ruling party setback in a Tokyo assembly election on Sunday, which some allies of Suga attributed to public anger over the Games, had forced the change of tack, sources said.
"Politically speaking, having no spectators is now unavoidable," a ruling party source told Reuters.
Japan will hold a parliamentary election later this year and the government's insistence that the Games - postponed last year as the virus was spreading around the world - should go ahead this year could cost it at the ballot box.
The Tokyo 2020 organising committee said restrictions on spectators would be based on the content of Japan's coronavirus state of emergency or other relevant measures.
The Games begin on July 23rd and end on August 8th.
-Reuters