The quake struck at 1.04am in Shizuoka Prefecture, west of Tokyo, the Japan Meteorological Agency said.
No tsunami warning was issued.
Authorities reported no deaths, but the quake shattered storefront windows and knocked items off shelves and caused transport disruption in parts of the country.
An emergency official in Aichi told AFP authorities had received 16 reports of mostly minor injuries.
Authorities in neighbouring Shizuoka said at least three women sought emergency medical care, including a woman in her nineties who injured her shoulder after falling from her bed during the shaking.
The local Chunichi Shimbun newspaper and Tokai Television separately reported at least 24 were hurt across the two regions.
In Toyohashi city in the Aichi region, windows of a car dealership were found shattered after the quake.
A large bookstore in Toyota city, also in Aichi, reported that its merchandise dropping from shelves.
The quake also forced temporary delays and suspensions of regional train services on Sunday morning, including the Shinkansen bullet trains.
It was felt as far west as Osaka, where a foreign resident was taken to hospital after falling from a second-floor apartment when the shaking startled them, local media reported.
Separately, a 4.3-magnitude quake struck the Kumamoto region at 10.53am on Sunday, the Meteorological Agency said.
There were no immediate reports of serious damage or injuries from the second quake.
The southwestern region is still trying to recover from a 7.1-magnitude tremor that killed 38 people, flattened homes and left tens of thousands without power in July. (AFP)