Pogacar attacked on an incline to the line to finish just ahead of Van der Poel with Jonas Vingegaard third.
It was the 26-year-old Slovenian’s 18th Tour de France stage win as he seeks a fourth overall triumph on the Tour.
Fans were treated to another Pogacar-Vingegaard head-to-head duel as five hills made the final 40km a roller-coaster.
But overall leader Van der Poel also entered the fray, narrowly failing to win the stage but keeping the yellow jersey he took from Alpecin teammate Jasper Philipsen, who quit the Tour injured after a nasty fall on Monday.
On the day’s final real climb, Pogacar dropped all his rivals with only Vingegaard offering a real fight.
But the big Dutch rider and the slender Dane both came back at Pogacar, making him fight all the way to the line in a thunderous finale.
The same trio top the overall standings, with Pogacar second and Vingegaard in third.
Unlike the opening three stages, there was hardly a puff of wind and not a drop of rain, but there were still plenty of falls.
Stage five will shake up the overall standings with a 33km individual time trial around Caen.
The stage is being billed as the day Remco Evenepoel will finally slip into the overall leader’s yellow jersey.
To do so the 25-year-old Belgian world and Olympic champion in the discipline will need to cover the 33km course almost a minute faster than Pogacar and Vingegaard.
The first mountains come as late as stage 10 over the volcanic landscape of the Puy de Dome, with two more colossal climbing days in the Pyrenees before the blockbuster final week in the Alps. (AFP)