In a video clip posted on mainland social media, the Putonghua-speaking man was seen carrying the post over a short distance before returning it to its original spot.
The Agriculture, Fisheries and Conservation Department (AFCD) said the M148 post was loose after the incident, and it has now been sealed off for repairs.
The AFCD noted it could be an offence for a person to willfully or negligently damage country park facilities.
Officials pointed out that the marker is used during emergencies to help rescuers locate those who need help.
They also called on hikers to protect the countryside.