The area, commonly known as Mission Hill or Woh Chai Shan, became a well-known destination after the recent reopening of the Ex-Sham Shui Po Service Reservoir.
Authorities announced a revitalisation scheme on Monday that would replace dilapidated private structures built on government land, including table tennis tables and exercise equipment.
The plan would also include safety upgrades and slope maintenance works.
Residents have until Wednesday to clear out the area, and notices have already been put up along the trail.
Kowloon West lawmaker Vincent Cheng welcomed the move, saying about 50 illegal structures have to be dealt with before proper recreational facilities can be built.
“Residents feel they need such amenities, because there’s a lack of recreational facilities in the community. But after all they’re illegally occupying government land, and these illegal structures pose certain dangers,” he told an RTHK programme.
“These recreational facilities are built on the mountainside. Especially with the recent rain, sometimes you see signs of soil erosion, so they pose certain dangers.”
The government said that depending on the progress of the works, the revitalisation will be completed in phases from the end of August to September.
It also invited religious leaders to assist in the removal of Buddhist statuettes in the area, and promised to maintain contact with relevant stakeholders.