Australia were dismissed for just 212 after South Africa captain Temba Bavuma won the toss in bowler-friendly conditions, with pace spearhead Kagiso Rabada taking 5-51.
But at stumps South Africa had slumped to 43-4 in reply, a deficit of 169 runs, as Starc finished the day with figures of 2-10 in seven overs.
South Africa’s Marco Jansen, who took 3-49, admitted it had been a mixed day for the Proteas.
“Bowling-wise I think we did really well,” he said. “They bowled well and we are going to have our work cut out for us tomorrow.”
Given the strength of both teams’ pace attacks, the key question was always likely to be which side could put enough runs on the scoreboard.
Australia’s total certainly looked a lot healthier when left-arm quick Starc had Aiden Markram playing on for a duck before dismissing Ryan Rickelton (16) to leave South Africa floundering at 19-2.
Novice number three Wiaan Mulder, dropped on one, was bowled by Australia captain Pat Cummins, having taken 44 balls to make six.
And shortly before the close, Josh Hazlewood bowled Tristan Stubbs.
Australia had earlier slumped to 67-4 at lunch.
But star batsman Steve Smith (66) and all-rounder Beau Webster (72) repaired the damage with a fifth-wicket stand of 79.
“Two good bowling outfits, it was tough work for the batters,” Webster told the BBC. “We’ve got a score on the board and to grab four scalps at the end of the day, I think we’re pretty happy.”
Webster, however, would have been lbw to Rabada for eight if South Africa had reviewed an original not out decision.
“There were a couple of reviews throughout the day which could’ve gone either way,” said Webster. “The degree of seam movement up the slope was tricky.”
Australia were in desperate trouble in the early overs, reduced to 16-2 after losing Usman Khawaja and Cameron Green in a single Rabada over.
The 30-year-old, who has more than 300 Test wickets, is playing his first Test after serving a one-month ban for cocaine use earlier this year.
“It means a lot for me playing for South Africa, I give my all each and every time. I am so happy to do the job out there,” Rabada told Sky Sports.
Australia have struggled to find an opening partner for Khawaja since David Warner’s retirement 17 months ago, with Marnus Labuschagne the latest batsman tried out.
The 30-year-old, without a Test century for nearly two years, battled hard for 17 off 56 balls before falling to towering left-armer Jansen.
Australia had been in similar trouble in the 2023 final against India across London at The Oval, only for Smith and Travis Head to both score hundreds as they turned the match in their favour.
Head, however, could only manage 11 before he glanced Jansen, with Kyle Verreynne holding an excellent diving one-handed catch.
Smith went to fifty before falling to part-time spinner Markram after Jansen, at slip, clung on at the third attempt.
Australia lost their last five wickets for just 20 runs, with Rabada ending the innings when he bowled Starc.
Rabada’s haul meant he surpassed Allan Donald’s tally of 330 Test wickets in his 71st match compared to the retired South Africa great’s 72 Tests.
“That’s awesome,” said Rabada. “Allan Donald, what a legend, it means a lot to get past someone like him.” (AFP)