Having lost their Group A opener, France looked to be heading out until former Marseille striker Gignac completed his treble from the penalty spot to level for a third time on 85 minutes, before Teji Savanier grabbed an injury-time winner.
Gignac, who currently plays for Tigres in Mexico, is now top scorer in Tokyo with four goals from two matches.
South Africa, whose hopes of qualifying have now been ended, led three times through second-half goals from Kobamelo Kodisang, Evidence Makgopa and Teboho Mokoena.
Hosts Japan made it two wins from two later on Sunday, beating Mexico 2-1.
Brazil held & Argentina have fresh hope
Defending champions Brazil were held to a goalless draw by Ivory Coast in Group D, after Aston Villa midfielder Douglas Luiz was sent off for a trip on former Leeds winger Max Gradel on just 13 minutes.
Luiz was initially booked, but the yellow card was upgraded to red by VAR, after he was deemed to deny a goalscoring opportunity.
Ivory Coast, who won their opener against Saudi Arabia, also had a player dismissed late on as Eboue Kouassi received a second yellow card.
Brazil, captained by former Barcelona defender Dani Alves, still top the group after thrashing Germany in their opening match.
Elsewhere, Facundo Medina gave Argentina their first win of the Games with a 1-0 victory over Egypt, while also in Group B, Mikel Oyarzabal’s late strike saw Spain beat Australia to open their account.
Burnley striker Chris Wood was on target in New Zealand’s 3-2 loss to Honduras and, in the same group, South Korea thrashed Romania 4-0, leaving all the teams on three points after two matches.