He replaces the little-known David van Rooyen who had only been in the job since Thursday.
Last week, President Jacob Zuma sacked previous Finance Minister Nhlanhla Nene in a widely-criticised move that sent the rand to record lows and caused the stock market to tumble.
The developments come amid concern over South Africa’s struggling economy.
Mr Gordhan was widely respected when he served as South Africa’s finance minister from 2009 until 2014.
BBC Africa business reporter Lerato Mbele says his re-appointment is designed to quell market discontent and restore some confidence.
It appeared to have an immediate effect with the currency rising, recovering from just over 16 rand to the dollar to about 15 by Monday morning, according to currency site xe.com.
The Johannesburg stock exchange also recovered some of last week’s losses.
But the new finance minister has a hard job with unemployment currently above 25%, growth sluggish and credit rating agency Fitch recently downgrading South Africa to one notch above “junk” status.
The brief tenure of Mr van Rooyen and the uncertainty it caused may have damaged South Africa’s reputation further, analysts say.
Mohammed Nalla, head of research at Nedbank Capital, said having a finance minister serve just a few days did not bode well.
“International investors are probably thinking: ‘Why didn’t the president make a much more considered decision in the first place?'” he said.
President Jacob Zuma’s decision to fire two finance ministers in the space of a week has been a colossal blunder.
Not only has it been recognised by opposition parties, who are calling for his resignation, but also by the general public, the financial markets and, by the weekend, the president himself, hence the change in mind.
But what is happening with the governing African National Congress?
The ANC leadership was not consulted and seemed to be hearing about the dramatic appointments at the same time as the rest of us.
There is no doubt that the continent’s oldest liberation movement is in disarray.
President Zuma will emerge weaker but the party will not lose votes in the medium term – as people remain loyal to the movement if not the individual.
Mr Nene’s reluctance to approve a plan to build several nuclear power stations at a cost of up to $100bn is thought to have contributed to his removal as finance minister.
But President Zuma’s move to get rid of him drew a lot of criticism from within the governing ANC.
Former Health Minister Barbara Hogan on Friday called on Mr Zuma to resign. The highest-profile ANC member to oppose Mr Nene’s removal, she said that the president had crossed a line and needed to be held to account.
Razia Khan, an analyst with Standard Chartered bank, said the turmoil was “perhaps the first instance since 2007 that Zuma has come under severe pressure within the party”.
A statement from Mr Zuma’s office said he had “received many representations” to reconsider his decision to appoint Mr van Rooyen.
“As a democratic government, we emphasise the importance of listening to the people and to respond to their views,” it added.
Fitch said on Thursday that Mr Nene’s sacking “raised more negative than positive questions”.