It is the latest in a series of high-profile departures from President Trump’s team.
There has been speculation that Mr Cohn, a supporter of free trade, was angered by Mr Trump’s plans to impose tariffs on aluminium and steel imports.
In a statement released by the White House, Mr Cohn said it had been “an honour to serve my country”.
The 57-year-old former president of the Goldman Sachs bank had helped Mr Trump push through his sweeping tax reforms late last year.
Why has Mr Cohn gone?
Gary Cohn and President Trump were never believed to be close.
Mr Cohn wasn’t specific about the reasons, saying in a statement it had been “an honour to serve my country and enact pro-growth economic policies to benefit the American people, in particular the passage of historic tax reform”.
Once that mission had been achieved, a number of differences may have prompted the departure, including the possible looming trade tariff war and his differences on that issue with trade adviser Peter Navarro and Commerce Secretary Wilbur Ross.
Mr Cohn had reportedly set up a meeting between Mr Trump and business executives who opposed the tariffs move. But Mr Trump pulled out of that meeting and on Tuesday reportedly asked Mr Cohn in the Oval Office to back the tariffs publicly. Mr Cohn did not answer, sources told Bloomberg.
In August last year, Mr Cohn had also criticised Mr Trump over his reaction to a far-right rally in Charlottesville, Virginia, saying the administration “can and must do better”. He was reported to have drafted a resignation letter after the event.
An official said: “For several weeks Gary had been discussing with the president that it was nearing time for him to transition out.”
The exact departure date had yet to be determined.
In a statement, Mr Trump said: “Gary… did a superb job in driving our agenda, helping to deliver historic tax cuts and reforms and unleashing the American economy once again.
“He is a rare talent and I thank him for his dedicated service to the American people.”
And the markets?
Analysts were pointing to the resignation of Mr Cohn, a free market advocate, as one reason behind a drop in shares across Asia on Wednesday. The Nikkei closed 0.77% down and the Hang Seng 1.03%.
Rick Meckler of LibertyView Capital Management told Reuters that Mr Cohn was “very credible” and the resignation announcement “certainly causes short-term downward pressure”.
The dollar continued its retreat against the yen, down from 113 at the start of the year to 105.6 on Wednesday.
European stocks also opened lower, the FTSE 100 and pan-Europe STOXX 600 falling about 0.5% after opening.
Does this herald a new direction?
Analysis by Anthony Zurcher, BBC News, Washington
Gary Cohn was a bit of a stranger in a strange land. He was a Democrat in a Republican White House; an economic globalist working for a president who campaigned on economic nationalism. Now, it seems, Donald Trump’s protectionist bent has pushed the top administration economic adviser to the exit.
This was not an unexpected development. By many accounts, there had been a contentious White House fight over whether to impose sweeping sanctions on US steel and aluminium imports – a tug-of-war that was settled, precipitously, by the president himself last week.
There were the rumours that Mr Cohn was only sticking around to see last year’s tax bill over the finish line, after his extreme discomfort following the president’s warm words about some of the white nationalist marchers involved in violent clashes in Charlottesville last August.
Mr Cohn was reportedly viewed by many Trump loyalists in the White House as an unwelcome interloper. Some on the outside, particularly in the financial world, welcomed him as a moderating influence – along with son-in-law Jared Kushner and daughter Ivanka.
Now the former is leaving and the latter two seem greatly weakened. All this could mark a sharp new direction in White House policy.
Last week, Mr Trump announced he would be imposing steep tariffs on steel and aluminium imports – 25% and 10% respectively. He is yet to sign them into effect.
He has regularly argued that other countries have been “taking advantage of” the US on trade for decades.
Trading partners reacted angrily. The EU, which says the steel and aluminium tariffs could cost it €2.8bn ($3.48bn; £2.5bn) a year, has now drawn up a $3.5bn hit list of retaliatory tariffs.
There are now fears of a global trade war. Mr Trump said on Tuesday: “When we’re behind on every single country, trade wars aren’t so bad.”
But there is disquiet at his proposals even among members of Mr Trump’s Republican party.
House Speaker Paul Ryan was one of those urging Mr Trump to have a “smarter” plan that was “more surgical and more targeted” and avoided the “unintended consequences” of a trade war.
Who will replace Mr Cohn?
Mr Trump tweeted that he would pick Mr Cohn’s replacement “soon”.
“Many people wanting the job – will choose wisely!” he added.
Possible candidates mooted by US media include Peter Navarro and Larry Kudlow, a conservative commentator and 2016 campaign adviser.
President Trump tweeted that there was no chaos at the White House but there were “still… some people that I want to change”.
Who else has left the White House this year?
A number have gone, adding to the string of senior figures who have left since Mr Trump took office.
Hope Hicks – Mr Trump’s press secretary quit a day after testifying to a congressional panel investigating Russian influence on the 2016 election, telling them she had occasionally told “white lies” for her boss
Rob Porter – The White House staff secretary resigned after two of his ex-wives publicly accused him of physical and emotional domestic abuse
Andrew McCabe – The FBI deputy director was forced out amid repeated criticism from Mr Trump that he was impartial in the Russia investigation