The Hercules C-130 plane hit two houses and a hotel before bursting into flames creating a huge fireball.
Air Force head Agus Supriatna visited the crash site and told the AFP news agency that there were no survivors among the 113 people on board.
Around 50 bodies have been taken to hospital – others are being recovered.
A major rescue operation is under way at the site which was covered in flames and thick black smoke.
The BBC’s Alice Budisatrijo in Jakarta says that only the tail of the aircraft is still recognisable; the rest has been reduced to debris.
Our correspondent says that there are reports that people are trapped inside wrecked buildings and the nearest hospital is continuing to receive bodies arriving from the crash site.
It is too early to know exactly how many people were killed in the disaster, our correspondent says, or what caused it.
Large crowds watched the emergency services search the flaming wreckage.
The plane had just taken off when it ran into trouble.
“It passed overhead a few times, really low,” a witness told the Reuters news agency.
“There was fire and black smoke. The third time it came by it crashed into the roof of the hotel and exploded straight away.”
Air Force Chief of Staff Agus Supriatna said that the pilot had asked to return to base because of technical difficulties.
“The plane crashed while it was turning right to return to the airport,” he said.
The Hercules transport plane was manufactured in 1964, but a military spokesman said he was convinced that it was in good condition.
Correspondents say that it is the second time in 10 years that a plane has crashed in Medan.
In September 2005, a Boeing 737 came down in a crowded residential area shortly after take-off from Medan’s Polonia airport, killing 143 people including 30 on the ground.
The latest crash in Medan is one of several involving military aircraft since 2009:
- April 2015: An F-16 fighter jet catches fire as it takes off from an airbase in Jakarta,
- March 2015: Two air force planes from an Indonesian aerobatics team crash during a practice session
- November 2013: An army helicopter crashes near the Indonesian-Malaysian border, killing 13 people
- August 2013: A door from a training helicopter falls onto a residential area of Jakarta
- 2012, nine people killed after an Indonesian air force plane crashed into a housing complex in Jakarta
- 2009 military transport plane carrying troops and their families crashed in Java, killing 98 people