“Vengeance can wipe out an entire generation,” he said, in an address to the nation on Sunday, urging calm.
A presidential spokesman told the BBC it was too early to blame anyone for the killing of Gen Adolphe Nshimirimana in a rocket and gun attack on his car.
Mr Nkurunziza won a third term last month, amid deadly protests over his decision to run for office again.
Opponents argued this contravened the constitution and there was a failed coup attempt in May.
“Adolphe Nshimirimana was the regime’s number two, not officially, but unofficially. This assassination is a very serious blow to Burundi’s government,” International Crisis Group analyst Thierry Vircoulon told BBC Afrique.
The attackers reportedly targeted the general’s car with machine guns and rocket launchers in the Kamenge district of the capital Bujumbura.
Presidential spokesperson Gerve Abehayo rejected suggestions that the attack could have been carried out by elements within Burundi’s own security apparatus, after witnesses reported four men in military uniform spraying the general’s car with bullets.
“The government is not losing support… the army remains strong and united… one general was killed, but this does not mean the whole military has been wiped out of this country,” he told the BBC’s Newsday programme.
President Nkurunziza has called for an investigation into the general’s killing to be concluded within a week.
The African Union, European Union and the US have all condemned the attack, with AU chairperson Nkosazana Dlamini Zuma describing it as a “barbaric act that is likely to further destabilise the country”.
Prominent Burundian journalist Esdras Ndikumana, who works for AFP news agency and Radio France Internationale (RFI), says he was detained by intelligence officers and badly beaten after trying to take photos at the scene of the attack.
More than 70 people have been killed since the unrest began in April, and 180,000 have fled the country, according to the UN refugee agency.
Mr Nkurunziza came to power in 2005, after 300,000 were killed in a 12-year conflict between ethnic Hutus and Tutsis.