His main rival Edward Lowassa has rejected the official results that gave him 40% of the ballots cast.
The opposition Ukawa coalition candidate earlier claimed he had won with 62% of the vote.
The elections on Sunday were the most fierce the governing party faced after 54 years in power.
John Magufuli is celebrating his 56th birthday so the presidency is a perfect gift for him. He was never a CCM insider and confounded many when he was elected as the ruling party’s presidential candidate.
As works minister in the outgoing government, Mr Magufuli was reputed to be a no-nonsense, results-driven politician. He became known as “The Bulldozer” for driving a programme to build roads across the country.
He campaigned for the presidency on a platform of hard work, and will now have to tackle far bigger problems facing the East African state. This includes constant power outages, and corruption – an issue which led to many people turning against CCM in the election.
In Zanzibar, elections for the semi-autonomous archipelago’s parliament and president were annulled on Wednesday.
Zanzibar’s election chief Jecha Salum Jecha said the poll had been marred by gross irregularities, including rigging and physical fights between rival election commissioners.
CCM supporters have been celebrating Mr Magufuli’s victory outside CCM’s headquarters in Tanzania’s main city, Dar es Salaam.
- John Magufuli, CCM: 8,882,935 votes – 58.46%
- Edward Lowassa, Ukawa: 6,072,848 votes – 39.97%
President Jakaya Kikwete, who is standing down after two terms in office, retweeted a CCM photo of Mr Magufuli and the accompanying words: “Our next Commander-in-Chief, Dr John Pombe Magufuli, the President-elect of The United Republic of Tanzania.”
Abdallah Safari, vice-president of Chadema, one of the four opposition parties that make up Ukama, told the BBC that Tanzanians “have been robbed of their victory”.
BBC Tanzania analyst Zuhura Yunus says the result is a big blow for Mr Lowassa after four opposition parties put their faith in him, uniting for the first time to field a single candidate.
She says Mr Lowassa is convinced he won and the question now is whether he will challenge the result further, or throw in the towel.
European Union observers said that the elections were “generally well organised” but “with insufficient efforts at transparency from the election administrations”.
Teams from the African Union and southern African regional body Sadc said that the vote had largely been “free and fair”, despite all groups raising concerns over the subsequent annulment of Zanzibar’s local elections.