The council condemned the test, saying “a clear threat to international peace and security continues to exist”.
This is the North’s fourth nuclear test since 2006, but if confirmed would be the first of an H-bomb.
However, the US has joined nuclear experts in questioning whether the blast was large enough for such a test.
US White House spokesman Josh Earnest said “initial analysis was not consistent with North Korea’s claims of a successful hydrogen bomb test”.
He added: “Nothing that happened overnight has changed our view of North Korea’s nuclear capabilities.”
The Security Council held an emergency session on Wednesday. It was called by the US, Japan and South Korea.
Uruguay’s UN Ambassador Elbio Rosselli, current president of the council, said: “The members… recalled that they have previously expressed their determination to take further significant measures in the event of another [North Korea] nuclear test.
“In line with this commitment and the gravity of this violation, [they] will begin to work immediately on such measures in a new Security Council resolution.”
Mr Earnest said North Korea’s isolation had “deepened as they have sought to engage in increasingly provocative acts”.
Japan’s ambassador to the UN, Motohide Yoshikawa, called for a swift and robust new UN resolution.
He said: “The authority and credibility of the Security Council will be put in question if it does not take these measures.”
However, he and other members have not spelled out what they will be or when the resolution could be adopted.
Russia’s UN ambassador said it would be going “too far” to say Moscow supported further sanctions.
North Korea’s tests in 2006, 2009 and 2013 triggered UN sanctions, with 20 entities and 12 individuals on a UN blacklist.
‘Bang would have been bigger’
If an H-bomb test were confirmed, it would mark a major upgrade in North Korea’s nuclear capabilities.
Hydrogen bombs are more powerful and technologically advanced than atomic weapons, using fusion – the merging of atoms – to unleash massive amounts of energy.
Atomic bombs, like the kind that devastated two Japanese cities in World War Two, use fission, or the splitting of atoms.
Bruce Bennett, an analyst with the Rand Corporation, was among those casting doubts on Pyongyang’s test: “The bang they should have gotten would have been 10 times greater than what they’re claiming.”
A South Korean politician, Lee Cheol-woo, said he was briefed by the country’s intelligence agency that the blast “probably falls short” of a hydrogen detonation.
Suspicions that North Korea had carried out a nuclear test were raised when an earthquake was registered near the Punggye-ri nuclear site in North Korea at 10:00 Pyongyang time (01:30 GMT), with the tremors rattling Chinese border cities.
Hours later, in a surprise announcement, a newsreader on North Korean state TV said: “The republic’s first hydrogen bomb test has been successfully performed at 10:00 am on January 6, 2016.”
A note signed by North Korea leader Kim Jong-un authorising the test said 2016 should begin with the “stirring explosive sound” of a hydrogen bomb.
China and Japan are reported to have been trying to detect radiation.