Nikki Haley told an emergency meeting of the Security Council in New York that the US did not want a war but its patience was "not unlimited".
The US will table a new UN resolution shortly to toughen sanctions.
China, the North's main ally, has called for a return to negotiations and Switzerland has offered to mediate.
Meanwhile South Korea's navy carried out live-fire naval drills on Tuesday, warning that if the North provoked them "we will immediately hit back and bury them at sea", reported Yonhap news agency.
It comes a day after the South's military simulated a missile attack on the North's nuclear test site.
Reports suggest the North is preparing new test missile launches.
On Sunday, it tested a bomb underground, which was thought to have a power range from 50 kilotonnes to 120 kilotonnes. A 50kt device would be about three times the size of the bomb that destroyed Hiroshima in 1945.
'Time for dialogue'
Ms Haley argued that only the strongest sanctions would enable the problem to be resolved through diplomacy.
"War is never something the United States wants," she said. "We don't want it now but our country's patience is not unlimited."
China's envoy to the UN, Liu Jieyi, reiterated a call for all sides to return to negotiations.
"The peninsula issue must be resolved peacefully," he said. "China will never allow chaos and war on the peninsula."
Speaking in Berne, Swiss President Doris Leuthard pointed to her country and Sweden's long record in neutral and discreet diplomacy.
"I think it really is time for dialogue," she said. "We are ready to offer our role for good services as a mediator. I think in the upcoming weeks a lot will depend on how the US and China can have an influence in this crisis."
China is key but it is a conflicted party. On the one hand it does not want to see a nuclear-armed North Korea and it has made its view clear to Pyongyang on many occasions.
However, it does not want to see the North Korean regime swept away. This would result in millions of refugees flooding into China and would probably result in a unified Korea very much in the US orbit. This is seen in Beijing as worse than having a difficult nuclear neighbour.
If China were to take the view that the coincidence of a rapidly advancing North Korean nuclear programme and the uncertainties of the Trump administration's diplomatic capabilities means that there is a very real risk of misunderstanding and catastrophe, then maybe it might bring much greater pressure to bear on Pyongyang.
North Korea is a very isolated country and China is both its major ally and economic prop. There is a lot more that China can do. North Korea's recent testing has been as much an embarrassment to China as it has angered the US. But the Chinese have a difficult diplomatic calculation to make.
What other sanctions remain?
Last month, the Security Council voted unanimously to ban North Korean exports and limit investments in the country.
Ms Haley did not spell out what additional measures might be taken but diplomats have suggested an oil embargo would have a crippling effect.
There could also be a ban on the North's national airline, curbs on North Koreans working abroad, and asset freezes and travel bans on officials.
On Tuesday, South Korean Foreign Minister Kang Kyung-wha said after a conversation with her Chinese counterpart that she believed Beijing "could be open to more sanctions".
How big was the latest test?
It was North Korea's sixth nuclear test to date. The US Geological Survey recorded a resulting tremor at 6.3 magnitude.
Kim Jong-un was pictured on camera being shown what state media said was a new type of hydrogen bomb.
South Korea said it was now presumed that the North had reduced its nuclear warhead in size to below 500kg, and would be able to attach one to an intercontinental ballistic missile (ICBM).
But analysts have said the North's claims about miniaturisation should be treated with considerable caution.
Earlier on Monday, a South Korean defence ministry official said there were signs of "possibly more ballistic missile launches", including an ICBM.
How else are the South and its US ally responding?
Monday's drills simulated the targeting of the Punggye-ri nuclear site in Kilju County, where North Korea carried out its bomb test. Missiles were fired from the ground and rockets from fighter jets.
Seoul has said there will be more live-fire drills this month, as well as a naval battle group training exercise and a joint anti-submarine warfare exercise with the US.
South Korea and the US have also agreed after talks to scrap a warhead weight limit on the South's missiles, which are currently capped at 500kg (1,100lb).
The defence ministry in Seoul also told parliament the US would seek to deploy a nuclear-powered aircraft carrier to seas off the Korean peninsula.
Four more launchers of the US Thaad (Terminal High-Altitude Area Defence) missile defence system - strongly opposed by China and Russia - would also be deployed to join two already at a site in Seongju, south of Seoul.