For days the government had blocked them from travelling by train, but it relented after many began walking towards the border on Friday.
About 4,000 people arrived in Austria, with many moving on to Germany. Meanwhile a second group has set off on foot from Budapest towards the border.
Austria says the migrants can apply for asylum there or carry on to Germany.
The move comes as European Union countries are struggling to agree on how to deal with a surge of asylum seekers – many of whom have fled conflict in Syria.
Hungary’s government eased restrictions on transit after many frustrated migrants overwhelmed police cordons.
Overnight, buses picked up migrants from Keleti station in Budapest where thousands had been camped.
Vehicles were also sent to take those who had decided to walk along a motorway to Austria.
On Saturday Hungarian government spokesman Zoltan Kovacs told the BBC there would be no more buses or trains to take the migrants on to Austria.
He said the transport had been arranged as a one-off, because of fears for the migrants’ safety.
But the BBC’s Ben Brown reports from Budapest that the government’s move seems to have encouraged migrants who later streamed into the station.
The second group of about 1,000 migrants is now on its way to the border.
As thousands crossed the border on foot, some Austrians displayed welcome signs.
Austrian Red Cross workers at a makeshift centre greeted the migrants with blankets and tea.
“I feel [at] home,” said Ayaz Morad, one of the first to arrive. “This is a great land – nice people, nice government.”
The migrants were being taken to Vienna by bus and rail. Special trains to the capital are leaving the Austrian border town of Nickelsdorf every half hour.
Many immediately boarded trains for Germany. Several hundred arrived at Munich station later on Saturday.
The German government has said anyone coming from Syria can apply for asylum. It expects about 6,000 people to arrive over the weekend.
Germany has said it expects to take in 800,000 asylum seekers this year.
Austria’s Chancellor Werner Faymann said that after talks with his German counterpart Angela Merkel, they would allow in the migrants.
But he said he expected Hungary to respect any EU quotas for asylum seekers – something Hungary, along with the Czech Republic, Poland and Slovakia, has rejected.
Hungary, a major transit point for those fleeing the Middle East and Africa, has taken measures that have underlined divisions within the EU over ways to tackle the crisis.
The government of Prime Minister Viktor Orban is building a barbed-wire fence along the border with Serbia, where it also plans to deploy extras troops and police.
On Friday the Hungarian parliament approved tougher border controls and penalties for migrants.
On Saturday, EU foreign ministers discussed the crisis at a meeting in Luxembourg.
EU foreign policy chief Federica Mogherini said they had agreed to strengthen co-operation in five areas:
- Protection of asylum seekers
- Border controls
- Fighting trafficking networks, particularly at sea
- Helping transit countries outside the EU deal with the influx
- Addressing the root causes of the crisis, notably instability in Libya and the Syria conflict.