Hundreds of people fled from the main town in Pakistan's Swat valley on Tuesday after a government official urged residents in some neighbourhoods to seek safety as fresh fighting with Taliban militants could erupt.

A February peace pact aimed at ending Taliban violence in the Swat valley northwest of the capital has all but collapsed as the government comes under US pressure to get tough with the militants rather than appease them.

President Asif Ali Zardari, former prime minister Benazir Bhutto's widower, is due to meet US President Barack Obama and Afghan President Hamid Karzai in Washington on Wednesday for talks on the growing militant threat in the region.

The top government official in Swat, Khushal Khan Khattak, said militants had infiltrated five districts of Mingora, the main town in Swat, and begun attacking security forces and government installations.

He said security forces could soon attack the militants and  urged people to get out of harm's way. But he later said the fear of fighting had passed and people could stay home, while the army said it was in control of the town.

That did not reassure the people cramming into and on top of buses to get out of Mingora, 130 km (80 miles) from Islamabad.

'I'm taking my family to Peshawar because if there's any fighting, no one can protect us,' said Mohammad Karim, as he searched for a bus heading out of the valley to Peshawar, the main city in the northwest.

Some residents estimated thousands were fleeing. Hundreds of thousands of people have fled from fighting in different parts of the northwest since last August, putting another burden on an economy being propped up by a $7.6 billion International Monetary Fund loan.

Increasing violence and the Taliban's spread have raised alarm in the United States about the ability of the nuclear-armed country, whose help is vital in efforts to stabilise Afghanistan, to stand up to the militants.

Residents of Mingora said earlier militants had surrounded a paramilitary force base at a power station in the town and others had taken up positions on buildings and were patrolling streets.

The military said in a statement security forces had beaten back an attack on the camp but a senior military official in the region said an operation might be launched to rescue 46 paramilitary soldiers besieged there.

'We're acting with restraint because they're using civilians as a shield but we'll go after them if the situation gets worse,' said the military official, who declined to be identified.

Pakistani stocks ended up but off the day's highs as investors remained cautious amid mounting expectation the military would step up its operations against the militants.

The February peace pact, under which authorities agreed to a Taliban demand for introduction of Islamic sharia law in the former tourist valley, led to accusations from critics both at home and abroad that the government was caving in to militancy.

The Taliban refused to give up their guns and pushed into Buner district, only 100 km (60 miles) northwest of Islamabad, and another district adjacent to Swat last month.

US Secretary of State Hillary Clinton last month accused Islamabad of abdicating to the Taliban while Obama expressed grave concern the government was 'very fragile' and unable to deliver basic services.

Amid the mounting concern, security forces launched an offensive to expel militants from Buner and another district on April 26. About 180 militants have been killed, according to the military, although there has been no independent confirmation.

The operation in Buner was going smoothly and troops were consolidating their positions, the military said.

At this week's Washington talks, Zardari is expected to do his utmost to convince Obama the government is on<