They captured the suburb of Kinama, but the army said at the weekend it had retaken the district.
It said 10 soldiers had died in the fighting. There has been no independent confirmation of either death toll.
He said that moves towards peace in the Democratic Republic of Congo could have a destabilising effect on other countries in the region.
Uganda and Rwanda have both pulled troops out of DR Congo in the past several weeks.
Locals allowed home
The Hutu-dominated rebel National Liberation Forces (NLF) launched an attack on Bujumbura on 24 February, the biggest offensive in Burundi's seven-year-old civil war.
They captured the suburb of Kinama, but the army said at the weekend it had retaken the district.
A curfew remains in place, Radio Rwanda reported on Monday.
A BBC correspondent allowed into the suburb on Monday said crews in protective clothing were disinfecting the ground where dead bodies had lain.
She said a Roman Catholic church where rebels had set up a machine-gun position was almost completely destroyed.
However, damage elsewhere was more limited, she said, and corpses had largely been cleared away.
Rebels withdraw
The army said it would allow most residents to return to their homes Monday, although the commander of the armed forces Group for the Defence of the Capital (Godec) said some neighbourhoods had been too badly damaged for people to return.
The rebels of the NLF are said to have withdrawn to hills around Bujumbura.
Bujumbura's airport virtually ceased to function last week after three carriers suspended flights to Bujumbura.
The companies are reported to have received threats, to the effect that if they did not cancel their flights, their planes risked being shot at.
There were rumours on Sunday that Regional Air would resume flights to Bujumbura as soon as Monday, but the airline denied that report on Monday, according the French news agency AFP.
More than 200,000 people, mostly civilians, have been killed in Burundi's civil war since 1993.
The latest fighting does not bode well for the next round of Burundi peace talks, scheduled to take place in the Tanzanian town of Arusha later this month.