Police say the rebels, from the United Liberation Front of Assam (Ulfa) were shot dead in the Tinsukia district, about 560 km east of the state capital, Guwahati.
One of those killed was a senior Ulfa commander, Pradeep Gogoi.
Ulfa rebels have been fighting for a separate homeland and have refused to follow the example of smaller insurgency groups and surrender.
Monday's encounter lasted only 30 minutes after security forces surrounded the rebels and opened fire.
"Security forces raided a house where the five Ulfa militants were sheltering and on seeing the troops the rebels fired," Tinsukia police chief, PC Haloi, is quoted by the AFP news agency as saying.
"The police and soldiers retaliated, killing them all," he said.
Surrender
Last month, more than 500 rebels laid down their weapons in Assam, in what the authorities described as the biggest mass surrender ever.
Although many of the rebels were said to belong to Ulfa, a commander of the rebel group rejected the move, describing it as a "stage-managed" exercise.
Ulfa believes Assam should be rid of non-ethnic Assamese - including Bengali-speaking migrants from West Bengal and neighbouring Bangladesh.
More than 10,000 people are reported to have been killed in 20 years of fighting in Assam.