"Hindu population is reducing in India because Hindus never convert people. Minorities in India are flourishing unlike some countries around (sic)," he tweeted.
The Minister of State for Home's comment came after the Arunachal Pradesh Congress Committee accused the Narendra Modi-led BJP government of trying to convert Arunachal Pradesh into a Hindu state.
"Why is Congress making such irresponsible statements? People of Arunachal Pradesh are unitedly living peacefully with each other (sic).
"Congress should not make such provocative statements. India is a secular country. All religious groups enjoy freedom & living peacefully (sic)," he said in a series of tweets, responding to the APCC's charge.
Rijiju hails from Arunachal Pradesh and is a practising Buddhist.
Reacting to his statement, All India Majlis-e-Ittehadul Muslimeen (AIMIM) chief Asaduddin Owaisi said he should remember that he is a "minister of India for all Indians not for Hindus only".
"Remember your oath as minister," the Hyderabad MP said.
"Wht have the minorities India got to do with minorities of 'other' countries. It is Constitution which guarantees rights (sic)," he tweeted.
According to the 2011 Census, Hindus make up India's 79.80 per cent of population, Muslims 14.23 per cent, Christians 2.30 per cent, Sikhs 1.72 per cent, Buddhists 0.70 per cent and Jains 0.37 per cent.
The country's Hindu population as per the 2001 Census was 80.5 per cent, while that of Muslims was 13.4 per cent, Christians 2.3 per cent, Sikhs 1.9 per cent, Buddhists 0.80 per cent and Jains 0.4 per cent.