"They are asking me to go to Pakistan. Who has given them the right to decide that I am not a Hindu like them and so I cannot live in India?
"The BJP's talk of 'Hindu Rashtra' is really dangerous and will destroy this nation. Has Talibanisation started in Hinduism?" the Congress MP told workers of Kerala's opposition United Democratic Front (UDF) at a protest march against alleged neglect of the state by the Centre on Tuesday.
Members of the BJP's youth wing-- Bhartiya Janata Yuva Morcha (BJYM)-- had defaced Tharoor's office on Monday while demanding an apology for his "Hindu Pakistan" comments.
Tharoor had said the BJP, if voted to power again, will rewrite the Constitution and pave the way for the creation of a "Hindu Pakistan".
The Congress leader on Wednesday said he will not withdraw any of his statements and demanded an apology from those who attacked his office in Kerala.
Asked about claims that he was trying to break the country with his remarks, Tharoor told reporters outside Parliament that it was not so.
"If you make India for one religion, that can probably break the country. When everyone will be able to live together, how will that break the country," he said.
"Those who attacked my office should apologise. I am not withdrawing anything," he added when asked to comment on demands that he should apologise.
The Congress had distanced itself from his comments and cautioned its leaders to choose words carefully while rejecting the saffron party's "hatred".
Five BJYM workers were arrested and released on bail on Tuesday in connection with the incident of vandalism at Tharoor's office.
BJP workers had shown black flags to the former Union minister on Tuesday.
(With inputs from PTI)