Ansari is a Member of the Parliament representing Bihar in the Rajya Sabha, the upper house of the Parliament.
"Joining hands with BJP is unacceptable", he said.
Meanwhile, RJD supporters, reportedly, blocked Mahatma Gandhi Setu, connecting north Bihar with Patna, over Nitish Kumar's resignation and forming government with the BJP.
Less than a day after resigning as Bihar Chief Minister, Nitish Kumar would take oath at 10 a.m. on Thursday as the head of the government that would include BJP and its allies.
Kumar met Governor Keshari Nath Tripathi late on Wednesday with NDA leaders and staked claim to form a new government with support of 132 MLAs.
Talking to the reporters after meeting the Governor, BJP leader and former Deputy Chief Minister Sushil Kumar Modi said that Nitish Kumar had been invited to take oath at 10 a.m. on Thursday.
"We gave letters of support of 132 MLAs," he said.
Sushil Modi said they had requested the trust vote should take place at the earliest.
Asked if meeting the Governor in late hours reflected a fear that some MLAs could leave and join the RJD-Congress alliance, Sushil Modi said "others may have fear, not us."
Sushil Modi said the letter of support had been given of MLAs belonging to JD-U, BJP, LJP, RLSP, HAM and two independents.
State BJP chief Nityanand Rai told reporters that Nitish Kumar and Sushil Modi will take oath on Thursday.
Meanwhile, RJD, which had sought time to meet Governor to stake claim to form government and was asked to meet him at 11 a.m., questioned the rationale of asking Nitish Kumar to take oath at 10 a.m.
"Governor gave us time of 11 a.m. and now suddenly has asked NDA for oath ceremony at 10 a.m. Why so much hurry & rush Mr. Honest & Moral?" Tejashwi Yadav tweeted.
He said the Governor had initially asked Nitish Kumar to take oath at 5 p.m. on Thursday.
"Why is the Governor changing his decision in the night. The swearing is taking place at 10 a.m. instead of 5 p.m. We are going to Raj Bhavan. We are going to Governor house to stage the dharna if not called for being the single largest party.
"A Governor should act as custodian of constitution, not as Centre's stooge. He can't ignore single largest party and largest pre-poll alliance," Tejashwi said.
He alleged that Nitish Kumar had gone to Raj Bhavan at midnight as half of JD-U MLAs were "in our touch", he added.
"If Nitish Ji is proud of his moral values and honesty, he would not had been in midnight rush to stake claim for government. Honest man does not fear," the RJD leader said.
In dramatic and fast-paced developments in Bihar, Nitish Kumar on Wednesday evening resigned as Chief Minister dumping the RJD and Congress to stitch a new alliance with BJP, which quickly announced support to a new government under him and said it will join it too.
Nitish Kumar ended a 20-month-long grand alliance coalition that defeated the BJP in 2015 and gave his resignation claiming he can't work in the "current circumstances" following corruption charges against Deputy Chief Minister Tejashwi Yadav -- son of RJD chief Lalu Prasad.
(With inputs from agencies)