Rishi Sunak defended his decision to launch an ethics inquiry into Nadhim Zahawi rather than sacking him, at yesterday's Prime Minister's Questions.
Sir Keir Starmer claimed the PM was "hopelessly weak" for not firing the ex-chancellor.
"Is he starting to wonder if this job is just too big for him?" the Labour leader asked.
Mr Sunak said it was Sir Keir who was weak because "he has no principles - just petty politics".
Downing Street initially said Mr Sunak's tax arrangements were "confidential" when asked by Labour if he had ever paid a penalty to the UK tax authorities, like Mr Zahawi.
But the PM's spokesperson later confirmed that he had not, saying: "The prime minister has never paid a penalty to HMRC."
Tax returns
The PM will publish his tax returns "in due course", Downing Street has said.
In the House of Commons, Mr Sunak said it would have been "politically expedient" to sack Mr Zahawi as a minister before PMQs got under way at noon but he believed in "proper due process".
That was why, he said, he had asked ask his ethics adviser to investigate whether the Conservative Party chairman had broken ministerial rules.
It will be up to the PM to decide whether to sack Mr Zahawi if his ethics adviser says he has broken the ministerial code.
The BBC reports that Mr Zahawi was chancellor at the time the estimated £4.8million settlement was agreed with HMRC.
Sir Keir asked Mr Sunak why he had said at last week's PMQs that Mr Zahawi had "addressed this matter in full".
More information
,"Since I commented on this matter last week more information, including a statement from the minister without portfolio (Mr Zahawi), has entered the public domain which is why it's right that we do establish the facts," the prime minister said.
He accused Sir Keir of "simple political opportunism" for urging him to appoint an ethics adviser then wanting a decision before they had investigated the case.
And he claimed "the difference between him (Sir Keir) and me is I stand by my values and my principles even when it is difficult," accusing the Labour leader of indulging in "petty politics".
Sir Keir said the PM's "failure" to sack Mr Zahawi showed "how hopelessly weak he is - a prime minister overseeing chaos, overwhelmed at every turn".