The allegations are the subject of an investigation by cabinet office civil servant Sue Gray, a former director of propriety.
It comes as campaign group the Good Law Project warns the Met Police it will take legal action if the force does not investigate the gatherings.
The cops have so far said they will investigate the events if Ms Gray finds evidence of potential criminality.
The Labour mayor told London assembly members: “I think there is an issue about policing by consent… about the public having confidence that everyone abides by the rules.”
He said if it appeared “that there is one rule for them and one rule for the rest of us, at a time when they appear to be laughing at us, that doesn’t bode well for trust and confidence”.
While he said assembly members (AMs) who asked questions about the Met’s response to the claims were “articulating in a very polite way the anger felt by millions across the country.”
The mayor said: “Not only will I, but the policing and crime committee and the home affairs select committee, be ensuring that justice occurs and justice is seen to occur.
“It does affect the confidence or lack of confidence in the police going forward.”
Mr Khan stopped short of saying he would call on the Met to investigate the parties.
He told the assembly: “There’s a very good reason why the police and crime commissioner shouldn’t be directing what the police do but also [shouldn’t] give the impression of bias on the part of the police who do police without fear or favour.”
Labour AM Len Duvall OBE asked Mr Khan: “Can you ask the police, please explain to members of the public why you’re not taking action on these occasion?”
He said: “What is really galling and makes people really angry, [is that] during the time of these parties weddings were broken up - rightly because they were breaking rules.
“The police need to walk the extra mile to explain to the public why no action is being taken.
“It can’t be right that the great and the good get away with it while the ordinary public are taken to the courts, while people in charge in the national government flout the rules and flagrantly break them.
“I don’t think that’s politically partisan, I’d be saying that if Labour was in control.
“They should be brought to task.”
Caroline Pidgeon, Liberal Democrat AM, asked Mr Khan if he was “satisfied with the way the Met has responded to reports of parties in Conservative HQ and No10 in recent weeks?
The prime minister has previously apologised in the House of Commons for attending a gathering he said he “implicitly believed” was a work event for 25 minutes in May 2020.
And he has urged opponents to wait for the findings of Ms Gray’s inquiry, which is widely expected to be released next week.