The Lions hosted the Eagles in a south London derby on Saturday in the cup, with Palace coming from behind to win 2-1.
Young winger Michael Olise was the hero for the Eagles, assisting and scoring for the Croydon outfit.
One bottle hit Olise in the head, with the forward gesturing to the home fans that he was not intimidated.
Scotland Yard confirmed it was investigating the incident, and released images of two men that officers wish to speak to.
Insp Matt Ashmead, from the central football unit, said: “The vast majority of football fans go to matches simply to enjoy themselves and support their team.
“Sadly, a small minority are intent on causing trouble and resorting to violence.
“We are committed to identifying them and ensuring they face appropriate sanctions.
“We have a close relationship with clubs across London and in this case, we have worked with Millwall to identify those involved.
“I would appeal to the public to come forward with any information that could help us to identify the two men.”
Anyone with information should call police on 101, giving the reference 3200839/22. Information can also be provided to Crimestoppers, anonymously, on 0800 555 111.