RIO DE JANEIRO (AP) — Two American Olympic swimmers were taken off their flight from Brazil to the U.S. by local authorities amid an investigation into a reported robbery targeting 12-time medalist Ryan Lochte and his teammates.

A lawyer for the two athletes said Thursday they will not be allowed to leave Brazil until they provide testimony to investigators, who are still searching answers about how the swimmers were robbed Sunday morning.

Jack Conger and Gunnar Bentz left the Rio de Janeiro airport early Thursday after being taken off the plane Wednesday. They departed for an unspecified location in Brazil and had yet to testify, said attorney Sergio Viegas. The swimmers did not speak to reporters as they left the airport and were shuttled away in a black car waiting outside.

The action comes amid increasing tension between Brazilian authorities and the American swimmers over the ever-changing account of the robbery. Police say they have found little evidence so far to support their claims, and that the swimmers were unable to provide key details in interviews.

Lochte has said he was with Conger, Bentz and Jimmy Feigen when they were robbed at gunpoint in a taxi by men with a police badge as they returned to the athletes village from a party, several hours after the final Olympic swimming events were held.

Lochte has returned home to the U.S. Feigen, along with Conger and Bentz, remains in Brazil.

While he's medaled often in the Olympic pool, Lochte's accomplishments have long been overshadowed by teammate Michael Phelps — the most decorated Olympian in history. He won a gold in Rio in a relay race alongside Phelps.

NBC reported Wednesday night that Lochte backed off some of his earlier claims about the robbery. He now says the taxi wasn't pulled over by men with a badge, but that they were robbed after stopping at a gas station, NBC reported. Lochte also said the assailant pointed a gun at him rather than putting it to his head.

Patrick Sandusky, U.S. Olympic Committee spokesperson, said Conger and Bentz were detained and "released by local authorities with the understanding that they would continue their discussions about the incident on Thursday. James Feigen is also communicating with local authorities and intends to make further statements regarding the incident on Thursday as well."

Representatives from the U.S. consulate arrived at the airport shortly after the swimmers were stopped from leaving.

A Brazilian judge earlier ordered the seizure of Lochte and Feigen's passports. The office of Judge Keyla Blanc said in a statement that there were discrepancies in their statements.

The U.S. Olympic Committee said police went to the athletes village to try to collect the passports, but the swim team had already moved out.

Lochte's father told The Associated Press by phone from his Florida home that his son called him Tuesday after arriving in the United States. The 32-year-old swimmer was going to pick up his car and buy a new wallet to replace the one that he said was stolen in the robbery.

"I'm just happy he's safe," Steve Lochte said. "It was an unfortunate experience for him and the other three. I don't know what all the controversy is. They were basically taken out of the taxi and robbed. The main thing is he's very lucky that he's safe and that all they got was his cash and wallet."

The group did not call police, authorities said, and officers began investigating once they saw media reports in which Lochte's mother spoke about the robbery.

A police official with knowledge of the investigation told the AP that police cannot find their taxi driver or witnesses from the robbery. The official spoke on condition of anonymity because the investigation is ongoing.

Police interviewed Lochte and one other swimmer, who said they had been intoxicated and could not remember what type and color of taxi they rode in or where the robbery happened, the police official said. The swimmers also could not say what time the events occurred.

Lochte's attorney, Jeff Ostrow, has said there is no question the robbery occurred.

"Why would anybody fabricate anything?" Steve Lochte said. "It's just ridiculous."

Lochte described the incident to NBC's "Today" show Sunday.

"We got pulled over, in the taxi, and these guys came out with a badge, a police badge, no lights, no nothing just a police badge and they pulled us over," Lochte said. "They pulled out their guns, they told the other swimmers to get down on the ground — they got down on the ground. I refused, I was like we didn't do anything wrong, so — I'm not getting down on the ground.

"And then the guy pulled out his gun, he cocked it, put it to my forehead and he said, 'Get down,' and I put my hands up, I was like 'whatever.' He took our money, he took my wallet — he left my cellphone, he left my credentials."

Lochte told USA Today that he and his teammates didn't initially tell the U.S. Olympic officials about the robbery "because we were afraid we'd get in trouble."

Steve Bentz, father of Gunnar Bentz, when reached by phone late Wednesday night, said: "I really don't want to say anything," and hung up.