Heavily armed police in Australia stormed a Sydney cafe where a gunman had held more than a dozen people hostage early Tuesday, ending a tense standoff that lasted more than 16 hours and saw at least three people die, including the gunman.

Police could be seen entering the cafe at about 2:30 a.m. local time on Tuesday, after a gunman held at 17 people in the central business district of Australia’s largest city. Gunshots could be heard ringing out on video feeds from the scene, and local media reports indicated injuries were sustained by both hostages and police.

The gunman died in the police raid, authorities said, and he was identified as Man Haron Monis, an Iranian-born man and a self-described cleric who has been on authorities’ radar in the past. A chaotic scene unfolded as police raided the cafe, with hostages running outside and an officer carrying out at least one ailing hostage.

New South Wales Premier Mike Baird said the gunman had carried out “horrendous vicious attacks.”

“We are a peaceful society which is the envy of the world,” he said in a televised news conference. “Today we must come together like never before.”

Two hostages died, five escaped and six were uninjured, police said. At least one police officer suffered a minor gunshot wound to the face, but was alive. Police wouldn’t say whether the hostages were killed by the gunman or in crossfire during the raid, the Associated Press reports.

“They believed if they didn’t enter there would have been many more lives lost,” New South Wales Police Commissioner Andrew Scipione said of the officers’ decision to raid the cafe. “Events that were unfolding inside of the premises led them to the belief that now was the time to deploy.”

It started Monday morning, when hostages were seen displaying a black and white flag in the window of the Lindt café in Martin Place—a major commercial precinct usually crowded with office workers and tourists and, at this time of year, Christmas shoppers.

The flag bore, in Arabic text, what was thought to be the shahada, or Muslim testimony of faith. The flag, which is commonly flown by Islamist terrorist groups, sparked fears that a terrorist attack was unfolding. But Sydney police had not said if it is a terrorist attack, and an official later described it as an “isolated incident.” Authorities confirmed earlier they had made contact with the perpetrator and were in negotiations. Several hostages had left the scene late Monday before the chaos that unfolded early Tuesday.

U.S. President Barack Obama was briefed overnight on the crisis.

“This is a very disturbing incident,” Prime Minister Tony Abbott said in a televised address earlier Monday. “It is profoundly shocking that innocent people should be held hostage by an armed person claiming political motivation.

“We are a free, open and generous people, and today we have responded to this in character,” he added.“Yes, it has been a difficult day. Yes, it has been a day which has tested us, but so far, like Australians in all sorts of situations, we have risen to the challenge.”

Australia’s paramount figure on Islamic law, the Grand Mufti Ibrahim Abu Mohamed, issued a statement “unequivocally” condemning the action. He said the Muslim community was “devastated” by the incident and said such actions are “denounced in part and in whole in Islam.”

Buildings in the area, including the famed Sydney Opera House and the U.S. consulate, had been evacuated, with office workers taken to nearby Hyde Park. Martin Place train station, one of Sydney’s busiest, was closed, as were major nearby roads.

“Sometimes here in Australia you think something like this would never happen so it’s pretty shocking to see,” said Kristina Ryan, who works nearby at Circular Quay. “It’s been really frustrating with the lack of information and how much longer can they expect us to just sit here without understanding why this is happening? I think that’s really adding to the fear people are feeling in the city.”

Ryan said that fear was amplified as she and her co-workers watched as authorities evacuated the Opera House.

“There are lots of government buildings around. It’s a very busy place, especially on a Monday morning,” Clarke Jones, a terrorism expert at Australian National University’s College of Asia and the Pacific, tells TIME.

The unfolding hostage crisis comes more than two months after Australian authorities foiled a terrorist plot by local ISIS supporters, who were reportedly planning to behead members of the public in Martin Place.

Days later, on Sept. 21, forces affiliated with the ISIS released a 42-minute audio recording calling on followers to attack non-Muslims in Australia. The call to arms appears to have been made in retaliation for Canberra’s deployment of military personnel and fighter jets to the Middle East to fight in the international coalition against ISIS and al-Qaeda-affiliated groups in Syria and Iraq.

According to Jones, officials believe that up to 60 Australian nationals and residents are believed to be currently fighting in jihadist ranks in the Middle East. Dozens are believed to have already returned to Australia.

“They haven’t been prosecuted, but we don’t know at this stage if they’ve come back with an added intent to continue the fight here in Australia,” says Jones. “It’s hard to know.”

As darkness fell on Monday evening, a crowd of around 200 people remained milling around the scene. “Everyone was quite calm,” says Victor Domni, who works at Macquarie Bank directly across from the Lindt cafe and was among the first to be evacuated this morning. “I wasn’t in a position to do anymore work so I was ready to go home. I’m quite hesitant to go into work tomorrow because it’s quite scary.”

Abbott urged his fellow Australians to remain coolheaded.

“The whole point of politically motivated violence is to scare people out of being themselves. Australia is a peaceful, open and generous society,” he said on Monday morning. “Nothing should ever change that.”

-Additional reporting by Courtney Subramanian / Sydney

Read next: Australians Use #IllRideWithYou Hashtag in Solidarity With Muslims During Sydney Siege