Here’s the thing about speech: It’s never free.

Billy Bob Thornton was right when accepting his Golden Globe this past January: “These days you get into a lot of trouble no matter what you say… you can say anything in the world and get in trouble. I know this for a fact. So I’m just going to say thank you.”

Being right isn’t the same as being smart.

Justine Sacco was fired from her PR job after a tactless tweet about white people not contracting AIDS. When Patricia Arquette said in her Oscar acceptance speech that it’s time for all gay and colored people to “fight for [women] now,” she came under fire for being white, privileged, and insensitive to minority intersectionality. Journalists and cartoonists at Charlie Hebdo were threatened and then attacked for drawing cartoons of the Prophet Mohammad.

All these are cause and effect. You have the right to say and do anything within the realm of the law, but can’t ignore that words and actions carry consequences, regardless of whether or not those consequences are justified or legitimate. From internet shamers to fanatic extremists, the consequences can include public humiliation, loss of employment, and even violent and fatal attacks, which leads to the intersection of free speech, hate speech, and responsible speech; being right versus being smart.

Consequences are just that — not necessarily an attack on free speech.

In response to the attacks on Charlie Hebdo, Pamela Geller — co-founder and president of the American Freedom Defense Initiative, a right-wing organization the Southern Poverty Law Center considers an Islamophobic hate group — organized a Mohammad-drawing contest in Garland, Texas. Despite warnings by police of violent threats, Geller’s group went ahead with the contest, which also provided a $10,000 reward to the winner. Two Muslim brothers showed at the event and opened fire, injuring one police officer before being killed by police. “Freedom of speech is under violent assault here in our nation,” Geller said in response. “The question now before us is: Will we stand and defend it, or bow to violence, thuggery, and savagery?”

One person who decided to “stand and defend it” was former-Marine Jon Ritzheimer, who organized on May 29 a “Freedom of Speech” rally —featuring its own Mohammad-drawing contest — in Phoenix outside a mosque where the two shooters in Garland had spent time worshipping. Ritzheimer admitted the rally was a provocation and said to Anderson Cooper on his show the aim of the rally was to expose that “true Islam is terrorism.” The rally passed with a violence-free counter protest.

Both of these events in Garland and Phoenix were done under the umbrella of Freedom of Speech and fighting against its infringement. But Freedom of Speech is not under attack. Here’s an example of the system — protecting freedom of speech — at work: A few months ago, after the MTA refused to put up ads purchased by Geller that the MTA thought offensive to Muslims, a Manhattan federal judge ruled in her favor and forced the MTA to run the ads. That’s a free speech issue. An organization denied Geller’s free speech; a judge ruled in her favor; Geller got her way.

This example is different from the contest: No one stopped Geller from having the rally; people just decided to punish her for it. But Geller and her group had the right to protest. They protested. Then two Islamic extremists showed up and committed a crime, and they were shot and killed by police. That, also, is the system at work.

The term “free speech” is erroneously assigned.

It has become that if one person kills another because of his race, religion, or ethnicity, then it’s a hate crime. But if one person kills another because that person offends his religion, race, or ethnicity, then that becomes an act against freedom of speech. And this has become especially — if not solely — true when it concerns Muslims and Islam. But if I called someone an asshole and they shot me dead, that wouldn’t be a violation of my freedom of speech. It would be a consequence of my choice of words and the action of a violent person.

The problem also stems from the way the First Amendment is interpreted. Originally, the Bill of Rights was established to limit government power, which is to say it was meant to stop the government from infringing the individual’s freedom of speech and religion, and right to peaceful assembly. “Congress shall make no law… prohibiting… the right of the people peaceably to assemble.” The government can’t stop people from rallying and protesting, and in the case of Garland and Phoenix, the government didn’t. There were, however, fanatic individuals who didn’t like the nature of the protest and threatened violence and, in Garland, executed. That doesn’t make it okay, it just makes it an issue different from that of the First Amendment.

The term “free speech” is also often abused.

That isn’t to say there isn’t a problem here; there is. Once you’re legitimately afraid to step outside and speak your mind for fear of being killed, there is, in a sense, an infringement on your freedom to speak. Especially since it can become a slippery slope. But part of what deters this issue from really being about freedom of speech is the way in which Geller and Ritzheimer went about the protest; guising it under an umbrella of freedom of speech, while bordering on ignorance and hate speech so much that it becomes hard to take them seriously.

Geller told the New York Times she and members of her organization decided to hold the event in the same conference room they had heard a Muslim group had used for a conference after the attack on Charlie Hebdo. Her past projects include calling for the Dome of the Rock to be removed from the Temple Mount in Jerusalem, getting a Muslim principal in Brooklyn to resign, and stopping a mosque and Islamic cultural center from being built near Ground Zero in 2010.

Ritzheimer’s decision to protest outside a mosque during evening prayers while wearing a t-shirt that says “Fuck Islam,” makes him more provocative, spiteful, and hateful than an emblem of democracy and free speech at its finest. When he says that the point of the protest is to “show the true colors of Islam,” he’s essentially calling them out, taunting and daring Islamic extremists to show “Islam’s true face.” That’s probably also the reason why the facebook page event encouraged the six hundred anticipated participants to bring their guns, so that they can “utilize (their) second amendment right at this event just (in case) [their] first amendment comes under the much anticipated attack.” (And in fact, several protesters carried weapons.)

Words Matter.

There is an obsession in the United States with the First Amendment; the God-given right to know, say, and do anything. Manipulating this obsession, as Geller and Ritzheimer have done, is counterproductive to both the freedoms at hand and the real problem concerning Islamic and other extremists. Words matter; naming the problem something it’s not only hinders the solution and fans unnecessary flames.

The ironic thing is that the two shooters in Garland played directly into Geller’s hands and proved her point by doing exactly what she was hoping they would do. And both Geller and Ritzheimer (along with all their protestors), whether realizing it or not, played right into the hands of terrorist groups such as ISIS and Al-Qaeda (the latter took responsibility for the attack at Charlie Hebdo), perpetuating exactly what they’re trying to establish: a war between Islam and the West, the notion you can’t be Islamic without being anti-Western. Throughout all of this, groups like ISIS are the big winners, getting exactly what they want without having to lift a finger; American Islamophobes, rather, doing the dirty work for them, all the while claiming it’s entirely in the name of America’s most sacred freedom.