What's the issue?
People associate Islam with terrorists thus a Muslim place of worship seems scary. Why? Media, media, media. The problem is we can't generalize the acts of a small group of extremists who were in the wrong onto a huge population of people. It's just not logical.
Think of this. Of all the psychologists in the American Psychological Association (APA) 5 of them go and commit some sort of attack on some group. Are we then going to say the APA is a hub for terrorists and we should condemn them all? No, absolutely not. The actions of that group of individuals were wrong and they should be condemned. End of story.
And anyhow the building of a mosque would be the last thing extremists want to do. Why? Because a mosque is a place of prayer, tranquility, community, and understanding. It is a house of God and in it are people who believe in the actual tenets of Islam and practice them in their everyday lives. Park 51 would do wonders in New York City. It would be a hub of interfaith dialoge and community building.
A lot of bloggers have written excellent posts on the topic. Check out Sabrina's Slice of Lemon for an awesome article on the issue.
The Daily Show had an awesome segment on it too!