Rocco,
Something worth considering, and that might account for both yours and Jan's comments:
While it's true that the problems are generally isolated, and not universal within the city limits, it is also true that the areas they are isolated to change, shift, and grow.
For people who have historically visited a city and are used to visiting certain parts of it that were once safe, a shift in the patterns of dangerous incidents feels like the city is being taken over -- because the parts that they once loved and felt were safe ARE being taken over.
This pattern is true whether the dangerous incidents are growing or merely shifting location. And, worse; an area that was once safe becoming dangerous has a much greater psychological impact on people than an area that was once dangerous becoming safe.
Also, even within moderately dangerous areas, the psychological difference between a "dangerous" area you have traversed safely for ten years and one where you were mugged last week is enough to spoil one's experience of a city. At that point you have to put in quite a bit of mental work to partition that experience into the small context where it occurred so that you can get back to enjoying the rest.
J