Really, I think the difference for the modern/abstract art and music lover, or at least for me anyway, is that while most people experience art and music in a fairly surface, sensory manner, and therefore gravitate, quite reasonably, toward art that's comfortable and pleasant feeling, I tend to experience art from a far more argumentative, analytical perspective. Most people prefer stuff that calms the senses; I, and a minority of other cantankerous folks (many of whom tend to be critic-types) prefer material that riles the mind. This is often a source of frustration for critic types who feel that everyone should follow their experience, and although I don't propose a solution, it does seem to me that critics and others of similar disposition should generally refrain from castigating general audiences for not getting something. (Other critics, however, are fair game.) [emp. mine]
I think this points to something that most everyone acknowledges during casual conversations about the arts, but that doesn't often make its way into on-the-page (or screen) critical writing: your preferences, background, etc. are going to play a huge role in determining what kinds of movies, books, music, etc. engages you, what kinds bore you, and what kinds just put you off.
For example, it doesn't surprise me at all that Grant Morrison was named the #1 comics writer by comics bloggers: his comics are written as if their primary purpose is to be blogged about. That is, they require and reward a certain kind of allegorical interpretation and symbolic puzzle solving that is enhanced by the cross-blog-pollination of interpretations and solutions.
Another favorite example (which I've stolen from Michael Blowhard), the canonization of Ulysses by academics probably has something to do with how well that book rewards academic-style reading and criticism.
I suppose I'd, er, prefer it if more critics would be up front about their own preferences. I mean, I know it's best to read any kind of value judgment in a piece criticism as if the author had stuck an "In my opinion" in front of it, but revealing personal preferences gives your audience a better handle on the way you approach art and makes your value judgments a lot more useful.