It's not just the comments, I'm more concerned about the submissions. Look at the front page right now, and see how many items are not "Hacker News" even with a very lax definition of "stuff that's interesting specifically to hackers."
It's clear why this happens. The people interested in fuzzy, quasi-political or "outrage" issues are passionately engaged with them. People are less passionate about the debate over the worst-case complexity of hash table lookups. At a step's remove, it's evident that the topics HN was intended to cover are systemically disadvantaged compared to whatever the moment's advocacy topics are --- and every moment has at least one advocacy topic: the TSA, SOPA, Occupy, Wikileaks, police misconduct, Asian manufacturing, ebook licensing, DRM, &c &c &c.
More passion, less thought, less tolerance for conflicting views, worse threads.
Paul Graham seems cautiously optimistic about comment quality. I think he's being too generous. I suspect that advocacy energy is cumulative; it doesn't go away when the topic fades, but sticks around and waits to bind to subsequent advocacy topics, causing the site to steadily crud up with them.
I don't think it's really an issue of passion. Rather, it is an issue of demographics: almost everybody is affected or at least mildly interested in politics; only smaller subsets of the population are interested in Haskell/Erlang/complexity/aglorithms...etc. Frankly, I bet some people are as "passionate" (for a loose definition of "passionate") about, say, functional programming as others are about politics; there are just more people with at least a casual interest in the latter than the former.
Also, as HN gets more readers, it becomes more diverse. I've seen doctors, medical students, lawyers...etc post. It's no surprise somebody like that is going to skip a post about using abstract mathematics to model programming or about optimizing lunch at your startup; they are probably gravitating towards the political articles. On the other hand, just being interested in start-ups and CS does not preclude looking at politics.
So really, I think this is a function of diversity: in a purely democratic system--everybody has an equal vote--less specialized things really have an advantage. This is why I don't bother reading the newspaper or front page of reddit, but still enjoy /r/haskell.
I'm sure there are also plenty of confounding variables; I just think it's more broad appeal than passion. (Seriously, have you read the responses to people's blindly disparaging functional programming? At least as passionate as most political posts ;).)
There are geopolitical/economics issues that are as important to hackers as algorithms. This past week, of SOPA, PIPA, OPEN, ACTA and other less beloved acronyms are of at least some interest for hackers because they directly impact our ability to hack things, to the point of having pg reminding us that the long-entrenched entertainment industry is ripe for disruptive innovation.
Here we find people of diverse tastes. I am an engineer and, given the possibility, I'd be designing processor architectures and instruction sets. I also like programming - and do it for a living, but I have lost some of my taste for algorithms and data structures and prefer to generate competitive differentials elsewhere on the stack. But I am also very worried by the political background, both here in South America, where I live, in Europe, where my ancestors came from, and with the US, which is a very fine country where a lot of dear friends live and that we shouldn't allow to be ruined by a bunch of crazy people. When I was a teenager, I read 1984 and the idea something like that could happen scared me a lot. It still does. Bad politics is contagious.
I think all is fine, as long as we remain civilized and don't forget to disagree in constructive ways. Community and relationships are important too - I have only one nickname, after all, one which is easily identifiable. All my friends know who I am, where I came from and what my opinions tend to be. I like to think that, in the many times we disagreed about things, we never ourselves a bad example.
===== Why It Happens Theory #1 - "Community Size Syndrome" ====
I am guilty of this. And I think I know why I/we do it. When a topic comes up, and you feel your voice isn't/won't be heard, it tends to make you over-emote, yell, and stomp though the place to get more attention / vent frustration.
Think about it. It wasn't like this when Hacker News was smaller, because when HN was smaller there were less people, less comments, and everyone had a voice that could be heard. Now that the place is bigger your "voice" is easily lost, your comment ends up at the bottom, you feel like your opinion won't matter. This stresses you out and pisses you off so you go and piggyback off of someone else's comment near the top and just vent.
We see this in political ralleys, demonstrations, protests. When humans feel they are NOT being listened to, they get loud and angry.
I think these are related. The kind of people who like to comment on some things — say, politics — tend to be less tactful. This drags the average down. Then, unless you're very lucky, people readjust their norms and start acting ruder themselves — and then you have yourself a vicious circle. I've seen it happen in a lot of places. It makes me sad every time.
I'd strongly suggesting policing OT posts more proactively. Like PG said in his essay about starting Hacker News, this is a social problem, not a technical one. We need to refrain from doing things that make jerks feel like this is the place for them.
My question was asking if there may be a correlation between the quality of the content and the actual name of Hacker News being misrepresentative due to the ambiguity in the definition of "hacker."
Which I asked because my thoughts seemed to wander a bit too far from what you were asking.
I agree, Diego. I now find a few articles a day on the front page interesting, whereas before this seemed much higher.
It seems that hacker news may be getting too popular for it's own good. At which point, you start getting submissions from people who are more interested in promoting themselves or some agenda and finding ways to game the ranking system to get content on the front page.
For example: http://news.ycombinator.com/item?id=3497331