Unfortunately, the world isn’t necessarily organized in that fashion.  Few would believe there are such categorical distinctions.  Prospective dates have some degree of positive and negative qualities, rather than attributes being merely present or absent.  Are people either qualified or not for a job?  Most people instead belong along a continuum of professional ability, with some being very qualified (way above being merely adequately qualified) and some people being just below and just above the border of qualification.  Politicians aren’t uniformly liberal or conservative and we routinely see partisans on both sides upset at those who aren’t extreme enough and who toe the partisan line.

This may seem obvious, but the reason I bring it up now is that while most everyone would agree with this fact, when thought about more carefully, still many people continue to argue as if things are categorical.  There are two recent examples on the yourmorals blog.

First, the comment section of this post has become a debate (for many) over whether psychology is objective (science) or subjective (art).  Allow me to quote Gene, from this thread:

there is SOME objective knowledge that comes from psych research (anything that can be experimentally shown, is predictive, even if only statistically, it has value).

If you want to get really nitty gritty, even physics is not completely “objective”…it’s merely instrumental to understanding objectivity (see here: http://en.wikipedia.org/wiki/Instrumentalism)

Most things are not completely objective or completely subjective, especially where human affect, behavior, and cognition is concerned.  Yes, psychology is less objective than physics…but it’s more objective than sculpture.  If I think that Paul McCartney sings better than I do, is that an objective or a subjective fact?  It’s objective in so far as a survey of people would detect a very large statistically significant difference between perceptions of our singing.  But it’s subjective in so far as it may not be true for a particular person (e.g. my wife and my mom).

What complicates things further is that many people who read psychology don’t really care about what happens to most people, but rather how the research applies to them.  Consider this very useful overview of how changing our consumption patterns can make people happier.  One of the recommendations is something that I tell people often, that experiences lead to more happiness than material things, an opinion shared by 57% of a national sample (and shown to be true for most in experimental research).  Yet, 34% of those people disagree (and some don’t benefit in experiments).  So is the statement that “buying experiences leads to more happiness than buying things” an objective or a subjective fact?  It’s true for a majority of people, but not for a significant minority.  It’s likely true for many groups, but certainly not all groups.  Yet many people still think we can definitively decide if psychology is objective or subjective, even though humans, unlike inanimate objects, don’t react predictably to situations, except perhaps in aggregate (e.g. we have free will or at least the illusion of it).  I can find truths that apply to all rocks or all electrons, but not for all humans.  But I can find truths that apply to many humans or most humans, and that might give someone insight into themselves, which is a valuable thing.

A second instance of categorical thinking on the yourmorals blog of late is Pigliucci’s critique of Haidt’s recent SPSP speech.  Haidt pointed out that there is underrepresentation of conservatives in social psychology compared to the population and cites both self-selection and discrimination as issues to varying degrees.  Many people (understandably) focus on the sexier charge of discrimination, and Pigliucci answered that he “suspect(s)  the obvious reason for the “imbalance” of political views in academia is that the low pay, long time before one gets to tenure (if ever), frequent rejection rates from journals and funding agencies, and the necessity to constantly engage one’s critical thinking skills naturally select against conservatives.”  But what if causality was continuous and not categorical.  Pigliucci may be entirely right about his obvious reason, yet there still could be some amount of discrimination.  Indeed, if there is one student somewhere whose ideas are supressed (and there was at least one in Haidt’s talk), then there is at least some degree of both self-selection and discrimination, meaning that a debate over what statistically causes underrepresentation misses the point.  Bear in mind that these are not just data points, but actual human beings.  One human being discriminated against is one human being we could serve better, even if the vast majority of under-representation is due to self-selection.

I’m obviously biased in the above debate, but these thoughts are not a response to that debate, but rather a response to almost every debate and decision I see in psychology.  Some other things that are continuous, and not categorical:

Journal Publication – Editors have to make categorical decisions to accept or reject papers, yet many papers that are accepted never get cited, while other papers are published through sheer persistence down the chain of  journal prominence.

Statistical Significance – A 94.9% chance of being right is not that different than a 95.1% chance of being right, yet it is treated as a categorical distinction called “significance” because we need to be able to say whether something is true or not, when in reality, all we have is some evidence toward the truth, that varies to some degree.  Even the best paper does not definitively prove anything and even the worst paper is some evidence toward something.

Authorship – Many people work on papers (often undergraduate research assistants) and are not authors, while others do fairly little and receive authorship.  Sometimes the first author does 90% of the work and sometimes they do 51%.  Yet they still receive the categorical distinction of first author.

Psychological conditions – Few psychological clinical conditions are categorical.  In reality, people have some degree of anxiety, rather than having or not having an anxiety disorder.  Yet, for insurance reasons, people have to be diagnosed categorically as having a particular condition.

Psychological constructs – Is shame the same as guilt or different?  Is shame the same as sadness?  Is shame the same as happiness?  The truth is that shame is somewhat like some of these constructs and less like others of these constructs.  Categorical distinctions between such constructs are useful for publications, but don’t really reflect the continuous nature of the real world.

I am sure that if I thought more, I could come up with many more examples of things that are continuous, but treated as categorical. In academia, perhaps we can eventually change our systems, leveraging technology, to acknowledge the continuous nature of things.  My real-world hope, as someone who believes that a world with less conflict is better than a world with more conflict, is that perhaps seeing things as continuous, rather than categorical, means that people will be less likely to make harsh judgments of others based on the idea that their beliefs are the categorical caricatures that we make them out to be.

- Ravi Iyer

What interested me was not the data, but the critique of how psychologists attempt to illuminate the human condition.  A few quotes from the chapter summarize the points I’d like to emphasize.

Psychologists often study phenomena in isolated, artificial environments, which allows researchers to necessarily isolate variables of interest, but….

Aiming to isolate phenomena, scholars in this research enterprise are prone to devise somewhat peculiar and overly constrained assessments of moral functioning that are remote from everyday moral experience.

A gold nugget in Gilligan’s (1982) critique of moral psychology was her skepticism concerning such constrained dilemmas and her advocacy for assessing moral judgment more naturalistically, tapping moral problems from individuals’ own experience.

Another paradigmatic assumption to which we draw attention asserts that people are psychologically “cut from the same cloth,” uniformly operating by the same moral psychological

processes. This assumption is manifest in the frequent reliance on a single type of research participant (e.g., undergraduate students garnering course credit), a lack of consideration for

individual differences, and a homogenizing “people” label.

Our proposal contends that lab experimentation should be balanced with real-world observation of socially significant affairs and that morally relevant aspects of personality should

be tapped across all levels of personality description. Different methodologies should be mutually informative. Multiple lenses on the same phenomena contribute to a more comprehensive understanding, whereas divergent findings across methodologies hearken our attention.

So what can psychology tell us about moral reasoning that literature and the humanities, or simply reading the newspaper thoughtfully, cannot?  I would say not much, but rather that psychology can help buttress what can be learned by other methods and vice versa. They both get at the same questions. A colleague of mine once shared that he thinks of psychology studies as statistical parables, in the same way that stories of the real or fictional world provide us with different kinds of parables. Anyone who has read a really good novel might believe Ralph Waldo Emerson’s quote that “Fiction reveals truth that reality obscures.”

The authors I quote above want us to use multiple lenses to understand the human condition, referring to the lenses that psychologists might use (different samples, different methods). I would further extend that analogy to all fields that attempt to understand the human condition, such as literature and the humanities, but also just reading the news. This is not to say that there is not something powerful about quantitative analysis and methodologically rigorous psychological research. But as I step back from the research, I find that I’m only convinced by findings where there is a web of evidence, of the type that one researcher, paper, study, method, or discipline, could never produce…where the statistical parable has been replicated in other ways by other people and is echoed in situations I’ve faced and news stories I’ve read about. Fortunately, the internet and semantic web technologies promise to make it easier to discover such webs of evidence…but that’s a subject for another post.

If you have the patience, it’s worth reading the results of the conference in Herzilaya, but if not, perhaps I’ll make a practice of summarizing some of the other chapters as I read them. Social psychology can be unfortunately unintelligible, in ways that literature is not.

- Ravi Iyer

Academic publishing is not easy. In psychology (though we’d be happy to publish outside of psychology), it’s not enough just to have a valid results, but the results often have to be novel as well. Therefore, many replication studies may not be publishable or may only be publishable in lesser known journals or just on this blog. That doesn’t necessarily make that endeavor unworthwhile, as replication, or the failure to replicate, is an essential part of the scientific method, but we want people to know what they are getting into. We’re open to anyone who is motivated to publish in peer reviewed journals, and there is no inherent reason that limits this to academics. However, it’s a labor intensive process with no monetary reward, so it’s quite possible that only those with an eye toward building an academic CV might be interested.

Here is a running list of potentially publishable results which are in our publication queue, but there are many more possibilities. We are open to proposals on a variety of topics. Some of you might be interested in a specific topic and might find this list of measures useful in determining if we have data on that topic.  Data might potentially serve as the 1st study in a 3 study package where a community sample reinforces the results of a lab experiment, or as convergent evidence in something you already are working on. In rare cases, we may even be willing to collect new data using additional measures, even including experimental methods, if your ideas are compelling enough. However, there are only so many resources we have and the degree of effort required is definitely a consideration, balanced against the contribution which could be made. Also bear in mind that some number of papers are already in progress, and it may be possible that your idea is already being worked on.

If you are interested, please use this form to contact me as it has important questions to be answered. Beginning any publication process is a commitment and we would obviously like to work on projects that have successful conclusions. Thanks for your potential interest.

- Ravi Iyer

At the same time, I’ve been working with colleagues on a paper about experiential vs. material purchasing styles, for which we have found convergent correlations all suggesting that experiential purchasers are dispositionally motivated towards seeking new, stimulating experiences to promote positive emotion, while material purchasers often seek to avoid negative emotions. This is supported by the fact that, in the YourMorals.org dataset, experiential purchasers report higher levels of openness to experience, lower levels of neuroticism (both measured by the Big Five Personality Inventory), and lower levels of disgust (as measured by the Disgust Scale). The disgust finding does not necessarily fit with the idea that experiential purchasing is related to seeking new experiences, unless one looks at the literature on disgust. In particular, this study theorized about such a relationship and confirmed it by reporting correlations between disgust and big five personality dimensions.

It occurred to me that I could contribute to the original studies’ findings, by examining the same correlations in our dataset, using a more diverse and far larger sample, and perhaps even including some internal cross-validation.  The results are summarized in the table below.

Disgust Scale Correlations with Big Five Personality Traits

The main hypothesis of the original study actually dealt with the two robust relationships found in our dataset, specifically that disgust is negatively related to openness to experience and positively related to neuroticism. In all, these two relationships stand out as robust across groups and in both studies. Interestingly, the correlation between openness to experience and disgust is weaker in the two most ‘rational’ groups, edge.org and libertarians, which might be worth pursuing later. Given the smaller sample size and restricted diversity of the original study, I’d be inclined to say that conscientiousness and agreeableness are not robust correlates of disgust, though this could be an effect of the fact that yourmorals.org uses a different measures of Big Five personality traits from the original study.

Can I publish this finding? It’s only correlational and says nothing about causality. It really doesn’t say much that is new, but rather confirms the original study, more or less. Still, the 26 papers which cited the original study would be slightly more improved if they could cite this finding as well, since it’s the same basic study with a different (larger and more diverse) sample. This is where the discussion of the peer review system converges with this analysis. According to this paper,  ”many natural science fields operate on a norm that submissions should be accepted unless they are patently wrong.” In contrast, psychology papers are often rejected, not because they are wrong, but because they are not interesting or novel enough.

The paper and the listserve discussion bring up many points related to this, but one relevant one to this finding is that it is hard to build a cumulative science when you don’t reward replication, but instead reward novelty. The end result is that you end up with a series of slightly different perspectives on the same subjects, all named differently, where authors are constantly trying to come up with something new rather than building on something existing. This may help academics, but it makes it very difficult for these theories to be used in the real world. Any research on humans is likely flawed in some way. Can anybody do double-blind experiments on representative samples of people with behavioral measures? The public is wisely skeptical of any social science finding as are academics…but the solution might lie in publishing more replications rather than in restricting the publication process toward the mythical goal of the perfect, novel study. No single study proves anything when dealing with research on people. It’s the convergence of lots of studies that might potentially be convincing enough to outsiders.

- Ravi Iyer

ps. if anyone wants to write this up and publish it traditionally, feel free to contact me