Survey of Caste in America: Another Ideologically Driven Crusade Predisposed to Malign Hindu Culture

About a month ago, I was forwarded a link to complete a survey “about caste in the South Asian Diaspora in the United States.” I wondered if the survey would be an effective approach to a potentially real, but likely highly attenuated, imported phenomenon; or perhaps it would simply be another ideologically driven crusade, repeatedly and misleadingly presented in credulous publications as evidence of the evils of South Asian (read: Indian, Hindu) culture. As I walked through it, I realized it would sadly fall squarely on the latter side. The survey may have good intentions but is plagued with methodological problems which will bias its conclusions and render it worthless except for generating headlines.

Convenience Sampling 

To start with, the survey uses a convenience sample — that is, it is not drawing in a random or even attempted random way from the underlying population. The link, which was here but has now apparently been taken down, is being forwarded, for others to then forward.

Even random samples are prone to selection bias. Those ideologically affiliated with your position may be more (or less) likely to respond to a survey, with the direction of effect unpredictable. Convenience samples compound this problem enormously by affecting not just who responds to a solicitation, but who receives one in the first place. 

A study sample obtained in this way can never be representative of the underlying population — something the Equality Labs survey of a few years ago dishonestly elided, repeatedly, in its presentation of its own findings (link above) — and can at most generate hypotheses to be studied. 

It is unclear if the Caste Scale Development survey here will follow this route, but even if we charitably assume it is merely generating hypotheses, the scales so developed will be flawed–perhaps applicable only to those who answer the survey.

Unfortunately, the flaws run deeper.

Minor Issues of Survey Design

The wording of the questionnaires and choice of answers is repeatedly ambiguous. For example, on the first page, I was asked “Country of Origin”. I was born in the U.S., which is maybe what is being asked here; though it could as easily mean the ancestral homeland, which in my case would be India. A better approach might have been to offer a drop-down — there are only so many countries in South Asia if this was the intent — and/or reword it to mean “country of family origin” with an explanation (e.g., “Going back three generations, which country would your grandparents have hailed from, assuming current boundaries?”, or alternatively whatever was intended here.) 

This is not mere pedantry; an obvious possible subanalysis of this survey might include how those of different national ancestry perceive caste differently. That opportunity is lost when the question is asked poorly, as it is here. 

A similar issue attends the citizenship and religious affiliation questions. For both, there is only a limited range of possible answers; these should simply be listed out, with possibly a box to indicate “other” with a free response. Leaving it open-ended in this way virtually ensures analytic cross tabbing will not be possible.

Asking demographic questions at the beginning of the survey, as this study does, is also not considered a best practice, given these are often perceived to be more invasive and/or less interesting than other questions and are best when the survey respondent has been primed to answer them.

Asking About Caste Group

Getting to the next page, the survey does make one important contribution, which is that it does not force the respondent to select a varna upfront; rather, it allows the respondent to choose a jati, biradari, kula, etc. It is implied in the question wording that this might be mapped to a larger category later on (which may itself be fraught with errors, since last names as stand-ins for caste can be changed or in some cases or even mean different castes in different areas), but this approach allows for far more nuance than asking respondents about whether they are Brahmins, Shudras, etc.

Scales

However, that minor good news cannot outweigh the obviously flawed construction of the next several sections, where, inexplicably, the order choice is Strongly Disagree, Somewhat Disagree, Disagree, Agree, Somewhat Agree, and Strongly Agree. 

The survey continues with questions such as “It is important that people think of themselves as South Asian or South Asian American and not upper or lower caste” — a potentially reasonable or highly controversial question depending on if the survey is geared at South Asians or at Indians. 

In most respondents’ minds an unmodified “Agree” or “Disagree” likely falls between “Strongly” and “Somewhat”, but the visual metaphor contradicts this here, leading to potentially great confusion in how the question is answered and thus how the data is analyzed and interpreted. 

One can only assume that the “Somewhat” categories were included to allow those with stronger beliefs to be separated from those with weaker ones, possibly in a sensitivity analysis; but this becomes possible only for those who Strongly Agree or Strongly Disagree given the possible confusion about the two internal categories. Moreover, this is clearly not a simple typographical error since it occurs repeatedly and is also restated in the instructions in the same order.

Questions

Coming to the questions, the wording is frequently biased and/or designed to have a right answer, despite the bolded, italicized, and underlined assertion that There are no right or wrong answers. For e.g. “Talking about caste leads to divisions” is a negative framing of a commonly held belief (misguided or not) that rising above caste-based divisions and discrimination involves paying less attention to caste. The point here is that the survey authors are biasing the survey respondents against the underlying construct they are likely trying to study. Similarly “Lower caste people are more to blame for caste discrimination than upper caste people” may be a proxy for any number of potentially mutually contradictory beliefs: a casteist belief that the upper castes are “truly” better, an anti-caste belief that caste-based discrimination would end if not for constant attention to caste and reservation politics, etc. The folks who hold the latter belief may not answer affirmatively to this question, which would in turn damage the construct validity of the question and the latent variable it is attempting to assess. 

A similar issue exists with very strongly-worded statements like “Caste discrimination does not exist in the US”. Such a strong statement — a universal negative — is likely not representative of the underlying idea that “Systemic caste discrimination is rare in the US”, which is likely a more interesting question to the researchers and to those who might read their work.

The survey continues with questions such as “It is important that people think of themselves as South Asian or South Asian American and not upper or lower caste” — a potentially reasonable or highly controversial question depending on if the survey is geared at South Asians or at Indians. 

In completing the survey the authors ask about both South Asia and about India; but no question comes up about Pakistan, Bangladesh, Sri Lanka, Nepal, the Maldives, or Bhutan, so I presume the primary audience is actually not South Asians at all and is rather about Indians. Given that the very term South Asian is contested in the Indian imagination (as noted in the Carnegie study of last year, ~ 7 times as many Indian Americans identify as Indian, Indian American, or Asian Indian over South Asian, 75% vs. 10%), it is disingenuous to label the survey about South Asia when it is clearly aimed at one community in particular.

Yet another issue: questions are often repeated but in subtly different ways, with significant ambiguity about the end result. 

For example, the question above about identifying as South Asian rather than by caste is mirrored several questions later with the same question but this time about identifying as Indian. 

Having chosen a frame for their survey, the authors ought to stick to it, with South Asian terms throughout. As it is a respondent might disagree that people should identify as South Asian, implying they should identify with a caste instead, but they may also disagree that people should identify as Indian. 

This is of course a problem for the South Asian term as a whole, where people from specific parts of South Asia often have very strong beliefs about people from other parts of South Asia, and may well think that in their corner of South Asia there is no casteism, but in other parts, there might be.

Moving on, the survey touches a live wire with its questions about meat, which it manages to ask in a remarkably obtuse way. 

Many Hindus, Jains, and Sikhs of all castes are vegetarian for religious and moral reasons, but might object to the charged language that “eating meat is immoral” and especially the squirm-inducing idea (for some) that “eating meat is impure.” A confrontational way of asking the same questions in the opposite direction might be to assert “I am vegetarian because I object to the killing of animals.” A more neutral approach might be to say “vegetarianism is preferable to non-vegetarianism” or vice versa. Incidentally, the vice versa approach is a better one than rewording the question–asking if “vegetarianism is preferable to non-vegetarianism,” then if “non-vegetarianism is preferable to vegetarianism,” can help tease out those who are answering at random if they agree with both statements.

The survey then gets to a final point which further confuses the issue — there are compound statements where a person may object to part of a statement but agree to another part of it. So, an ethical vegetarian may think that eating meat is immoral but not that it is “impure”. But this was already addressed in the separate questions, so why are the two being asked together? What does it mean if someone agrees that meat-eating is immoral, disagrees that it is impure, yet agrees that it is both immoral and impure? How will this be interpreted?

Lost opportunity

Finally, the survey is a lost opportunity for teasing out differences between the individual respondent and their community. These questions are best reflected together rather than apart, else the respondent may get confused on whose behalf they are responding. “I recognize casteism is a problem but I cannot separate it from my culture” is followed immediately by “Caste identity is a source of pride”. Is the latter a description of me, or of my community but not me? Grouping my personal attitudes about caste and separating them from my perception of overall attitudes about caste could have been an interesting side aspect of this study.

Conclusion

This study is poorly designed and unlikely to yield even hypothesis-generating results. The non-random sample, the numerous issues with question-wording, the constant shifts between Indian and South Asia (but not between South Asia and any other South Asian country), the answer scales being mixed up throughout, the construct validity of many of the underlying concepts, and the poorly thought through demographics mean difficulties if not impossibilities for interpreting the results or applying to any context. It’s a shame because the underlying concepts may well be worth studying, but this is not the study that will get us there. 

---

Raman Khanna, MD is an associate professor of clinical medicine, physician, and informaticist at the University of California, San Francisco. Since majoring in religion in college he maintains an avid interest in religion and history. He is a member of the Hindu American Foundation’s National Leadership Council, where he helps with special projects and data analysis.