4.28.2007

Gender Bias in Language Research?

On the WPA-L, someone sent around the link to The Gender Genie, a program that uses an algorithm to determine if the author of a text was male or female. First of all, I think the whole thing is hugely dubious. It is based on statistical analysis of the British National Corpus, so there is nothing remotely innate about the correlation between gender and "the use of pronouns and certain types of noun modifiers" (Argamon, Koppel, Fine, Shimon [.pdf document]). The conclusions are rightly stated as "socialization of gender."

However, even this claim needs to be tempered. I ran my own little test using the Gender Genie. When submitting blocks of text, one must choose the type of text it is from broad genre categories of fiction, non-fiction, or blog. Since I don't really have a great deal of fiction to submit, I first took a piece of this blog and ran it through the Genie. It claimed the author was female. As a comparison, I took a piece of my dissertation. It claimed the author was male. Why the difference?

It maybe that the samples I submitted were too short. The blog was 289 words and the section of my dissertation was 382 words. So, I ran another test, this time with a different blog entry and a different section of my dissertation. They were 816 and 900 words, respectively. I achieved the same results with these samples as with the first test. My blog entry was seen as "female" and my dissertation was seen as "male."

My dissertation discusses theory and empirical data while my blog tends to remain more theory oriented. However, I would think this would actually score a difference the other way around with real experience scoring as more "feminine," assuming the Gender Genie reinforces the stereotype of abstract discourse as a penchant for males and experiential or intuitive discourse as something innately female. Moreover, since both blog entries and my dissertation involve academic thinking about discourse, the fact that any difference was detected strikes me as patently odd.

Now, to be fair, my dissertation involves more conclusive theorizing than the two blog entries I used previously. So, I ran a third test, this time involving only one strongly argued (and firmly believed) blog entry since we know the dissertation is "male." Based on only 441 words, mind you, the Gender Genie found this one to be "male" as it did my dissertation.

Conclusions? Beyond that the Gender Genie is absolute bunk (like you needed me to tell you)? That what is being measured here are stereotypes. The dissertation sections and the lone blog entry that scored as male did so because of the conviction and assured manner in which they were argued, not the more tentative and speculative manner of my blog entries. As such, the algorithm picks out a degree of certainty with which the author uses language to relate the reader to the subject matter being written. That is it. There is another step required to make the leap from that measure to the conclusion that these are indicative of the ways in which British or even Western societies socialize along gendered lines. The authors of the Gender Genie makes that step, equating a self-assured manner of writing with gender characteristics. But, in doing so only reinforces sexist stereotypes about women's and men's ways of writing.

Don't believe me? As a final check, I ran two more texts: Gwen Ifill's op-ed about Don Imus' derogation of the Rutger's women's basketball team and this blog entry. Ifill's op-ed that took Imus to task was 765 words and was seen as male. This entry? 618 words and written by a male.

3 comments:

k8 said...

Funny thing. It's been out there for a few months, but I finally decided to play with it a few nights ago - and then blogged about it. The next day or so, the WPA discussion started up. Something's in the air.

Oh, and apparently I'm mostly a boy.

I do find it interesting (regardless of how problematic the whole 'genie' is) is how it ascribes male attributes to the type of writing traditionally gendered masculine. But I've decided, after only a tiny bit of consideration, that I won't pontificate on gender and writing this time around, even though it does speak to these issues. Most people seem to be laughing at its conclusions. I'm not yet sure how I feel about them. Or why I felt compelled to try it out, for that matter.

Anonymous said...

Well I agree but I about the post should acquire more info then it has.

Anonymous said...

I inclination not approve on it. I assume precise post. Particularly the designation attracted me to be familiar with the intact story.

Not always theoretical... not even always academic.. but always written..