If you’ve been following these blog entries on the evolution of my article for the Journal of Marketing Research, you will likely have noticed that the last round of reviews was not very positive.
In fact, I knew after reading those reviews that this was do or die time.
First, I had a new editor. I didn’t know Wagner Kamakura. I had never met him (I don’t believe I have met him to this day). This editor had a new set of standards, and it looked like he had seriously raised the bar on me. Russ Winer had expressed, probably signaling to reviewers, his interest in the topic and in publishing something that covered this topic. I received no such signal from Prof. Kamakura. In fact, Prof. Kamakura’s cover letter let me know that the paper was deficient in a number of ways. He also signaled that he wanted me to take more of a content analytic, quantitative, numerical approach to accounting for my data. I didn’t like it, but there it was. I could either count the number of postings I was using, and the users, or pretty much put myself out of the game. In the process, though, I thought that maybe I could educate the Editor a little bit about the differences in approach between ethnography and other methods. For example, that comments about providing ‘objective, unbiased’ information. That indicated that ethnography might not be an area that the good Editor was very familiar with.
But the good thing, the surprising thing really, was that Prof., Kamakura could easily have rejected the paper, and he had not. Reviewer B had not moved in his/her opinion. In fact, that reviewer seemed to hate the paper more now that ever. She or he had given the Editor a good reason to bail on me if the Editor had wanted to. Rev B questioned everything about it-the ethical stance, the method, the focus, the topic, my skills. In their closing shot in the review, Rev B implied that the paper and method were useless. And I guess that meant that the last two years of my life had been wasted working on these topics.
Now, I have to tell you that, as a junior faculty member with nothing on my CV save a relatively obscure European journal article, this was more than disheartening. It was, at first, like someone had dropped a baby grand piano on my back. I felt flattened. I felt roughed up. Okay, you saw the pictures. That’s what I felt like. But then, as I read through the comments over and over, I realized something.
It was clear to me that Reviewer B didn’t understand what I was doing at all.
Reviewer C pretty much got it. She or he was still a tad snarky, but they could see what I was doing and they were offering comments that were raising the bar and generally helpful. They were seeing that there was improvement, progress here. They were working with me.
Of course, even Reviewer C gave me pause. What did they mean by the manuscript ‘improved again’?-this was the first revision of it they had seen.
At this point, I had a major decision to make. I needed to make a complete rewrite of this paper. This wasn’t a time for minor modifications-the paper would be shot down almost instantly if I was to try that.
But I didn’t know how to deal with Reviewer B’s negativity. It seemed like there was absolutely nothing I could do to please this reviewer.
So, after consulting with a few faculty members around Kellogg, I decided to ask the Editor to speak to me on the phone. I’m glad that I did. In that phone call, Prof. Kamakura and I discussed the paper and what I could do to improve it. We straightened out the requirements for a good revision of the paper.
And I came to that phone meeting armed to the teeth with references, citations, and all that stuff I had been ruminating about. Overwhelming force, or as much of it as I could muster. In the process, I sought to convince Prof. Kamakura that I knew a lot more about my topic matter than Reviewer B. Reviewer B was, I still believe, the wrong person to be reviewing this manuscript. I politely mentioned this to Prof. Kamakura. I also politely suggested that he might replace this reviewer with someone more qualified to review my manuscript.
Although you are getting the scans of these reviews, I got paper copy of Reviewer B’s reviews, which were typed on an old typewriter and then photocopied on some ancient 1940s model typewriter. This was clearly someone who was stuck in another age and just didn’t get anything about technology. And they clearly didn’t want to get it.
So I argued about some of the main points that I later raised in my reviewer notes. Online ethics are obvious and a non-issue? I don’t think so! And here are three solid references and three excellent reasons why that is so….
I grew as an academic during this process. Although I was junior, I knew more about my topic than any of these reviewers. And I aimed to show it to them. I had to take a stand to do that. I had to take some risks.
For my example of netnography, I was also profoundly challenged. My last revision’s example was definitely lame, no doubt about it. But I was hamstrung by that 30-page limit. How could I express the richness of real ethnography in three journal pages? I felt like paraphrasing Scotty on Star Trek: “Jim, it’s against the quantum laws of ethnographic writing–it just can’t be done.” [Yeah, yeah, another Star Trek reference, I know…]
Add to that the fact that I had only written one ethnography before, in my thesis and in the paper coming out of it for JCR. I had started my Burning Man ethnography by that time as well, but in terms of being an expert in ethnographic writing, that I most definitely was not. I had been collecting data for a while for another project, which I thought would be a separate paper at some point, a study of coffee community and culture online. The only thing to do was to sacrifice it up for this JMR piece, to add the richness and detail I had to the table, see if I could condense it into something meaningful, and illustrate what this new approach could yield in terms of real marketing and business insight.
So, after all of that, here are my Reviewer Notes. Notice that they reflect my growth as a scholar. They are a bit more confident, a lot more assertive, and they reflect someone who realizes that, when you know more than your reviewers do, you have to educate them. And when they don’t want to be educated, you need to out their critiques as unreasonable, an alternative point of view that you don’t subscribe to or, sometimes, as just plain wrong.
Revision Notes RE: JMR MS# 1205-9-3: “The Field Behind the Screen: Using Netnography to Research Market-Oriented Virtual Communities”
First, I would like to thank Prof. Kamakura and the 2 Reviewers for their helpful comments. The enclosed manuscript has undergone a major revision (almost a complete rewrite) and a new positioning in an attempt to fully incorporate the feedback from the past round of review. The illustrative example is new (I had discussed this with Prof. Kamakura in a phone conversation and he supported my decision to change the example so that I could draw on a richer pool of data). The example is now much more fully realized, and takes up approximately half of the revised manuscript’s space (before references). To make room for it, the background information on cyberculture, as well as all tables and figures, have been jettisoned. These notes will overview the specific changes that have been made in response to the reviews. The notes will respond in considerable detail to each of the reviews in turn.
A. Editors Comments
Your comments were very clear, directive, constructive, and helpful. This revision reflects an attempt to implement all of your requirements. As you noted in your letter, this did “require a complete re-write of the manuscript.” It was very challenging to try to convey the richness of ethnography, along with the thoroughness of methodological description, and references in 30 pages total. The original coffee ethnography was over 25 pages, and could easily have been over 60.
In general, the paper was more clearly positioned as a methodological piece in this revision. It omits almost all of the background information and theorizing on cyberculture and virtual communities present in past versions of the manuscript. It also provides much more detail on how to apply the method to marketing research, its advantages over competing qualitative marketing research methods. Finally, the example is much more fully developed than in the prior manuscript. Specific comments follow:
1. The positioning of this manuscript has evolved through the review process. This version conceptualizes netnography as a technique useful to the study of market-oriented virtual communities. This is a much more specific, and for that reason, defensible, positioning for the method and the manuscript. In the manuscript, it is explained that many virtual communities exist that have market-oriented concerns. Many of these virtual communities are quite sizeable (many in the 100,000 range, quite a few in the 500,000-1 million range, or more), comparable in the number of readers they might have to many magazines, newspapers, or cable TV shows. In addition, they are more specifically consumer-to-consumer, and so they might carry considerable weight in consumer decision-making if, as the Almquist and Roberts (2000) article cited in the paper might suggest, consumer advocacy is an important facet influencing consumers’ choice of one brand over another. Netnography is positioned in a manner that is entirely consistent with the theory and practice of ethnography in anthropology, sociology, cultural studies and consumer research. That is, (as described further in the test on p. 4) it is positioned as a study of the local and the particularistic (cultures, communities, communications) that can be, but need not be, generalized to other groups. I have tried to convince readers of this manuscript that these groups are, in and of themselves, populous and important.
2. The general discussion of CMC and virtual communities has been almost completely removed. What remains is placed in the introduction section and is very brief. It has also been positioned so as to be of more interest and relevance to marketing research practitioners as well as academics.
3. The introduction clearly distinguishes netnography from ethnography. This is aided by the new positioning (i.e., it would be difficult, and probably mostly irrelevant, to study virtual communities in-person). Netnography is also detailed as being far less intrusive, less resource intensive and providing quicker research turnaround than ethnography. Netnography is also compared with focus groups and interviews, and detailed as more naturalistic, less susceptible to demand bias, less intrusive, and less resource intensive. Again, positioning the method as useful for the study of market-oriented virtual communities ameliorates many of these concerns about comparison. To use focus groups or personal interviews to study virtual communities seems less logical than to use netnography.
4. This draft provides a much more specific and detailed description of the steps and procedures involved in conducting netnography. It can now act, as you suggest, “as a blueprint to the reader interested in applying the method on [sic] her own research.” As explained in the text, the inherent flexibility of the method is consistent with the positioning of the method in anthropology and other fields. This manuscript’s description of netnography emphasizes methodological rigor. However, because of the importance of interpretive methods, and the researcher’s role “as instrument” (to use John Sherry’s turn of phrase), the method can not be said to lead to “unbiased, objective information” any more than can personal interviews, focus groups, or market-oriented in-person ethnography. This build’s upon Reviewer 3’s suggestion to more fully develop the “flexibility” of netnography, and is explicated further in the text.
5. The empirical illustration attempts to address your comments fully. While it is extremely condensed, and must leave out much analysis and description because of the limited space available, it tries to provide the reader with a lucid example of a netnographic application as well as a sense of its general flavor. On pp. 15-18, the various steps you mention in your letter are detailed (and in some cases expanded):
6. which newsgroups were chosen
7. what criteria for newsgroup selection was used
8. the number of messages downloaded for interpretation
9. the message threads downloaded for interpretation
10. what criteria for message thread selection was used
11. the #distinct informants or messages posters involved n those messages
12. some classification of the types of messages found (in addition, in several other places throughout the paper, e.g., p. 8, p. 16, different categorization schemes were provided that might be useful to other researchers)
13. amount of raw data utilized
14. methods of determining trustworthy messages/ exclusion of suspect messages
15. interpretive methods (this was a very brief overview, with references, many articles and books are available on the interpretive methodology that was used here)
16. some discussion of the more appropriate employment of the term and concept of research “trustworthiness” (rather than the inappropriate term and notion of “validity” applied to positivist or quantitative research)
17. methods of verifying information and conclusions with informant member checks
18. types of alterations to the text made in response to member checks
In addition, it is worth noting that I have been diligent in error-checking the ethnography with literary sources as well as with experts. In an extended communication with a coffee entrepreneur and author of 2 books about coffee production, this person confirmed the correctness of my netnographic analysis and the potential utility of some of the conclusions to those in the coffee business. This person (who requested anonymity) also provided numerous comments about the specifics of coffee production that helped to improve the manuscript.
B. Reviewer #1 Comments
Thank you for your helpful comments. I hope that this extensive revision meets some of the objections to the prior revision that you identify and describe.
19. The positioning has been changed, as detailed in 1, above. The attempt has been made to address your concerns about a lack of focus and overclaiming.
20. Theoretical descriptions of the differences between online and offline communication have been removed in this version in order to make room for the empirical example of netnography. That there are difference between online and offline communication seems an important concern, and one that has been treated in a range of articles and books. In this article, the focus is considerably more pragmatic than directly theorizing and specifying these differences (a matter I leave to other articles, and/or other scholars). Therefore, there are far fewer “constructs” in this paper, and hopefully less vagueness and less confusion. The article focuses on the method and the illustrative example of the method.
21. Building on the knowledge that (1) there are a multitude of virtual communities that deal with consumption and marketing related concerns (like coffee consumption), (2) these groups are growing in number and probably in influence, and (3) the numbers of people reading these groups is in the tens of thousands, oftentimes hundreds of thousands, and sometimes millions, it seems useful to marketing researchers to have a methodological tool (some guidelines) with which to study these social phenomena. While ethnography is venerable and effective, there are some specifics to virtual community that conventional ethnographic practice simply does not discuss. These are all in the realm of the specific configuration of virtual communities, which I describe in the revised manuscript. For example, how does one choose a site or newsgroup? Which site or newsgroup to choose? Where to find lists of these groups? What types of messages to read, save and download? What types of informants are present on these groups? What types of classifications may be useful for this type of data? What types of interpretations may be useful for this type of data? How do we handle the particular ethical concerns raised by the online medium? And so on….
22. Your insight that “online research is useful for studying net cultures,” was helpful to the new positioning of the paper. As you suggest, the prior version’s more general claims to represent other types of groups was a less defensible position than that of the current revised version.
23. Your critical comments on the “Trust and Rapport” section were useful, and this section has been eliminated because, as you point out, it only applied to a particular kind of netnographic research (participatory). In addition, the notion of informant identity is rendered far less problematic if we conceptualize it as an issue of trustworthiness that can be treated by altering our unit of analysis from the individual to the interaction (as with G.H. Mead’s social psychological methods). This also answers your question about “what is imaginary and what is real”? Every interaction in a market-oriented virtual community is real. These insights are in the revised text on p. 10.
24. Your critical comments on the “Interview” and “Digital Projectives” sections were useful, and these sections have been eliminated due to space constraints.
25. You contend that “the whole matter of public versus private is a non-issue” and that “electronic eavesdropping” is not likely to give rise to psychological harm. Both points are contestable. The latter point appears tangential to the topic at hand. Netnography is a method for researching virtual communities and as such it will often have impacts far beyond the mere act of observation. What is more salient than data collection (“eavesdroppping”) is the potential impact resulting from the publication of information that was disclosed in an online forum. If I overhear someone discussing his or her private matters online and then publish it, is there the potential for harm?
With all due respect, your position appears overly theoretical. My own thinking on the matter has been informed by over 13 years of experience as a virtual community participant and over 5 years as a virtual community researcher. In my research, I regularly come across people who do not want their information shared with others or published (even after promising them anonymity). I have published this finding elsewhere, but in deference to your comment I now include it in revised manuscript (on p. 12).
Here is an online example of the type of trouble that researchers can cause. The following quote comes from someone who posted to a “public” forum on suicide and felt compelled to write a web-site conveying her feelings about being researched:
I am NOT a guinea pig. I am not sitting here, writing everything so that I can become part of someone’s report. . . I am, however, pissed off.-Dana-Christene Umanetz, posted on http://ash.xanthia.com/researchers.html
While this is a strong case, these types of comments are common among newsgroup participants. In my coffee ethnography for this article, people told me they were uncomfortable with the “lab rat” aspect of being observed. This sort of comment is very familiar to ethnographers of all stripes, but in person one has the distinct advantages of usually being able to see one’s observer. Not so in cyberspace. For the alt.coffee ethnography, I had people who required considerable explanation (and insisted on seeing copies of the manuscript, in order to judge how their quotes were being represented) before they would give me permission to use their postings.
If this manuscript is published in JMR, it might be used by graduate students and professional researchers, as well as researchers in disciplines outside of marketing. The potential that they might study something sensitive, like drug use, pornography, dealing with handicaps and illness, addictions, stigmatic groups or beliefs, or other social concerns (that can and may well have marketing implications) is very real. In addition, as an anthropologist, I have a strong concern for the cultures and culture members that I study. I believe that they and their beliefs must be treated with dignity and respect. I feel strongly that a section on ethics that recommends the ethical “high ground” is imperative for this article. I would not want it published without it.
The differences between netnography and ethnography arise exactly because of misconceptions such as the one that you proffer. Because cyberspace is not clearly public or private, netnography must have methodological provisions that specify procedural solutions to ethical concerns that face-to-face ethnography does not face. This is the “new news” you solicit. That it is distinct from in-person ethnography is specified clearly in this version on p. 13. Because these matters are contestable, frequently being contested by academics, and of major practical concern, this element could be one of this manuscript’s most important contributions.
26. As noted in comments 1 and 2, above, notions of generalizability have been replaced by a more defensible positioning of netnography as a method for studying specific and particular virtual communities.
27. The manuscript has removed most of the “participative” elements of netnographic methodology (such as interviews), mainly to conserve space, and to help prevent the sort of confusion that this Reviewer identifies.
C. Reviewer #3 Comments
Your comments were very honest, insightful, directive, constructive, and helpful. I truly appreciate the positive tone and the constructiveness of your review and have tried to implement all of your suggestions.
26. As outlined in comments 1 and especially 2, above, the paper’s positioning as a methods piece has been substantially altered in this major re-write. This version realizes your suggestion by portraying netnography as much more attuned to and related to the openendedness and flexibility of ethnographic methodology. The sections on p. 3-4 in the introduction to the method try to convey this. Even more importantly, the illustrative example exemplifies this openendedness by providing rich verbatims and (abbreviated, but hopefully still containing some interpretive spirit) analysis of the rich data provided in the <alt.coffee> newsgroup. This reflects my attempt to, as you suggest, “push” the method and “try to explain to a relatively naïve audience the nuanced nature of the interpretive act.” This effort is present throughout the major revision, particularly in the “Ethnography and Netnography” section, and towards the end of the “Applying Netnographic Methodology” section. In addition, building on your suggestion, you will find some comments comparing netnography with conversation analysis on p. 10 of this manuscript.
27. As noted in 4 and 5, above, the illustrative example has been subject to radical revision. I have taken your suggestion to ‘really build it up.’ It is now the centerpiece of this article. It is also now much more “ethnographic” as you suggest it should be. I do not believe that this ethnography could have come as a result of any other method. I do not think the religious evocations of voices calling unto coffee preparers, and “god shots” would have been likely outcomes of focus groups and conjoint. Show you the money? That’s what I was aiming for.
28. The conclusion section has been reined in to curtail overclaiming, while still trying to convey that this might be a useful method, one that marketing researchers may wish to utilize.
29. USA Today’s source is contained in footnote 1. As well, the Arbitron data (Reid 1995) may be of interest. Some search engines (like google.com) have archived copies of these pages, whose links may be expired.
30. Laurel’s quote is an exaggeration, but I think there’s still some truth to it, in a contingent sense. The cybergeeks still remain constituents of almost every newsgroup (true, certainly of the alt.coffee group), and their early conventions still drive CMC online. Yes, things have progressed and the innovators no longer have the playing field to themselves, but the people who are now there are still very much playing by the rules that the geekforce originally set. I think this is especially obvious in the use of conventions like emoticons and abbreviations. You’re quite right that the quote as it stands is contestable, and I have removed it as I jettisoned the cyber-theory sections of the paper. But this is a very interesting point. Thanks for the insight.
31. Yes, culture’s problematized nature is illustrated by my diagram. When trying to draw diagrams from my text, I often end up drawing concentric circles. Perhaps I should just stick to text. That’s what I’ve done in this revised version.
32. “Social inhabiting of imaginative animal…or alien bodies”…quite right. My enthusiasm was showing there. That section, along with much else, was slashed.
33. Your insight that F2F triangulation need not occur at all is, I believe, right on target. However, I struggled with your position vis-à-vis Reviewer 1’s position and (I believe) Prof. Kamakura’s. My solution was conveyed on p. 10 and 11 and represents a bit of a compromise. Because netnography is interested in the particular and the specific (now that it has been repositioned as dealing only with the study of virtual communities), and because we can think of the unit of analysis as interactions, chunks of text that represent implied/intended/actual social dynamics (or Wittgensteinian “language games”), we need not triangulate outside the method itself. However, it might be important to recognize that marketing researchers are probably going to want to generalize outside of the virtual community (whether we advise them to or not). They are going to try to apply findings to wider audiences assuming (perhaps rightly) that the vast bulk of “lurkers” is more like a general population whose characteristics might be useful to have but are very difficult to determine. Therefore, I have included a section that states that if netnography is going to be more generally repurposed then it would benefit from triangulation. In addition, triangulation might be useful when the stakes of the research are high, and someone’s neck is on the block. I completely agree triangulation isn’t strictly necessary, but I want to make sure JMR readers are aware it might be prudent for some occasions, and necessary for some kinds of claims.
34. As you suggest, the various methodological discussions (e.g., ethics, member checks) are now much more focused on netnography.
Thank you again, Prof. Kamakura and Reviewers, for your ideas and assistance.
Here is the abstract of the revised paper, the third revision:
The Field Behind the Screen: Using the Method of Netnography
To Research Market-Oriented Virtual Communities
ABSTRACT
This article develops netnography as an online marketing research technique for providing consumer insight. Netnography is market-oriented ethnography conducted on virtual communities dedicated to marketing-relevant topics. As a method, netnography is faster, simpler, and less expensive than ethnography, and more naturalistic and unobtrusive than focus groups or interviews. It provides information on the symbolism, meanings, and consumption patterns of online consumer groups. Guidelines are provided that acknowledge the online environment, respect the inherent flexibility and openness of ethnography, and provide rigor and ethics in the conduct of marketing research. As an illustrative example, a netnography of an online coffee newsgroup is provided and its marketing implications discussed.
Here is the entire third round revision: field_behind_round3.pdf
