I had worked and poured my heart into my revision of the JMR manuscript and had sent it back to Russ Winer by the deadline, my mind filled with hope and optimism about the possibility of publishing in JMR.
It was the end of summer, and the weather in Chicago was still quite warm. But the response I got from the Editor and Reviewers at JMR stopped me cold and chilled me through. The manuscript was still alive, but it was being eaten alive. Enduring a slow death, its fate was hanging by an extremely thin thread. It was bad news, much worse than before.
Have a look.
Journal of Marketing Research
September 13, 2000
Professor Robert V. Kozinets
Kellogg graduate School of Management Northwestern University
Leverone Hall
Evanston IL 60208-2008
Dear Robert,
RE: MS 1205-9-2 “The field behind the screen: Using Netnography to research market-oriented virtual communities.”
The above-referenced manuscript was read by the two reviewers who had recommended rejection of the original version. Their current recommendations regarding the disposition of this manuscript are as follows: Reviewer #1: reject; Reviewer #3: invite revision.
I see this manuscript as a methodological piece. It introduces marketing researchers to a new approach for collecting data from Web communities and for analyzing these data. However, the current version of the manuscript is way too long on background and too short on details about how the method can be actually applied, about its advantages over traditional research methods and about the empirical illustration.
Typical JMR readers would want to know when “Netnography” would be appropriate and useful, and how exactly they would go about applying it. I believe your manuscript could be of value to JMR readers, but it must be positioned and written properly for this audience. First, I suggest that you replace the discussion about computer-mediated communications and virtual communities of consumption with the proper citation of seminal articles and let readers seek them if they need an elaborate discussion on these topics. Your focus on the introduction should be on distinguishing “netnography” from ethnography, and clearly demonstrating the potential value of the former in marketing research, relative to competing methods (focus groups, and other face-to-face methods).
Second, you should provide a more specific and detailed description of the steps and procedures involved in conducting “netnography.” This should be a blueprint to the reader interested in applying the method on her own research, explaining why the steps are necessary and how they lead to unbiased, objective information to the marketer.
Third, your empirical illustration should provide the reader with a clear idea of how the proposed approach is applied and of its methodological rigor. The two experts find that the description of your application does not make a strong case for ethnography, and in fact might give readers the impression that it lacks rigor. The current application gives the erroneous impression that netnography consists of a simple content analysis of news group discussions. I would like to know each one of the steps that you have followed in this application, from selecting the sample, collecting the data, interpreting the results, cross-validating the interpretation, and the managers’ use of these results. For example, I would like to know:
• which particular news groups were chosen
• what criteria were used for this selection
• how many messages were interpreted
• how many users were involved in these messages
• some classification of those messages into general categories (my own experience with newsgroups is that there is a lot of idle, useless talk in them.)
• how were the messages validated (product-related newsgroups are known to be visited by manufacturers, often anonymously).
• the method used in interpreting these messages
• how the interpretations were cross-validated (multiple interpreters? Crossvalidation?)
• does this provide better and richer information than a series of focus groups, or indepth interviews? Why?
I hope you will be able to revise your manuscript along the lines discussed above, and responding to the two reviewers. This revision will require a complete re-write of the manuscript. I also want you to reduce the total length of the manuscript to no more than 30 pages (for example, you can drop the two figures and glossary, and reduce the citations to those strictly necessary). The success of this revision will depend heavily on the detailed description of the methodology and on the quality and rigor of the empirical illustration.
Sincerely,
[Wagner]
Wagner A. Kamakura
A quarterly publication of the American Marketing Association / 311 S. Wacker Drive, Suite 5800/ Chicago, Illinois 60606
“JMR MS# 1205-9-X1
“The Field Behind the Screen ….. ” Reviewer # 1
The revision is a workmanlike job that shortens the paper as the editor and reviewers suggest. Nonetheless, cosmetic changes (e.g., deleting “scholars in diverse disciplines” in the first sentence and replacing it with” marketing researchers” or replacing “interest” with “important”) do not represent a “more focused positioning,” as the authors claim in Note 3. The issue of overclaiming identified by Reviewer 3 comes out even more clearly in the revised version.
The article claims to build on “humanistic” research (par 2, p 1) to theorize “fundamental differences” between ways that researchers can access online vs. offline behaviors. This seems a rather pedestrian undertaking, and it is up to the authors to convince the readers not only that there ARE differences, but that they are sufficiently noteworthy to provide added value to researchers. To this end, the authors have included an empirical application of the method. The reviewer’s task remains that of assessing the revised claims of “new and different” as well as that of evaluating the empirical evidence.
A major problem that surfaces at the outset is weakness of construct definition. To begin, the authors note (p.3) the “frequent incoherence of the term community.” Next, the issue of whether or not CMC is impoverished (p. 4) or “richly detailed” is confusing. Footnote 5 is not helpful, and the authors’ stance is itself equivocal. Finally, the laundry list of special characteristics of CMC is “consolidated” (p. 5) in a way that makes it impossible for readers to understand. How did the 12 characteristics morph into the 3 that the authors’ identify? The presence of so many vague and patchy constructs suggests a glossing-over of fundamentals essential to theorizing.
Lack of clarity about CMC vs. f2f in terms of similarities/differences makes the section on CMC mediation of social representation (p. 7) confusing: which is it that the authors claim -differences or similarities? The discussion is still off the mark, because cmc has changed the nature of “writing” far beyond what Derrida et al. considered. The relationship between the written/spoken word is so complex that merely mentioning putative differences and/or similarities is obfuscating.
That online research is useful for studying net cultures (p. 10, “entree”) is rather obvious. Likewise, so is its application for the study of virtual “communities,” however defined, with an eye towards “providing a contextualized understanding” of behavior. This squarely places it in the realm of interpretive research, which, admittedly, has been sparsely represented in JMR. However, the global claim of applicability to marketing research is not borne out. From the lists of group types on (pp. 12 ff.), the material covered is simplistic and obvious to those in market research. For example (p. 15), the presence of enormous amounts of data requires data management; anyone who has dealt with transcripts of taped interviews is well aware of this.
“Trust and Rapport” assumes participatory rather than observational research, and informant identity (pp 15-16) does likewise. This is essentially ethnographic research, and not newsworthy. The alternative of identity-construction, discussed in terms of Haraway’s quote (p. 16), is most problematical. How is the researcher to evaluate what is imaginary and what is real? And what is a “lived material-semiotic world?”
The section on “interviews” are admitted to be “like their face-to-face counterparts” (p. 17), and the promise that digital projective tasks offer more “profound” levels of cultural knowledge gives rise to two questions: first, “more profound” than what? And second, what supports the claim of “more X” than anything?
The whole matter of public versus private is a non-issue: electronic eavesdropping on public communication is no more likely to give rise to “psychological harm” than is eavesdropping on a conversation held in a restaurant. King (bottom, p. 19) points this out. Where, exactly, can one not distinguish private from public (p. 20)? Indeed, the conclusion (bottom p. 20) is a nonstarter, for the first situation is routine ethnographic procedure, and the second (observation) is non-invasive. What is new news here?
The empirical portion (pp. 21 ff.) is what this reviewer was awaiting. However, the first paragraph exemplifies destroys any pretense of generalizability, and cannot be explained away by mentioning it. As Wells reminds us, merely stating limitations does not make them go away. Further, mining the news group appears to be observational — thereby rendering most of the preceding sections (those dealing with participatory researchers) irrelevant. The dilemma here is that the technique is shown to be neither innovative nor of real value to researchers. If the “downloads and observations” are anonymously generated, how is the market researcher supposed to discriminate between the real and imaginary selves of the respondent? The research outcome is disappointing from the perspective of value added: f2f interviews yield the same information (useful in market segmentation), without the identity mystery. In market positioning, it is essential to know who the market segments are!
Thus, the implications (p. 25) are neither “major,” as claimed, nor– potentially — even useful.
Notes to Authors of “netnograhpy. [sic]” –1205-9-2 Rev. #3
General: First, I want to say that this manuscript has again improved. It has also moved in the direction requested by the reviewers and the editor. It is also clearly an important topic. We’ve come a long way from the first submission.
I really want to acknowledge that.
There are still, however, problems. They are:
Problems:
One: The paper, positioned as a methods piece, is still under-specified, or (alternatively) simply reflects the open and inherently under-specified nature of ethnography. I tend to believe the later. Ethnography, at least to me, represents the most open form of inquiry. This does not mean it is sloppy or purely a make things up as you go meta-method either. This point seems lost in our current discourse, and in this paper. The author gets caught in this dammed-if-you do, damned if you don’t dilemma. But, I don’t want to let him or her entirely off the hook. Right now the paper reads like a very-under-specified scientific positioning of ethnography. This isn’t working for me very well. I wish the author would either push it fully in that direction, or more explicitly speak of its flexibility as a virtue, and try to explain to a relatively naive audience the nuanced nature of the interpretive act. This is not easy, I know that.
Two: The example is a fairly weak. There’s really nothing ethnography (specifically) did here in order to recommend it above any other method. This is the weakest part of the manuscript. Either it should be cut out entirely, or really built up. JMR readers are going to rightfully say: I don’t see anything special about this: standard focus group/concept testing accompanied by conjoint would have done a much better job … and they are right. Either show us the money, or drop it. I prefer the former. Show us what this method can do above and beyond any other.
Three: conclusion is a bit of a job of over-claiming ..
Page by page:
p., par I: what was USA Today’s source?
P 1: I still think this is essentially the same (or very similar) to what the interpersonal communications people are doing with on-line communication: it’s conversation analysis. And even prior to the net, when conversations were tape recorded, transcribed and analyzed … isn’t this similar?
P 2: I don’t agree with Laurel (1990), and I assume you as well: “all virtual communities exist as ‘villages of activity within the larger cultures of computing.'” I suppose this is quite debatable, but there is good theoretical reason to believe that as we go further down the diffusion curve, the medium and it’s culture becomes more transparent. People who listen to the radio in their cars, are not like early Marconi (sp?) (users). My guess is that by this point many users are not really cyberculture members at all (Whatever that is), but simply users of a rapidly diffusing technology. But, this is just a matter of opinion. You are entitled to yours.
P2: I think what you discuss here and illustrate in Figure 1 beautifully illustrates the problematized nature of the term “culture.” As you no doubt know, the notion of a set thing called culture is not particularly viable these days. I see these concentric circles as much too tightly drawn and formal. Cultures are built, tom down, and re-invented constantly. But, if this is how you want to do it.
P4: extra word, first sent.
p. 5: I’m not convinced that the limited cue theories have been discredited. Maybe, but I’d like to see more evidence.
P.6; grammar: addition
p. 8: “for reversion of gender, or ungendered being, and even the social inhabiting of imaginative animal, mechanical or alien bodies.” … no doubt this is true, but I think we may be getting a bit far afield from the modal application here.
P 11: I’m not sure FtF triangulation has to occur at all. They are all different methods. For example, surveys don’t NEED experiments.
P12: again, I think unobtrusive data may stand alone.
P 16: extra “identity”
P 17- I we are now trading in very general terms about surveys and interviews, I thought this was about this unique form of ethnography. This doesn’t belong here.
P 19-20 this discussion of ethics should be more focused on your particular method. Very general discussion.
P 21: more on the idea of member checks, and its particular value here.
P22: total postings ==7
Pps 21-25: Illustrative Example: this really didn’t work for me, at all. Two problems: There is no methodological detail here, or really much of anything about METHOD, which is really the promise of the paper, and (2) I really don’t see what this yields beyond the standard focus group/concept tests by a commercial supplier, and maybe a conjoint. Given your focus on segments, why wouldn’t a conjoint do as well, and provided comparable attribute metrics, you could call them “netrics. “? Don’t get me wrong, I’m a fan of ethnography, but its value is not shining through here. Do it justice, or drop it.
P 26: missing word.
