This is, by the way, an actual unretouched photograph of what I looked like when I received my reviews back.
In the last post I gave you access to my initial submission to JMR. I sent it off on November 23, 1999, with a cover letter explaining the background for what I was doing, requesting a few appropriate reviewers, and asking the editor not to send it to a few reviewers I feared might savage the piece (doing the latter was a suggestion of one of my mentors).
Everything in those days was done on paper. I think there were 5 paper copies I had to send out. I initially was over their 50 page limit total, found out too late, and had to toss my 5 paper copies of 53 pages each into the garbage while I whittled the document down to 50 pages total.
Then, I waited for a paper package with the reviews in it to come in the mail. Pretty much everything is electronic and happen via email and pdf files now.
So, in February, I heard back. Everything from the reviews I am reprinting from the paper copies I kept in my files. Everything was scanned in with OCR software. I’ve checked it a few times, but some errors may remain that are my own. Some of the reviews had errors in them, and those I’ve marked with the convention [sic] so that you know the errors were in the original copies.
Here in the blog is the scan of the Editor’s cover letter, and the reviews. I have asked Russ Winer’s permission to use this letter in its complete, intact form, and he has kindly consented.
In the next post, I will elaborate and interpret the reviews and the letter a little bit, and then I will present my response, which was contained in my reviewer notes.
Journal of Marketing Research
Russell S. Winer, Editor Haas School of Business
University of California at Berkeley Berkeley, CA 94720-1900
February 15, 2000
Professor Robert V. Kozinets
Kellogg Graduate School of Management Northwestern University
Evanston, IL 60208-2008
Dear Rob:
Re: MS#1205-9-1: “The Field Behind the Screen: Using Netnography to Research Market-Oriented Virtual Communities”
The above-referenced paper was read by three knowledgeable reviewers. Their recommendations concerning the disposition of the paper are the following: Reviewer #1: Reject, despite merit; Reviewer #2: Unconditionally accept; Reviewer #3: Reject, despite merit.
This is obviously an unusual manuscript for JMR as neither do I see too many ethnographic studies nor many papers at all without empirical work. In order to get varied opinions on the paper, I sent it to one reviewer from your suggested list, one from your “no” list whose opinions I trust, and a third not on either list but who is very familiar with ethnographic studies.
Let me state at the outset that I am interested in publishing a paper on this topic. JMR editors should publish and always have published papers demonstrating the value of new methodologies. Of course, what is particularly interesting and topical about netnography is that its domain is the Internet. The three reviewers also feel that the paper has merit, although Reviewers # 1 and #3 think that the promise of the work is more in the future than with the present paper.
As you know, I am not an expert in this area of research. Besides the comments made by #1 and #3, I can see two major issues with the work. The first is editorial in nature. I found the conceptualization aspect of the paper to be interesting and worth keeping. However, the section “Ethnography and Netnography” carries on for too long and causes the reader (at least this one) to lose interest. This section needs be more tightly focused on the netnography method itself. The second major issue is that I would like to see an application of the method. An obligation of an author espousing a new method is to show how it can be applied in practice; particularly in a journal that is read by many practitioners.
Reviewers #1 and #3 have made some very useful comments. Reviewer #1 ‘s main point is that you have not put the netnographic method into terms that marketing researchers can use. This is a valid point; some of this problem would be ameliorated with an empirical application. Reviewer #3 feels that you have not made a sufficient case for how the netnographic method is new and different. While I would be satisfied with a good explication of the approach with an application (i.e., it does not have totally break new ground), this reviewer obviously feels that your claims have been overstated.
Overall, I am inviting you to revise and resubmit the paper for possible publication. However, the paper needs a significant editorial overhaul and a good example of its application. In addition, at 38+ pages, it is too long. I would like you to shoot for a 30- page paper, excluding exhibits and references. I will send the paper back to Reviewers #1 and #3; thus, if you choose to revise the paper, you should prepare detailed notes about how you addressed their comments.
After June 30, 2000, please send any new or revised manuscripts to:
Professor Wagner A. Kamakura, Editor
Journal of Marketing Research
Henry B. Tippie College of Business
The University of Iowa
108 John Pappajohn Business Bldg.
Iowa City, IA 52242-1000
Thank you for sending your work to JMR, and please continue to do so in the future.
Sincerely,
[Russ]
Russell S. Winer
A quarterly publication of the American Marketing Association / 250 S. Wacker Drive / Chicago, Illinois 60606
MS# 1205-9-1 “The Field Behind the Screen …..
Reviewer # 1
Abstract: The concept of “netnography” is set forth as the subject of this ethodological [sic] contribution to market research.
General Comments
The opening paragraph would be clearer if “market-oriented interests” were briefly described, especially in view of “virtual communities.” The focus is ambiguous, for the paper’s focus on market-oriented choice is not well-defined. Does this mean information-gathering via web sites (a solo, not a social activity), which is marketer-driven? And/or information-exchange in chat rooms (social), which is not marketer-driven? Or both? Given that the paper examines online vs. offline environments, the precise ones should be stated up front.
The object of study seems to be CMC (rather than consumer choice?). However, the signs used to replicate face-to-face communication are extensions of linguistic/syntactical techniques generally considered under the rubric of “tone” in discussions of print communications mimetic of human speech. Thus, “differences” between online speech and f2f speech should be considered in terms of print media as well.
At times (p. 7, for example), the use of jargon overwhelms clear presentation of the material. This paragraph (and others like it), do not lead the reader anywhere — what is an “obtrusive and opaque cultural experience”?
The differences do not necessarily hold firm: time-lags also characterize f2f communication, with silences and delayed responses long considered valuable communication devices (notably in terms of endowing speakers with power). The writing/speech relationship has been discussed at length in deconstructive criticism (Derrida, especially), and those arguments should be consulted. Even though online selves are considered mutable, personality changes are very much a feature of f2f communication as well — one puts on a self to serve the rhetorical purpose of the speech act.
The last sentence in par. 1 on p. 11 seems to be a summary of the market-oriented behaviors that the paper will consider, However, the distinction between the first two (consumer-driven) and the last one (marketer-driven) problematizes the notion of “community.” Is this the consumer community (WOM, online chat groups) or the marketing community (advertisers on the net) or both? The study of media fan and game-playing communities (p. 12) seems more limited than the above communities. The injunction to be careful regarding “social-specificity” in researching general marketing and consumer behavior is bewildering — Jenkins’s phrase is, again, a bit of jargon that is not readily identifiable by all marketing researchers
Fly on the wall observational techniques do not resemble focus groups –the former are consumer-driven, whereas the latter are marketer-driven. The hidden observer technique seems far less likely to involve social construction of the _ audience under study. The discussion of types of virtual communities does not tie into consumer behavior very clearly — how exactly are they all marketing-oriented?
This raises the issue, again, of the focus of netnotgraphy [sic]: is it the subject of the consumer discourse, the nature of the discourse itself, or both? The first does not necessarily lead to generalizable knowledge; the second may not be possible because of fragmented discourse types; the third possibility is confusing.
On p. 17, the authors discuss the accessibility of data (observational netnography). This is a major advantage of netnography, but its advantageousness depends on the researcher’s focus. There is little new about this observation, and the authors shy away from the issue of computer-coding vs. hermeneutics-by-hand. The section on treating on1ine data seems at odds with introspective “noting” by the ethnographer. Why is this a necessary step in observational netnography? The line between observation and participation has to be rethought very carefully in terms of standard ethnographic practices vs. online research — as presented, it is not convincing.
The study of anonymous users (who may be recreating themselves anew) vs. the study of trusted informants is reminiscent of the study of the write vs. the man/woman. Here, a major source is the philosophical and literary debate about reality: which one is “really real”? These debates would shed light on the paper’s discussion of the issue _p.22.
The mechanics of the CMC interview (like those in many interpretive studies) reveal the threat of response bias: interviewees are those who agree to be interviewed rather than a random sample of the universe in question. The issue of e-mail surveys appears to be different from that of interviewing chat room participants, but the paper is not clear on this point.
How does the section on research ethics relate to strictly “observational” netnography? It seems as if electronic eavesdropping is about the same as eavesdropping on public conversations, crowd comments, sports fans at a game, and so forth. Whether or not eavesdropping is ethical does not depend on whether or not the subjects give consent if one assumes that anything said in public is not privileged matter.
The section on representation might be enriched by consulting those works in marketing/consumer behavior that deal with representation in a traditional way — Stern’s edited book, for example, with the useful article by Price and Arnould.
To sum up, the topic is most interesting, but there are problems in different sections that need to be resolved. Perhaps most problematical in the lack of a clear marketing research focus. The study of consumer groups (and perhaps, marketing communications) is not fleshed out, and the objects of netnography __ consumer choice, -consumption communities — aren’t made compelling to marketing researchers. The paper is nicely written and organized but it needs a much more specific focus to make it relevant to marketing.
Comments to Author of Netnography. Reviewer #3
Like many, I’m very interested in this topic. You offer some very insightful and thoughtful comments. I am impressed with that. And, I do think that it demonstrates a familiarity and a level of expertise that sets you apart from many. So, I applaud the effort, the insight, and the challenge. At the end of the day (and this review) I want to encourage you to go forward.
However, I am not favorably disposed to the paper as written. I think it over-claims, and underdelivers. I am not saying it is without merit, or potential. But, I am saying as currently conceived and written, I simply don’t think it works.
Here’s why:
It’s a bit of a fireside chat. While making some fairly grand claims, it only offers a recitation of things many which many people have recognized, written about, and pondered. It ponders well, but does little in a concrete way.
While CMC present legitimately new research challenges, a poor case was made for how this author’s methods (not explicated particularly well either), were going to significantly engage the new problems, address the new opportunities, and actually contribute something tangible and concrete in reaction.
Many of the underlying assumptions are good rhetoric for positioning one’s work, but highly questionable. For example, on page 1: “they (market researchers) have a weak appreciation of some of the more radical discontinuities in social formation these developments represent.” Quite the contrary, I think many have a very good appreciation for the discontinuities, as well as for the facets of CMC which are not radically different at all from other forms of human communication.
Most critically and damning, there is absolutely nothing to the idea that the methods proposed here are “discontinuous innovations.” NO WAY. In fact, I don’t really see fully developed and explicated method here. And most of what I do see, is not really new. Add to this situation, modest empirical grounding and then there’s just not much here in the way of methodological discontinuities. Yes, all media are different, and have unique properties, and this is true of the Internet as well. But there is nothing you propose here that is A DIFFERENT METHOD, not really. You trade on the novelty of the internet to advance this claim, but you never show anything beyond an adaptation, extension, or for that matter direct application of other textual, interpersonal communication, organizational communication, linguistic, or general field work technique. There is simply no discontinuous methodological innovation here. I’m not sure there needs to be, but I simply don’t see it.
Now, I could be wrong. You could try to convince me, show me. Or, maybe you want to see your contribution as more one of integration, synthesis, extension. I don’t know, that’s really between you and the editor, but one thing is sure: the discontinuous innovation claim is simply not working.
I do wish you well.
