In this long and detailed series of blog postings, I have tried to bring out the “backstage,” behind-the-scenes elements of what goes into a top-tier journal publications, from the initial submission, through the review process, to the final, accepted version of the paper.
I have done this for a paper that was historically very important to my career, a pivotal paper that marked the clearest and most widely-distributed explanation of the new approach of netnography, a rigorous approach to online ethnography. And an article that broke ethnography into the pages of the psychy and economics paradigm dominated JMR–something very few culturally-oriented research articles are able to achieve.
If you have been following the postings, you will have seen that this was not exactly an easy process. The blog posts were a good way for me to reveal the hidden story behind this new qualitative or ethnographic methodology.
One of the big stories here is that the paper was accepted because it had two champions, two people who believed in it enough to go against the grain and try to publish it. Russ Winer and Wagner Kamakura, the two editors of JMR, were those champions. They were supportive of the paper, but insisted on changes that made it better. They were able to judge the reviewers’ comments and to argue against them when they felt it was necessary.
In the first round, Prof. Winer was confronted with three reviewers. One said accept the paper unconditionally, basically as is. The other two said to reject it. Prof. Winer stated in his letter that he wanted to see a paper on this topic, citing his belief in JMR’s mandate to publish new methodological articles and approaches.
In the final round, which got the paper accepted, Prof. Kamakura was confronted with the two remaining reviewers. One said to accept the paper conditionally, meaning with some changes. The other said to reject it. That reviewer was consistently saying reject every single round of the paper.
In his cover letter, Prof. Kamakura went on record saying that he saw the contribution of this paper as different from Reviewer B. He said that, although he didn’t know the area of ethnography well, he thought that this adaptation had value, and he listed several sensible criteria to back that up, including the fact that it was applied to a real marketing and marketing research set of questions (i.e., the example helped to make that point convincingly).
These two champions of netnography are both from outside the qualitative consumer research field. They have training in economics, and are considered experts in the type of marketing research commonly referred to as ‘modeling.’
The surprising part of the story might be that that two remaining reviewers-who are both qualitative “CCT” people, likely senior ones, with apparently strong records of publication–were less than supportive. I see one (Reviewer B) as downright hostile, trying repeatedly to slam the JMR door in my article’s face.
In this last round, Reviewer B pulls out all the stops to try to argue that JMR absolutely must not publish this article. In his/her review, Rev. B keeps talking about what JMR readers would want to see, how expert they are, how simplistic the approach is, how little value it would have for the typical JMR reader, what a “mismatch” there is between JMR’s readership and “the paper’s content.” In other words, Reviewer B has it in for this paper. Reject, reject, and definitely-you-absolutely-must-reject this article. Not innovative. Not newsworthy. Not useful. Not for this audience.
This level of hostility and single-mindedness is, fortunately, pretty rare in my experience. As a writer, reviewer, and associate editor, I don’t often see reviewers who are quite this focused on killing papers, at least not nearly as obviously so.
I am grateful that Prof. Kamakura explicitly argued against Reviewer B’s comments,stating, quite simply, that he didn’t see things that way.
Reviewer C’s comments continue in this round to be somewhat helpful. Reviewer C pushed me to do better, to deliver on the promise of the topic, and that is what a Reviewer should do. This person did move in their evaluations and recognize the value in the paper. If not for them, acting as the fulcrum point, this paper would likely not have been published.
Reviewer B’s comments and approach, however, provide some food for thought. Do qualitative consumer researchers, or CCT people, have a reputation for ‘eating their young’? Have they, for a long time, been shooting down the work of young scholars? Are they failing to do the proper job of mentoring new people and new ideas into the field? Something must have explained the dearth of new people in this area for many years. I think this has changed, and continues to change, but certainly reviews like these should give everyone pause.
In the next blog posting, I will offer a few key takeaway points from this set of blog postings about academic publishing.
To complete the file itself, here are my Reviewer notes to the 4th revision. You will probably notice that I was just as meticulous and careful with this set of notes as I was with the last two. I did not want to jeopardize the paper’s full acceptance at this stage. Conditional acceptances can still be revoked if you don’t take the comments seriously.
Revision Notes RE: JMR MS# 1205-9-4: “The Field Behind the Screen: Using Netnography For Marketing Research in Online Communities”
First, I would like to thank you, Prof. Kamakura, as well as the 2 Reviewers for all of your helpful comments in these rounds of revision. I have revised the enclosed manuscript in line with the minor revisions in your suggestions, and the reviewer comments, as contained in your letter dated March 2, 2001.
I enclose this note to overview the specific changes that have been made. The notes will respond in detail to each point in your letter of March 2.
Responses to Editor’s Requested Changes
1. Hard-core users: since this seemed to be an important concern raised in your comments, in the comments of Reviewer #3, and also in exchanges with a range of people interested in the research, this revision treats these concerns in detail. On p. 12 of the manuscript, there is an extended section in which careful triangulation and long-term immersion are described as necessary to help marketing researchers distinguish hardcore, marginal extremists from a more typical group of consumers (which directly addresses your question about differentiating typical consumers from extremists). Directly addressing your question about avoiding being misled, rigorous methodology is demonstrated in the empirical example on p. 19 where the importance of sampling a range of conversational participation to avoiding “being misled or unduly influenced by a minority of unrepresentative and vocal extremists” is shown. The revised manuscript revisits and details this point about avoiding being misled by vocal extremists on p. 28 in the conclusions section and links it to distinguishing typical consumers from (as your medical metaphor puts it) “‘hard core’ pathological cases.”
So that readers do not get the impression that extreme or hard core perspectives are the norm in online community environments, the manuscript proceeds to explain (also on p. 12) that there are internal social mechanisms keeping this behavior in check. On p. 17, it is noted that even the hard core user may contribute to creative ideation, an important role of lead users (who could very easily be described as marginal, extremist, or hard core in their own right). A follow up to this appears on p. 28, with an assertion that links ostensibly extreme consumer behavior (i.e., “innovators” in the diffusion of innovation literature) with eventual diffusion into the mainstream. The assertion is that, while it is important to be conscious about whether or not extreme perspectives are being expressed, they still can be treated as useful and interesting information for marketing research.
2. Generalizability: The hard-core user issue is related to the issue of generalizability. As you suggested in your letter, this revised manuscript now covers issues of generalizability in several places, discussing it both in the presentation of the method and in the empirical section of the paper, giving specific guidelines. On p. 4, it is noted that generalizing the findings of a “netnography” to other groups must involve careful evaluations of similarity and employ multiple methods for triangulation. This point is again made and elaborated upon in some detail on p. 12.
3. Opinion Leaders: On p. 18, the background of the opinion leaders categorization of the prior manuscript is explained more clearly. The group of members are termed “insiders” who are frequently quoted and referenced by other community members, deferred to by existing and new members, and mentioned by members as important arbiters of taste. In the manuscript, this is termed “an informal type of network analysis” and it is noted that “these insiders seem to be usefully conceptualized as opinion leaders” in the local context of this particular online community. The assertion here is that the way in which these persons’ opinions are repeated and referenced by other online community members makes conceptualizing them as opinion leaders a useful categorization.
4. Distinguishing Netnography from Traditional Ethnography: As you suggested, netnography has been redefined. On p. 3 of the manuscript, netnography has been defined as a specialized adaptation of traditional ethnography. This revisions also attempts to carefully distinguish the particular need for netnography in a range of places within the revised manuscript, as suggested in Reviewer #1’s many comments to this effect, and your direction to address the issue. On the top of p. 3 has been added a very important paragraph that should help to clarify the positioning of this method in the two main literature streams it addresses (ethnographic methodology and marketing research method). This paragraph begins with a very concise description of the novel context of online community (a much-abridged version of material originally presented in revision 2 and later excised for brevity). Then, the manuscript proceeds to note that the difficult context may have challenged and hampered investigation of online communities by researchers -which sets up what I believe (and have observed in my exchanges with students and industry practitioners) is a very real problem. The manuscript then provides numerous references from the anthropology, sociology, and marketing research literatures where scholars have noted that rigorous guidelines are needed to adapt existing ethnographic research techniques to online contexts (evidence disputing and indirectly addressing Reviewer #1’s many concerns that a new adaptation of ethnography is patently obvious, nothing new and not needed). Finally, the manuscript describes (as you suggest in your letter) exactly what its contribution is in relation to standard ethnography. As stated on p. 3, “Although it does not break entirely new ground methodologically, this paper addresses this important need by providing researchers with a rigorous methodology adapted to the unique characteristics of online communities.”
Throughout the revised manuscript, in every section of the descriptions of methodology (and in the empirical example), netnography is distinguished and differentiated from traditional ethnography. As noted above, this is not the sort of quantum leap radical breakthrough that seems to be what Reviewer #1 is looking for. It is, however, an important and much-needed extension and adaptation. So through this revised version, there are many references to the differences between “netnography” and traditional ethnography. In every case, the adaptations are important and non-obvious. For example, on p. 6, it is noted note that “Distinct from traditional ethnographies, in the identification of relevant communities online search engines will prove invaluable.” On p. 9, it is noted that “Dealing judiciously with instantaneous information overload is a much more important problem for ‘netnographers’ than for traditional ethnographers.” On p. 10, it is noted that “in a sharp break from traditional ethnography, a rigorous netnography could be conducted using only observation and downloads, and without writing a single fieldnote.” On p. 11, the importance of procedures that are adapted to the purely textual nature of much online information is detailed, noting that utilizing online data can requires “a radical shift from traditional ethnography.” On p. 12, in the summary, these differences are again emphasized. On pp. 12-13, the texts emphasizes that “One of the most important differences between traditional ethnography and ‘netnography’ may be in issues of research ethics,” and then proceeds to elaborate a set of guidelines distinct from usual online practice (i.e., web-surfing and searching) and also from traditional ethnographic procedure.
5. Quotation Marks: As you requested, “netnography” now appear only within double quotation marks in the manuscript. Following this, “netnographer,” and “netnographic” have been placed in quotation marks as well. In addition, double quotes are also now used for the typology of newsgroup users (“minglers,” “tourists,” etc.).
6. Title Change. The title of the manuscript has been changed in line with your astute comment. It now more clearly portrays the nature of the contribution, which is to describe the use of netnography to conduct marketing research on online communities.
Responses to Changes Requested By Reviewer #1
All of Reviewer #1’s points have been addressed in my comments #4: Distinguishing Netnography from Traditional Ethnography, above.
Responses to Changes Requested by Reviewer #3
7. Virtual Community: On p. 1 of the revised manuscript, the colloquial use of the word virtual community is explained in order to acquaint readers properly with the topic of this paper (since the term is in common parlance). Reviewer #3’s correct assertion that the term may be misleading is then addressed by, citing appropriate (and earlier) literature. Thereinafter, in this revised version, the term “online community” is used to distinguish communities formed through computer-mediated communication, a topic central to the methodology.
8. Lead Users: On p. 17, the term “lead users” is defined and then the concept is compared and contrasted with online community participants. The essence of the assertion is in lead user’s creativity, inventiveness, and knowledge, which may (or may not) be evidenced by online community members. On p. 28 and 29, the text demonstrates how treating online community information as similar to lead user information can be useful to marketing researchers when accompanied by the proper critical evaluation and rigorous analytic procedure. Since the utility of lead user analysis is in invention and creativity, notions of representativeness (implied by Reviewer #3’s invocation of “marginal” and “inconsequential”) are not relevant to this classification. They are, however, useful points in and of themselves, and have been dealt with in my response #1: Hard-core users, above.
9. I respond to the comments on “opinion leaders” in response #3: Opinion Leaders, above.
10. “Maing” has been changed to “making”
11. “On these Internet-based” has been changed to “In these Internet-based”
12. Muniz and O’Guinn (forthcoming) has been changed to Muniz and O’Guinn (2001), although the issue has not yet been published and the page numbers are not available.
13. “Virtual” has been explained and the term “virtual community” changed to “online community” throughout, as explicated in detail in response #7: Virtual Community, above.
14. Opinion/Politics: On p. 13 I note the contentious, contestable, and still-under-debate nature of the online ethical marketing research discussions. As I wrote in considerable detail about this topic in my previous note responding to JMR reviewers, I will be brief here and state simply that I believe erring on the side of being overly cautious and considerate of online community member’s wishes is the most ethical course of action. Legality does not come into my methodological consideration set, because many things in the online environment are so new that they are only now being considered in a legislative and regulatory context (e.g., Napster, consumer privacy). Simply because Reviewer #3 says it is a public medium, this does not mean that this long-standing and important debate is settled. I politely but very adamantly differ in my opinion from Reviewer #3 on this matter. I believe that my “taking the high ground” treatment of research ethics in this paper may be an important contribution to the discourse on online marketing research.
15. Influence: The prior confusing reference to influence has been removed. The nature of the opinion leadership has been elaborated upon, as stated above. Influence was evidenced by quotations and references to the opinions and tastes of particular members of the online community, some of which have been represented in the illustrative example.
Final Comment
16. In closing these comments, it must be noted that each of these requested changes required the addition of textual material, often explicitly in several places (for example, on p.2 of your letter you direct that the revision be “improved…on page 3-15” by adding emphasis of its distinctions, and that generalizability “should be discussed both in the presentation of the method and in the empirical section”). There were no directions cut back on existing material. The prior version was cut to the bone. In the last revision, considerable material was excised (such as distinguishing the method from traditional ethnography and face-to-face qualitative research methods), some of which needed to be added back intro the manuscript in a more focused manner. Cutting back on existing material would, in this author’s opinion, impair the ability of this manuscript to make a contribution by either cutting short the explication or impairing the persuasiveness and depth of the illustrative example. It should be noted that consumer ethnographies (for example, the one I will be publishing in JCR this summer) can easily run over 70 pages -and contain little or no methodological explication. The current length of this revised manuscript -which contains a new method, guidelines for researchers, and a concise-yet-detailed ethnographic example- is thus by no means excessive.
Adding the additional material to this manuscript has considerably raised its contribution level to JMR. It has improved its ability to address certain fundamental questions of importance to marketing researchers about the generalizability of the technique and its differentiation from traditional ethnography. With this added material, netnography can make a much clearer contribution to the marketing research literature.
Here is the Abstract:
ABSTRACT
This article develops “netnography” as an online marketing research technique for providing consumer insight. “Netnography” is market-oriented ethnography conducted on online communities dedicated to marketing-relevant topics. As a method, “netnography” is faster, simpler, and less expensive than ethnography, and more naturalistic and unobtrusive than focus groups or interviews. It provides information on the symbolism, meanings, and consumption patterns of online consumer groups. Guidelines are provided that acknowledge the online environment, respect the inherent flexibility and openness of ethnography, and provide rigor and ethics in the conduct of marketing research. As an illustrative example, a “netnography” of an online coffee newsgroup is provided and its marketing implications discussed.
And here is the Round Four Manuscript: field_behind_round4.pdf
