Monthly Archives: July 2009

The Academic Journal Review Process: Nine Suggestions for Scholarly Researchers

number-nines.jpgAs many of you know, the JMR paper whose progress we have carefully followed in the past several blog postings has gone on to become a fairly well known article.

If you look at the JMR’s list of their most highly cited articles, this one appears #9 on their list of all articles from 2000-2007. It is number one on in its category of 2002-2004, and has been in the top spot in its rolling 3-year class since the top-rated figures have been kept.

So, reading through the reviews, I thought that there might be several lessons in the story that might benefit junior scholars of all stripes, and maybe even all publishing academic scholars in general.

1. Don’t be afraid to research something really new and interesting. Believe in yourself. Innovate. Take risks. The reviewers may not recognize your contribution; they’re only human. Focus on making a contribution. Be the kind of academic and the kind of scholar that you admire.

The greater audience of scholars, the academic community, often recognizes works of value, over time. But you need to get it out there. Do not be afraid to do something risky and different. We need more of that kind of big-thinking scholarship. But you will need to handle resistance and critique. Expect it to take longer. Expect it to be more difficult. Expect to get frustrated. But expect that you will grow in myriad and immeasurable ways as a scholar because of that challenge.

2. Be the right mix of stubborn and flexible as you respond to Reviewers. What’s the right mix? Well, that depends. If this is really something new, a new area, new construct, a new context, a genuinely new theory, or a new methodological approach (as here), then you may well know more than your reviewers and editor, and you may need to enlighten them if they haven’t got it yet. If this is a well-mined area, you may not know as much as your reviewers and editor(s), and you might be best off learning more from them. In that case, keep your mouth shut and be flexible and eminently responsive. You need to know when to resist, when to simply act on advice, and when to just be grateful that these smart, experienced people are mentoring-you-by-mail.

3. Be polite. If you are actually enlightening others, even if you are angry, hurt, and frustrated by the process and the processors, do your utmost to be polite to the others who are involved. Even if they are not polite to you (especially if they are not), try to maintain a deferent, humble tone that keeps the whole thing professional and collegial.

Reviewers and Editors do this work for free. And, oftentimes, others have taught them their (bad) habits. Try to see it through their eyes, and approach them and the process itself with a positive, grateful, upbeat tone. Do anything else, and it’s a slippery, nasty slope.

4. If there’s a big problem with the process, talk to the Editor. They are good people generally and, in my experience, a phone call can be very helpful. It doesn’t matter if you are junior or senior as a scholar–as someone who is in the review process at that journal you all should count the same.

However, don’t abuse this option. Only ask them if you can schedule a call if there is something really, really wrong. I have only done this one other time in my career.

5. Ask for specific, constructive critique on your paper if you aren’t getting it. You will notice that the big difference between Reviewer B and Reviewer C in these reviews is that Reviewer B only criticizes, and never offers actual suggestions for improvement. Any intended improvement is only shot down in the subsequent round. Reviewer C, on the other hand, sets the bar high, but is specific enough that the directions can be discerned and followed. Reviewer C notes and recognizes the improvement, even when s/he is asking for further refinement of it.

I wrote and verbally asked the Editor to try to ensure that Reviewer B be more constructive in his/her critiques. After that, as the Reviewer continued to be unhelpful, I believe that they may have lost credibility with the Editor, and this may have led to their advice to reject the paper being more heavily discounted. Even as a junior author with no other publications, I was able to make my voice heard.


6. Remember that the process can be painful when you are sitting on the other side, and act accordingly when you review.
Criticism or faint praise can discourage and dishearten newcomers. Remember that when you write reviews of your own. Try to be constructive, positive and encouraging in all of your reviews. Write with the Golden Rule in mind. And always be constructive, and tell the author what they need to do to improve the paper (see point # 5 above). Even if this is a rejection-especially if it is a rejection—be concrete and constructive. Tell them exactly what they need to do to get the paper to publishable quality.

7. Appreciate and listen to the advice of the Editor or Associate Editor who synthesizes comments, particularly ones that are divergent, and give you specific guidance about the major revisions you need to make to your paper. At that point, you don’t always need to take their guidance (see points #1 and 2, above). But you benefit greatly from knowing exactly what that guidance is. And if you don’t take that advice, you need to clearly and convincingly state why you are doing something different. In probably 99 times out of 100, you want to take a good, involved Editor’s advice as offered.

8. Realize that journal articles don’t spring fully-formed from the minds of their authors the way Athena did from the forehead of Zeus. The vast majority of them take a lot of time, blood, sweat, struggle, and angry, bitter nights and weekends to get to be the way that they finally appear. New authors: don’t be discouraged if your first drafts don’t look exactly like the published articles. Neither did my first drafts. As you can see.

9. Persist. Persist. Persist. Never give up. Rise to the challenge. Make your case. Persist. Perform your revisions and changes as quickly as you can, keep it fresh, keep it moving. Be efficient and do the best job you can. Be thorough. Keep coming back. Stay in the game. Don’t give up. Work through the disappointment. Refuse to go away. Persist.

There are probably many more takeaways and many more interesting stories. I would be happy to hear your stories and share them and also work through any questions you might have.

That’s enough journal publishing stuff to last me a while. Stay tuned for lots more new blog material, coming your way soon.

Anatomy of an Academic Article, Round 4: End Game

end_game.jpgIn this long and detailed series of blog postings, I have tried to bring out the “backstage,” behind-the-scenes elements of what goes into a top-tier journal publications, from the initial submission, through the review process, to the final, accepted version of the paper.

I have done this for a paper that was historically very important to my career, a pivotal paper that marked the clearest and most widely-distributed explanation of the new approach of netnography, a rigorous approach to online ethnography. And an article that broke ethnography into the pages of the psychy and economics paradigm dominated JMR–something very few culturally-oriented research articles are able to achieve.
If you have been following the postings, you will have seen that this was not exactly an easy process. The blog posts were a good way for me to reveal the hidden story behind this new qualitative or ethnographic methodology.

One of the big stories here is that the paper was accepted because it had two champions, two people who believed in it enough to go against the grain and try to publish it. Russ Winer and Wagner Kamakura, the two editors of JMR, were those champions. They were supportive of the paper, but insisted on changes that made it better. They were able to judge the reviewers’ comments and to argue against them when they felt it was necessary.

In the first round, Prof. Winer was confronted with three reviewers. One said accept the paper unconditionally, basically as is. The other two said to reject it. Prof. Winer stated in his letter that he wanted to see a paper on this topic, citing his belief in JMR’s mandate to publish new methodological articles and approaches.

In the final round, which got the paper accepted, Prof. Kamakura was confronted with the two remaining reviewers. One said to accept the paper conditionally, meaning with some changes. The other said to reject it. That reviewer was consistently saying reject every single round of the paper.

In his cover letter, Prof. Kamakura went on record saying that he saw the contribution of this paper as different from Reviewer B. He said that, although he didn’t know the area of ethnography well, he thought that this adaptation had value, and he listed several sensible criteria to back that up, including the fact that it was applied to a real marketing and marketing research set of questions (i.e., the example helped to make that point convincingly).

These two champions of netnography are both from outside the qualitative consumer research field. They have training in economics, and are considered experts in the type of marketing research commonly referred to as ‘modeling.’

The surprising part of the story might be that that two remaining reviewers-who are both qualitative “CCT” people, likely senior ones, with apparently strong records of publication–were less than supportive. I see one (Reviewer B) as downright hostile, trying repeatedly to slam the JMR door in my article’s face.

In this last round, Reviewer B pulls out all the stops to try to argue that JMR absolutely must not publish this article. In his/her review, Rev. B keeps talking about what JMR readers would want to see, how expert they are, how simplistic the approach is, how little value it would have for the typical JMR reader, what a “mismatch” there is between JMR’s readership and “the paper’s content.” In other words, Reviewer B has it in for this paper. Reject, reject, and definitely-you-absolutely-must-reject this article. Not innovative. Not newsworthy. Not useful. Not for this audience.

This level of hostility and single-mindedness is, fortunately, pretty rare in my experience. As a writer, reviewer, and associate editor, I don’t often see reviewers who are quite this focused on killing papers, at least not nearly as obviously so.

I am grateful that Prof. Kamakura explicitly argued against Reviewer B’s comments,stating, quite simply, that he didn’t see things that way.

Reviewer C’s comments continue in this round to be somewhat helpful. Reviewer C pushed me to do better, to deliver on the promise of the topic, and that is what a Reviewer should do. This person did move in their evaluations and recognize the value in the paper. If not for them, acting as the fulcrum point, this paper would likely not have been published.

Reviewer B’s comments and approach, however, provide some food for thought. Do qualitative consumer researchers, or CCT people, have a reputation for ‘eating their young’? Have they, for a long time, been shooting down the work of young scholars? Are they failing to do the proper job of mentoring new people and new ideas into the field? Something must have explained the dearth of new people in this area for many years. I think this has changed, and continues to change, but certainly reviews like these should give everyone pause.

In the next blog posting, I will offer a few key takeaway points from this set of blog postings about academic publishing.

To complete the file itself, here are my Reviewer notes to the 4th revision. You will probably notice that I was just as meticulous and careful with this set of notes as I was with the last two. I did not want to jeopardize the paper’s full acceptance at this stage. Conditional acceptances can still be revoked if you don’t take the comments seriously.

Revision Notes RE: JMR MS# 1205-9-4: “The Field Behind the Screen: Using Netnography For Marketing Research in Online Communities”

First, I would like to thank you, Prof. Kamakura, as well as the 2 Reviewers for all of your helpful comments in these rounds of revision. I have revised the enclosed manuscript in line with the minor revisions in your suggestions, and the reviewer comments, as contained in your letter dated March 2, 2001.

I enclose this note to overview the specific changes that have been made. The notes will respond in detail to each point in your letter of March 2.

Responses to Editor’s Requested Changes

1. Hard-core users: since this seemed to be an important concern raised in your comments, in the comments of Reviewer #3, and also in exchanges with a range of people interested in the research, this revision treats these concerns in detail. On p. 12 of the manuscript, there is an extended section in which careful triangulation and long-term immersion are described as necessary to help marketing researchers distinguish hardcore, marginal extremists from a more typical group of consumers (which directly addresses your question about differentiating typical consumers from extremists). Directly addressing your question about avoiding being misled, rigorous methodology is demonstrated in the empirical example on p. 19 where the importance of sampling a range of conversational participation to avoiding “being misled or unduly influenced by a minority of unrepresentative and vocal extremists” is shown. The revised manuscript revisits and details this point about avoiding being misled by vocal extremists on p. 28 in the conclusions section and links it to distinguishing typical consumers from (as your medical metaphor puts it) “‘hard core’ pathological cases.”

So that readers do not get the impression that extreme or hard core perspectives are the norm in online community environments, the manuscript proceeds to explain (also on p. 12) that there are internal social mechanisms keeping this behavior in check. On p. 17, it is noted that even the hard core user may contribute to creative ideation, an important role of lead users (who could very easily be described as marginal, extremist, or hard core in their own right). A follow up to this appears on p. 28, with an assertion that links ostensibly extreme consumer behavior (i.e., “innovators” in the diffusion of innovation literature) with eventual diffusion into the mainstream. The assertion is that, while it is important to be conscious about whether or not extreme perspectives are being expressed, they still can be treated as useful and interesting information for marketing research.

2. Generalizability: The hard-core user issue is related to the issue of generalizability. As you suggested in your letter, this revised manuscript now covers issues of generalizability in several places, discussing it both in the presentation of the method and in the empirical section of the paper, giving specific guidelines. On p. 4, it is noted that generalizing the findings of a “netnography” to other groups must involve careful evaluations of similarity and employ multiple methods for triangulation. This point is again made and elaborated upon in some detail on p. 12.

3. Opinion Leaders: On p. 18, the background of the opinion leaders categorization of the prior manuscript is explained more clearly. The group of members are termed “insiders” who are frequently quoted and referenced by other community members, deferred to by existing and new members, and mentioned by members as important arbiters of taste. In the manuscript, this is termed “an informal type of network analysis” and it is noted that “these insiders seem to be usefully conceptualized as opinion leaders” in the local context of this particular online community. The assertion here is that the way in which these persons’ opinions are repeated and referenced by other online community members makes conceptualizing them as opinion leaders a useful categorization.

4. Distinguishing Netnography from Traditional Ethnography: As you suggested, netnography has been redefined. On p. 3 of the manuscript, netnography has been defined as a specialized adaptation of traditional ethnography. This revisions also attempts to carefully distinguish the particular need for netnography in a range of places within the revised manuscript, as suggested in Reviewer #1′s many comments to this effect, and your direction to address the issue. On the top of p. 3 has been added a very important paragraph that should help to clarify the positioning of this method in the two main literature streams it addresses (ethnographic methodology and marketing research method). This paragraph begins with a very concise description of the novel context of online community (a much-abridged version of material originally presented in revision 2 and later excised for brevity). Then, the manuscript proceeds to note that the difficult context may have challenged and hampered investigation of online communities by researchers -which sets up what I believe (and have observed in my exchanges with students and industry practitioners) is a very real problem. The manuscript then provides numerous references from the anthropology, sociology, and marketing research literatures where scholars have noted that rigorous guidelines are needed to adapt existing ethnographic research techniques to online contexts (evidence disputing and indirectly addressing Reviewer #1′s many concerns that a new adaptation of ethnography is patently obvious, nothing new and not needed). Finally, the manuscript describes (as you suggest in your letter) exactly what its contribution is in relation to standard ethnography. As stated on p. 3, “Although it does not break entirely new ground methodologically, this paper addresses this important need by providing researchers with a rigorous methodology adapted to the unique characteristics of online communities.”

Throughout the revised manuscript, in every section of the descriptions of methodology (and in the empirical example), netnography is distinguished and differentiated from traditional ethnography. As noted above, this is not the sort of quantum leap radical breakthrough that seems to be what Reviewer #1 is looking for. It is, however, an important and much-needed extension and adaptation. So through this revised version, there are many references to the differences between “netnography” and traditional ethnography. In every case, the adaptations are important and non-obvious. For example, on p. 6, it is noted note that “Distinct from traditional ethnographies, in the identification of relevant communities online search engines will prove invaluable.” On p. 9, it is noted that “Dealing judiciously with instantaneous information overload is a much more important problem for ‘netnographers’ than for traditional ethnographers.” On p. 10, it is noted that “in a sharp break from traditional ethnography, a rigorous netnography could be conducted using only observation and downloads, and without writing a single fieldnote.” On p. 11, the importance of procedures that are adapted to the purely textual nature of much online information is detailed, noting that utilizing online data can requires “a radical shift from traditional ethnography.” On p. 12, in the summary, these differences are again emphasized. On pp. 12-13, the texts emphasizes that “One of the most important differences between traditional ethnography and ‘netnography’ may be in issues of research ethics,” and then proceeds to elaborate a set of guidelines distinct from usual online practice (i.e., web-surfing and searching) and also from traditional ethnographic procedure.
5. Quotation Marks: As you requested, “netnography” now appear only within double quotation marks in the manuscript. Following this, “netnographer,” and “netnographic” have been placed in quotation marks as well. In addition, double quotes are also now used for the typology of newsgroup users (“minglers,” “tourists,” etc.).

6. Title Change. The title of the manuscript has been changed in line with your astute comment. It now more clearly portrays the nature of the contribution, which is to describe the use of netnography to conduct marketing research on online communities.

Responses to Changes Requested By Reviewer #1

All of Reviewer #1′s points have been addressed in my comments #4: Distinguishing Netnography from Traditional Ethnography, above.

Responses to Changes Requested by Reviewer #3

7. Virtual Community: On p. 1 of the revised manuscript, the colloquial use of the word virtual community is explained in order to acquaint readers properly with the topic of this paper (since the term is in common parlance). Reviewer #3′s correct assertion that the term may be misleading is then addressed by, citing appropriate (and earlier) literature. Thereinafter, in this revised version, the term “online community” is used to distinguish communities formed through computer-mediated communication, a topic central to the methodology.

8. Lead Users: On p. 17, the term “lead users” is defined and then the concept is compared and contrasted with online community participants. The essence of the assertion is in lead user’s creativity, inventiveness, and knowledge, which may (or may not) be evidenced by online community members. On p. 28 and 29, the text demonstrates how treating online community information as similar to lead user information can be useful to marketing researchers when accompanied by the proper critical evaluation and rigorous analytic procedure. Since the utility of lead user analysis is in invention and creativity, notions of representativeness (implied by Reviewer #3′s invocation of “marginal” and “inconsequential”) are not relevant to this classification. They are, however, useful points in and of themselves, and have been dealt with in my response #1: Hard-core users, above.

9. I respond to the comments on “opinion leaders” in response #3: Opinion Leaders, above.

10. “Maing” has been changed to “making”

11. “On these Internet-based” has been changed to “In these Internet-based”

12. Muniz and O’Guinn (forthcoming) has been changed to Muniz and O’Guinn (2001), although the issue has not yet been published and the page numbers are not available.

13. “Virtual” has been explained and the term “virtual community” changed to “online community” throughout, as explicated in detail in response #7: Virtual Community, above.

14. Opinion/Politics: On p. 13 I note the contentious, contestable, and still-under-debate nature of the online ethical marketing research discussions. As I wrote in considerable detail about this topic in my previous note responding to JMR reviewers, I will be brief here and state simply that I believe erring on the side of being overly cautious and considerate of online community member’s wishes is the most ethical course of action. Legality does not come into my methodological consideration set, because many things in the online environment are so new that they are only now being considered in a legislative and regulatory context (e.g., Napster, consumer privacy). Simply because Reviewer #3 says it is a public medium, this does not mean that this long-standing and important debate is settled. I politely but very adamantly differ in my opinion from Reviewer #3 on this matter. I believe that my “taking the high ground” treatment of research ethics in this paper may be an important contribution to the discourse on online marketing research.

15. Influence: The prior confusing reference to influence has been removed. The nature of the opinion leadership has been elaborated upon, as stated above. Influence was evidenced by quotations and references to the opinions and tastes of particular members of the online community, some of which have been represented in the illustrative example.

Final Comment

16. In closing these comments, it must be noted that each of these requested changes required the addition of textual material, often explicitly in several places (for example, on p.2 of your letter you direct that the revision be “improved…on page 3-15″ by adding emphasis of its distinctions, and that generalizability “should be discussed both in the presentation of the method and in the empirical section”). There were no directions cut back on existing material. The prior version was cut to the bone. In the last revision, considerable material was excised (such as distinguishing the method from traditional ethnography and face-to-face qualitative research methods), some of which needed to be added back intro the manuscript in a more focused manner. Cutting back on existing material would, in this author’s opinion, impair the ability of this manuscript to make a contribution by either cutting short the explication or impairing the persuasiveness and depth of the illustrative example. It should be noted that consumer ethnographies (for example, the one I will be publishing in JCR this summer) can easily run over 70 pages -and contain little or no methodological explication. The current length of this revised manuscript -which contains a new method, guidelines for researchers, and a concise-yet-detailed ethnographic example- is thus by no means excessive.

Adding the additional material to this manuscript has considerably raised its contribution level to JMR. It has improved its ability to address certain fundamental questions of importance to marketing researchers about the generalizability of the technique and its differentiation from traditional ethnography. With this added material, netnography can make a much clearer contribution to the marketing research literature.

Here is the Abstract:

ABSTRACT

This article develops “netnography” as an online marketing research technique for providing consumer insight. “Netnography” is market-oriented ethnography conducted on online communities dedicated to marketing-relevant topics. As a method, “netnography” is faster, simpler, and less expensive than ethnography, and more naturalistic and unobtrusive than focus groups or interviews. It provides information on the symbolism, meanings, and consumption patterns of online consumer groups. Guidelines are provided that acknowledge the online environment, respect the inherent flexibility and openness of ethnography, and provide rigor and ethics in the conduct of marketing research. As an illustrative example, a “netnography” of an online coffee newsgroup is provided and its marketing implications discussed.

And here is the Round Four Manuscript: field_behind_round4.pdf

Anatomy of an Academic Article, Round 4: Back from the Brink

Journal of Marketing Research

 

Wagner A. Kamakura, Editor
Henry B. Tippie College of Business The University ofIowa
338 John Pappajohn Business Bldg. S Iowa City, IA 52242-1000

March 2, 2001

Professor Robert Kozinets
Kellogg Graduate School of Management Northwestern University
Leverone Hall
Evanston IL 60208-2008

Dear Bob,

RE: MS# 1205-9-3 “The field behind the screen: Using the method of netnography to research market-oriented virtual communities.”

The above-referenced manuscript was read by the two remaining reviewers who evaluated its previous versions. As you recall, the third reviewer (#2) had signed-off earlier, by recommending unconditional acceptance. Reviewer #1 recommends that the manuscript be rejected, while Reviewer #3 recommends conditional acceptance.

As you can see, I have two conflicting expert opinions on a subject area I have little knowledge about. Reviewer #3 characterizes the present version as “not particularly brilliant”, but worth publishing. Reviewer #1 is as highly critical of this version as with the previous ones.

My reading of reviewer #1 ‘s comments is that you sell “netnography” as a methodological advance while this reviewer sees it as nothing more than an adaptation of ethnography to the new data-collection medium. Consequently, this reviewer is not convinced that “the paper adds sufficient value to the research stream.” My own view about your contribution is somewhat different from this reviewer. Since I am not trained in ethnography, nor knowledgeable about the literature in this area, I represent probably more than 95% of all JMR readers. Therefore, I am certainly not able to evaluate the contribution of this manuscript to the ethnography literature, but can clearly appreciate the value of your adaptation of it for marketing research in computer-mediated environments. The fact that you provide an empirical application in the actual market environment, and take care to discuss the implications of the method to the practice of marketing research makes it particularly valuable to an important segment of JMR readers, marketing consultants and marketing researchers, who represent more than half of our subscribers (and actually use what is published by JMR in their practice), without diminishing its value to academic researchers. However, I agree with this reviewer that you must be more upfront about what exactly your contribution is, in relation to standard ethnography.

Therefore, I am pleased to conditionally accept your manuscript, pending the following:

* As requested by Reviewer #3, explain in the text how you know the lead users, and that they are not marginal “hard-core” fanatics. In fact, the generalizability of the findings obtained with “netnography” is something that should be discussed both in the presentation of the method and in the empirical section of the paper. How do you avoid being misled by a minority of vocal extremists? How do you differentiate the typical consumer from “hard-core” pathological cases?

* Explain how you can identify “opinions leaders” in your analysis without a more formal network analysis.

* Make all the minor corrections requested by Reviewer #3.

* Improve the presentation of “netnography” on pages 3-15, emphasizing more strongly its distinctions from traditional ethnography. You should also define “netnography” more explicitly, as an adaptation of ethnography to computer-mediated communications. Throughout the text, you must refer to “netnography” within quotes, to indicate its status as an adaptation of ethonographic research to a new medium.

* You must also make sure to use double quotes when referring to your typology of newsgroups users (“tourists”, “posters”, “minglers”, etc.) to avoid confusing the reader.

* I also ask that you change the title of the manuscript. The current title gives the impression that “netnography” is a method for studying market-oriented virtual communities, when in fact, you are proposing the use of market-oriented virtual communities as a laboratory for qualitative marketing research.

I hope you will make the changes indicated above, which only requires a re-write of portions of your manuscript, clarifying some critical issues and making a more explicit statement about your contribution. I will not send the revision back to the reviewers, but ask that you submit a detailed account of how you have handled the comments from the two reviewers and myself.

Sincerely,

[Wagner]

Wagner A. Kamakura

MS# 1205-9-1

 

“The Field Behind the Screen ….. “

 

Reviewer # 1

The revision is more concise, and the authors are to be commended for sticking to the editor’s request for a 30-page paper. The focus is narrower, and much of the overclaiming has been deleted. The authors have deleted theoretical material that was problematical, and have reconstructed the manuscript as a methods paper, including an lllustrative method.

Still, the introduction of “netnography” seems to be but an add-on to ethnography — an “adaptation” (p. 5) — that does not break new ground methodologically. The summary of ethnographic procedures (pp. 3~5) and the application of these procedures to online research (pp.5-14) does not convince this reviewer that the paper adds sufficient value to the research stream. The summary points out that “no two ethnographies have ever been conducted in exactly the same manner” (p. 4), which suggests that online ethnography is merely a modification of a familiar method, not something that provides “an unprecedently new level of access to the heretofore unobservable behaviors of interacting consumers.”

Beginning on p. 5, the authors present specific guidelines for netnography based on the 6 ethnographic procedures listed on p.4. The “entree” section seems quite simplistic for the research audience at which JMR aims. It might well be a tutorial for market researchers (that is, practitioners), but does not take into account the expertise that JMR readers bring to their research.

The section on data collection and analysis is also wide of the mark for this audience, for it is again too obvious to need discussion. It is unlikely that a JMR reader does not understand where to find data or how to collect it. In fact, the elements cited on p. 7 are familiar ethnographic ones. Classification of message-posters may be “useful for marketing strategy formulation,” but this is relevant to practitioners rather than researchers. Further, the authors admit (p. 9), that the long suit of ethnography is not meticulous classification, which makes one wonder why the method is linked to but one system.

In the “trustworthy interpretation” section (pp.9-10), the authors state that “data, analysis, and finds are similar to conversation analysis” (uncited, which again leads one to ask, “what’s new?”

The “ethics” section reviews the controversy about public vs. private online discourse, again linking this to the “history of ethnography” (p. 12). The guidelines sound like standard fare ordinarily requested by research review boards and ordinarily complied with by ethnographers.

So far, by the time the authors reach “member checks,” also a familiar procedure in both ethnographic and phenomenal research, there is no indication that netnography is anything more than a modification of those methods in accordance with the change of medium from face-to-face spoken and recorded text to computer-mediated written text. This resembles the kind of information more often found in textbooks, whose audience is different from the research community.

To sum up at the midway point, the paper does not appear to have made a successful case for its value to JMR readers. This does not imply that it is valueless, for a more appropriate audience may be marketing or research practioners who need some “how-to” advice.

The introduction to the example (pp. 14-17) illustrates the netnography procedure in terms of the 6 ethnographic procedures treated in the first part of the paper. The organizational flow is clear and logical, but it no more breaks new ground than does the general material that precedes it.

The empirical portion (pp. 17-24) is shorter and crisper, and its generalizability seems equivalent to that of ethnographic observations.

It is in the implications section (pp. 24-26) reveal the mismatch between the paper’s content and the JMR audience. On p. 25, the authors point out that the implications “for wise coffee marketers are considerable.” The point is well-taken, but managerial implications are more appropriate for JM (but one example), which aims at an audience of practitioners as well as researchers. Likewise, practical implicactions [sic] for “professional marketing researchers” who work for “particular marketing research clients” (p. 25, bottom) and managers in charge of “positioning and branding strategies” (p. 26) seem better suited to a practitioner audience.

Thus, despite a heroic rewrite, the technique does not come off as innovative and newsworthy to the designated audience. Thus, the redone paper is a much better one, but might better be aimed at a less research-savvy audience.

The Field Behind the Screen: Using the Method of Netnography to Research …

 

MS# 1205-9-3 Reviewer #3

To Author:

The paper is much improved. I think it will provide a good impetus for conversation on both the substantive and methodological planes.

Remaining concerns:

1.) key undefined terms: virtual community. First, why is it virtual? Many sociologists as well as Muniz and O’Guinn have demonstrated that these are communities. For about 75 years now, most sociologists have not required physical proximity as a requisite of community. See Muniz and O’Guinn (current issue of JCR) for a discussion. “Lead users” … just exactly how do you know these are lead users and not just purely marginal and inconsequential fanatics? Is there a difference? What is it? How do you know?

2.) Related to # 1 : you claim (on page 15) that these groups contain “opinion leaders.”

How do you know that without the kind of network analysis that Wellman and other do? We simply do not know the social or market significance of these aggregations of consumers. I am not saying that they are unimportant. .. personally I think they are … but I think you go too far in your assertions sans analysis. Exacerbating this is the issue of social stratifications markers … , which have historically been important in network models of influence. Still, I am willing to trade that for the unobtrusive nature of the data … , which I still find as the feature that recommends it most.

Minor:

Page 1, line 1: “maing” I don’t know this word.

p. 1, line 5 :”In these Internet based” rather than “on”

P 1, Muniz and O’Guinn 2001 is out.

I like pages 2 through 3.

p. 3: why “virtual?” I know it’s a term of art, but sociologists really don’t approve of it. It’s misleading.

Page 12: Well it’s a matter of opinion/politics. but legally, and as far as I’m concerned ethically, if it’s posted on a public medium, it’s public.

Pps 15-16: you really don’t have any analysis to support claims or even mild assertions of influence here. You don’t have an “effects” kind of analysis. It would be nice if you did but you don’t.

Anatomy of an Academic Article, Round 3: Coming Out Swinging

risen_again.jpg

If you’ve been following these blog entries on the evolution of my article for the Journal of Marketing Research, you will likely have noticed that the last round of reviews was not very positive.

In fact, I knew after reading those reviews that this was do or die time.

First, I had a new editor. I didn’t know Wagner Kamakura. I had never met him (I don’t believe I have met him to this day). This editor had a new set of standards, and it looked like he had seriously raised the bar on me. Russ Winer had expressed, probably signaling to reviewers, his interest in the topic and in publishing something that covered this topic. I received no such signal from Prof. Kamakura. In fact, Prof. Kamakura’s cover letter let me know that the paper was deficient in a number of ways. He also signaled that he wanted me to take more of a content analytic, quantitative, numerical approach to accounting for my data. I didn’t like it, but there it was. I could either count the number of postings I was using, and the users, or pretty much put myself out of the game. In the process, though, I thought that maybe I could educate the Editor a little bit about the differences in approach between ethnography and other methods. For example, that comments about providing ‘objective, unbiased’ information. That indicated that ethnography might not be an area that the good Editor was very familiar with.

But the good thing, the surprising thing really, was that Prof., Kamakura could easily have rejected the paper, and he had not. Reviewer B had not moved in his/her opinion. In fact, that reviewer seemed to hate the paper more now that ever. She or he had given the Editor a good reason to bail on me if the Editor had wanted to. Rev B questioned everything about it-the ethical stance, the method, the focus, the topic, my skills. In their closing shot in the review, Rev B implied that the paper and method were useless. And I guess that meant that the last two years of my life had been wasted working on these topics.
Now, I have to tell you that, as a junior faculty member with nothing on my CV save a relatively obscure European journal article, this was more than disheartening. It was, at first, like someone had dropped a baby grand piano on my back. I felt flattened. I felt roughed up. Okay, you saw the pictures. That’s what I felt like. But then, as I read through the comments over and over, I realized something.

It was clear to me that Reviewer B didn’t understand what I was doing at all.

Reviewer C pretty much got it. She or he was still a tad snarky, but they could see what I was doing and they were offering comments that were raising the bar and generally helpful. They were seeing that there was improvement, progress here. They were working with me.

Of course, even Reviewer C gave me pause. What did they mean by the manuscript ‘improved again’?-this was the first revision of it they had seen.

At this point, I had a major decision to make. I needed to make a complete rewrite of this paper. This wasn’t a time for minor modifications-the paper would be shot down almost instantly if I was to try that.

But I didn’t know how to deal with Reviewer B’s negativity. It seemed like there was absolutely nothing I could do to please this reviewer.

So, after consulting with a few faculty members around Kellogg, I decided to ask the Editor to speak to me on the phone. I’m glad that I did. In that phone call, Prof. Kamakura and I discussed the paper and what I could do to improve it. We straightened out the requirements for a good revision of the paper.

And I came to that phone meeting armed to the teeth with references, citations, and all that stuff I had been ruminating about. Overwhelming force, or as much of it as I could muster. In the process, I sought to convince Prof. Kamakura that I knew a lot more about my topic matter than Reviewer B. Reviewer B was, I still believe, the wrong person to be reviewing this manuscript.  I politely mentioned this to Prof. Kamakura. I also politely suggested that he might replace this reviewer with someone more qualified to review my manuscript.

Although you are getting the scans of these reviews, I got paper copy of Reviewer B’s reviews, which were typed on an old typewriter and then photocopied on some ancient 1940s model typewriter. This was clearly someone who was stuck in another age and just didn’t get anything about technology. And they clearly didn’t want to get it.

So I argued about some of the main points that I later raised in my reviewer notes. Online ethics are obvious and a non-issue? I don’t think so! And here are three solid references and three excellent reasons why that is so….

I grew as an academic during this process. Although I was junior, I knew more about my topic than any of these reviewers. And I aimed to show it to them. I had to take a stand to do that. I had to take some risks.

For my example of netnography, I was also profoundly challenged. My last revision’s example was definitely lame, no doubt about it. But I was hamstrung by that 30-page limit. How could I express the richness of real ethnography in three journal pages? I felt like paraphrasing Scotty on Star Trek: “Jim, it’s against the quantum laws of ethnographic writing–it just can’t be done.” [Yeah, yeah, another Star Trek reference, I know...]

Add to that the fact that I had only written one ethnography before, in my thesis and in the paper coming out of it for JCR. I had started my Burning Man ethnography by that time as well, but in terms of being an expert in ethnographic writing, that I most definitely was not. I had been collecting data for a while for another project, which I thought would be a separate paper at some point, a study of coffee community and culture online. The only thing to do was to sacrifice it up for this JMR piece, to add the richness and detail I had to the table, see if I could condense it into something meaningful, and illustrate what this new approach could yield in terms of real marketing and business insight.

So, after all of that, here are my Reviewer Notes. Notice that they reflect my growth as a scholar. They are a bit more confident, a lot more assertive, and they reflect someone who realizes that, when you know more than your reviewers do, you have to educate them. And when they don’t want to be educated, you need to out their critiques as unreasonable, an alternative point of view that you don’t subscribe to or, sometimes, as just plain wrong.

Revision Notes RE: JMR MS# 1205-9-3: “The Field Behind the Screen: Using Netnography to Research Market-Oriented Virtual Communities”

First, I would like to thank Prof. Kamakura and the 2 Reviewers for their helpful comments. The enclosed manuscript has undergone a major revision (almost a complete rewrite) and a new positioning in an attempt to fully incorporate the feedback from the past round of review. The illustrative example is new (I had discussed this with Prof. Kamakura in a phone conversation and he supported my decision to change the example so that I could draw on a richer pool of data). The example is now much more fully realized, and takes up approximately half of the revised manuscript’s space (before references). To make room for it, the background information on cyberculture, as well as all tables and figures, have been jettisoned. These notes will overview the specific changes that have been made in response to the reviews. The notes will respond in considerable detail to each of the reviews in turn.

A. Editors Comments

Your comments were very clear, directive, constructive, and helpful. This revision reflects an attempt to implement all of your requirements. As you noted in your letter, this did “require a complete re-write of the manuscript.” It was very challenging to try to convey the richness of ethnography, along with the thoroughness of methodological description, and references in 30 pages total. The original coffee ethnography was over 25 pages, and could easily have been over 60.

In general, the paper was more clearly positioned as a methodological piece in this revision. It omits almost all of the background information and theorizing on cyberculture and virtual communities present in past versions of the manuscript. It also provides much more detail on how to apply the method to marketing research, its advantages over competing qualitative marketing research methods. Finally, the example is much more fully developed than in the prior manuscript. Specific comments follow:

1. The positioning of this manuscript has evolved through the review process. This version conceptualizes netnography as a technique useful to the study of market-oriented virtual communities. This is a much more specific, and for that reason, defensible, positioning for the method and the manuscript. In the manuscript, it is explained that many virtual communities exist that have market-oriented concerns. Many of these virtual communities are quite sizeable (many in the 100,000 range, quite a few in the 500,000-1 million range, or more), comparable in the number of readers they might have to many magazines, newspapers, or cable TV shows. In addition, they are more specifically consumer-to-consumer, and so they might carry considerable weight in consumer decision-making if, as the Almquist and Roberts (2000) article cited in the paper might suggest, consumer advocacy is an important facet influencing consumers’ choice of one brand over another. Netnography is positioned in a manner that is entirely consistent with the theory and practice of ethnography in anthropology, sociology, cultural studies and consumer research. That is, (as described further in the test on p. 4) it is positioned as a study of the local and the particularistic (cultures, communities, communications) that can be, but need not be, generalized to other groups. I have tried to convince readers of this manuscript that these groups are, in and of themselves, populous and important.

2. The general discussion of CMC and virtual communities has been almost completely removed. What remains is placed in the introduction section and is very brief. It has also been positioned so as to be of more interest and relevance to marketing research practitioners as well as academics.

3. The introduction clearly distinguishes netnography from ethnography. This is aided by the new positioning (i.e., it would be difficult, and probably mostly irrelevant, to study virtual communities in-person). Netnography is also detailed as being far less intrusive, less resource intensive and providing quicker research turnaround than ethnography. Netnography is also compared with focus groups and interviews, and detailed as more naturalistic, less susceptible to demand bias, less intrusive, and less resource intensive. Again, positioning the method as useful for the study of market-oriented virtual communities ameliorates many of these concerns about comparison. To use focus groups or personal interviews to study virtual communities seems less logical than to use netnography.

4. This draft provides a much more specific and detailed description of the steps and procedures involved in conducting netnography. It can now act, as you suggest, “as a blueprint to the reader interested in applying the method on [sic] her own research.” As explained in the text, the inherent flexibility of the method is consistent with the positioning of the method in anthropology and other fields. This manuscript’s description of netnography emphasizes methodological rigor. However, because of the importance of interpretive methods, and the researcher’s role “as instrument” (to use John Sherry’s turn of phrase), the method can not be said to lead to “unbiased, objective information” any more than can personal interviews, focus groups, or market-oriented in-person ethnography. This build’s upon Reviewer 3′s suggestion to more fully develop the “flexibility” of netnography, and is explicated further in the text.

5. The empirical illustration attempts to address your comments fully. While it is extremely condensed, and must leave out much analysis and description because of the limited space available, it tries to provide the reader with a lucid example of a netnographic application as well as a sense of its general flavor. On pp. 15-18, the various steps you mention in your letter are detailed (and in some cases expanded):

6. which newsgroups were chosen
7. what criteria for newsgroup selection was used
8. the number of messages downloaded for interpretation
9. the message threads downloaded for interpretation
10. what criteria for message thread selection was used
11. the #distinct informants or messages posters involved n those messages
12. some classification of the types of messages found (in addition, in several other places throughout the paper, e.g., p. 8, p. 16, different categorization schemes were provided that might be useful to other researchers)
13. amount of raw data utilized
14. methods of determining trustworthy messages/ exclusion of suspect messages
15. interpretive methods (this was a very brief overview, with references, many articles and books are available on the interpretive methodology that was used here)
16. some discussion of the more appropriate employment of the term and concept of research “trustworthiness” (rather than the inappropriate term and notion of “validity” applied to positivist or quantitative research)
17. methods of verifying information and conclusions with informant member checks
18. types of alterations to the text made in response to member checks

In addition, it is worth noting that I have been diligent in error-checking the ethnography with literary sources as well as with experts. In an extended communication with a coffee entrepreneur and author of 2 books about coffee production, this person confirmed the correctness of my netnographic analysis and the potential utility of some of the conclusions to those in the coffee business. This person (who requested anonymity) also provided numerous comments about the specifics of coffee production that helped to improve the manuscript.

B. Reviewer #1 Comments

Thank you for your helpful comments. I hope that this extensive revision meets some of the objections to the prior revision that you identify and describe.

19. The positioning has been changed, as detailed in 1, above. The attempt has been made to address your concerns about a lack of focus and overclaiming.

20. Theoretical descriptions of the differences between online and offline communication have been removed in this version in order to make room for the empirical example of netnography. That there are difference between online and offline communication seems an important concern, and one that has been treated in a range of articles and books. In this article, the focus is considerably more pragmatic than directly theorizing and specifying these differences (a matter I leave to other articles, and/or other scholars). Therefore, there are far fewer “constructs” in this paper, and hopefully less vagueness and less confusion. The article focuses on the method and the illustrative example of the method.

21. Building on the knowledge that (1) there are a multitude of virtual communities that deal with consumption and marketing related concerns (like coffee consumption), (2) these groups are growing in number and probably in influence, and (3) the numbers of people reading these groups is in the tens of thousands, oftentimes hundreds of thousands, and sometimes millions, it seems useful to marketing researchers to have a methodological tool (some guidelines) with which to study these social phenomena. While ethnography is venerable and effective, there are some specifics to virtual community that conventional ethnographic practice simply does not discuss. These are all in the realm of the specific configuration of virtual communities, which I describe in the revised manuscript. For example, how does one choose a site or newsgroup? Which site or newsgroup to choose? Where to find lists of these groups? What types of messages to read, save and download? What types of informants are present on these groups? What types of classifications may be useful for this type of data? What types of interpretations may be useful for this type of data? How do we handle the particular ethical concerns raised by the online medium? And so on….

22. Your insight that “online research is useful for studying net cultures,” was helpful to the new positioning of the paper. As you suggest, the prior version’s more general claims to represent other types of groups was a less defensible position than that of the current revised version.

23. Your critical comments on the “Trust and Rapport” section were useful, and this section has been eliminated because, as you point out, it only applied to a particular kind of netnographic research (participatory). In addition, the notion of informant identity is rendered far less problematic if we conceptualize it as an issue of trustworthiness that can be treated by altering our unit of analysis from the individual to the interaction (as with G.H. Mead’s social psychological methods). This also answers your question about “what is imaginary and what is real”? Every interaction in a market-oriented virtual community is real. These insights are in the revised text on p. 10.

24. Your critical comments on the “Interview” and “Digital Projectives” sections were useful, and these sections have been eliminated due to space constraints.

25. You contend that “the whole matter of public versus private is a non-issue” and that “electronic eavesdropping” is not likely to give rise to psychological harm. Both points are contestable. The latter point appears tangential to the topic at hand. Netnography is a method for researching virtual communities and as such it will often have impacts far beyond the mere act of observation. What is more salient than data collection (“eavesdroppping”) is the potential impact resulting from the publication of information that was disclosed in an online forum. If I overhear someone discussing his or her private matters online and then publish it, is there the potential for harm?

With all due respect, your position appears overly theoretical. My own thinking on the matter has been informed by over 13 years of experience as a virtual community participant and over 5 years as a virtual community researcher. In my research, I regularly come across people who do not want their information shared with others or published (even after promising them anonymity). I have published this finding elsewhere, but in deference to your comment I now include it in revised manuscript (on p. 12).

Here is an online example of the type of trouble that researchers can cause. The following quote comes from someone who posted to a “public” forum on suicide and felt compelled to write a web-site conveying her feelings about being researched:

I am NOT a guinea pig. I am not sitting here, writing everything so that I can become part of someone’s report. . . I am, however, pissed off.-Dana-Christene Umanetz, posted on http://ash.xanthia.com/researchers.html

While this is a strong case, these types of comments are common among newsgroup participants. In my coffee ethnography for this article, people told me they were uncomfortable with the “lab rat” aspect of being observed. This sort of comment is very familiar to ethnographers of all stripes, but in person one has the distinct advantages of usually being able to see one’s observer. Not so in cyberspace. For the alt.coffee ethnography, I had people who required considerable explanation (and insisted on seeing copies of the manuscript, in order to judge how their quotes were being represented) before they would give me permission to use their postings.

If this manuscript is published in JMR, it might be used by graduate students and professional researchers, as well as researchers in disciplines outside of marketing. The potential that they might study something sensitive, like drug use, pornography, dealing with handicaps and illness, addictions, stigmatic groups or beliefs, or other social concerns (that can and may well have marketing implications) is very real. In addition, as an anthropologist, I have a strong concern for the cultures and culture members that I study. I believe that they and their beliefs must be treated with dignity and respect. I feel strongly that a section on ethics that recommends the ethical “high ground” is imperative for this article. I would not want it published without it.

The differences between netnography and ethnography arise exactly because of misconceptions such as the one that you proffer. Because cyberspace is not clearly public or private, netnography must have methodological provisions that specify procedural solutions to ethical concerns that face-to-face ethnography does not face. This is the “new news” you solicit. That it is distinct from in-person ethnography is specified clearly in this version on p. 13. Because these matters are contestable, frequently being contested by academics, and of major practical concern, this element could be one of this manuscript’s most important contributions.

26. As noted in comments 1 and 2, above, notions of generalizability have been replaced by a more defensible positioning of netnography as a method for studying specific and particular virtual communities.

27. The manuscript has removed most of the “participative” elements of netnographic methodology (such as interviews), mainly to conserve space, and to help prevent the sort of confusion that this Reviewer identifies.

C. Reviewer #3 Comments

Your comments were very honest, insightful, directive, constructive, and helpful. I truly appreciate the positive tone and the constructiveness of your review and have tried to implement all of your suggestions.

26. As outlined in comments 1 and especially 2, above, the paper’s positioning as a methods piece has been substantially altered in this major re-write. This version realizes your suggestion by portraying netnography as much more attuned to and related to the openendedness and flexibility of ethnographic methodology. The sections on p. 3-4 in the introduction to the method try to convey this. Even more importantly, the illustrative example exemplifies this openendedness by providing rich verbatims and (abbreviated, but hopefully still containing some interpretive spirit) analysis of the rich data provided in the <alt.coffee> newsgroup. This reflects my attempt to, as you suggest, “push” the method and “try to explain to a relatively naïve audience the nuanced nature of the interpretive act.” This effort is present throughout the major revision, particularly in the “Ethnography and Netnography” section, and towards the end of the “Applying Netnographic Methodology” section. In addition, building on your suggestion, you will find some comments comparing netnography with conversation analysis on p. 10 of this manuscript.

27. As noted in 4 and 5, above, the illustrative example has been subject to radical revision. I have taken your suggestion to ‘really build it up.’ It is now the centerpiece of this article. It is also now much more “ethnographic” as you suggest it should be. I do not believe that this ethnography could have come as a result of any other method. I do not think the religious evocations of voices calling unto coffee preparers, and “god shots” would have been likely outcomes of focus groups and conjoint. Show you the money? That’s what I was aiming for.

28. The conclusion section has been reined in to curtail overclaiming, while still trying to convey that this might be a useful method, one that marketing researchers may wish to utilize.

29. USA Today’s source is contained in footnote 1. As well, the Arbitron data (Reid 1995) may be of interest. Some search engines (like google.com) have archived copies of these pages, whose links may be expired.

30. Laurel’s quote is an exaggeration, but I think there’s still some truth to it, in a contingent sense. The cybergeeks still remain constituents of almost every newsgroup (true, certainly of the alt.coffee group), and their early conventions still drive CMC online. Yes, things have progressed and the innovators no longer have the playing field to themselves, but the people who are now there are still very much playing by the rules that the geekforce originally set. I think this is especially obvious in the use of conventions like emoticons and abbreviations. You’re quite right that the quote as it stands is contestable, and I have removed it as I jettisoned the cyber-theory sections of the paper. But this is a very interesting point. Thanks for the insight.

31. Yes, culture’s problematized nature is illustrated by my diagram. When trying to draw diagrams from my text, I often end up drawing concentric circles. Perhaps I should just stick to text. That’s what I’ve done in this revised version.

32. “Social inhabiting of imaginative animal…or alien bodies”…quite right. My enthusiasm was showing there. That section, along with much else, was slashed.

33. Your insight that F2F triangulation need not occur at all is, I believe, right on target. However, I struggled with your position vis-à-vis Reviewer 1′s position and (I believe) Prof. Kamakura’s. My solution was conveyed on p. 10 and 11 and represents a bit of a compromise. Because netnography is interested in the particular and the specific (now that it has been repositioned as dealing only with the study of virtual communities), and because we can think of the unit of analysis as interactions, chunks of text that represent implied/intended/actual social dynamics (or Wittgensteinian “language games”), we need not triangulate outside the method itself. However, it might be important to recognize that marketing researchers are probably going to want to generalize outside of the virtual community (whether we advise them to or not). They are going to try to apply findings to wider audiences assuming (perhaps rightly) that the vast bulk of “lurkers” is more like a general population whose characteristics might be useful to have but are very difficult to determine. Therefore, I have included a section that states that if netnography is going to be more generally repurposed then it would benefit from triangulation. In addition, triangulation might be useful when the stakes of the research are high, and someone’s neck is on the block. I completely agree triangulation isn’t strictly necessary, but I want to make sure JMR readers are aware it might be prudent for some occasions, and necessary for some kinds of claims.

34. As you suggest, the various methodological discussions (e.g., ethics, member checks) are now much more focused on netnography.

Thank you again, Prof. Kamakura and Reviewers, for your ideas and assistance.

Here is the abstract of the revised paper, the third revision:

 

The Field Behind the Screen: Using the Method of Netnography
To Research Market-Oriented Virtual Communities

 

ABSTRACT

This article develops netnography as an online marketing research technique for providing consumer insight. Netnography is market-oriented ethnography conducted on virtual communities dedicated to marketing-relevant topics. As a method, netnography is faster, simpler, and less expensive than ethnography, and more naturalistic and unobtrusive than focus groups or interviews. It provides information on the symbolism, meanings, and consumption patterns of online consumer groups. Guidelines are provided that acknowledge the online environment, respect the inherent flexibility and openness of ethnography, and provide rigor and ethics in the conduct of marketing research. As an illustrative example, a netnography of an online coffee newsgroup is provided and its marketing implications discussed.

Here is the entire third round revision: field_behind_round3.pdf

Anatomy of An Academic Article, Round 2: Pierced to the Hilt

jmr-cross-swords.jpgI had worked and poured my heart into my revision of the JMR manuscript and had sent it back to Russ Winer by the deadline, my mind filled with hope and optimism about the possibility of publishing in JMR.

It was the end of summer, and the weather in Chicago was still quite warm. But the response I got from the Editor and Reviewers at JMR stopped me cold and chilled me through. The manuscript was still alive, but it was being eaten alive. Enduring a slow death, its fate was hanging by an extremely thin thread. It was bad news, much worse than before.

Have a look.

Journal of Marketing Research

September 13, 2000

Professor Robert V. Kozinets
Kellogg graduate School of Management Northwestern University
Leverone Hall
Evanston IL 60208-2008

Dear Robert,

RE: MS 1205-9-2 “The field behind the screen: Using Netnography to research market-oriented virtual communities.”

The above-referenced manuscript was read by the two reviewers who had recommended rejection of the original version. Their current recommendations regarding the disposition of this manuscript are as follows: Reviewer #1: reject; Reviewer #3: invite revision.

I see this manuscript as a methodological piece. It introduces marketing researchers to a new approach for collecting data from Web communities and for analyzing these data. However, the current version of the manuscript is way too long on background and too short on details about how the method can be actually applied, about its advantages over traditional research methods and about the empirical illustration.

Typical JMR readers would want to know when “Netnography” would be appropriate and useful, and how exactly they would go about applying it. I believe your manuscript could be of value to JMR readers, but it must be positioned and written properly for this audience. First, I suggest that you replace the discussion about computer-mediated communications and virtual communities of consumption with the proper citation of seminal articles and let readers seek them if they need an elaborate discussion on these topics. Your focus on the introduction should be on distinguishing “netnography” from ethnography, and clearly demonstrating the potential value of the former in marketing research, relative to competing methods (focus groups, and other face-to-face methods).

Second, you should provide a more specific and detailed description of the steps and procedures involved in conducting “netnography.” This should be a blueprint to the reader interested in applying the method on her own research, explaining why the steps are necessary and how they lead to unbiased, objective information to the marketer.

Third, your empirical illustration should provide the reader with a clear idea of how the proposed approach is applied and of its methodological rigor. The two experts find that the description of your application does not make a strong case for ethnography, and in fact might give readers the impression that it lacks rigor. The current application gives the erroneous impression that netnography consists of a simple content analysis of news group discussions. I would like to know each one of the steps that you have followed in this application, from selecting the sample, collecting the data, interpreting the results, cross-validating the interpretation, and the managers’ use of these results. For example, I would like to know:

• which particular news groups were chosen

• what criteria were used for this selection

• how many messages were interpreted

• how many users were involved in these messages

• some classification of those messages into general categories (my own experience with newsgroups is that there is a lot of idle, useless talk in them.)

• how were the messages validated (product-related newsgroups are known to be visited by manufacturers, often anonymously).

• the method used in interpreting these messages

• how the interpretations were cross-validated (multiple interpreters? Crossvalidation?)

• does this provide better and richer information than a series of focus groups, or indepth interviews? Why?

I hope you will be able to revise your manuscript along the lines discussed above, and responding to the two reviewers. This revision will require a complete re-write of the manuscript. I also want you to reduce the total length of the manuscript to no more than 30 pages (for example, you can drop the two figures and glossary, and reduce the citations to those strictly necessary). The success of this revision will depend heavily on the detailed description of the methodology and on the quality and rigor of the empirical illustration.

Sincerely,

[Wagner]

Wagner A. Kamakura

A quarterly publication of the American Marketing Association / 311 S. Wacker Drive, Suite 5800/ Chicago, Illinois 60606

“JMR MS# 1205-9-X1

 

“The Field Behind the Screen ….. ” Reviewer # 1

The revision is a workmanlike job that shortens the paper as the editor and reviewers suggest. Nonetheless, cosmetic changes (e.g., deleting “scholars in diverse disciplines” in the first sentence and replacing it with” marketing researchers” or replacing “interest” with “important”) do not represent a “more focused positioning,” as the authors claim in Note 3. The issue of overclaiming identified by Reviewer 3 comes out even more clearly in the revised version.

The article claims to build on “humanistic” research (par 2, p 1) to theorize “fundamental differences” between ways that researchers can access online vs. offline behaviors. This seems a rather pedestrian undertaking, and it is up to the authors to convince the readers not only that there ARE differences, but that they are sufficiently noteworthy to provide added value to researchers. To this end, the authors have included an empirical application of the method. The reviewer’s task remains that of assessing the revised claims of “new and different” as well as that of evaluating the empirical evidence.

A major problem that surfaces at the outset is weakness of construct definition. To begin, the authors note (p.3) the “frequent incoherence of the term community.” Next, the issue of whether or not CMC is impoverished (p. 4) or “richly detailed” is confusing. Footnote 5 is not helpful, and the authors’ stance is itself equivocal. Finally, the laundry list of special characteristics of CMC is “consolidated” (p. 5) in a way that makes it impossible for readers to understand. How did the 12 characteristics morph into the 3 that the authors’ identify? The presence of so many vague and patchy constructs suggests a glossing-over of fundamentals essential to theorizing.

Lack of clarity about CMC vs. f2f in terms of similarities/differences makes the section on CMC mediation of social representation (p. 7) confusing: which is it that the authors claim -differences or similarities? The discussion is still off the mark, because cmc has changed the nature of “writing” far beyond what Derrida et al. considered. The relationship between the written/spoken word is so complex that merely mentioning putative differences and/or similarities is obfuscating.

That online research is useful for studying net cultures (p. 10, “entree”) is rather obvious. Likewise, so is its application for the study of virtual “communities,” however defined, with an eye towards “providing a contextualized understanding” of behavior. This squarely places it in the realm of interpretive research, which, admittedly, has been sparsely represented in JMR. However, the global claim of applicability to marketing research is not borne out. From the lists of group types on (pp. 12 ff.), the material covered is simplistic and obvious to those in market research. For example (p. 15), the presence of enormous amounts of data requires data management; anyone who has dealt with transcripts of taped interviews is well aware of this.

“Trust and Rapport” assumes participatory rather than observational research, and informant identity (pp 15-16) does likewise. This is essentially ethnographic research, and not newsworthy. The alternative of identity-construction, discussed in terms of Haraway’s quote (p. 16), is most problematical. How is the researcher to evaluate what is imaginary and what is real? And what is a “lived material-semiotic world?”

The section on “interviews” are admitted to be “like their face-to-face counterparts” (p. 17), and the promise that digital projective tasks offer more “profound” levels of cultural knowledge gives rise to two questions: first, “more profound” than what? And second, what supports the claim of “more X” than anything?

The whole matter of public versus private is a non-issue: electronic eavesdropping on public communication is no more likely to give rise to “psychological harm” than is eavesdropping on a conversation held in a restaurant. King (bottom, p. 19) points this out. Where, exactly, can one not distinguish private from public (p. 20)? Indeed, the conclusion (bottom p. 20) is a nonstarter, for the first situation is routine ethnographic procedure, and the second (observation) is non-invasive. What is new news here?

The empirical portion (pp. 21 ff.) is what this reviewer was awaiting. However, the first paragraph exemplifies destroys any pretense of generalizability, and cannot be explained away by mentioning it. As Wells reminds us, merely stating limitations does not make them go away. Further, mining the news group appears to be observational — thereby rendering most of the preceding sections (those dealing with participatory researchers) irrelevant. The dilemma here is that the technique is shown to be neither innovative nor of real value to researchers. If the “downloads and observations” are anonymously generated, how is the market researcher supposed to discriminate between the real and imaginary selves of the respondent? The research outcome is disappointing from the perspective of value added: f2f interviews yield the same information (useful in market segmentation), without the identity mystery. In market positioning, it is essential to know who the market segments are!

Thus, the implications (p. 25) are neither “major,” as claimed, nor– potentially — even useful.

Notes to Authors of “netnograhpy. [sic]” –1205-9-2 Rev. #3

General: First, I want to say that this manuscript has again improved. It has also moved in the direction requested by the reviewers and the editor. It is also clearly an important topic. We’ve come a long way from the first submission.

I really want to acknowledge that.

There are still, however, problems. They are:

Problems:

One: The paper, positioned as a methods piece, is still under-specified, or (alternatively) simply reflects the open and inherently under-specified nature of ethnography. I tend to believe the later. Ethnography, at least to me, represents the most open form of inquiry. This does not mean it is sloppy or purely a make things up as you go meta-method either. This point seems lost in our current discourse, and in this paper. The author gets caught in this dammed-if-you do, damned if you don’t dilemma. But, I don’t want to let him or her entirely off the hook. Right now the paper reads like a very-under-specified scientific positioning of ethnography. This isn’t working for me very well. I wish the author would either push it fully in that direction, or more explicitly speak of its flexibility as a virtue, and try to explain to a relatively naive audience the nuanced nature of the interpretive act. This is not easy, I know that.

Two: The example is a fairly weak. There’s really nothing ethnography (specifically) did here in order to recommend it above any other method. This is the weakest part of the manuscript. Either it should be cut out entirely, or really built up. JMR readers are going to rightfully say: I don’t see anything special about this: standard focus group/concept testing accompanied by conjoint would have done a much better job … and they are right. Either show us the money, or drop it. I prefer the former. Show us what this method can do above and beyond any other.

Three: conclusion is a bit of a job of over-claiming ..

Page by page:

p., par I: what was USA Today’s source?

P 1: I still think this is essentially the same (or very similar) to what the interpersonal communications people are doing with on-line communication: it’s conversation analysis. And even prior to the net, when conversations were tape recorded, transcribed and analyzed … isn’t this similar?

P 2: I don’t agree with Laurel (1990), and I assume you as well: “all virtual communities exist as ‘villages of activity within the larger cultures of computing.’” I suppose this is quite debatable, but there is good theoretical reason to believe that as we go further down the diffusion curve, the medium and it’s culture becomes more transparent. People who listen to the radio in their cars, are not like early Marconi (sp?) (users). My guess is that by this point many users are not really cyberculture members at all (Whatever that is), but simply users of a rapidly diffusing technology. But, this is just a matter of opinion. You are entitled to yours.

P2: I think what you discuss here and illustrate in Figure 1 beautifully illustrates the problematized nature of the term “culture.” As you no doubt know, the notion of a set thing called culture is not particularly viable these days. I see these concentric circles as much too tightly drawn and formal. Cultures are built, tom down, and re-invented constantly. But, if this is how you want to do it.

P4: extra word, first sent.

p. 5: I’m not convinced that the limited cue theories have been discredited. Maybe, but I’d like to see more evidence.

P.6; grammar: addition

p. 8: “for reversion of gender, or ungendered being, and even the social inhabiting of imaginative animal, mechanical or alien bodies.” … no doubt this is true, but I think we may be getting a bit far afield from the modal application here.

P 11: I’m not sure FtF triangulation has to occur at all. They are all different methods. For example, surveys don’t NEED experiments.

P12: again, I think unobtrusive data may stand alone.

P 16: extra “identity”

P 17- I we are now trading in very general terms about surveys and interviews, I thought this was about this unique form of ethnography. This doesn’t belong here.

P 19-20 this discussion of ethics should be more focused on your particular method. Very general discussion.

P 21: more on the idea of member checks, and its particular value here.

P22: total postings ==7

Pps 21-25: Illustrative Example: this really didn’t work for me, at all. Two problems: There is no methodological detail here, or really much of anything about METHOD, which is really the promise of the paper, and (2) I really don’t see what this yields beyond the standard focus group/concept tests by a commercial supplier, and maybe a conjoint. Given your focus on segments, why wouldn’t a conjoint do as well, and provided comparable attribute metrics, you could call them “netrics. “? Don’t get me wrong, I’m a fan of ethnography, but its value is not shining through here. Do it justice, or drop it.

P 26: missing word.

Anatomy of an Academic Article: Back on the Horse

jmr_mount_12.jpgSo what did I make of these reviews?

Well, it really was a mixed bag, and as a junior scholar I wasn’t particularly well-equipped to interpret the results. I had the advantage of having a strong mentor in John Sherry, who gave me a lot of guidance. But there’s only so much you can do when the data point to paradox.

I think it’s pretty unusual that one reviewer on the first round says unconditionally accept. That certainly hasn’t happened to me very much (maybe one or two other times). Reviewer 2, whoever you are, I thank you. I never got to see the review. I don’t even know if they wrote one. But it certainly didn’t hurt innovation to only have to deal with two reviewers for the rest of the way through rather than three (or even, worse, four or five as I have sometimes seen & experienced).

Then, the other two reviews are both rejections. Now, that makes it interesting for the Editor. The Editor, at that point, has grounds to reject the paper. Forget the “Despite Merit” part, that’s just being polite. Those are two rejections, plain and simple. Those reviewers want this thing down and out–they don’t want to see it again.

Rejecting the paper would have been, I think, the most conventional, sensible, defensible, logical, easy decision here. Had the Editor been interested in keeping out qualitative research, or new methods, he easily could have simply said something like ‘We are open to new stuff, but the Reviewers have spoken, 2 out of 3 said reject, sorry, try another journal.’

But there seem to be at least two reasons why Russ Winer didn’t do that.

First, and I’m reading behind and between the lines to do this, I think this is a somwhat unusual set of reviews. He said that he sent the paper to one on my ‘don’t send’ list, one on my ‘do send’ list, and one other person he trusted. I think that the person from my send list gave the unconditional accept. However, I think the one on my ‘don’t send’ list was Reviewer #3, and I think Reviewer #3 was significantly more positive towards the paper than Reviewer #1 (but still rejected it). So Reviewer #1, who the Editor chose, would have been the most negative one towards the paper.

Secondly, Prof. Winer makes a very telling admission in his letter: “Let me state at the outset that I am interested in publishing a paper on this topic.” I read that statement then, and still do now, as very encouraging to me, and as a clear signal to reviewers. In fact, that whole paragraph is as much for the reviewers as it is for me. He continues: “JMR editors should publish and always have published papers demonstrating the value of new methodologies. Of course, what is particularly interesting and topical about netnography is that its domain is the Internet.”

Again, this says to me that this Editor is taking his job as gatekeeper in a top-tier publication very seriously, and weighing it in relation to the advancement of knowledge. He realizes that novel methods are going to be resisted by the old guard. And he sees his job as encouraging, fostering innovation, making sure that those flickering new flames aren’t prematurely snuffed out. And the new game in town for Marketing Research is the Internet, and new Internet-based methods should be of interest to JMR. This is the right place for this research–that’s my interpretation. And when he says he is interested in publishing a paper on this topic, I see that as very positive. Because I am very interested in writing a paper on this topic and, as far as I know, there was no one else doing so at that time.

So he asks me to take into consideration some of Reviewer #1 and #3’s comments, and then asks for a paper that is thoroughly edited, contained a good example of an application, and is shortened.

That sounded quite do-able to me. The example would take a bit of time, though.

Maybe the scariest part of this was that I was teaching a full load of demanding Kellogg courses until well into May. That included my Hollywood course on Entertainment Marketing, which finished with a weeklong trip to Hollywood with my students. Pretty intense.

It was very unlikely that I would finish the revisions by June 30th. So I may be looking at sending my revision to a new editor, an unknown editor, an editor who might not be as fair-minded or as open to new and Internet-based methods as the former one.

Timing is everything. As we will see.

* * *

I did make my revision, and my revision notes. Revision notes, if you aren’t familiar with them, are the notes that you are required to provide with your revision to point out exactly how you addressed the comments of the Reviewers and the Editor.

The example I chose to add into the shortened paper at this point, as you will see, is a kitchen appliance search that I performed. This was part of some innovation work I was doing with a company, and there was only so much of it that I could share, so the example is pretty sparse and spare.

Here are the revision notes, in their entirety. Reading through them now, I would say that they are not the best set of notes I’ve devised, but they certainly do indicate that I tried to make the major revisions asked of me, and many of the minor ones, too.

To be honest, I didn’t really know what to do with a lot of Reviewer #1’s comments. They seemed like nit-picks from someone who didn’t understand the method, the Internet, or what I was trying to do. Reviewer #1 seemed to be trying to put me into a familiar box. Deconstruction?

Reviewer #3 wrote the review in sort of an oddly confrontational style, but had some good points, and I thought about them a lot and tried to address them. That thinking was productive. But was the experience of reading that abrasive tone enjoyable in the least? NO WAY.

Here are my revision notes:

Revision Notes RE: JMR MS# 1205-9-1: “The Field Behind the Screen: Using Netnography to Research Market-Oriented Virtual Communities”

First, I wish to thank the Editor and reviewers for their insightful, helpful and encouraging comments. I have sought in these revisions to meet all of the reviewers and Editor’s comments. These notes will overview in detail the changes that have been made to address the comments provided.

1. As suggested by the editor, the paper has been shortened significantly. The section entitled “Ethnography and Netnography” has been overhauled and now focuses more precisely on the method of netnography. The paper has been carefully edited to remove redundancies. In addition, sub-sections and topics whose orientation appeared either redundant (e.g., cultural exit) or of mainly theoretical interest (e.g., representation) were either removed or trimmed back substantially. The result is a 30-page paper, excluding exhibits and references.

2. Within these 30 pages is included a 5-page illustrative example that demonstrates how netnography can be (actually, has been) applied in practice. Adding this “capstone” empirical application has been one important way in which this revised version of the earlier manuscript brings the method of netnography into a form that may be much more useful to marketing researchers than the previous version. In addition, the entire manuscript has been substantially revised to more overtly and clearly reflect the interests and objectives of marketing researchers (Editor’s comments).

3. To clarify the positioning of the paper, the opening paragraph now clearly describes and details the meaning of market-oriented interests, specifying them as social information that is not marketer-driven (Reviewer#1 comments). This more focused positioning is also reflected at other places within the revised manuscript.

4. This revision of the paper repositions netnography as an extension of current methods, in line with the comments of Reviewer #3. The current version presents netnography as more of an integrative method that extends and synthesizes current ethnographic methodology into a form that may be more suitable to some of the unique contingencies of the Internet environments (for example, there are new ethical issues, completely unobtrusive “lurking” is possible, informant accessibility is of an entirely new order). The revised version attempts to demonstrate how certain current ethnographic approaches must be modified for the online environment. These techniques are not in themselves a new methodology, but they do (arguably) represent a potentially useful extension of current techniques to a new and sometimes very different social environment. The claims for discontinuity no longer appear in this version. I agree with the reviewer that these claims were not convincing, and are not necessary (Reviewer #3 comments).

5. Footnote 4 has been added in order to raise the point that the discussion of the signs used in CMC in order to replicate face-to-face communication are similar to the differences between print media and f2f speech (Reviewer#1 comments).

6. The object of study of netnography has been overtly described in this revised version as dealing with the content of CMC as it may affect consumer choice (for example, on p. 4) (Reviewer#1 comments).

7. The use of “jargon” has been greatly reduced on p. 7, and throughout the paper. Potentially obfuscating phrases such as “obtrusive and opaque cultural experience” and “social specificity” have been cleaned up or removed outright (Reviewer#1 comments).

8. The differences between CMC and f2f time lags have been developed further, as presented on page 7 and Table 2 (Reviewer#1 comments).

9. The writing/speech relationship has been developed further, through citing relevant work in deconstructive criticism, on page 8 (Reviewer#1 comments).

10. The mutability of identity concept in CMC has been developed further as a more extreme manifestation of what often occurs during f2f communication, on p. 8 (Reviewer#1 comments).

11. The focus of the market-oriented behaviors considered in the paper has been clarified as those which are consumer- (rather than marketer-) driven (Reviewer#1 comments).

12. The focus of market-oriented netnography has been clarified as the topic of the discourse itself, as demonstrated in page 1 and 2’s revised definition and description of the method, and elsewhere in the revised manuscript. These revisions attempt to make clear that one of the important limitations of the method is that (as with any ethnography) it is not guaranteed to produce generalizable knowledge (Reviewer#1 comments).

13. Spurred by these insightful comments, the important issues regarding the accessibility of data have been reformulated. This has resulted in a rethinking and repositioning of the observation and participation comments throughout the manuscript on the basis of these helpful comments (Reviewer#1 comments).

14. The discussion of anonymous users has been revised, and has introduced an ontological discussion (Reviewer#1 comments).

15. More attention has been paid to the discussion of response bias, random selection and comparison of netnography and online surveys, as this version reflects my attempts to make these points clearer (see page 19) (Reviewer#1 comments).

16. As with the comment addressed in point 13, above, the Ethics section has been substantially revised to reflect the interesting comments regarding the role of strictly observational ethnography, and to directly address and further develop the ideas contained in Reviewer #1’s comments on the matter.

In summary, addressing these comments has resulted in a substantially revised paper that is more firmly focused on applications relevant to marketing researchers and practitioners, contains an empirical application, does not overstate the newness of the method, is considerably clearer, develops ideas more thoroughly and is 30 pages long excluding exhibits and references.

Thank you again, Reviewers and Editors, for your ideas and assistance.

* * *

This is the abstract for the Round 1 revision, entitled : “The Field Behind the Screen: Using the Method of Netnography
To Research Market-Oriented Virtual Communities”

ABSTRACT

Computer-mediated communication (CMC) has fostered cybercultures and virtual communities, many of which are market-oriented in their focus. This article develops netnography as an online ethnographic technique for marketing research. Netnographic techniques are adapted to the characteristics of CMC. Netnography presents flexible guidelines for conducting online fieldwork that adaptively address central ethnographic concerns, while providing practical marketing research. An illustrative example is provided to demonstrate the utility of using netnography for new product development in the small kitchen appliance market. The example demonstrates how netnographic observations inform product attribute and positioning decisions. Implications of the method are discussed in a concluding section.

Here is the complete Round 1 Revision: field_behind_round1_rev.pdf

And, in the next post in this series, you’ll see just how this revision was received by JMR.

Anatomy of an Academic Article: First Blood

wound_surgery.jpgThis is, by the way, an actual unretouched photograph of what I looked like when I received my reviews back.

In the last post I gave you access to my initial submission to JMR. I sent it off on November 23, 1999, with a cover letter explaining the background for what I was doing, requesting a few appropriate reviewers, and asking the editor not to send it to a few reviewers I feared might savage the piece (doing the latter was a suggestion of one of my mentors).

Everything in those days was done on paper. I think there were 5 paper copies I had to send out. I initially was over their 50 page limit total, found out too late, and had to toss my 5 paper copies of 53 pages each into the garbage while I whittled the document down to 50 pages total.

Then, I waited for a paper package with the reviews in it to come in the mail. Pretty much everything is electronic and happen via email and pdf files now.

So, in February, I heard back. Everything from the reviews I am reprinting from the paper copies I kept in my files. Everything was scanned in with OCR software. I’ve checked it a few times, but some errors may remain that are my own. Some of the reviews had errors in them, and those I’ve marked with the convention [sic] so that you know the errors were in the original copies.

Here in the blog is the scan of the Editor’s cover letter, and the reviews. I have asked Russ Winer’s permission to use this letter in its complete, intact form, and he has kindly consented.

In the next post, I will elaborate and interpret the reviews and the letter a little bit, and then I will present my response, which was contained in my reviewer notes.

Journal of Marketing Research
Russell S. Winer, Editor Haas School of Business
University of California at Berkeley Berkeley, CA 94720-1900
February 15, 2000

Professor Robert V. Kozinets
Kellogg Graduate School of Management Northwestern University
Evanston, IL 60208-2008

Dear Rob:

Re: MS#1205-9-1: “The Field Behind the Screen: Using Netnography to Research Market-Oriented Virtual Communities”

The above-referenced paper was read by three knowledgeable reviewers. Their recommendations concerning the disposition of the paper are the following: Reviewer #1: Reject, despite merit; Reviewer #2: Unconditionally accept; Reviewer #3: Reject, despite merit.

This is obviously an unusual manuscript for JMR as neither do I see too many ethnographic studies nor many papers at all without empirical work. In order to get varied opinions on the paper, I sent it to one reviewer from your suggested list, one from your “no” list whose opinions I trust, and a third not on either list but who is very familiar with ethnographic studies.

Let me state at the outset that I am interested in publishing a paper on this topic. JMR editors should publish and always have published papers demonstrating the value of new methodologies. Of course, what is particularly interesting and topical about netnography is that its domain is the Internet. The three reviewers also feel that the paper has merit, although Reviewers # 1 and #3 think that the promise of the work is more in the future than with the present paper.

As you know, I am not an expert in this area of research. Besides the comments made by #1 and #3, I can see two major issues with the work. The first is editorial in nature. I found the conceptualization aspect of the paper to be interesting and worth keeping. However, the section “Ethnography and Netnography” carries on for too long and causes the reader (at least this one) to lose interest. This section needs be more tightly focused on the netnography method itself. The second major issue is that I would like to see an application of the method. An obligation of an author espousing a new method is to show how it can be applied in practice; particularly in a journal that is read by many practitioners.

Reviewers #1 and #3 have made some very useful comments. Reviewer #1 ‘s main point is that you have not put the netnographic method into terms that marketing researchers can use. This is a valid point; some of this problem would be ameliorated with an empirical application. Reviewer #3 feels that you have not made a sufficient case for how the netnographic method is new and different. While I would be satisfied with a good explication of the approach with an application (i.e., it does not have totally break new ground), this reviewer obviously feels that your claims have been overstated.

Overall, I am inviting you to revise and resubmit the paper for possible publication. However, the paper needs a significant editorial overhaul and a good example of its application. In addition, at 38+ pages, it is too long. I would like you to shoot for a 30- page paper, excluding exhibits and references. I will send the paper back to Reviewers #1 and #3; thus, if you choose to revise the paper, you should prepare detailed notes about how you addressed their comments.

After June 30, 2000, please send any new or revised manuscripts to:

Professor Wagner A. Kamakura, Editor
Journal of Marketing Research
Henry B. Tippie College of Business
The University of Iowa
108 John Pappajohn Business Bldg.
Iowa City, IA 52242-1000

Thank you for sending your work to JMR, and please continue to do so in the future.

Sincerely,

[Russ]

Russell S. Winer

A quarterly publication of the American Marketing Association / 250 S. Wacker Drive / Chicago, Illinois 60606

MS# 1205-9-1 “The Field Behind the Screen …..

Reviewer # 1

Abstract: The concept of “netnography” is set forth as the subject of this ethodological [sic] contribution to market research.

General Comments

The opening paragraph would be clearer if “market-oriented interests” were briefly described, especially in view of “virtual communities.” The focus is ambiguous, for the paper’s focus on market-oriented choice is not well-defined. Does this mean information-gathering via web sites (a solo, not a social activity), which is marketer-driven? And/or information-exchange in chat rooms (social), which is not marketer-driven? Or both? Given that the paper examines online vs. offline environments, the precise ones should be stated up front.

The object of study seems to be CMC (rather than consumer choice?). However, the signs used to replicate face-to-face communication are extensions of linguistic/syntactical techniques generally considered under the rubric of “tone” in discussions of print communications mimetic of human speech. Thus, “differences” between online speech and f2f speech should be considered in terms of print media as well.

At times (p. 7, for example), the use of jargon overwhelms clear presentation of the material. This paragraph (and others like it), do not lead the reader anywhere — what is an “obtrusive and opaque cultural experience”?

The differences do not necessarily hold firm: time-lags also characterize f2f communication, with silences and delayed responses long considered valuable communication devices (notably in terms of endowing speakers with power). The writing/speech relationship has been discussed at length in deconstructive criticism (Derrida, especially), and those arguments should be consulted. Even though online selves are considered mutable, personality changes are very much a feature of f2f communication as well — one puts on a self to serve the rhetorical purpose of the speech act.

The last sentence in par. 1 on p. 11 seems to be a summary of the market-oriented behaviors that the paper will consider, However, the distinction between the first two (consumer-driven) and the last one (marketer-driven) problematizes the notion of “community.” Is this the consumer community (WOM, online chat groups) or the marketing community (advertisers on the net) or both? The study of media fan and game-playing communities (p. 12) seems more limited than the above communities. The injunction to be careful regarding “social-specificity” in researching general marketing and consumer behavior is bewildering — Jenkins’s phrase is, again, a bit of jargon that is not readily identifiable by all marketing researchers

Fly on the wall observational techniques do not resemble focus groups –the former are consumer-driven, whereas the latter are marketer-driven. The hidden observer technique seems far less likely to involve social construction of the _ audience under study. The discussion of types of virtual communities does not tie into consumer behavior very clearly — how exactly are they all marketing-oriented?

This raises the issue, again, of the focus of netnotgraphy [sic]: is it the subject of the consumer discourse, the nature of the discourse itself, or both? The first does not necessarily lead to generalizable knowledge; the second may not be possible because of fragmented discourse types; the third possibility is confusing.

On p. 17, the authors discuss the accessibility of data (observational netnography). This is a major advantage of netnography, but its advantageousness depends on the researcher’s focus. There is little new about this observation, and the authors shy away from the issue of computer-coding vs. hermeneutics-by-hand. The section on treating on1ine data seems at odds with introspective “noting” by the ethnographer. Why is this a necessary step in observational netnography? The line between observation and participation has to be rethought very carefully in terms of standard ethnographic practices vs. online research — as presented, it is not convincing.

The study of anonymous users (who may be recreating themselves anew) vs. the study of trusted informants is reminiscent of the study of the write vs. the man/woman. Here, a major source is the philosophical and literary debate about reality: which one is “really real”? These debates would shed light on the paper’s discussion of the issue _p.22.

The mechanics of the CMC interview (like those in many interpretive studies) reveal the threat of response bias: interviewees are those who agree to be interviewed rather than a random sample of the universe in question. The issue of e-mail surveys appears to be different from that of interviewing chat room participants, but the paper is not clear on this point.

How does the section on research ethics relate to strictly “observational” netnography? It seems as if electronic eavesdropping is about the same as eavesdropping on public conversations, crowd comments, sports fans at a game, and so forth. Whether or not eavesdropping is ethical does not depend on whether or not the subjects give consent if one assumes that anything said in public is not privileged matter.

The section on representation might be enriched by consulting those works in marketing/consumer behavior that deal with representation in a traditional way — Stern’s edited book, for example, with the useful article by Price and Arnould.

To sum up, the topic is most interesting, but there are problems in different sections that need to be resolved. Perhaps most problematical in the lack of a clear marketing research focus. The study of consumer groups (and perhaps, marketing communications) is not fleshed out, and the objects of netnography __ consumer choice, -consumption communities — aren’t made compelling to marketing researchers. The paper is nicely written and organized but it needs a much more specific focus to make it relevant to marketing.

Comments to Author of Netnography. Reviewer #3

Like many, I’m very interested in this topic. You offer some very insightful and thoughtful comments. I am impressed with that. And, I do think that it demonstrates a familiarity and a level of expertise that sets you apart from many. So, I applaud the effort, the insight, and the challenge. At the end of the day (and this review) I want to encourage you to go forward.

However, I am not favorably disposed to the paper as written. I think it over-claims, and underdelivers. I am not saying it is without merit, or potential. But, I am saying as currently conceived and written, I simply don’t think it works.

Here’s why:

It’s a bit of a fireside chat. While making some fairly grand claims, it only offers a recitation of things many which many people have recognized, written about, and pondered. It ponders well, but does little in a concrete way.

While CMC present legitimately new research challenges, a poor case was made for how this author’s methods (not explicated particularly well either), were going to significantly engage the new problems, address the new opportunities, and actually contribute something tangible and concrete in reaction.

Many of the underlying assumptions are good rhetoric for positioning one’s work, but highly questionable. For example, on page 1: “they (market researchers) have a weak appreciation of some of the more radical discontinuities in social formation these developments represent.” Quite the contrary, I think many have a very good appreciation for the discontinuities, as well as for the facets of CMC which are not radically different at all from other forms of human communication.

Most critically and damning, there is absolutely nothing to the idea that the methods proposed here are “discontinuous innovations.” NO WAY. In fact, I don’t really see fully developed and explicated method here. And most of what I do see, is not really new. Add to this situation, modest empirical grounding and then there’s just not much here in the way of methodological discontinuities. Yes, all media are different, and have unique properties, and this is true of the Internet as well. But there is nothing you propose here that is A DIFFERENT METHOD, not really. You trade on the novelty of the internet to advance this claim, but you never show anything beyond an adaptation, extension, or for that matter direct application of other textual, interpersonal communication, organizational communication, linguistic, or general field work technique. There is simply no discontinuous methodological innovation here. I’m not sure there needs to be, but I simply don’t see it.

Now, I could be wrong. You could try to convince me, show me. Or, maybe you want to see your contribution as more one of integration, synthesis, extension. I don’t know, that’s really between you and the editor, but one thing is sure: the discontinuous innovation claim is simply not working.

I do wish you well.