Anatomy of an Academic Article: Back on the Horse

So what did I make of these reviews?

Well, it really was a mixed bag, and as a junior scholar I wasn’t particularly well-equipped to interpret the results. I had the advantage of having a strong mentor in John Sherry, who gave me a lot of guidance. But there’s only so much you can do when the data point to paradox.

I think it’s pretty unusual that one reviewer on the first round says unconditionally accept. That certainly hasn’t happened to me very much (maybe one or two other times). Reviewer 2, whoever you are, I thank you. I never got to see the review. I don’t even know if they wrote one. But it certainly didn’t hurt innovation to only have to deal with two reviewers for the rest of the way through rather than three (or even, worse, four or five as I have sometimes seen & experienced).

Then, the other two reviews are both rejections. Now, that makes it interesting for the Editor. The Editor, at that point, has grounds to reject the paper. Forget the “Despite Merit” part, that’s just being polite. Those are two rejections, plain and simple. Those reviewers want this thing down and out–they don’t want to see it again.

Rejecting the paper would have been, I think, the most conventional, sensible, defensible, logical, easy decision here. Had the Editor been interested in keeping out qualitative research, or new methods, he easily could have simply said something like ‘We are open to new stuff, but the Reviewers have spoken, 2 out of 3 said reject, sorry, try another journal.’

But there seem to be at least two reasons why Russ Winer didn’t do that.

First, and I’m reading behind and between the lines to do this, I think this is a somwhat unusual set of reviews. He said that he sent the paper to one on my ‘don’t send’ list, one on my ‘do send’ list, and one other person he trusted. I think that the person from my send list gave the unconditional accept. However, I think the one on my ‘don’t send’ list was Reviewer #3, and I think Reviewer #3 was significantly more positive towards the paper than Reviewer #1 (but still rejected it). So Reviewer #1, who the Editor chose, would have been the most negative one towards the paper.

Secondly, Prof. Winer makes a very telling admission in his letter: “Let me state at the outset that I am interested in publishing a paper on this topic.” I read that statement then, and still do now, as very encouraging to me, and as a clear signal to reviewers. In fact, that whole paragraph is as much for the reviewers as it is for me. He continues: “JMR editors should publish and always have published papers demonstrating the value of new methodologies. Of course, what is particularly interesting and topical about netnography is that its domain is the Internet.”

Again, this says to me that this Editor is taking his job as gatekeeper in a top-tier publication very seriously, and weighing it in relation to the advancement of knowledge. He realizes that novel methods are going to be resisted by the old guard. And he sees his job as encouraging, fostering innovation, making sure that those flickering new flames aren’t prematurely snuffed out. And the new game in town for Marketing Research is the Internet, and new Internet-based methods should be of interest to JMR. This is the right place for this research–that’s my interpretation. And when he says he is interested in publishing a paper on this topic, I see that as very positive. Because I am very interested in writing a paper on this topic and, as far as I know, there was no one else doing so at that time.

So he asks me to take into consideration some of Reviewer #1 and #3’s comments, and then asks for a paper that is thoroughly edited, contained a good example of an application, and is shortened.

That sounded quite do-able to me. The example would take a bit of time, though.

Maybe the scariest part of this was that I was teaching a full load of demanding Kellogg courses until well into May. That included my Hollywood course on Entertainment Marketing, which finished with a weeklong trip to Hollywood with my students. Pretty intense.

It was very unlikely that I would finish the revisions by June 30th. So I may be looking at sending my revision to a new editor, an unknown editor, an editor who might not be as fair-minded or as open to new and Internet-based methods as the former one.

Timing is everything. As we will see.

* * *

I did make my revision, and my revision notes. Revision notes, if you aren’t familiar with them, are the notes that you are required to provide with your revision to point out exactly how you addressed the comments of the Reviewers and the Editor.

The example I chose to add into the shortened paper at this point, as you will see, is a kitchen appliance search that I performed. This was part of some innovation work I was doing with a company, and there was only so much of it that I could share, so the example is pretty sparse and spare.

Here are the revision notes, in their entirety. Reading through them now, I would say that they are not the best set of notes I’ve devised, but they certainly do indicate that I tried to make the major revisions asked of me, and many of the minor ones, too.

To be honest, I didn’t really know what to do with a lot of Reviewer #1’s comments. They seemed like nit-picks from someone who didn’t understand the method, the Internet, or what I was trying to do. Reviewer #1 seemed to be trying to put me into a familiar box. Deconstruction?

Reviewer #3 wrote the review in sort of an oddly confrontational style, but had some good points, and I thought about them a lot and tried to address them. That thinking was productive. But was the experience of reading that abrasive tone enjoyable in the least? NO WAY.

Here are my revision notes:

Revision Notes RE: JMR MS# 1205-9-1: “The Field Behind the Screen: Using Netnography to Research Market-Oriented Virtual Communities”

First, I wish to thank the Editor and reviewers for their insightful, helpful and encouraging comments. I have sought in these revisions to meet all of the reviewers and Editor’s comments. These notes will overview in detail the changes that have been made to address the comments provided.

1. As suggested by the editor, the paper has been shortened significantly. The section entitled “Ethnography and Netnography” has been overhauled and now focuses more precisely on the method of netnography. The paper has been carefully edited to remove redundancies. In addition, sub-sections and topics whose orientation appeared either redundant (e.g., cultural exit) or of mainly theoretical interest (e.g., representation) were either removed or trimmed back substantially. The result is a 30-page paper, excluding exhibits and references.

2. Within these 30 pages is included a 5-page illustrative example that demonstrates how netnography can be (actually, has been) applied in practice. Adding this “capstone” empirical application has been one important way in which this revised version of the earlier manuscript brings the method of netnography into a form that may be much more useful to marketing researchers than the previous version. In addition, the entire manuscript has been substantially revised to more overtly and clearly reflect the interests and objectives of marketing researchers (Editor’s comments).

3. To clarify the positioning of the paper, the opening paragraph now clearly describes and details the meaning of market-oriented interests, specifying them as social information that is not marketer-driven (Reviewer#1 comments). This more focused positioning is also reflected at other places within the revised manuscript.

4. This revision of the paper repositions netnography as an extension of current methods, in line with the comments of Reviewer #3. The current version presents netnography as more of an integrative method that extends and synthesizes current ethnographic methodology into a form that may be more suitable to some of the unique contingencies of the Internet environments (for example, there are new ethical issues, completely unobtrusive “lurking” is possible, informant accessibility is of an entirely new order). The revised version attempts to demonstrate how certain current ethnographic approaches must be modified for the online environment. These techniques are not in themselves a new methodology, but they do (arguably) represent a potentially useful extension of current techniques to a new and sometimes very different social environment. The claims for discontinuity no longer appear in this version. I agree with the reviewer that these claims were not convincing, and are not necessary (Reviewer #3 comments).

5. Footnote 4 has been added in order to raise the point that the discussion of the signs used in CMC in order to replicate face-to-face communication are similar to the differences between print media and f2f speech (Reviewer#1 comments).

6. The object of study of netnography has been overtly described in this revised version as dealing with the content of CMC as it may affect consumer choice (for example, on p. 4) (Reviewer#1 comments).

7. The use of “jargon” has been greatly reduced on p. 7, and throughout the paper. Potentially obfuscating phrases such as “obtrusive and opaque cultural experience” and “social specificity” have been cleaned up or removed outright (Reviewer#1 comments).

8. The differences between CMC and f2f time lags have been developed further, as presented on page 7 and Table 2 (Reviewer#1 comments).

9. The writing/speech relationship has been developed further, through citing relevant work in deconstructive criticism, on page 8 (Reviewer#1 comments).

10. The mutability of identity concept in CMC has been developed further as a more extreme manifestation of what often occurs during f2f communication, on p. 8 (Reviewer#1 comments).

11. The focus of the market-oriented behaviors considered in the paper has been clarified as those which are consumer- (rather than marketer-) driven (Reviewer#1 comments).

12. The focus of market-oriented netnography has been clarified as the topic of the discourse itself, as demonstrated in page 1 and 2’s revised definition and description of the method, and elsewhere in the revised manuscript. These revisions attempt to make clear that one of the important limitations of the method is that (as with any ethnography) it is not guaranteed to produce generalizable knowledge (Reviewer#1 comments).

13. Spurred by these insightful comments, the important issues regarding the accessibility of data have been reformulated. This has resulted in a rethinking and repositioning of the observation and participation comments throughout the manuscript on the basis of these helpful comments (Reviewer#1 comments).

14. The discussion of anonymous users has been revised, and has introduced an ontological discussion (Reviewer#1 comments).

15. More attention has been paid to the discussion of response bias, random selection and comparison of netnography and online surveys, as this version reflects my attempts to make these points clearer (see page 19) (Reviewer#1 comments).

16. As with the comment addressed in point 13, above, the Ethics section has been substantially revised to reflect the interesting comments regarding the role of strictly observational ethnography, and to directly address and further develop the ideas contained in Reviewer #1’s comments on the matter.

In summary, addressing these comments has resulted in a substantially revised paper that is more firmly focused on applications relevant to marketing researchers and practitioners, contains an empirical application, does not overstate the newness of the method, is considerably clearer, develops ideas more thoroughly and is 30 pages long excluding exhibits and references.

Thank you again, Reviewers and Editors, for your ideas and assistance.

* * *

This is the abstract for the Round 1 revision, entitled : “The Field Behind the Screen: Using the Method of Netnography
To Research Market-Oriented Virtual Communities”

ABSTRACT

Computer-mediated communication (CMC) has fostered cybercultures and virtual communities, many of which are market-oriented in their focus. This article develops netnography as an online ethnographic technique for marketing research. Netnographic techniques are adapted to the characteristics of CMC. Netnography presents flexible guidelines for conducting online fieldwork that adaptively address central ethnographic concerns, while providing practical marketing research. An illustrative example is provided to demonstrate the utility of using netnography for new product development in the small kitchen appliance market. The example demonstrates how netnographic observations inform product attribute and positioning decisions. Implications of the method are discussed in a concluding section.

Here is the complete Round 1 Revision: field_behind_round1_rev.pdf

And, in the next post in this series, you’ll see just how this revision was received by JMR.