Article Critiques

Introduction

One of the primary goals of Proseminar is to develop your skills to read and write critically about empirical research. Simply put: You cannot generate knowledge without learning first how to critically evaluate extant work.

To be clear, thinking critically does not mean focusing only on what might be “wrong” about something. All research – published and not – has limitations, holes, and sometimes, out-right errors, and you will only grow and develop as a scholar when you learn to find these limitations, identify unanswered questions, and point directly to errors. All these things can be done respectfully and in the spirit of pursuing the truth, not simply identifying problems.

Questions to Guide Each Critique

The questions each article critique should address are:

A. Theoretical Perspective (about 3 pages)

  1. Critique the author’s conceptual framework. (NOTE: This means the conceptual logic the authors use to introduce their study. Usually starting broad and narrowing down to something specific
  2. Comment on the need for this study and its importance. (NOTE: What do the authors say about why this study is needed? Is it compelling? Non-existent? Other
  3. How effectively does the author tie the study to relevant theory and prior research? (Do they cover the right literature? Anything missing? Anything not needed?
    How comprehensive? Tied to their conceptual framework?
  4. Evaluate the clarity and appropriateness of the research questions or hypotheses. (NOTE: Either their explicit or implied questions. Appropriateness can take many forms, but especially its’ connection to 1-3 above)

B. Research Design and Analysis (about 4 pages)

  1. Critique the appropriateness and adequacy of the study’s design in relation to the research questions or hypotheses. (NOTE: Try to name a design — pre-post; longitudinal, quasi-experimental; experimental; ethnographic; qualitative (which kind?); etc. Then try to talk about if that design is or is not well-suited to answer the type of research questions that were stated.
  2. Critique the adequacy of the study’s sampling methods (e.g., choice of participants) and their implications for generalizability.
  3. Critique the adequacy of the study’s procedures and materials (e.g., interventions, interview protocols, data collection procedures). (NOTE: Sometimes it’s helpful to think of “procedures” as all the steps taken to get to the point of data analysis. Example: obtained consent; randomly sampled, filled out questionnaire, etc.)
  4. Critique the appropriateness and quality (e.g., reliability, validity) of the measures used. (NOTES: Google reliability and validity. To what extent do the authors discuss these issues and what implications does this have for their study?)
  5. Critique the adequacy of the study’s data analyses. For example: Have important statistical assumptions been met? Are the analyses appropriate for the study’s design? Are the analyses appropriate for the data collected? (NOTE: Consider also if the analyses actually answer the questions being asked)

C. Interpretation and Implications of Results (about 3 pages)

  1. Critique the author’s discussion of the methodological and/or conceptual limitations of the results. (NOTE: Just like we did in class)
  2. How consistent and comprehensive are the author’s conclusions with the reported results? (NOTE: Do they over-state or understate their findings? Does their conclusions “go beyond the data.” Have they not talked about something they should have?)
  3. How well did the author relate the results to the study’s theoretical base? (NOTE: The theoretical base (conceptual framework + literature review) were established in the introduction. Do they return to this framework and talk about how their data fits — or doesn’t — within this framework?)
  4. In your view, what is the significance of the study, and what are its primary implications for theory, future research, and practice? (Remember: Be specific)

Rubric

We assign 8 points to each of the 13 article critique questions.

For each question, we look for three elements in your response:

  1. A brief summary of what the authors did/reported.
  2. Your critique / evaluation of what the authors did.
  3. And support for your evaluation (e.g., reasons, rationale, evidence, etc.).

Low, average, and strong responses differ as follows:

  • Strong (scores ~ 7-8) – Do all three with some real conceptual substance and, at times, insight.
  • Above Average (scores ~ 5-6) – Do two of three explicilty – i.e., clear writing, clear evaluation and/or clear rationale/evidence.
  • Average (scores ~ 4) – Do two of three (e.g., ‘a’ and ‘b’ or ‘a’ and ‘c’), but not very explicitly.
  • Below Average (scores ~ 2-3) – Do one of the 3, often vaguely.
  • Low (scores ~ 0-1) – Fail to address the questions

Article Critique #1

In this assignment, you will learn about what makes for a good article critique by analyzing past article critiques and deriving the criteria that makes one article critique better than another. Specifically, in this first assignment we’ve provided you with three sample critiques of an article’s theory, methods, and results, each scored High, Average, and Low. But we aren’t going to tell you which example corresponds with which score. That’s your assignment!

Step 1 – Background

Before you begin, first make sure you understand the article critique process, its central importance in the 1st year development of students, and the role your critiques play in the preliminary exam [note: link takes you to MSU EPET page, click on program handbook, see page 19 for exam information]. Also make sure you read the 13 questions that each article critique should answer.

Step 2 – Read the Article

Before evaluating the critiques, first read the article being critiqued: Zhang, S., & Duke, N. (2008). Strategies for Internet Reading with Different Reading Purposes: A Descriptive Study of Twelve Good Internet Readers. Journal of Literacy Research, 40(1), 128-162. doi: 10.1080/10862960802070491.

Step 3 – Review Example Critiques

Next, review these example article critiques. Each of these is just a portion of a full critique — specifically these are responses to the first 4 (of 13 total) questions.

Examine the rubric we use to to evaluate critiques, and see if you can see differences in these examples in terms of the rubric.

Step 4 – Individually

Order the three examples from best to weakest. Why did you rank them this way. Be specific in terms of the rubric (summarize, critique, support with evidence).

Step 5 – As a Group

In your group, reach consensus on the rankings, and the principles, rules, and rubrics. Work on the other sections (Methods, Discussion)

Step 6 – Instructor Scores

After students have completed the assignment, you will be given access to the instructors’ ratings of these same responses.

Article Critique #2

In this assignment, you will critique an article with the help of a partner.

Step 1 – Read the Article

First, read the article being critiqued: Lai, C., Yang, J., Chen, F., Ho, C., & Chan, T. (2007). Affordances of mobile technologies for experiential learning: the interplay of technology and pedagogical practices. Journal of Computer Assisted Learning, 23(4), 326-337. doi: 10.1111/j.1365-2729.2007.00237.x.

Step 2 – Individually

Write a a review of the article by yourself. Bold each of the 13 questions. Keep your responses to no more than 1800 words. Do not post your review anywhere, or turn it in anywhere.

Step 3 – Swap Critiques with a Partner

On your own, compare your review with a partner’s review (partners will be assigned in class). Read your partner’s review carefully. Make notes of points of agreement and disagreement.

Step 4 – With Your Partner

With your partner, complete a joint review of no more than 1800 words. Submit one review for both of you Google Drive. The same grade will be awarded to both you and your partner.

Article Critique #3

Guidelines

  • Critique THIS ARTICLE .
  • This is an INDIVIDUAL assignment.
  • Your critique should not exceed 1800 words.
  • Submit your article critique here
  • Also post a copy of your critique on your own web presence, as this is one of the required turn-ins for your first year prelim.

Article Critique #4

Guidelines

  • Critique THIS ARTICLE .
  • This is an INDIVIDUAL assignment.
  • Your critique should not exceed 1800 words.
  • Submit your article critique here
  • Also post a copy of your critique on your own web presence.

Article Critique #5

Guidelines

  • Critique THIS ARTICLE .
  • This is an INDIVIDUAL assignment.
  • Your critique should not exceed 1800 words.
  • Submit your article critique here
  • Also post a copy of your critique on your own web presence.