General Research Analysis Summary Jonathan R. Brown, Ph.D Name: Joe Strittmatter I. Introduction Title: A.

Title: The identification of a target(s) population and variables under study were clearly stated. 1. What I liked: The title of the research was enough to allow those interested in research on faculty websites to locate the study. By including the key information and a short description of what is being measured, the title fulfills the most basic of the needs of a title. 2. What I didnt like: While the title informs readers of the basics of the research, it does little to grab readers attention. Had I not been doing a specific search for faculty websites, chances are this is not a paper that would have caught my attention. 3. Suggestions: More information could be included in the title such as a better description of the population, instead of merely, faculty. The researcher could simply add Education Facultyto provide more detail. B. Abstract: The purposes, methods (subjects, instrument/s, design, procedures), important findings, implications of the study were clear, theoretical support, limitations, and recommendations for future research were given. 1. What I liked: The purpose of this study, to determine how many faculty have up-to-date websites, the purpose of the sites, and the content found on faculty web sites is clearly stated in the abstract. The findings of the study were also presented in a short, to the point sentence. 2. What I didnt like: While an abstract is supposed to be very succinct, and to the point, the author could have included many more details such as the methods, theoretical support, limitations, and recommendations for future research in this abstract. 3. Suggestions: Abstracts should generally be between 100-300 words and even providing all of the needed information in that space can be difficult. The

author wrote the abstract in 62 words, allowing plenty of room for more details. Body of the Work Begins as Introduction C. A clear statement of the problem was provided. 1. What I liked: It is clear that the author is searching for reasons why more members of her faculty are not implementing websites, as she details previous attempts to improve faculty websites as well as their shortcomings. Her purpose of study then is to determine whether her university is similar to others when compared on the characteristics of faculty web sites. 2. What I didnt like: While the author summarizes her problem at the end of the introductory paragraph, she doesnt describe the problem in detail until pg. 21. It would be helpful to know what her exact research question is prior to the results section of her paper. 3. Suggestions: By including her research question(s) in her introduction, readers would have a better understanding of the purpose of her study. D. The rationale for the study was logical and convincing. 1. What I liked: The rationale was extremely simple but important nonetheless. A recent school-wide web redesign effort led to the researcher examining education faculty websites. Too often, institutions sink valuable time and money into efforts without first researching the effectiveness of what they are implementing. 2. What I didnt like: There was nothing about the rationale that I did not like. 3. Suggestion: The author could have elaborated on this topic a little more. A teachers ability to have a web presence is a hot topic and ties in with her rationale. E. A review of literature was current, thorough, accurate, and related to the statement of the problem. 1. What I liked: Much of the authors references were reasonably current with her article. The article was published in 2009, meaning it was most likely written in 2008. Several of the authors sources were published as recently as 2007. 2. What I didnt like: Most of the literature cited by the author seemed to be randomly thrown into the paper without a clear description of what it was describing. Possibly it was my lack of background in the topic but this section was hard to read. It seems

as though none of the prior researchers agree on what needs to be included on faculty websites and maybe that is because the capabilities of what can be included on sites constantly changes but did not think this section was as coherent as it could have been in explaining the prior research. F. Terms were clearly defined. 1. What I liked: A short section on page 12 that describes the different classifications of websites was helpful, as I would have had no clue what the difference between an Enhanced Directory and a regular Directory would have been without it. Also, on page 11, the author provides her working definition for faculty website. This was important to me because she defined it as one hosted by school servers and accessible from the College of Education, which is more than I was originally thinking. There are many other free hosting sites now such as Google so it was important that the author described her definition. However 2. What I didnt like: Other than the short description of terms in page 12, there was no section in which terms were clearly defined. While I am familiar with most of the terms used in the research paper, there were several that I could have used a definition for. Thankfully, the author usually included them in the same sentence. For example on page 23, the author describes the word trialability in the same sentence in which it appears. Also, while the differences between an associate professor, an assistant professor, and a full professor may be obvious to a university professor, I would have appreciated a description the differences between each. Suggestions: I would have preferred a section devoted to the definition of terms. In one specific case, I found myself wondering what exactly the author meant by using the term student centered. G. The specific purpose of study, research questions, or hypotheses were logical extensions of the problem, rationale, and literature review. 1. What I liked: 2. What I didnt like: Suggestions: H. Theoretical support for research was provided and supported.

1. What I liked: The theoretical support section was well written. It introduces and then elaborates on five different theories related to implementing new technology. Those theories are then referenced again on pages 23 and 24. 2. What I didnt like: There was nothing that I did not like. I. The article was clear and well written. 1. What I liked: The article really started to take shape once the author began to describe the theoretical support and was mostly well written. 2. What I didnt like: While I think the research was completely worthwhile, the author could have done a better job of selling its importance by being more clear in the introduction.

II. Method A. Subjects 1. Protection of human subjects review was clearly described. a. What I liked: Nothing described b. What I didnt like: I was unable to locate anything that describes how confidentiality was maintained. In fact, on page 37, the author provides the names of eight professors that demonstrated exemplary websites. c. Suggestion: Include any review process or protection for subjects involved. 2. Subject selection and exclusion criteria were clearly described. a. What I liked: The author made it perfectly clear that the intention was research colleges comparable in size, which was within 500 students. She even provided possible inhibitors to the reliability of the research if she did not exclude schools that were much bigger or smaller than hers in the research. b. What I didnt like: I feel the author did a good job explaining her selection and exclusion criteria. 3. Random selection of subjects and random assignment to groups was described. a. What I liked: To serve the purpose of her research, the researcher chose to use non-random sampling. She did this by selecting a group of ten schools that met her criteria and then examined the web sites of all associate, assistant, and full education

professors in those schools. b. What I didnt like: While the non-random sample greatly diminishes the external validity of the study, it served the researches needs by allowing her to compare her university to similar universities and she did learn more about the web sites of faculty at institutions similar to hers. 4. Differential selection of subjects and threats to internal validity were clearly controlled. a. What I liked: All of the schools with education departments were chosen for her study. She didnt exclude any schools that fit the design of her study. b. What I didnt like: Since the selection of participants was non-random, there is a threat to the internal validity of the study. 5. Subjects selected on basis of extreme scores were not present. a. What I liked: The researcher described that a university was eliminated from the research since it did not have an education department. Also, all visiting professors were excluded as they most likely would not have a website. b. What I didnt like: There was nothing I did not like. 6. Interaction of subject selection and threats to external validity were clearly controlled. a. What I didnt like: There was little description of the procedure itself. Is it possible that faculty members updated their website as a result of the study itself? 7. Materials used that were unique to the study. a. What I liked: The materials used were based on the literature review. The researcher checked the presence of elements that were chosen as desirable by previous researchers. b. What I didnt like: I would have preferred more details on the actual materials they used to check the websites. I assume the researcher used a simple checklist to determine if the elements were present or not. 8. The text was clear and well written. a. What I liked: While I would have appreciated a little more detail, I was able to follow the scope of the research enough to understand how the data was collected.

B. Instrumentation/Measurement 1. The instrumentation was appropriate.

a. What I liked: I assume that the instrument used in this study is the researcher, who determines whether the websites studied show the presence of the desired elements or not. The researcher mentions that also mentions that an Excel spreadsheet was used for calculating descriptive statistics. b. What I didnt like: Other than the Table #5, which was the most helpful for me to see an overview of what exactly was being measured. 2. The instruments were calibrated for the population sampled.

a. What I liked: Since the researcher is the instrument and developed the criteria being identified, there is no other instrument that can provide a more accurate measurement. 3. The instruments were reliable and valid.

a. What I liked: Since the researcher was able to measure an intended area, I believe the content validity. Also, the researcher included a wide scale of elements found on websites for her research. b. What I didnt like: The external validity of the measurement, due to a small, non-random sample is low for this study. While the reliability of the study should be quite high since the researcher would likely make the same observations consistently, there was no mention of preventative measures taken to assure reliability. Suggestion: Include the time period between the studies. If I were to observe a faculty member at one institution one day and another faculty member at the same institution several days later, is it possible that they would have had a training or instruction on web sites in that timeframe? 4. Experimenter bias was controlled.

a. What I liked: Much of the experimenter bias in this study appears to be tied to the selection of criteria to be observed, which is to be expected in qualitative research. b. What I didnt like: There was nothing in this section I did not like.

5.

Test environment was controlled.

a. What I didnt like: There was no explanation of steps taken to control the environment. 6. Instructions to subjects were controlled.

b. What I didnt like: There was no discussion to let this reader know that the faculty members knew their web sites were being observed. 7. Adequate selection and measurement of independent variable was described.

a. What I liked: The independent variables, the faculty websites, were described in detail on page 12. I appreciated that the web sites were classified by type. It was also help to be able to view the Table 3 on page 15 for a visual. 8. Adequate selection and measurement of dependent variable was described. a. What I liked: The dependent variable, which was the desired elements of a web site, were described adequately. The researcher explained everything they would be measured on from email to subtle aspects of web sites such as feedback forms. 9. The text was clear and well written. a. What I liked: The methods section was clear and well written. It was easy to see the direction the study was heading towards after reading this section. b. What I didnt like: I would have appreciated more information on the instruments. C. Materials used were clearly described. a. What I didnt like: Other than a brief sentence on pg. 11 that states elements and other information were recorded in an Excel spreadsheet there was very little description of the materials that were used. D. Procedures 1. The research design was appropriate for the study. a. Since the study does not randomly assign groups and uses multiple elements in the measurement, this research design is classified as a quasi-experimental research design. 2. The procedures (research design) reduced threats to internal validity:

a. history b. maturation c. pretesting d. mortality e. Hawthorne effect f. Interaction of any of the above E. The text was clear and well written. There was no section that I could find that clearly defined the procedures. This would have made it easier for someone investigating research papers for the first time to better understand the procedure. III. Results A. Data tabulation procedures were clear. 1. For me, the tables were the easiest way to decipher the data tabulations. B. A clear and measurable question (questions) was asked. 1. The question(s) were clear and measurable. They were: What percentage of Education faculty at comparable institutions have web sites and how up-to-date are they? What purposes do they serve? What information is included on the sites? Do faculty members set an example by have exemplary web sites? Is there a correlation between faculty rank and having a website? D. Tables and figures were clearly integrated into text. 1. The figures were extremely helpful while going over the research. E. Summary statistics were used appropriately. 1. The author frequently cited percentages in the results of her study. F. Organizational data was clear. 1. G. Statistical analysis was appropriate: 1. level of measurement 2. number of observations 3. types of samples 4. shape of distribution

H. I.

Appropriate use of correlationnal and inferential statistics was used. There was a clear presentation of significant and no-nsignificant differences and/or correlations.

J. K. L.

The research question(s) was answered. The results clearly related to research problem. The text was clear and well written.

IV. Discussion, Summary, and Conclusions A. The discussion clearly related to the research problem. B. Limitations of the method were discussed.

C. Conclusions were drawn directly and fairly from results. D. Reasonable explanations were given for unusual, atypical, or discrepant results. E. The results were related to various theoretical explanations. F. The results were externally valid. G. Implications for application of findings were discussed. H. Suggestions for future research were given. I. The text was clear and well written.

V.

References A. B. References in text were cited in reference listing. References followed APA (or journal required).

VI.

The article/thesis/dissertation was well written, clearly and neatly presented, and the information was communicated efficiently (quantity of words used to communicate) effectively.

Comments: This was as challenging assignment as I can ever recall completing. I can appreciate all of the work that goes into this process. I initially was under the impression the article that I

chose was simple and straightforward but as I critiqued the article, it seemed to get more and more complex. I was under the impression that research is done with one research question in mind, however, it seemed as though the research was juggling several research questions at once. I definitely need some more practice at this.