Mad research leads to mad public:
some fallacies of student research
By Marina Sergeyeva, scientist and author of Upstairs and Nikki & Nick are great friends to pick!
written on 9/10/2012, posted
Coincidentally, the day I finally get an opportunity to write a post on a topic I’ve been long meaning to, also happens to be the day of something special and seemingly contradicting to my topic, and Geneva holds the secret to it. That’s right, Sep. 10th four years ago (in 2008) tiny matter made big flashes, into the future and of the past, as the Large Hadron Collider at CERN fired its first beam of protons, and has been running since with minor downtimes. Although scientific teamwork at CERN seems to be on track, especially with the Higgs boson recent news attention, a lot of the researching world has been flooded with subpar analysis. This issue has been considered by scientists and several critics [see b. & c. below] in the past, but I want to discuss a few points that might not have been covered, from a somewhat psychological perspective. I have firsthand experience with college-level research, and since a large volume of modern research arises from institutions, these points could be helpful to note the next time one relies on a study. This is not to say that quality research doesn’t exist, because quality research is what practitioners and other scientists should keep at hand when considering a problem, to make better evaluations and decisions.
Consider an environment where you are placed under influential command and have to carry out a task to make yourself and the person over you look good. This is not hard to imagine because at some point we deal with this either in school or at work (even home), where someone else is a boss. Naturally, humans strive to be perceived as valuable by a peer or a superior. This is especially true of highly intelligent individuals, as they tend to reveal their thoughts/findings only after absolute confidence, so they expect a greater praise because they are held under higher standards. Supportive and jealous peers both contribute, whether unknowingly, to a strong level of self-criticism that develops during school years in smart students. This is the positive feedback loop: being considered valuable (either positive or negative “labels” of smarts, such as ms. cheatsheet, mr. brainiac, etc.) and wanting more of that response, even if peers’ expectations are sensitized and the bar is raised higher, so raising self-expectations and performance. No one wants to disappoint people’s established perception of one’s mental or physical abilities. Thus, by the time this student gets into an ivy league school, self-perceptions have long been molded by outsiders. Affirmatory results might also be associated with this perception, since raising the bar usually leads to greater effort. So yes, good work might come out of these determined students, but all methods will be exhausted to assure such success, including approaches frowned upon in the scientific community. “If you can’t join them, beat them” type of approach, not something normally practiced. This could also partly explain contradictory results between research teams from different institutions. (There are other factors playing a role in inconsistencies, which is a topic for another blog.)
The public reading of studies done from some prestigious colleges identifies with a trusted name, yet they will never know the exact details under which such a study was performed, in fact no one but the unsupervised student may know. Difficulties in the process are not described. What typically happens with research at institutions is the PI (principal investigator, usually a professor), who’s ultimately responsible for a specific topic, assigns tasks (the background of which is a student’s responsibility) to capable, eager students, and “overseas” any difficulties students may have. The PI and all students working on tasks even over the years make up the research team. The professor has many responsibilities and sometimes several investigations going on simultaneously so can’t babysit each student during minor issues, encouraging independent thought. The team of shifting students works on the thesis until a favorable, publishable result is obtained. This is where shortcuts begin. From a student’s perspective, who probably seeks to become a full-time researcher, great performance on this first dip into research feels crucial. A lot of trial and tribulation of one’s desires and abilities can be learned at this stage. Respect, confidence, possible publication, scholarship funding, and sometimes a grade all depend on the outcome. What is a student to do here with all these pressures when something doesn’t come as expected? Surveys on falsification of data reported a good number of students committing such no-no’s. These smart students may know the outcome and can “fit” the data nicely to a results or interpret things to their liking, kind of like what Hawk mentioned in his post on fitting equations to events, and rush to conclude the study before next set of students grabs onto and expands on their work.
This mostly contrasts to the way research is done outside of schools. I suppose the difference here is similar to how an experienced private surgeon and a hospital setting with residents swarming over the patient differ. Cation and careful understanding should be taken with either case, but the amount of contradictory information that comes out from the closed doors of science into the careless treatment of scientific information by the media is enough to make any sane person go mad with the science!
Here are some informative, some humorous information on how to deal with reports: