ALLEA defines ‘research integrity’ too narrow

ALLEA is All European Academies of Science – including the humanities. Together with the European Science Foundation they defined and maintain the European Code of Conduct of Research Integrity (2011).

The first observation is: there might be a subtle distinction between scientific integrity and research integrity.

For, we are in the realm of law and litigation now. ALLEA & ESF employ and represent hundreds of universities and research institutes, and one of their key issues is reputation. Parliaments will not want to squander taxpayer money on dubious institutes. Given the often abstract nature of scientific output, one of the main substitute criteria is trust. In other words: the boat should not be rocked. One of the pillars for public trust is the wise use of law. Scientists might claim that trust can be built upon science, but the lawyers employing them will explain that science itself on occasion can rock the boat – like saying that the Earth isn’t flat – so the legal pillar is sturdiest.

The European Code of Conduct for Research Integrity

The European Code of Conduct for Research Integrity (2011)

How ALLEA / ESF see the world

ALLEA / ESF (page 10) distinguish ethics relating to the external world (say, when you work on an atomic bomb) and ethics internal to the scientific process (say, you discover plagiarism w.r.t. that work for an atomic bomb). For the latter they use the notion of the scientific record. When you deal with that scientific record then there is research. ALLEA are like librarians maintaining an archive: when that archive is tainted, then there is a breach of internal reseach integrity. They state:

“There is, of course, no perfect watershed between the two categories. (…) but in the light of a discussion on a Code of Conduct the distinction may be clarifying.” (page 6) (my emphasis)

The weblog will hold that the distinction cannot be clarifying when the borders are vague. The “may be” is unwarranted, since the vague distinction can only be confusing. It is demagoguery to require a “perfect” watershed. We only require a distinction that works. Conclusion: ALLEA is on a wrong track.

Let us follow their track to see where it leads. Let us suppose that a scientist is busy 90% of the time with unavoidables like reading up, refereeing, meetings and conventions, email, discussions with research colleagues, writing research proposals or judging them, and 10% with writing and editing own research papers. ALLEA uses the term “research integrity” that deals with the supposed scientific record, and thus focuses on this 10%. Nobody will argue that this 10% is unimportant but focusing on it neglects an important part of scientific activity. A scientist might be a bully in discussions with colleagues but when the research papers are okay then ALLEA will legally support him on the issue of ‘research integrity’.

Subsequently, ALLEA formulates a code of conduct for this narrowed-down ‘research integrity’. The positive elements can be listed but are not so interesting legally. Few people would sue a professor for doing the proper things, though it might be an issue in some countries. Legally interesting is only the misconduct.

Subsequently ALLEA uses its magic wand and narrows this ‘research misconduct’ down to five issues (page 6). PM. My background on this issue is methodology, and in that field the term “falsification” is opposite to “confirmation”, which issue refers to Popper’s criterion of falsifiability. It is very strange to see the term “falsification” used in the sense below. The ALLEA / ESF paper wasn’t written by people coming from a background in methodology. My suggestion is that “manipulation” is a better term.

ALLEA / ESF code page 6

European Code, page 6

Nicole Janz (2015) wonders whether these categories aren’t too simple. She presents the following diagram by Rene Custers. ALLEA / ESF leave out the Questionable Research Practices. 

Slide1-data-secrecyBut this still is a weak criticism. In supplement to Janz and Custers, I can remark in stronger criticism of ALLEA.

  • A major case of misconduct is to assume authority in a field when you don’t have it. Check Linus Pauling and high doses of vitamin C. When this isn’t seen as misconduct then scientific institutes cannot be blamed when their employees step out of line.
  • ALLEA’s reference to whistleblowers may be incorrect. Whistleblowing applies to non-scientific employees who normally would remain silent. A scientist however has to speak, and the employer might try to enforce silence: which is censorship.
  • There is this case in which a “book review” is abused for misrepresentation and slander. When a scientist should give a fair evaluation when refereeing an article for a journal, then this should also hold for a book review that is published directly. A book review allows more freedom of judgement, but misrepresentation and slander don’t belong to that. However, commissions of integrity at TU Delft and KNAW / LOWI allowed this to happen – see here.

This Dutch LOWI is called in Dutch “Landelijk Orgaan Wetenschappelijke Integriteit”. This name is fishy and comes with the continuous need to explain that they do something else than looking into scientific integrity:

  • Google Translate of LOWI gives “National Board for Scientific Integrity” – and in this case I agree. You would expect that they do this. But they don’t.
  • LOWI itself has an English webpage, giving “The Netherlands Board on Research Integrity” – which plays into the distinction between “science” and “research”. In English they already better specify what they are doing, once you have grasped the language game that they are playing.
  • My suggestion is that they should call themselves “Dutch Office for Specifically Identified Breaches of Scientific Integrity”, since they only look into three breaches (fabrication, manipulation, plagriarism) at only some institutes (e.g. not at national planning bureaus). This proper name clarifies that there is still a lot to be done. This issue of a name is not just the choice between “Tom” and “Harry”: we are discussing the title of a book.

ALLEA has been neglecting the protection of individual scientists (who meet with censorship) and has been focusing on protecting the Institutes: (a) to generate the legal environment to resolve obvious cases of fabrication, manipulation and plagiarism that cause public worry, (b) by allowing institutes to censor criticism by employees, (c) by neglecting breaches that would cause a stir if they were tackled.

This weblog found these cases of assumed authority:

  1. Research mathematicians (RM) can claim authority in the field of mathematics education (ME) while they have no training for it. An old case in Holland is Hans Freudenthal (1905-1990), a recent case in Holland is Jan van de Craats. The ALLEA rules might maintain that Van de Craats didn’t do anything wrong in his field of research mathematics (RM). The problem however is that he embarked upon another field that he has no training for, namely the didactics of mathematics in primary education. ALLEA might dismiss it as a “context in which research takes place”. ALLEA then makes a deliberate choice to neglect serious breaches of science. (It so happens that Van de Craats wrote a paper for such a journal Tijdschrift voor Orthopedagogiek 2009, and one might say that this article isn’t properly researched as it should have been. See namely here, of which I informed him in 2008 (but I have no license for primary education either). But the real breach is that Van de Craats has been presenting his assumed authority all over, e.g. to Parliament where representatives might think that a professor of mathematics would know about these issues.)
  2. Another case is Klaas van Egmond, in this presentation for Dutch Parliament. He introduces himself as an “environmental scientist” (“milieukundige”) and then proceeds with discussing monetary and financial economics. Listeners might think that economics is included in “environmental science”. Thus he should have said: “I am no economist” – whence everyone would have been alerted to the question why his personal confusions on money and banking would be so important to inform Parliament about.
Robert Merton (1910-2003) and Adriaan de Groot (1914-2006)

Thus ALLEA / ESF are off-track on scientific integrity. What would be the proper track ?

The ALLEA / ESF code page10 refers to Robert Merton (1973), The sociology of science: theoretical and empirical investigations. I would prefer to also refer to Adriaan de Groot (1982) Academie en Forum, with his development of Forum Theory. See these two links on this weblog: one and two.

The key point is this:

  • ALLEA / ESF are on the wrong track since they define cases of misconduct as exceptions while they are rather extremes of common practice.
  • Thus, the key approach is to improve the Forum. When the Forum works well, the extremes are prevented (caught in the early stages of development). This will happen for a wider range than the few selected by ALLEA / ESF – and litigation would be reduced. Protection of the individual scientist against abuse within the Forum should allow the individual to function better in the Forum.

Intermezzo December 29 2015
(1) It appears that Vittorio Busato developed basically the same view, though not explicitly linking to the ALLEA code.
(2) For Dutch readers, there is A.D. de Groot & H. Visser (2003), Het forumwaarmerk van wetenschap, at KNAW
(3) For English readers, there is only A.D. de Groot (1984), The theory of the science forum: subject and purport.
Methodology and Science, 17, 230-259, which journal ISSN 0543-6095 was published in 1968-1995 (J.H. Kok, Kampen).

When research papers and books disappear behind (publisher) pay walls, then this reduces the possibility of criticism. Instead, open source publishing should be the norm, peer review can take place after publication, and criticism can be used to determine ratings. That the scientific community is so slow in changing from closed shop publication to open source publication, can be explained by the distaste for criticism.

Some researchers plea for open source publication, and a recent proposal by Arjen van Witteloostuijn is to restore Popper’s pride of place: “Change the way we conduct, report and publish our research” with a manifesto. However, the simple notion would be that the university or its library would provide for the publication outlet. The employer is also the publisher. There is really no need to set up new publishers. What is needed is to develop the Forum. (Read De Groot’s texts on this.)

PM. Incidently, Van Witteloostuijn’s embrace of Popper may be ill-advised. See here for Popper’s oversight. See here how Tilburg University (where Van Witteloostuijn is employed) sacked in indecent manner a fine expert in methodology of science, Bert Hamminga.

Pieter Drenth (1935), president of KNAW 1990-1996, president of ALLEA 2000-2006

How could it have happened, that ALLEA / ESF got off-track on scientific integrity ?

The ALLEA / ESF code came about by inspiration by Pieter Drenth (1935) (now 80 years of age), president of ALLEA 2000-2006, and former president of Dutch KNAW 1990-1996. ALLEA / ESF explain this on page 19, where they speak about “scientific and research integrity” which elsewhere is narrowed down.

European Code, page 19

European Code, page 19

Quite to my surprise, Pieter Drenth appears to be a psychologist, interested in experiments and testing, and appears to be acquainted with Adriaan de Groot (1914-2006), whom we have been discussing in two recent weblog entries.

Drenth’s interest in good science is obvious. In this “digital memory” interview from 2014 (in Dutch) he explains that he rejected pseudo-science like graphology, and instead relied on statistics and testing. In minute 19 we see reference to Adriaan de Groot and Otto Selz, whom we mentioned too.

Interview with Pieter Drenth (1935) in 2014 (screenshot)

Interview with Pieter Drenth (1935) in 2014 (screenshot)

This is Drenth’s book on testing theory, 4th edition. We observed that psychometricians in Holland have been testing arithmetic skills at elementary schools while neglecting to study the relevant field of science, namely didactics of mathematics. I have not checked how Drenth’s book deals with validity, perhaps it can never be clear enough.

Quite to my regret, Drenth was president of KNAW in 1990-1996, when I sent KNAW these letters (in Dutch) about the breach of integrity of science by the directorate of the Dutch Central Planning Bureau (CPB), i.e. the reason why this weblog advises to a boycott of Holland till the issue is resolved.

The letters are addressed to the board of KNAW. This would include its president Drenth. Reply came from director Moen. (A contact-person has also been Dirk de Hen, now webmaster at ENRIO: European Network of Research Integrity Officers.) (For these letters, see here under “KNAW”.)

Back then, I only had digital copies of my own letters, and it wasn’t yet so easy to scan the hardcopy replies. I haven’t found the time for this scanning and wonder about the use of it. The reply generally states: that KNAW does not see itself as an involved party, within a juridical issue between an employer (CPB) and an employee (me).

[Update 2015-12-07: Drenth informs me that he has no recollection of having seen these letters or issue, and that he presumes that director Moen has dealt with them administratively according to established procedure: which he was responsible for in 1990-1996 and which he still agrees with.]

  • This KNAW reply fully misses the notion that the CPB directorate abuses employment law to enforce this breach in scientific integrity.
  • This KNAW reply doesn’t explicitly state that KNAW deals with the universities and not with institutes like CPB at the central government. However, KNAW could have taken a lead role in giving advice to the government and the judges, even unasked for, about what matters in cases of integrity of science. Thus Drenth was at an ALLEA conference in 1996 speaking about “freedom and responsibility” but he wasn’t in the courtroom explaining to judges that putting me in a separate room and blocking discussion of my papers isn’t exactly freedom of scientific research.
  • [Update 2015-12-07: That Drenth says that he didn’t see these letters doesn’t change his responsibility for the policy of neglect, and thus there is no reason to change the observation of 2015-11-26:] Drenth’s dismal hypocrisy may have set the example for later KNAW presidents like Robbert Dijkgraaf (now at IAS, Princeton).
  • This KNAW reply in 1992-1995 doesn’t specify whether the board has asked for informal advice from economists, e.g. the fellows at KNAW itself. Drenth at VU might have asked VU professor of economics F.A.G. den Butter. Den Butter happened to be involved in a wrong manner, see here 1991 and here 2013. There is no information about such potential contacts. But it seems rather clear that Den Butter didn’t ask KNAW to get involved and protect the integrity of science.

Thus, unfortunately:

  • Drenth’s presidency of KNAW 1990-1996 and of ALLEA 2000-2006 was a failure and no success.
  • Drenth’s international prestige as a defender of scientific integrity makes it very difficult for me to explain to the rest of the world that there is such a case in Holland and that Drenth has been neglecting it. Strong points in my argument are: (1) The ALLEA code of conduct is deficient in both logic and understanding of scientific integrity, (2) It isn’t so difficult (psychometrically) to test that there is censorship of science by the directorate of CPB: see here and here (in Dutch).

PM 1. It isn’t known whether the situation in other countries w.r.t. integrity of science was so dismal that Drenth caused an improvement anyhow. PM 2. It isn’t known either whether other countries were hesitant to join in. The approach to limit “research misconduct” to only some categories may have pushed the wrong buttons to persuade others.

Update 2015-12-07: Drenth’s reaction allows to reformulate the following aspects more clearly.

I am trying to understand what is happening. How can someone with an interest in science neglect this censorship ? Watchers of the video who wonder about this question too are advised to pay attention to the following aspects.

(i) When asked to become secretary at KNAW, Drenth says that it is a body that doesn’t fight for a single university but

“for science for the whole of The Netherlands, you may say”.
(Dutch text (minute 55): “Het leek mij toch ook wel weer aardig om de KNAW [te doen], dat is een geheel nieuw orgaan [voor mij] waarin je niet vecht voor één universiteit maar voor de wetenschap van heel Nederland, zeg maar.”)

Indeed, when there is censorship of science in Holland then one would expect to hear a protest by KNAW. But there wasn’t. This weblog text substitutes “President KNAW 1990-1996” with “Pieter Drenth”. There is no objective to evaluate professor Drenth’s scientific career and achievements, but only to clarify that there is dismal hypocrisy w.r.t. the censorship of science by the CPB directorate. It is important to make the substitution, because Drenth proceeded as president ALLEA 2000-2006 and has become an icon of scientific integrity: which is a huge misconception.

(ii) The interview shows Pieter Drenth as someone who always worked hard and didn’t have to apply for jobs since others asked him whether he would like a job, do a Ph. D., become professor, dean, rector magnificus, president. (iii) He always worked in the tailwind of the mainstream developments in psychological testing. (iv) His knowledge of mathematics is likely to be at what is now a bachelor level, but it would still have gained the admiration of those around him who didn’t have this, and he has been at risk of some vanity. (v) His “conflict” with the “powerful” graphologists was quite a sensation for him since it seemed “huge” but the facts are a bit different – see below. (i) – (v) For him it might be conceptually difficult to understand what it is, when a directorate turns against science, abuses its power, and censors science.

Ad (iii): Judgement about tailwind or headwind requires knowledge of the history of psychology as a science in Holland. I do not have that knowledge. My intention is only to alert viewers to the issue. My own impression is that there is a tailwind. I am responding largely to this video itself. The career itself is indicative too. I consider it a tailwind when Snijders and De Groot ask him to write a book on Test Theory, since this meant from the beginning a support from two key figures. (At minute 41 in the video there is talk about translating APA rules, that causes Drenth to remark that there is a greater need for an introductory book, and then they ask why he doesn’t write it himself. It is not clear why they didn’t use an American book, or translate it, though English was already the language of science.)

Ad (iv): I have done no magic test at distance of Drenth’s math skills in past or present. I just warn viewers of the video: when the video speaks about mathematical skills, surpassing those of his contemporary fellow students in psychology around 1945, then viewers should be critical about what that level actually was, and that it might suffice to be at the level of a current bachelor. I never saw the book “Testtheorie”. It is intended as an introduction. It is used e.g. in the 2nd year of pedagogy at RUG.  It might be a valuable book for bachelor students. I am not judging professor Drenth’s mathematical skills by this book, and neither of co-author prof. dr. K. Sijtsma (currently dean in Tilburg). Authors of books for bachelors tend to have more knowledge of the field than bachelors, and for mathematics this might now be professor Sijtsma.

I didn’t say that Drenth is vain but at risk of vanity. In the video interview at minute 43 he states with a smile that students liked new editions of his textbook on Test Theory less when there appeared more formulas in them: why would this be a cause for smiling ? My effort as a teacher of mathematics is that students should be comfortable with formulas. (This very weblog text alerted Drenth to the problem that research mathematicians (RM) pontificate on mathematics education research (MER) that they are not qualified for. His reaction is that this is annoying but he doesn’t hold this to be a breach of ALLEA research integrity. Again we see the difference between a focus on so-called “research integrity” and proper scientific integrity.)

Ad (v) In the video he dates his “crusade against graphology” in 1968-69, but he informs me that it was in 1963. Originally I stated: “the graphologists were already losing from international science and its computers”. We can subtract 5 years from this statement. In 1963, we are still speaking 18 years after the end of World War 2. There had been quite some technological development, see this other video on the invention of “time sharing“. Yes, graphology was still “important” in Human Resources Departments around 1963 but a check shows that A.D. de Groot did some critical tests in 1947 and that the approach was stopped by the thesis by Abraham Jansen (1963), “Toetsing van grafologische uitspraken: een experimentele studie”, written under supervision by A.D. de Groot. (Skepsis, interview De Groot 2004 page 19 second column). There is also a report of a working group in that year, that likely relates to this thesis. I think that Drenth has had tailwind.

Video minutes 43:30-45:00 gives his assumption that his own “crusade” against graphology helped in eradicating the approach. He states that his crusade started by three large articles by graphologist Wolters in “NRC-Handelsblad” (though this newspaper came about only by a merger in 1970), with his reply in three large articles. He “got later support from the thesis by Jansen”. Everyone who stops for a moment of critical consideration should agree that it was rather De Groot in 1947 who started this, and that the scientific evidence given by the thesis by Jansen would have weighed more than the three newspaper articles by Drenth, or his later public discussion with Wittenberg, chair of the association of graphologists.

Remember that this is not a discussion about graphology. It is a discussion on Pieter Drenth not admitting: “I was deluded and didn’t do anything about the censorship of science by the CPB-directorate since 1990 while KNAW should have done so. Let me inform the current president of KNAW that they don’t make the same mistake now.”

Cover of the 1973 English translation of Jansen 1963

Cover of the 1973 English translation of Jansen 1963

Addendum 2015-12-16

Professor Drenth replied to the above on December 2, which gave me the opportunity to enhance clarity (see the edits). I gave him bilaterally my draft response to other points that still caused questions, so that he could rephrase a final reply. Drenth rejects a change in his reply. Thus here is the additional memo:

Why ALLEA doesn’t see censorship of science in Holland
Appendix A: reply by Drenth, Appendix B: email exchange

Drenth rejects the idea that he misjudged the censorship of science in Holland. Drenth doesn’t focus on the suggestion of reviving Forum Theory (A.D. de Groot). He states explicitly that he regards the final section of this weblog text as a personal attack. My response is that this latter statement is a breach of scientific integrity. Denouncing the critique as a personal attack may cause other people to neglect the criticism. There is no personal attack. There is objective reason to discuss the role of Drenth as president KNAW and ALLEA, with his preference for the legal approach and my preference for the scientific approach. One wishes to understand where his policy preference comes from. The video that I found is not helpful, since it is not critical, and thus there are ample reasons to warn viewers about points to be critical about. It is still a mystery why Drenth, who claims to defend science, doesn’t do so in the cases that I mention: CPB and education of mathematics.

Advertisements

Comments are closed.

%d bloggers like this: