Quality, Methodology, and Politics in Doctrinal Legal Scholarship

• 1. Introduction

  Doctrinal legal research has been declared dead before and would,¹x Posner, E., ‘The Rise of Interdisciplinary Approaches in American Legal Scholarship’, keynote address at the University of Tilburg Symposium, Towards a European Legal Methodology?, quoted in: van Gestel and Micklitz (2014). according to less pessimistic scholars, be in a state of crisis (Smits, 2012, introduction). Meanwhile, law journals and legal publishers around the world keep publishing this type of research. Hence, one could ask: Is there really a problem? The answer to this question probably depends on the perspective one takes. On the one hand, there are indeed sufficient outlets to publish doctrinal legal research and this type of research remains to be highly valued by legal practice. On the other hand, there appears to be a downfall in certain countries, including the Netherlands, with respect to the academic appreciation of doctrinal research in comparison with more multidisciplinary and empirical work.²x See about the relationship between doctrinal research and empirical legal studies: Davies (2022). Finally, there has also been debate about the extent to which the methodology and quality criteria for doctrinal legal publications should be made more explicit.³x See for example van Gestel (2017).
  Regarding the latter, perhaps the most difficult problem is how to take the normative character of doctrinal legal research sufficiently into account. In terms of methodology, for example, one cannot require from legal scholars who commit to doctrinal work that their research should, for example, be replicable (Hutchinson & Duncan, 2012), simply because legal scholars may attach different weight to the same legal arguments, which does not necessarily affect the quality of these studies because there might not be one ‘right’ answer and it is ultimately up to the forum of legal academics to decide which answer they find the most convincing. Simultaneously, though, this does not imply that in doctrinal legal research ‘anything goes’. Avoiding flawed or untruthful claims is also vital for doctrinal legal scholarship, especially in what some have called a ‘post-truth era’ (Keyes, 2004), in which disinformation, fake news and scholarly activism may threaten people’s faith in academic (legal) research (Khaitan, 2022).
  In light of the task to uphold public trust in legal scholarship, one should require from all legal academics to be more open and explicit about their theoretical perspective and research methods. For example, there is no reason why doctrinal legal scholars (hereafter doctrinalists) could not present an explicit research question, explain what they want to add to the body of knowledge⁴x The suggestion made by some that explicating a research question and/or explaining some basic methodological steps would not be possible in doctrinal research is in my view demonstrably false. Otherwise: Tjong Tjin Tai & Verbruggen (2022) referring to van der Burg & Taekema (2020). Also the fact that one sometimes needs to adjust one’s research questions during the course of the research occurs on a regular basis in both doctrinal and empirical research and is therefore not idiosyncratic for doctrinal research. and ensure that their conclusions are based on a thorough examination of legal materials without leaving aside relevant sources that may disconfirm a desired outcome.⁵x In one of the first studies I participated in on methodology, I discovered that these basic criteria were often not followed by the authors of articles published in the science section of the biggest Dutch law journal. See: Vranken & van Gestel (2007). Simultaneously, I have doubts about the willingness of doctrinalists to become more transparent about how they actually conduct their research. The reason is that I have heard plenty of excuses over the past decades why it would not make sense for doctrinalists to open up about this such as: there is no methodology in doctrinal scholarship other than textual interpretation, doctrinal legal research belongs to the humanities and other disciplines within this field are also not explaining their methods, and the quality of doctrinal scholarship is so interwoven with the use of language and the way practitioners view the law that one cannot separate the methods of legal scholarship from those of practitioners. I believe these excuses are falsehoods and attempts to escape accountability. A relevant question is therefore:

  What may explain the fact that the quality criteria, and especially those related to methodology, for doctrinal legal research are not being made more explicit, thereby enabling journals, publishers and research assessment bodies, to filter out substandard research?

• 2. Approach and Order of the Argument

  Before addressing this research question, I want to express my own position in the debate, which will hereafter primarily be based on the experience with Dutch legal scholarship, although I think that much of what I am going to share may also apply to doctrinal legal scholarship in other European countries. My own starting point is the belief in a multiform legal discourse. This implies that I have as much esteem for doctrinal legal scholarship as for empirical legal scholarship, comparative legal research, law and economics or any other type of legal research. I am not aware of belonging to any particular ‘school of thought’ and would qualify myself as a methodological pragmatist, who happens to believe that doctrinal legal scholarship may not survive in the long run if doctrinalists remain reluctant to explain their research methods and quality standards to members of other academic disciplines. Simultaneously, I do not see much added value in a renewed debate about the ‘true’ nature of law as a discipline, which is closely linked to the question whether law should be seen as a science or a craft. In my view, the answer to the question whether law is a science, and if so, what kind of science, ultimately depends on one’s definition of science. In a hard science view, law probably does not qualify, while from a broader soft science perspective, it does. Various books and articles have been written about this, but I do not see much progress in this debate (see for example: van Hoecke, 2011; Feldman, 2009; Alda, 1999).
  From a more pragmatic perspective, it is a given that law schools across the world occupy scholars conducting different types of research: doctrinal, empirical, legal-historical, comparative, law and economics, etc. To label some of these subdisciplines as academically irrelevant does not make sense to me, simply because they are there and provide a lively discourse for thousands of academics in universities. One might just as well argue that it is a sign of a blooming discipline that new subfields of law arise on a regular basis. More important to me is that the choice of research methods should follow one’s research questions. In my opinion, there cannot be one exclusive legal research method because the sort of questions that legal scholars try to answer differs. As a legal scholar, one might have a personal preference for a certain type of questions such as more applied or theoretical, more normative or empirical, more descriptive, explanatory or philosophical and so on. However, the research question a scholar chooses to answer should steer the methodology and not one’s personal preferences, beliefs or convictions.
  A normative legal ‘should question’, for instance, can never be answered satisfactorily by adopting a socio-legal or empirical approach that looks to explain differences between the law in the books and the law in action and a question about the effects of the law on human behaviour cannot be answered via a normative approach regarding the legality or legitimacy of this behaviour. This is what I call methodology with a big M; certain types of research are (un)suited for particular sorts of legal research questions. In addition, there is methodology with a small m, meaning that even within one particular approach (e.g. doctrinal, historical, comparative, empirical), certain methods and techniques can be more or less suited to answer a specific research question. In case one would like to learn whether certain lower courts deviate from the jurisprudence of the Supreme Court, a case law analysis makes more sense than studying legislation. Nonetheless, even within the realm of case law analysis, different methods are possible and some might be more suited than others depending on what one wants to get out of the case law. Studying one particular case in depth might very well be done by applying traditional textual interpretation methods, but if one needs to study thousands of cases in order to discover whether there are certain general patterns in the case law of a court, using coding techniques and computer software might be a better idea because a digital analysis of a huge amount of cases might be done more systematically and rigorously via coding techniques and the use of software than by using close reading in combination with traditional textual interpretation methods.⁶x More about the differences between traditional doctrinal case law analysis and methods such as structured content analysis can be found in Verbruggen (2021) and Salehijam (2018).
  What I am going to present hereafter is an essayistic sketch of the relationship between methodology and quality of doctrinal legal research (Section 3), for which I will lean on prior research conducted by Snel who has provided an overview of the quality criteria for doctrinal legal research derived from the literature, on the work by Vranken, being one of the pioneers in the Netherlands who put the methodology of doctrinal legal scholarship on the agenda,⁷x Most comprehensively in Vranken (2014). For a brief English summary of his argument, see: Vranken (2016). and on my own work on legal publishing over the years. Following Vranken’s footsteps, I am going to argue that the lack of attention for methodology in doctrinal legal research partly stems from the fact that many legal scholars still do not discern between methods of judicial law-making and methods of academic legal research (Section 4). This is also why doctrinal legal research is often wrongly associated with an internal (participants) perspective, although legal academics normally benefit more from adopting an external (observers) view towards developments in legal practice.
  This last aspect also has to do with a common flaw that one still finds in many doctrinal legal publications because of a too one-sided focus on making normative recommendations, which easily results in ‘advocacy scholarship’, which is basically a specific form of confirmation bias (Section 5). My argument is that more attention for methodology and theory-building may, to some extent, serve as an antidote to advocacy scholarship (Section 6). Last but not least, I will discuss some reasons for the still existing scepticism towards making the quality criteria and methodological ground rules for doctrinal legal research more explicit having to do with certain ‘politics’ in legal research due to the fact that law school managers try to please stakeholders with different interests (Section 7). Finally (Section 8), I will draw some conclusions.

• 3. Methodology and Quality of Doctrinal Legal Publications

  Snel has argued that there is something strange about the fact that, although law is one of the oldest academic disciplines, one finds a very limited amount of literature about the quality indicators relevant to assess doctrinal academic legal research (Snel, 2019, p. 2).⁸x I focus on academic doctrinal legal research here because doctrinalists also publish professional legal publications, which often lack the ambition to add knowledge to the state of the art and are usually focused on applying existing knowledge. Nevertheless, he has succeeded in summarizing some of the most important quality indicators for doctrinal legal publications:

  a traditional legal publication must (1) contain a concretely and precisely formulated research question or hypothesis that allows for (2) an original and (3) significant contribution to both society and academia and that (4) is substantiated with adequate methods and research techniques for answering it. Subsequently, in composing the answer to a traditional legal research question, the researcher must (5) use precise and correct references and justify important choices made, (6) be accurate in the sense that the arguments and materials used must actually contribute to answering the research question posed, (7) operate in a balanced way – a complete or representative use of research materials and a fair distribution of weight to the different arguments and insights present within these materials – and (8) achieve a high level of credibility in her descriptions, interpretations, evaluations and use of theoretical constructs. Finally, a traditional publication must also be (9) readable and (10) persuasive, and (11) may not – at least not unaccounted for – be influenced by the presuppositions or assumptions of the author herself. (Snel, 2019, p. 20)

  One of the problems with this summary is, unfortunately, that is does not make clear what ‘adequate methods and research techniques’ for doctrinal legal publications are. That is understandable because, as already mentioned, one cannot in general determine which small m methods and techniques are suitable for this type of research since that will depend on the particular research question(s) the author attempts to answer. However, this presupposes that doctrinal publications actually have an explicit research question, which has certain features such as that it is delineated and makes clear where the publication intends to add something to the ‘state of the art’ and indicates the type of research (e.g. comparative, descriptive, predictive etc.) that will be required. Otherwise, it would be impossible to evaluate whether the chosen methods are suited to answer the research question. What is also needed is that an author describes how the chosen research question is going to be answered in order for the reader to be able to roughly follow the path the author has travelled to reach a conclusion. Doing so will again enable the reader to assess if there are obvious mismatches between the research question and the methods and techniques used to answer them. In other words, without a proper research question, it makes no sense to be transparent about one’s research methods because it will be difficult, if not impossible, to assess whether the chosen methods are suitable to deliver a valid conclusion to the publication under scrutiny. This requires a certain level of detail. To give a practical and simple example, compare the following research questions:

  1) To what extent do digital messages count as forgery in the sense of Article 225 of the Dutch Criminal Code?

  Versus:

  2) What criteria have Dutch District courts and Appellate courts used from 1991 onwards to fill-in the criteria of readability and durability as developed by the Supreme Court regarding the interpretation of what may count as a ‘writing’ referred to in Article 225 of the Dutch Criminal Code and could Snapchat messages that are automatically being erased by the provider of this service after a while count as forgery according to the case law of these courts?

  Not only is the first question broader and less precise than the second one, it also appears to be a question that is merely asking to describe the current state of the law instead of adding knowledge to the state of the academic debate. The second research question, by contrast, does identify a gap in the body of knowledge. After all, there is case law from the Dutch Supreme Court setting criteria under which digital messages may, under certain circumstances, count as forgery, but it is unclear how this jurisprudence has been interpreted by lower courts in relation to how a particular form of digital messaging (Snapchats) relates to the Supreme Court criteria defining what counts as a writing (e.g. is a temporarily existing digital message sufficiently durable?) in the sense of Article 225 of the Criminal Code. The second question is also more informative about the expected methodology since it indicates that the author needs to study the interpretation by the District and Appellate courts of the criteria of readability and durability set by the Supreme Court in 1991. This is necessary to get more clarity about what they mean and how this would turn out when confronted with the technical features of Snapchat messages, including the fact that those messages are automatically erased after a certain time.⁹x HR 15 January 1991, ECLI:NL:HR:1991:AC3982.
  Apart from the fact that many doctrinal legal publications do not have an explicit research question, often they also do not possess a methodology paragraph. Even if there is one, it usually does not reveal very much with respect to what the author has actually done to answer the research question. For example, just mentioning that one has studied case law, legislation and literature is not very helpful. In the examples given above, the reader would probably like to know which cases have been selected (e.g. digital searches in certain databases using keywords or relying on cases mentioned in the literature or …?) and how these have been analysed (e.g. did the author apply some sort of assessment framework to systematically screen the case law?). Furthermore, how did the author deal with potential differences between the case law of District courts and Appellate courts (e.g. one strategy could be to use the District court’s case law in order to look for refinements in the Appellate courts case law and to detect potential criteria that have not been mentioned in the literature yet). The same holds true for literature and legislation. Which publications has been selected and how? The legislative history of Article 225 might certainly be relevant, but here one would also like to know whether the author has done more than studying the explanatory memorandum to the Criminal Code. After all, the explanatory memorandum represents the views of the government and not those of parliament, while ‘the legislature’ is a combination of both.
  A counterargument could be that the author’s footnotes reveal the source they used. Some believe that explaining the way the sources have been collected and studied would have little added value and diminish the legibility of legal publications. To a certain extent, the latter may even be true. Nieuwenhuis, for instance (Nieuwstadt & Schagen, 2007), has made the comparison between writing an article and building a house. In the case of a house, the builder would remove the scaffolds after completing the project. Something similar might apply to the methodology of a traditional legal publication, which should be left aside once the publication is finished. The problem with this analogy, however, is that buyers of a house can see for themselves whether the building has cracks or looks otherwise unstable, whereas in the case of a doctrinal legal publication, only another expert who has done similar research will be able to detect whether, for instance, certain relevant sources are missing in the footnotes. This requires a high degree of expert knowledge about the topic.
  Certainly not every potential reader of doctrinal publications will possess expert knowledge about the topic under investigation. Nevertheless, those readers may still want to have certain guarantees that the author did an honest, thorough and systematic analysis to answer the research questions. How is that possible if the author remains silent about what (s)he has actually done to answer the research questions? How important methodological accountability is can be witnessed by looking at Wijntjens’ study concerning apologies. She studied the legal consequences of apologies offered by doctors to patients in case of medical mistakes after treatment in a hospital. By analysing over 4000 cases applying a method called ‘structured content analysis’, which consisted of coding case law and systematically analysing patterns via the use of software, she was able to show that the claims made in a number of doctrinal publications that offering apologies could be seen as accepting civil liability by judges were largely false (Wijntjens, 2020). This is not only an important new insight. It may also say something about the case law selection and case law analysis by the doctrinalists making these claims.
  The importance of making implicit choices in the collection and analysis of legal sources in doctrinal legal research more explicit has become even more important since traditional national fields of law are increasingly influenced by supranational law (e.g. the case law of the CJEU or ECHR), which is not always immediately clear for everybody when it comes to answering a particular doctrinal legal research question that may look like a purely national legal matter at first sight. Therefore, a reader may want to know: (how) did the author check whether EU law or ECHR law applies to the research question at hand? This is particularly relevant because it is not always so easy to verify if there is no additional supranational law applicable in a certain case. It may sometimes feel like ‘looking for a black cat in the dark’.

• 4. Where Does the Current Lack of Attention for Methodology by Doctrinalists Come From?

  Several answers could be given why doctrinalists currently seem to have a disregard for methodology.¹⁰x To mention a few other possible answers: legal scholars do not consider themselves to be ‘scientists’ but as (legal) scholars (‘rechtsgeleerden’) standing in a hermeneutic and philosophical tradition, legal scholars seek recognition from practitioners who are less interested in methodology, legal scholars are focused on convincing others by arguments and are more practical and result-oriented than those who want to build new knowledge. The most important one, though, is probably the traditional interwovenness – and often mix-up – between professional and academic legal research. Vranken has shown that most doctrinalists in private law still identify themselves with the role model and the methodology of the judge who also interprets legislation, looks at existing case law and studies literature to answer legal questions.¹¹x An interesting example is the book by Schauer (2009), which is equating the reasoning of legal scholars with that of legal practitioners. He also argued, though, that this identification is misleading, if only because judges are not allowed to choose their own legal questions but are bound by the dispute that the parties bring to the table; judges are not focused on adding knowledge to the state of the art but concentrate on deciding disputes; judges do not collect their own data and evidence but lean on the litigating parties to do so and are bound by rules of evidence in terms of the ‘data’ they can and cannot use; judges also have less freedom in the research methods they apply.¹²x See for an English summary of Vranken’s main arguments: Vranken (2016).
  The identification between thinking like a legal scholar and thinking like a judge or another legal practitioner can also be witnessed in the debate concerning what should count as an academic legal publication instead of a professional one. This is caused, at least partly, by the fact that legal practitioners and academics may both teach law, often visit the same conferences and frequently publish in the same law journals in many European countries. Apart from this, there is a lack of consensus about what should set academic legal publications apart from professional publications (van Gestel & Snel, 2019). Is it (only) that academic legal publications should add knowledge to the state of the art or perhaps that academic research should be more thorough or methodologically sound? What complicates things is that universities may even have an interest in not drawing a sharp line between academic and professional publications, because it enables them to maintain a dialogue with legal practice. Moreover, in many European law schools, scholars are partly funded via contract research for administrative agencies, companies or NGOs. It would be difficult to maintain this if the outcomes of that research do not count as academic publications for the purpose of, for instance, institutionalized research assessment exercises with an impact on faculty funding (Kaltenbrunner & de Rijcke, 2017).
  The conflation of academic and professional research may also explain the internal point of view taken by many doctrinalists, who sometimes seem to act more like participants in the process of law-making than as external observers, trying to explain, compare, predict, or otherwise analyse what lawmakers are doing. Again, this contributes to the explanation why academic legal scholars are not used to make their implicit research methods more explicit, because practitioners are also not used to do this when trying to answer legal questions. Judges, for example, motivate their decisions by referring to legislation, prior case law and, sometimes, literature, but very rarely do they show how they have selected these legal sources or weighed the evidence brought by the litigants. Mostly they will restrict their motivation by explaining how the decision of the courts fits in the legal system. However, doctrinalists are not confined to an internal perspective on the legal system. They may choose to compare legal systems, they can focus on questioning why the law has developed in a certain way or introduce new concepts to change the law. In other words, the consistency and coherence of the legal system does not have to be the only, or even the most important perspective, for doctrinalists. In fact, for legal scholars there is no such thing as the legal system. There are countless legal subsystems that are constantly changing in our multi-layered legal order and doctrinalists can rethink these subsystems and doctrines. Other than judges, they are not limited by the boundaries of positive law, which also enables them to provide recommendations for law reform.
  To give an example of the latter, private (self)regulation is often not considered to be law because it does not fulfil some of the features of ‘regular law’. Nonetheless, legal scholars may defend that what counts as law today should no longer be limited to what the legislature or the courts recognize as law on the basis of formal criteria such as the involvement of democratic representatives in the process of law-making or the publication of laws in the official journal. Instead, scholars might argue that because of the increasing importance of self-regulation, for the business community to effectively address problems in transnational supply chains, we should be more open to accept new sources of law, especially when the parties in a civil law dispute acknowledge the relevance of these rules for their operations (Menting & Vranken, 2014). That way codes of conduct adopted by multinationals, for instance to show respect for fundamental rights, might be used in court to hold companies accountable for the violation of such rights even though these rights normally only apply in vertical relations between governments and private actors.¹³x A prominent recent Dutch example where this has happened is: Rb. Den Haag 26 May 2021, ECLI:NL:RBDHA:2021:5337 (Dutch text) and ECLI:NL:RBDHA:2021:5339 (English translation) (Milieudefensie et al. v. Royal Dutch Shell).

• 5. The Risk of Emphasizing the ‘Art of Persuasion’

  An underestimated risk in doctrinal legal research when scholars take the previously mentioned internal (participatory) perspective in combination with a strong focus in legal education on ‘authoritative’ sources,¹⁴x Meaning, for example, that appellate court decisions are considered to be more important than district court decisions and supreme court decisions more important than appellate court decisions and so on. on the building of arguments to persuade others and on the importance of providing normative recommendations (‘de lege ferenda’) to the legislature or the judiciary, is that advocacy becomes more important than independence and impartiality. Or as Spitzer has formulated it for the United States:

  In no other academic field of inquiry, whether the natural sciences, the social sciences, or the humanities, is the value of truth-seeking subordinated to the kind of values that are the foundation of legal training: to win an argument, to put the interests of the client/employer above those of the truth, or to maintain confidentiality regardless of its consequences. Law professor Anthony T. Kronman put it succinctly: ‘Scholarship … aims at the truth. Advocacy, by contrast, is concerned merely with persuasion.’ (Spitzer, 2008, p. 23)

  Of course one could argue that the Socratic method applied in U.S. law schools is quite different from the educational system in most European law schools. Nevertheless, I have doubts whether, especially in research, the situation here is so distinct from the one across the Atlantic. Particularly in fields with a very strong normative, ethical and policy-oriented dimension such as human rights law, environmental law, feminist law or migration law, there is an inherent risk that academic research turns into ‘advocacy scholarship’, which basically means that scholars are only looking for evidence to support the preferred answers to their research questions. Coomans, Grünfeld and Kamminga, for example, have written about methodology in the human rights discourse:

  human rights scholars tend to passionately believe that human rights are positive. Many of the scholars are activists or former activists in the field of human rights. Although seldom stated, the explicit aim of their research is to contribute to improved respect for human rights standards. They therefore risk ignoring the fact that the pursuit of human rights is not a goal in itself, but is merely one instrument designed to help improve respect for human dignity. They may forget that human rights standards are the result of compromises reached by states and may therefore be less than perfect. They may also overlook the fact that the mere adoption of resolutions by international bodies and the establishment of a new international institution will not necessarily result in the improvement of human rights on the ground. (Coomans, Grünfeld & Kamminga, 2010, p. 182)

  While this still seems a rather innocent statement – after all, is it so strange that human rights scholars are passionate about human rights? – the question is, more in general, what the emphasis by doctrinalists on persuasion, legal recommendations, and law reform in combination with a lack of attention for methodological rigour, does to the academic core values of objectivity and impartiality. Once again, U.S. legal scholarship might serve as a cautionary tale. Chilton and Posner, for example, studied whether they could find ideological preferences in American constitutional law scholarship by studying the extent to which professors from first tier law schools that donated to the Democratic Party favoured more liberal standpoints in their publications than professors donating to the Republican Party. They used a coding scheme to systematically screen liberal and conservative oriented positions in constitutional law. In doing so, Chilton and Posner discovered that, although most scholars claimed to be led primarily by the strength of doctrinal legal arguments, there is a correlation between the funding of political parties and liberal versus conservative views in constitutional law scholarship. This led them to conclude that:

  Law professors, unlike other types of academics, directly influence the law by writing articles that judges read and occasionally cite in judicial opinions. However, if their articles are seen as ‘rationalizations of their authors’ political ideology’ (Posner 2009), they may well lose whatever influence they have. (Chilton & Posner, 2015)¹⁵x Interestingly, though, Rachlinski also found a ‘mild correlation’ between the political beliefs of authors of empirical legal journal articles and the outcomes of their research. See: Rachlinski (2018).

  One might argue that this type of bias is more relevant for the politicized academic legal climate in the United States, but I do not see why the situation in European countries would necessarily be so different. In the Netherlands, for example, politicians from the Dutch party, Forum for Democracy, have argued that universities would be too leftist. The Royal Academy of Sciences (KNAW) wrote a report, downplaying this criticism, arguing that while political scientists might be leaning to the left side of the political spectrum, scholars in the field of economics and management would probably be more right-wing oriented (KNAW, 2018). The only threat the KNAW saw was that academics would become too dependent on sponsoring, because there is insufficient money for blue-sky research, which increasingly drives them towards contract research. In his opening address for the 2018 Diës Natalis of Leiden University, Rector Magnificus Stolker, however, shared his doubts concerning the conclusions drawn by the KNAW. He particularly mentioned law as an example of a discipline in which academics often have very strong normative, ethical and political views, where research is frequently commissioned and focused on presenting recommendations for ‘better law’. Moreover, Stolker shared his concerns about the impartiality of legal scholarship:

  In addition to being a descriptive science, law is also a prescriptive science, with a rather normative bias. On the one hand there is no single truth about Dutch or European or international civil law, criminal law or fiscal law. And on the other hand, the law in a society – the subject that lawyers at the academy are concerned with – is ultimately a balance between legal certainty, efficiency and justice. Yet, legal scholars are expected to provide answers that must have sufficient argumentative power: the objectification of positions beyond all political and social conflicts is part of the legal scholarly work.’ [translation RvG] (Stolker, 2018)

  It is hard to say to what extent Stolker’s worries about the susceptibility of legal scholarship for ideological biases are justified since there is hardly any empirical research on this. However, there have been quite a number of incidents lately that provide ample reasons for concern. Vleggeert, for instance, argued in his inaugural address that there would be a lack of independency and impartiality in the field of Dutch tax law because most professors would (also) be working for one of the major tax law firms. According to him many of the legal scholars in the field would not write critically about certain subfields in the area of tax law because that would negatively affect their clientele and conflict with the views of their employers (Vleggeert, 2020). Not everybody agrees with that (see for instance: Notenboom & Kooiman, 2021), but a study of the publications of tax law professors wearing ‘two hats’ has shown there does seem to be a lack of research in the field of tax law on certain controversial topics (Notenboom & Kooiman, 2021). Even members of parliament have been complaining that it would be increasingly hard to find truly impartial legal scholars to give independent advice on fraud and corporate tax evasion (see for instance Accountant, 2022 and Vleggeert, 2022).
  The problems with the independence and impartiality of tax law scholarship are not primarily caused by ideological biases. It is probably much more about how financial interests may affect the objectivity of tax law professors with dual roles. This does not imply, though, that ideological biases are not also a problem if most academics in this field seem to have a preference for studying particular topics (e.g. corporate tax) from a particular perspective (e.g. tax evasion), while ignoring other topics (e.g. childcare allowance) because they are less important for one’s clients or practice simply because of the different perspective.¹⁶x The Dutch ‘childcare benefits’ scandal (Kinderopvangtoeslagschandaal), which led to a parliamentary investigation and even a report by the Venice Commission ruling that the government had violated the rule of law by discriminating against certain parents with a different ethnic or cultural background, has led to widespread criticism that the Dutch academic legal community had totally overlooked this field of law, although if affected enormous amounts of citizens and caused grave harm for tens of thousands of families that were badly treated by the Dutch Tax Authority because they were falsely labelled as fraudsters. See about this scandal for example: ten Seldam & Brenninkmeijer, 2021; Heikkilä (2022); Council of Europe (2021).
  I believe it is also fair to say, to mention another example, that the Department of Encyclopaedia of the Law of Leiden Law School was until recently seen by many of their colleagues throughout the country as a strong right-wing bulwark with a preference for certain ideologically loaded topics such as immigration, identity politics, Islamic terrorism, etc. This has led to debate about the objectivity of the research and education by this group.¹⁷x See for example: de Quay (2020); Bormans (2021); Mare Online (2021); de Witt Wijnen & Wester (2021); NOS Nieuws (2020). This is not the place to take position in the debate about an alleged ideological bias of a specific law department of a specific law school, but an interesting defence from certain members of this Leiden group has been that the scholarship in Dutch universities in general would be too leftist, which would make their department a necessary counterweight (see: Korteweg, 2019). I find that a curious argument, because it appears to condone the very practices of which one accuses other scholars, namely being driven more by political convictions than by the values that scholars in general are supposed to uphold such as disinterestedness, honesty and organized scepticism.¹⁸x See about these and other academic values, for instance, Merton (1973).
  More generally, however, one may have doubts about how much room the rhetorical style of much doctrinal legal scholarship, combined with an implicit methodology and a lack of clear guidelines on how to use sources (van Boom, 2015, p. 13), leaves for a search for counter-evidence and falsification of hypotheses that may disprove one’s own normative perceptions of the law. This is a highly underestimated issue in legal scholarship, which could seriously affect the credibility of both conservative and liberal legal scholars conducting doctrinal research (Vranken, 2009, p. 88).

• 6. To What Extent Is Methodological Accountability an Effective Remedy Against Bias and Advocacy Scholarship?

  Although there are many different types of bias in academic legal research, confirmation bias is probably the most important threat to doctrinal legal research. It can be defined as ‘the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand’ (Nickerson, 1998, p. 175). This definition implies there is a strong relationship with one of the core values in academic research, namely impartiality, which can be found in most codes of conduct for scientific research. Researchers are deemed to be impartial when they do not allow themselves to be influenced by their preferences, sympathies, interests or personal prejudices in the execution of their academic work.¹⁹x See for instance the Belgium code of ethics for scientific research on p. 9: www.belspo.be/belspo/organisation/publ/pub_ostc/Eth_code/ethcode_nl.pdf. The Dutch code on scientific integrity sees impartiality as part of the principle of academic independence. See: www.nwo.nl/sites/nwo/files/documents/Netherlands%2BCode%2Bof%2BConduct%2Bfor%2BResearch%2BIntegrity_2018_UK.pdf.
  As mentioned above, much doctrinal legal research is focused on providing normative recommendations to create better law. The problem with that is at least threefold: 1) doctrinalists usually do not explicate their normative framework and criteria to determine what counts as ‘better law’ for the purpose of their particular research project; 2) in combination with the focus on ‘ought-questions’ instead of ‘is-questions’, there may be a tendency to focus on desirable outcomes to one’s research questions. I believe this is also part of the reason why one rarely finds doctrinal legal publications with a negative outcome, meaning: I thought that the answer to my research question would be A, but it turns out to be B; 3) because most doctrinalists do not explain how they collected, selected and analysed the sources they used to answer their research question(s), it is difficult, if not impossible, for a non-specialist on the topic to get a sense of the reliability and thoroughness of the research process. This creates an opportunity structure for scholars to look for support for one’s own normative conceptions and beliefs without paying any serious attention to find counter-evidence or counterarguments.
  Raising methodological awareness and accountability is a necessary, but simultaneously insufficient, remedy against confirmation bias. It is necessary because methodological awareness makes (especially starting) scholars more aware of the risks of biases and how to avoid these. Besides, methodological accountability presupposes transparency. Requiring from doctrinalists who want to provide normative recommendations to make (the selection of) their assessment criteria for what they see as improvements to the law explicit is an important precondition for others to both understand and criticize the research and simultaneously encourage the author to think about the basis for making normative claims. It also stimulates doctrinalists to think more often in terms of hypotheses, meaning: ‘what do I believe should be the answer to my research question, and what is the explanation for this?’ Especially this explanatory part may bring scholars towards theory-building since the reason for an expected answer to the research question is often the starting point for a theory. Let me provide two examples from my own work. One more traditional doctrinal and one that is a combination of doctrinal and empirical research. The second example is just to show there is often a fine line between purely doctrinal research and more law in context or socio-legal research.
  In 2012, the Dutch Supreme Court had to decide whether a reference to private (technical) standards in the Dutch Building Decree (DBD) transfers these private norms into public regulations.²⁰x Dutch Supreme Court 22 June 2012, ECLI:NL:HR:2012:BW0393. This was highly relevant since the Dutch Copyright Act rules that there can rest no copyright on laws, decrees or ordinances issued by public authorities. Two theoretical viewpoints were defended in the literature. The first one might be a called the ‘usurpation theory’ (for example defended by Elferink, 1998), which basically claimed that if a public bill refers to private standards for the interpretation of open texture in the law, the private standards transform into public regulations. The other theory, which can be labelled as the ‘competence theory’ (for instance defended by Evers, 1999), approached the status of the standards as a question of delegation. Since the organization responsible for the drafting of the standards (the Royal Netherlands Standardization Institute: NEN) had not received any competence to draft public regulations on behalf of the legislature, the technical standards must have remained private, which meant that the copyright stayed with the Standardization Institute.
  My hypothesis was that the Supreme Court would opt for the competence theory, simply because accepting the usurpation theory would imply that NEN would no longer be able to charge copyright fees to its clients for the use of technical standards as soon as the legislature would refer to them (van Gestel, 2012). This would probably have caused the NEN to stop producing these standards for public purposes. As a consequence, the production of the standards would from thereon need to be financed from the public purse. This would automatically make non-users of these norms future sponsors of technical standards even if they have no use for them. By introducing a formal opt-out possibility, also used in European directives referring to similar technical standards by European standardization institutes such as CEN and CENELEC, the Dutch legislature would be able to circumvent this economic risk. This enabled the Supreme Court to defend the competence theory arguing that because the Dutch Standardization Institute had no competence to enact legislation, the standards it produced should not change (legal) colour, simply because the legislature refers to the private standards, as long as there would be a possibility for private actors to opt for an alternative to show compliance with the law.²¹x See for the comparison with the European situation also: van Gestel & Micklitz (2013). This is also what happened: the Dutch Supreme Court basically adopted the same approach already followed by the European legislature, thereby corroborating the hypothesis.
  My second example deals with the opportunity for national courts to refer preliminary questions to the Court of Justice of the European Union (CJEU). While raising preliminary questions national courts also have the opportunity to offer the Luxembourg court so-called provisional answers, which represent the referring court’s own interpretation concerning how EU law should be interpreted in the particular case that is before the national court. However, few courts systematically use the opportunity to provide provisional answers. An interesting question for scholars working on EU law could be: Why is it that national courts seldom make use of the opportunity to offer the CJEU with provisional answers?
  One possible explanation could be that referring courts simply have no clue what the right interpretation of EU law should be, because otherwise they would not refer preliminary questions. Another explanation could be that national courts are afraid to get it wrong, because it would be embarrassing for them if the CJEU came up with an entirely different answer to the question of EU law, while a third potential explanation might be that national courts do not make use of this opportunity because it could make them look biased towards the litigating parties regarding what the referring court prefers to be the right interpretation of EU law that is relevant to their case. Studying the case law and the literature, perhaps with some additional interviews,²²x I realize that conduction interviews would go beyond traditional doctrinal research entering the field of empirical research but the example is still relevant because one cannot always know beforehand whether a purely doctrinal analysis will offer the most convincing answer to a normative question. It may also depend on what information the case law, legislative history and literature provide. may reveal what is the most likely answer to the question why the use of provisional answers is unpopular.²³x See for a discussion of the answers to this question: van Gestel & de Poorter (2019).
  These examples reveal the importance of trying to come up with different possible explanations or theories regarding the answer to one’s research question(s). Exploring and, where possible, ‘testing’ different hypotheses offers an additional safeguard against confirmation bias (Schumm, 2021, pp. 286-287), because it urges the author to consider more than one possible answer that he or she might prefer at first glance. I realize that real testing of hypothesis or falsification of normative questions is not possible in the way that happens in the natural sciences where researchers, for example, conduct experiments. However, this does not imply that doctrinalists cannot discipline themselves more in the process of looking for counter-evidence and counterarguments. Thinking in terms of hypothesis and challenging oneself to disprove certain assumptions regarding what probably should be the answer to a certain normative research question is also possible and useful in doctrinal research, as I have witnessed time and again working with students in methodology courses.
  Something similar applies with regard to enhancing transparency concerning the collection, selection and analysis of legal sources. I am not in favour of the introduction of an American blue book style of referencing where even the most futile claim needs to be supported by a footnote – I once read about an author who had to provide a footnote for the claim that Plato was an important philosopher – but that does not mean that doctrinalists could not benefit from a few more specific guidelines regarding when, why and how to use legal sources. Currently, the ‘rules’ that we have on referencing are mostly formalistic. Journals and book publishers normally require a certain style of referencing, but one finds surprisingly few guidelines concerning how legal scholars doing doctrinal work need to collect, select and analyse the ‘data’ they use and to what extent they should provide any justification about this process in, for example, a methodology paragraph of an article or book they want to publish. Nonetheless, the advantage of a set of guidelines could be that it would make legal scholars more aware of the fact that the answers to one’s research question need to be something more than a personal opinion supported by sources that happen to confirm that opinion without looking more systematically for sources that might disconfirm this answer.
  Of course I do realize that methodological rules and requirements to make the research process with regard to doctrinal publications more transparent are certainly not sufficient to prevent confirmation bias and advocacy scholarship. We also need to do more and other things. The most important thing, in my view, is to organize more (self-) criticism in the publication process. This is not only a responsibility for journals and publishers, responsible for the peer review of draft publications, but it should be something that becomes better ingrained in the training of law students and starting legal researchers. The best way to prevent substandard research from being published is to organize (self-)criticism via methodology training, peer panels and opportunities to present draft publications prior to submitting manuscripts to journals or publishers. Here, the proof of the pudding is probably in the eating. The sooner legal researchers start to realize that organizing criticism may benefit the quality of one’s scholarly publications, the more likely it becomes that peer review is going to be accepted as an opportunity for improvement instead of as a bureaucratic hurdle one simply needs to take to get published.
  I am certainly not claiming there are no law schools where students and PhDs are made familiar with peer review, but this is often left to the initiative of individual programme directors, vice deans and professors, while it should become a standard feature in the curriculum of every European law school. The same holds true for methodology courses, which are certainly still not a standard in every law school curriculum and even where there are such courses they differ tremendously in focus, ambition and practicality. Some schools emphasize searching techniques to find relevant legal sources, others focus more on academic legal writing skills, while few law schools in Europe offer hands-on methodology training programmes that make students acquainted with actually learning how to apply different research methods and techniques via tailor-made assignments guided by individual feedback from experts. As far as such courses exist, they are mostly focused on students in honours classes or PhD programmes.²⁴x For the last 15 years or so I have been visiting law schools in Europe to give lectures for (research) master students and PhDs from Spain to Sweden and from the UK to Switzerland and Slovenia on research design and methodology. Rarely did I encounter sophisticated methodology training programmes that offer tailor-made courses following the development of students step by step from the first year of a bachelor programme to the final year of a PhD training programme.

• 7. The Politics Behind Methodology Scepticism

  If it is such a good idea to make the implicit quality criteria and methodological requirements for doctrinal legal publications more explicit, then why has this not happened yet? What is holding us back? Part of the answer is probably that there is still no (inter)national consensus on what the quality indicators for doctrinal legal publications should be. Several attempts in the Netherlands, for example, to come up with journal rankings, quality indicators for legal research and a sharper delineation between professional and academic publications have failed so far.²⁵x See more in depth: van Gestel (2015, p. 243 ff). And for an international comparison: van Gestel & Lienhard (2019).
  Part of the reason for this seems to be that ‘academic politics’ stand in the way of it. The most likely factor seems to be the fear of many law schools to separate themselves too much from legal practice because that is where most law students will end up after graduation. It is hard to deny that only a tiny part of the total number of law students aspire to become legal academics, which explains why law school managers are mostly focused on trying to serve the needs of the future student employers. These often still seem to be most interested in law students with a thorough knowledge of ‘the state of the law’ and with the skills to remain up to date with respect to changes in the law. This is probably the main reason why in most European law schools, the curriculum has changed relatively little over the course of the last decades despite fierce criticism from the side of some academics (see for example Dyevre, 2017).
  Surely there is more attention for European and international law now than there was thirty years ago. Furthermore, new subfields such as environmental law, migration law and consumer law have emerged. In terms of methodology and skills training, however, the emphasis in most law schools across Europe still lies on textual interpretation methods with, only more recently, growing attention for digitization because of the rapidly growing influence data science and artificial intelligence are having on the law-making process. As far as research methods are concerned, empirical legal research is certainly on the rise, also in Europe, but this primarily has to do with the fact that institutions responsible for the distribution of funding, such as the European Research Council and national research foundations, seem to have developed a certain preference for more multi- and interdisciplinary research projects. Part of the reason for this is that the often mixed panels of scholars from different disciplines responsible for the assessment of legal research proposals have great difficulties recognizing the quality of doctrinal legal research proposals, partly because of both the lack of journal rankings, bibliometric (e.g. citation) indicators, and methodological justification in doctrinal legal research proposals. Next to this, there is the already mentioned intertwinement between academic and professional research in many European law schools.
  The problem for law school managers, such as deans and vice deans, is that they find themselves between a rock and a hard place when it comes to making the quality criteria and methodological requirements for doctrinal legal research more explicit. To a certain extent, it might take away scepticism from the side of funding bodies, international publishers and journal rankings, but as long as doctrinal legal research remains to be written in the language of authors from different countries, often dealing with national legal problems, without at least a strong comparative perspective, transnational consensus on what the future of doctrinal legal scholarship should look like will remain hard to reach. Attempts to turn the tide by, for example, switching to English as the lingua franca, and by encouraging legal scholars to focus more on global legal issues and theoretical perspectives, runs the risk that law schools will alienate themselves from national legal practice. We have even seen this in the United States after the 2008 credit crunch,²⁶x See about the effects of the credit crunch on law schools in the U.S.: Tamanaha (2012). when the criticism from both students and practitioners grew because they felt that law schools were ignoring the needs of legal practice. Telling is perhaps the sarcastic way Supreme Court Justice John Roberts spoke out about the legal publishing culture at the time:

  Pick up a copy of any law review that you see, and the first article is likely to be, you know, the influence of Immanuel Kant on evidentiary approaches in 18th Century Bulgaria, or something, which I’m sure was of great interest to the academic that wrote it, but isn’t of much help to the bar. (see Cassens Weiss, 2011).

  Others have argued that American top law schools should start practising what they preach by offering more meaningful research for legal practitioners and prepare students better for what awaits them in practice (Newton, 2010). Prominent legal scholars like Richard Posner even called for more research cooperation between legal scholars and judges in order to ensure that academic legal research at the top law schools in the United States would become relevant for the bar and the bench again (Posner, 2016). Most European law schools have not experienced these kinds of problems yet, but perhaps the American experience shows why law schools in Europe have mostly been reluctant to distance themselves from practice by, for instance, broadening the gap between professional and academic legal publications or by outsourcing certain types of publications such as handbooks, commentaries and case notes that remain highly valued by many practitioners, let alone because most of them do not have the time or the resources to provide an overview of relevant important new developments in the law and comment upon their potential relevance.
  The latter may explain why, in most European countries, university managers have undertaken few attempts to start a debate on making the methodological ground rules, quality criteria and assessment procedures by journals and publishers for doctrinal research more explicit. Apparently they are afraid to open up a Pandora’s box by doing so, because one does not know what the outcome of such a process might be. Moreover, in certain countries like Slovenia (see Hoinik, 2019, p. 341 ff.), where this has been tried, the result seems to have been a gradual marginalization of national legal publication outlets because legal scholars were forced by the government to adhere to the rankings, metrics and procedures of the social sciences. As I have argued above, though, doing nothing is not an option, especially now that academics are lying under a magnifying glass due to a growing scepticism towards expert knowledge and science more in general fuelled by populist political parties and conspiracy theorists, who claim that academics are part of an elite that block ‘genuine’ truth-seeking. Where this may lead to, we have already witnessed during the COVID-19 pandemic in which academics serving as advisers to the government increasingly suffered from personal attacks on social media and even in public.
  What should happen, in my view, is that we start by gathering best practices and exchanging information between law schools about experiences with methodology courses, study materials that are being used and compare this information with what is happening in terms of innovation in law firms, governmental legal training programmes and centres for permanent judicial education. In other words, learning from each other. Apart from that, it would be worthwhile to set up a European funding scheme for the modernization of legal education, which could start with a number of experiments to innovate traditional legal education by, for example, making use of digital methods and techniques for analysing case law and legislation that may have added value for both legal practice and academia.

• 8. Conclusion

  It is far from impossible to come up with a set of quality criteria for doctrinal legal publications, as Snel has shown. There is no reason whatsoever why a number of general requirements for academic publications such as the presence of an explicit research question, an explanation of the scientific relevance, and a match between conclusions and research questions would be less relevant for doctrinal legal publications. That an author explains what his or her publication adds to the state of art, that the research findings are based on a balanced and therefore not one-sided use of sources, and that one does not make claims or generalizations that go beyond the evidence that was found, to name but a few, are minimum requirements that every academic publication should be able to fulfil.
  More complicated is the criterion that publications should be built on ‘adequate methods and research techniques’ since there is no consensus within the scholarly legal community on what these are for doctrinal legal research. To some extent, this is caused by the fact that doctrinal research is a matter of argumentation and there is not really a generally accepted method to arguing. To put it differently, in doctrinal legal research, argumentation very often is the method. This is not the whole story though. The strength of one’s arguments is ideally built on something more than a personal opinion in academic legal research. Just as in other types of (legal) research, doctrinalists need to collect and process ‘data’, which in this case usually means legislation, case law and literature.²⁷x Fleuren calls the methodology of doctrinal legal research a rational reconstruction of what counts as the law regarding a certain subject or legal problem. See: Fleuren (2008).
  Here it becomes more difficult because, as Van Boom has argued: ‘when it comes to referencing, legal scholars are all over the place, we do not make anything explicit, and we explain to our students only half of what to do’ [translation RvG] (van Boom, 2015, p. 13). Some scholars still seem to claim that checking a publication’s footnotes should suffice to learn which evidence an author has used to support the answer to his research question(s). However, the problem with this approach is, among other things, that readers who are unfamiliar with the topic will normally not be able to discover which relevant sources have, for example, been left aside. Only experts with virtually identical knowledge as the author of the publication at hand will be able to verify whether the footnotes suffice to support the research findings. This already proves that, especially for non-experts, it will remain obscure to what extent the research process is trustworthy, as long as authors do not justify in any way which methods have been used to gather and analyse the evidence.
  The latter is even more problematic because of a set of characteristics that one finds in much doctrinal legal research, namely: researchers taking an internal (participants) perspective, often focusing on providing normative recommendations for how the law should read, relying heavily on the authority of certain legal sources (e.g. Supreme Court decisions) which they use to try to persuade readers without being transparent about the research methods that have been applied to answer certain research questions. This combination of features easily results in biased research outcomes or even advocacy scholarship, which disrespects some of the core principles of academic research such as disinterestedness, impartiality and transparency. As I have mentioned in the Introduction, the dangers of advocacy scholarship or ‘scholactivism’, as Khaitan calls it (see: Morijn, 2022). are especially grave in times where disinformation, conspiracy theories and populism are on the rise. It is one of the reasons why I believe that doctrinalists – in the long run – cannot afford to ignore spending more time and energy on methodological justification in their publications. Does this mean that more attention for methodology will completely filter out every form of bias, sloppiness or plain mistakes? Of course not! However, it will probably create more self-awareness when it comes to the importance of working in a more systematic, disciplined and transparent way. In combination with other measures such as thinking more in terms of hypothesis and organizing criticism in the research process prior to publication (e.g. peer review), I believe it could make doctrinal legal research more robust and trustworthy.
  Unfortunately, though, there are also politics involved that make it unlikely that investing in methodological accountability in doctrinal legal research will happen overnight. The main reason for this is that most law school managers – but also publishers and journals – have an interest in not alienating doctrinal legal scholarship too much from legal practice, because that could make them less relevant and credible in the eyes of the future employers of their students. In addition, law schools and (doctrinal) legal scholars have a financial interest in staying close to the publication and research practices of practitioners because, for example, contract research is an important source of income. Last but not least, law schools may be afraid to get colonized by the research assessment methods and publication practices in the (social) sciences where English is the lingua franca and methodological accountability and theorizing have sometimes gone off the rails to the detriment of the practical relevance of certain disciplines, while this relevance has always been a strength in the field of law and legal research. The challenge therefore is to find middle ground, because in a globalizing legal order with increasingly complex legal problems that require multidisciplinary work, old habits, like predominantly publishing in the national language and sticking closely to the textual interpretation methods that judges and attorneys use, will not suffice forever.

References

• Accountant (2022). Minister laat overzicht maken van ‘gesponsorde’ hoogleraren. www.accountant.nl/nieuws/2022/7/minister-laat-overzicht-maken-van-gesponsorde-hoogleraren/ (last accessed 29 December 2022).

• Alda, F. (1999). The law: An art or a science? American University Journal of Gender, Social Policy & the Law, 7(2), 355-372.

• Bormans, A. (2021). Een Leids onderzoek. www.groene.nl/artikel/een-leids-onderzoek (last accessed 29 December 2022).

• Cassens Weiss, D. (2011). Law prof responds after Chief Justice Roberts disses legal scholarship. www.abajournal.com/news/article/law_prof_responds_after_chief_justice_roberts_disses_legal_scholarship (last accessed 29 December 2022).

• Chilton, A. & Posner, E. (2015). An empirical study of political bias in legal scholarship. Journal of Legal Studies, 44, 304. doi:10.1086/684302.

• Coomans, F., Grünfeld, F. & Kamminga, M. (2010). Methods of human rights research: A primer. Human Rights Quarterly, 32, 179-186. doi:10.1353/hrq.0.0127.

• Council of Europe (2021). Netherlands – Report of the Childcare Allowance Parliamentary Inquiry Committee entitled ‘Unprecedented injustice’ (Ongekend onrecht). CDL-REF(2021)073-e. www.venice.coe.int/webforms/documents/?pdf=CDL-REF(2021)073-e (last accessed 29 December 2022).

• Davies, G. (2022). Taming law: The risks of making doctrinal analysis the servant of empirical research. In M. Bartl & J. Lawrence (Eds.). The politics of European legal research: Behind the method (pp. 124-138). Cheltenham: Edward Elgar.

• de Quay, M. (2020). Vaarwel Rechtsgeleerdheid, hallo Cliteur-kunde: waarom mijn prof niet zo moet zeuren. www.mareonline.nl/opinie/vaarwel-rechtsgeleerdheid-hallo-cliteur-kunde/ (last accessed 29 December 2022.

• de Witt Wijnen, P. & Wester, J. (2021). Wie het beste kan raden wat professor Kinneging wil horen. www.nrc.nl/nieuws/2021/02/05/raden-wat-professor-kinneging-wil-horen-a4030535 (last accessed 29 December 2022).

• Dyevre, A. (2017). Fixing European law schools. European Review of Private Law, 25(1), 151-168. doi:10.54648/ERPL2017008.

• Elferink, M. (1998). Verwijzingen in wetgeving. Over de publiekrechtelijke en auteursrechtelijke status van normalisatienormen. Deventer: Kluwer.

• Evers, G. (1999). Een normalisatienorm is geen algemeen verbindend voorschrift. Nederlands Juristenblad, 38, 1814.

• Feldman, R. (2009). The role of science in law. Oxford: Oxford University Press.

• Fleuren, J. W. A. (2008). Rechtswetenschap en rationele reconstructie. Een bijdrage aan het debat over methoden van rechtsgeleerd onderzoek. In Y. Buruma e.a. (Red.). Op het rechte pad. Liber amicorum Peter J.P. Tak (pp. 25-46). Nijmegen: Wolf Legal Publishers.

• Heikkilä, M. (2022). Dutch scandal serves as a warning for Europe over risks of using algorithms. www.politico.eu/article/dutch-scandal-serves-as-a-warning-for-europe-over-risks-of-using-algorithms/ (last accessed 29 December 2022).

• Hoinik, J. (2019). Evaluation of academic legal publications in Slovenia. In R. van Gestel & A. Lienhard (Eds.). Evaluating academic legal research in Europe (pp. 341-383). Cheltenham: Edward Elgar.

• Hutchinson, T. & Duncan, N. (2012). Defining and describing what we do: Doctrinal legal research. Deakin Law Review, 17(1), 116. doi:10.21153/dlr2012vol17no1art70.

• Kaltenbrunner, W. & de Rijcke, S. (2017). Quantifying ‘output’ for evaluation: Administrative knowledge politics and changing epistemic cultures in Dutch law faculties. Science and Public Policy, 2, 293. doi:10.1093/scipol/scw064.

• Keyes, R. (2004). The post-truth era: Dishonesty and deception in contemporary life. New York: St. Martin’s Press.

• Khaitan, T. (2022). On scholactivism in constitutional studies: Skeptical thoughts. International Journal of Constitutional Law, 20(2), 1-10. doi:10.1093/icon/moac039.

• KNAW (2018). Vrijheid van wetenschapsbeoefening in Nederland. Amsterdam: KNAW. https://www.knaw.nl/publicaties/vrijheid-van-wetenschapsbeoefening-nederland (last accessed 29 December 2022).

• Korteweg, A. (2019). Paul Cliteur: ‘Binnen elke discipline is een pluraliteit van standpunten nodig’. www.volkskrant.nl/mensen/paul-cliteur-binnen-elke-discipline-is-een-pluraliteit-van-standpunten-nodig~ba968595/ (last accessed 29 December 2022).

• Mare Online (2021). Studenten klagen over intimiderend gedrag van rechtsfilosoof Kinneging. www.mareonline.nl/nieuws/grensoverschrijdend-gedrag-kinneging/ (last accessed 29 December 2022).

• Menting, M. & Vranken, J. B. M. (2014). Gedragscodes in een meergelaagd privaatrecht in Europa en Nederland. In M. Menting, J. B. M. Vranken & M. W. Scheltema (Eds.). Gedragscodes in internationaal, Europees en privaatrechtelijk perspectief: Juridische betekenis, effectiviteit en handhaving (pp. 7-59). (Preadviezen Nederlandse Vereniging voor Burgerlijk Recht 2013). Zutphen: Paris.

• Merton, R. (1973). (Ed.). And Introd. By N.W. Storer, The normative structure of science. In The sociology of science (pp. 267-786). Chicago: University of Chicago Press.

• Morijn, J. (2022). The language of power – moving beyond restrictive conceptions of constitutional law scholarship. https://verfassungsblog.de/the-language-of-power/ (last accessed 29 December 2022).

• Newton, B. (2010). Preaching what they don’t practice: Why law faculties preoccupation with impractical scholarship and devaluation of practical competences obstruct reform in the legal academy. South Carolina Law Review, 62, 105-156.

• Nickerson, R. (1998). Confirmation bias: A ubiquitous phenomenon in many guises. Review of General Psychology, 2(2), 175-220. doi:10.1037/1089-2680.2.2.175.

• Nieuwstadt, H. & Schagen, E. van (2007). De juristen moeten de vragen stellen. Interview met Prof. Hans Nieuwenhuis. Ars Aequi, 56(11), 920-925.

• NOS Nieuws (2020). Cliteurs Leidse afdeling: ‘conservatieve kraamkamer’ en ‘vrijplaats voor debat’. https://nos.nl/artikel/2358883-cliteurs-leidse-afdeling-conservatieve-kraamkamer-en-vrijplaats-voor-debat (last accessed 29 December 2022).

• Notenboom, D. & Kooiman, W. R. (2021). Hoogleraren met een dubbele pet onafhankelijk? NLF-W 30 (2021): 1-15. Retrieved from https://www.nlfiscaal.nl/nlfiscaal-doc/me_nlfw2021_0035 (last accessed 29 December 2022).

• Posner, R. (2016). Diverging paths. The academy and the judiciary. Cambridge, MA: Harvard University Press.

• Rachlinski, J. (2018). The politics of legal empirics: Do political attitudes predict the results of empirical legal scholarship? Journal of Institutional and Theoretical Economics, 174, 1-12. doi:10.1628/093245618X15137509187753.

• Salehijam, M. (2018). The value of systematic content analysis in legal research. Tilburg Law Review, 23(1), 34-42. doi:10.5334/tilr.5.

• Schauer, F. (2009). Thinking like a lawyer. A new introduction to legal reasoning. Cambridge, MA: Harvard University Press.

• Schumm, W. (2021). Confirmation bias and methodology in social science: An editorial. Marriage & Family Review, 57(4), 285-293. doi:10.1080/01494929.2021.1872859.

• Smits, J. M. (2012). The mind and method of the legal academic. Cheltenham: Edward Elgar.

• Snel, M. (2019). Making the implicit quality standards and performance expectations for traditional legal scholarship explicit. German Law Journal, 20(1), 1-20. doi:10.1017/glj.2019.6.

• Spitzer, R. (2008). Saving the constitution from lawyers. Cambridge: Cambridge University Press.

• Stolker, C. (2018). Vrijheid van wetenschap – drie steentjes in een rustige KNAW-vijver. Science Guide. Retrieved from www.scienceguide.nl/2018/05/drie-steentjes/ (last accessed 23 May 2018).

• Tamanaha, B. (2012). Failing law schools. Chicago: University of Chicago Press.

• ten Seldam, B. & Brenninkmeijer, A. (2021). The Dutch benefits scandal: a cautionary tale for algorithmic enforcement. https://eulawenforcement.com/?p=7941 (last accessed 29 December 2022).

• Tjong Tjin Tai, E. & Verbruggen, P. (2022). Onderzoeksmethoden in de rechtswetenschap. Nederlands Juristenblad, (1), 10.

• van Boom, W. H. (2015). ‘Door meten tot weten’. Over rechtswetenschap als kruispunt. Den Haag: BJu.

• van der Burg, W. & Taekema, S. (2020). Inzicht, integriteit en NWO. In B. de Vries e.a. (Red.). Rechtstheorie en rechtspraktijk (liber A.M. Hol) (pp. 303-312). Den Haag: Boom juridisch.

• van Gestel, R. A. J. (2012). Hoge Raad is duur. RegelMaat, 27(4), 249-295.

• van Gestel, R. A. J. (2015). Kwaliteit van juridische publicaties, Preadvies NJV. Deventer: Kluwer.

• van Gestel, R. A. J. (2017). Ranking, peer review, bibliometrics and alternative ways to improve the quality of doctrinal legal scholarship. In R. A. J. van Gestel, E. Rubin & H. W. Micklitz (Eds.). Rethinking legal scholarship (pp. 351-398). New York: Cambridge University Press.

• van Gestel, R. & Lienhard, A. (Ed.). (2019). Evaluating academic legal research in Europe. Cheltenham: Edward Elgar.

• van Gestel, R. A. J. & de Poorter, J. C. A. (2019). In the court we trust. Cambridge: Cambridge University Press.

• van Gestel, R. A. J. & Micklitz, H. W. (2013). European integration through standardization: How judicial review is breaking down the club house of private standardization bodies. Common Market Law Review, 50(1), 145-181. doi:10.54648/COLA2013007.

• van Gestel, R. A. J. & Micklitz, H. W. (2014). Why methods matter in European legal scholarship. European Law Journal, 20(3), 292-316. doi:10.1111/eulj.12049.

• van Gestel, R. A. J. & Snel, M. V. R. (2019). Rehabilitatie van de vakpublicatie. Nederlands Juristenblad, (42), 3152-3160.

• van Hoecke, M. (Ed.). (2011). Methodologies of legal research: Which kind of method for what kind of discipline? Oxford: Hart Publishing.

• Verbruggen, P. W. J. (2021). Methoden van systematische rechtspraakanalyse. Den Haag: Boom juridisch.

• Vleggeert, J. (2020). Hoe onafhankelijk is de fiscale wetenschapper? (oratie, Universiteit Leiden). https://www.universiteitleiden.nl/binaries/content/assets/rechtsgeleerdheid/fiscaal-en-economische-vakken/belastingrecht/oratie-vleggeert.pdf (last accessed 29 December 2022).

• Vleggeert, J. (2022). Een burgerperspectief op belangenverstrengeling (oratie, Universiteit Leiden). http://www.universiteitleiden.nl/binaries/content/assets/rechtsgeleerdheid/feestrede-vleggeert-totaal.pdf (last accessed 29 December 2022).

• Vranken, J. B. M. (2009). Als een arrest niet bevalt, laten wij het gewoon weg. In A. G. Castermans e.a. (Red.). Ex Libris Hans Nieuwenhuis (pp. 77-94). Deventer: Kluwer.

• Vranken, J. B. M. (2014). Algemeen Deel. Een synthese. Deventer: Kluwer.

• Vranken, J. B. M. (2016). Methodology in legal research and legal practice. A comparison of judge and legal academic. In M. Groenhuijsen, E. H. Hondius & A. Soeteman (Eds.). Recht in geding II (pp. 15-25). Den Haag: Boom juridisch.

• Vranken, J. B. M. & van Gestel, R. A. J. (2007). Rechtswetenschappelijke artikelen. Naar criteria voor methodologische verantwoording. Nederlands Juristenblad, (24), 1448-1461.

• Wijntjens, L. (2020). Als ik nu sorry zeg, beken ik dan schuld? Den Haag: Boom juridisch.

Noten

• 1 Posner, E., ‘The Rise of Interdisciplinary Approaches in American Legal Scholarship’, keynote address at the University of Tilburg Symposium, Towards a European Legal Methodology?, quoted in: van Gestel and Micklitz (2014).

• 2 See about the relationship between doctrinal research and empirical legal studies: Davies (2022).

• 3 See for example van Gestel (2017).

• 4 The suggestion made by some that explicating a research question and/or explaining some basic methodological steps would not be possible in doctrinal research is in my view demonstrably false. Otherwise: Tjong Tjin Tai & Verbruggen (2022) referring to van der Burg & Taekema (2020). Also the fact that one sometimes needs to adjust one’s research questions during the course of the research occurs on a regular basis in both doctrinal and empirical research and is therefore not idiosyncratic for doctrinal research.

• 5 In one of the first studies I participated in on methodology, I discovered that these basic criteria were often not followed by the authors of articles published in the science section of the biggest Dutch law journal. See: Vranken & van Gestel (2007).

• 6 More about the differences between traditional doctrinal case law analysis and methods such as structured content analysis can be found in Verbruggen (2021) and Salehijam (2018).

• 7 Most comprehensively in Vranken (2014). For a brief English summary of his argument, see: Vranken (2016).

• 8 I focus on academic doctrinal legal research here because doctrinalists also publish professional legal publications, which often lack the ambition to add knowledge to the state of the art and are usually focused on applying existing knowledge.

• 9 HR 15 January 1991, ECLI:NL:HR:1991:AC3982.

• 10 To mention a few other possible answers: legal scholars do not consider themselves to be ‘scientists’ but as (legal) scholars (‘rechtsgeleerden’) standing in a hermeneutic and philosophical tradition, legal scholars seek recognition from practitioners who are less interested in methodology, legal scholars are focused on convincing others by arguments and are more practical and result-oriented than those who want to build new knowledge.

• 11 An interesting example is the book by Schauer (2009), which is equating the reasoning of legal scholars with that of legal practitioners.

• 12 See for an English summary of Vranken’s main arguments: Vranken (2016).

• 13 A prominent recent Dutch example where this has happened is: Rb. Den Haag 26 May 2021, ECLI:NL:RBDHA:2021:5337 (Dutch text) and ECLI:NL:RBDHA:2021:5339 (English translation) (Milieudefensie et al. v. Royal Dutch Shell).

• 14 Meaning, for example, that appellate court decisions are considered to be more important than district court decisions and supreme court decisions more important than appellate court decisions and so on.

• 15 Interestingly, though, Rachlinski also found a ‘mild correlation’ between the political beliefs of authors of empirical legal journal articles and the outcomes of their research. See: Rachlinski (2018).

• 16 The Dutch ‘childcare benefits’ scandal (Kinderopvangtoeslagschandaal), which led to a parliamentary investigation and even a report by the Venice Commission ruling that the government had violated the rule of law by discriminating against certain parents with a different ethnic or cultural background, has led to widespread criticism that the Dutch academic legal community had totally overlooked this field of law, although if affected enormous amounts of citizens and caused grave harm for tens of thousands of families that were badly treated by the Dutch Tax Authority because they were falsely labelled as fraudsters. See about this scandal for example: ten Seldam & Brenninkmeijer, 2021; Heikkilä (2022); Council of Europe (2021).

• 17 See for example: de Quay (2020); Bormans (2021); Mare Online (2021); de Witt Wijnen & Wester (2021); NOS Nieuws (2020).

• 18 See about these and other academic values, for instance, Merton (1973).

• 19 See for instance the Belgium code of ethics for scientific research on p. 9: www.belspo.be/belspo/organisation/publ/pub_ostc/Eth_code/ethcode_nl.pdf. The Dutch code on scientific integrity sees impartiality as part of the principle of academic independence. See: www.nwo.nl/sites/nwo/files/documents/Netherlands%2BCode%2Bof%2BConduct%2Bfor%2BResearch%2BIntegrity_2018_UK.pdf.

• 20 Dutch Supreme Court 22 June 2012, ECLI:NL:HR:2012:BW0393.

• 21 See for the comparison with the European situation also: van Gestel & Micklitz (2013).

• 22 I realize that conduction interviews would go beyond traditional doctrinal research entering the field of empirical research but the example is still relevant because one cannot always know beforehand whether a purely doctrinal analysis will offer the most convincing answer to a normative question. It may also depend on what information the case law, legislative history and literature provide.

• 23 See for a discussion of the answers to this question: van Gestel & de Poorter (2019).

• 24 For the last 15 years or so I have been visiting law schools in Europe to give lectures for (research) master students and PhDs from Spain to Sweden and from the UK to Switzerland and Slovenia on research design and methodology. Rarely did I encounter sophisticated methodology training programmes that offer tailor-made courses following the development of students step by step from the first year of a bachelor programme to the final year of a PhD training programme.

• 25 See more in depth: van Gestel (2015, p. 243 ff). And for an international comparison: van Gestel & Lienhard (2019).

• 26 See about the effects of the credit crunch on law schools in the U.S.: Tamanaha (2012).

• 27 Fleuren calls the methodology of doctrinal legal research a rational reconstruction of what counts as the law regarding a certain subject or legal problem. See: Fleuren (2008).