Guest Post: Hacking the Supreme Court in the Age of AI

(This is a guest post by Anand Venkatanarayanan).

—

Introduction

How do you bring down a democracy? In their seminal paper, Bruce Schneier and Henry Farrell argue that democracy is a political information system, which has the following characteristics.

• Common knowledge – the consensus beliefs that hold systems together.
• Contested knowledge – the knowledge that is contestable, where people may disagree.

They argue quite convincingly through the example of Russian election meddling in the US elections, that democratic systems have an inherent vulnerability that can be exploited to bring it down: this vulnerability comes into play when common knowledge becomes contested knowledge. For example, the disinformation campaign launched by the Russians included undermining knowledge assumptions about how voting works, spreading distrust about the candidates and also the political system in general, through a variety of other means.

This post builds on the above paper and further argues that dispensation of justice is also an information system in a democracy, and has the same characteristics. For instance, “settled law” can be viewed as common knowledge, while different interpretations of law by different benches of the Supreme Court of India (due to its polyvocal nature) can be regarded as contested knowledge.

When viewed through this lens, one way in which the Supreme Court could be brought down is by turning common knowledge into contested knowledge. For instance, a party that has lost a case in the court can ascribe extraneous motives to the judge who gave the judgement, and attempt to turn common knowledge (Court orders are binding) into contested knowledge (they are not binding and can be flouted because they are based on extra-legal factors).

While Courts have evolved contempt jurisdiction to handle external attempts to change common knowledge into contested knowledge, this post argues that the Supreme Court has become institutionally blind to how i’s recent judgements, in attempting to deliver substantive justice by disregarding procedure, are turning common knowledge (settled law) into contested knowledge, and thereby undermining public trust on it.

Procedure, Proof and Innovation

It is often said by Karl Popper that Science never proves, but only disproves. At the heart of his assertions is the simple observation that “criterion of the scientific status of a theory is its falsifiability, or refutability, or testability”. The difference between science and pseudo-science is that pseudo-science does not offer any testable experiments (natural or otherwise) that can disprove the hypothesis.

Testability by others, and not just by the claimants, hence is at the heart of scientific inquiry. For instance, while someone can make a claim that peacocks reproduce by crying, by virtue of their position and title (as a judge), it is deemed to be absurd because contrary evidence has been recorded by eyewitnesses and others for a very long time.

How does testability itself works in the scientific world? It is defined as a process that must be followed to ascertain the validity of the claims. This includes formal proofs, experimental observations, peer review through publishing in journals, feedback, revision and finally acceptance or rejection.

Thus process, procedure and innovation are deeply interconnected in furthering scientific knowledge. There are well known cases such as fraudulent stem cell research where all of this failed, but that was not because of lack of application of the due process – it was in spite of it. Thus, failures in applying due process are very rare and practically non-existent – and the necessity of due process itself gets strengthened through exceptions such as the fraudulent stem cell research case.

Scientific due process is hence an information system, which everyone agrees is essential (Common knowledge) to mediate substantive disputes (contested knowledge). Hence innovation (change) in due process, while permissible and welcome, is not arbitrary and accepted only when it aids on mediating disputes and in furthering scientific knowledge.

Evidence Act and other norms

In the legal system, the Evidence Act of 1872 can be viewed as the equivalent of scientific due process, since it provides a legally sanctioned framework, which covers in detail how to establish facts within the context of a legal dispute, and also issues of proof generation, and scrutinizing those proofs via cross-examination.

The Indian Evidence Act embodies the adversarial system of justice, where parties represent their case before judges, who attempt to determine the truth and pass judgment accordingly. In this system, the Courts are expected to follow due process (as set out in laws such as the Evidence Act) and other norms to decide the ratio decidendi of a particular case. This, in turn, implies that (subject to their own interpretation, of course) the rules set out under the Evidence Act and other norms can be thought of as “Common Knowledge”, which used to mediate substantive disputes (Contested knowledge). Innovation in common knowledge (i.e., modifying or bypassing due process) must be accepted only when it aids in delivering justice.

However, if common knowledge (due process and norms) can be converted into contested knowledge (due process and norms are not important and hence can be discretionary), it introduces an “attack surface” on the justice system itself, which is not patchable (in software terms, not patchable means that it can’t be fixed). What would be the implication if this attack succeeds?

The Salem witch trials offers us an example of this played out in medieval times: in these trials, “tribunals” admitted spectral evidence based on dreams and visions. Leaving aside the problematic outcomes of the witch trials, they resulted in results void of any reason and reflected the bias and prejudices of the population as well as the judge.

Modern-day effects are very similar, and a critical analysis of CJI Dipak Misra’s judgements pointed out how outcomes void of reason have become the norm. Sealed covers and power point “evidence” are the modern day variants of medieval spectral evidence. And whether it the non-existent CAG report in the Rafale judgement or the dodging of facts and inverting reality and claiming that something that was argued in open Court was never actually argued in the Aadhaar judgment, jettisoning reason has serious consequences and makes judgements unintelligible.

If reason and norms are abandoned, what replaces them? Both the Rafale judgement and the Aadhaar judgement show that it is replaced by excessive faith in the government of the day, and statements by government functionaries have higher evidentiary value, even when they are unsigned, not placed on affidavit and demonstrably false, when put under scrutiny.

Attack surfaces on the Supreme Court

The structural faults elaborated above exposes the Supreme Court to new attack vectors, which can be pulled off by actors in the age of Big data and Artificial intelligence (AI). State governments are turning towards AI systems for crime detection and even the court seems to believe that it can be used for medical education reform. Note that faith in technological regimes is not new, but even the creators of AI are not sure what they have built and how they work – in contrast to their employers, who believe that we can have, not just accurate AI but trusted AI as well. But if they manage to convince the executive and the government of the day to publicly pronounce their faith in new technological regimes, success is guaranteed against any litigation, since the court places excessive faith in the statements of the government and it’s functionaries. For example, as the creator of Aadhaar had so convincingly demonstrated, all that was required was to convince three judges that mattered, and Court’s belief in the government’s statements overrode reason, logic and other norms, including open contempt towards it’s interim orders.

This post predicts that this “trust us at the cost of established due process norms to establish contested knowledge” would be the new normal in how technology projects would be rolled out in the future, and litigation by citizens against those projects will always fail, unless the court reverses course.

Conclusion

This post made three claims based on the paper by Bruce Schneier and Henry Farrell

1. Justice is an information system in a democracy and hence has the same vulnerabilities: common knowledge attacks are deadly to its legitimacy.
2. The Supreme Court of India, in its quest to deliver substantive justice, has often ignored laws and norms which constitute common knowledge, and has hence unknowingly participated in an insider attack (in information security parlance) that has the potential to erode the court’s legitimacy in the public eye.
3. The vulnerability caused by this blindness will be ruthlessly exploited by marketers of technology projects which will use Big data and AI.

It is entirely possible for others to disagree with the above claims, but the claims are at least testable in the Popperian sense; and thinking about justice as an information system that has unique vulnerabilities might bring to the fore new perspectives about its relationship to security.