(This is the second post in a four part series by Anand Venkat examining the factual claims underlying the Majority judgments in Aadhaar.)
How do we know that a certain technology works as claimed? The steps to ascertain that it does indeed work, as expected, are fairly standard, and are neutral to the technology deployed, as described below:
- First a trial or a laboratory exercise is conducted, under pristine or controlled conditions, to see how the technology performs.
- It is a given at this stage that this is the best possible condition, which is not representative of the real world, because no small scale experiment can capture the complex reality of the real world in full.
- The results are then published, with precise descriptions of the laboratory conditions including an assessment of the assumptions.
- Then, extrapolations are made, based on the conditions necessary to extend it to the real world. These are usually accompanied by error bands or confidence intervals.
- The technology is then rolled out slowly, and the predictions are checked against real world measurements. The applicability is continuously determined
- The side effects of deviation from laboratory conditions are then addressed, either through improvements, or, if the deviations are relatively small, then manually.
The description above is essential for the reader to understand a key issue that the Majority in the Aadhaar judgement, dodged — all technology solutions are fallible, but what matters is the deviation from expectations, to determine applicability.
The Majority points out one key study in its judgement (Page 16), that was submitted by the UIDAI itself, titled “Role of Biometric Technology in Aadhaar Authentication”. The study points out on page 24, the following conclusion:
Finally 1.87% of residents participating in the study were found to have fingerprint Quality not sufficient for fingerprint authentication.
They were not included in the further authentication tests.
In essence, the headline accuracy figures of 99.3%, that the Majority quoted in its judgement, came after excluding those from the study that UIDAI conducted, for whom biometrics does not work at all.
This specific facet was argued by the petitioners in depth, which the Majority chose to ignore. Instead, the judgement specifically used a headline figure of 99.76% (Paragraph 314, Page 384), but this is one that even the UIDAI did not claim, in its own study.
The Authority has claimed that biometric accuracy is 99.76%. It was, however, submitted that where more than 110 crores of persons have enrolled themselves, even 0.232% failure would be a phenomenal figure, totalling 27.60 lakh people.
It might be possible to argue that this is nitpicking on numbers, but let us deconstruct the basic technology issues here, from elementary probability the kind that is taught in 10th standard schooling. For biometric authentication to work reliably, the following parameters are required.
- Fingerprint quality of the resident itself.
- Fingerprints/IRIS scans captured at a good quality during enrolment.
- Working scanners during authentication.
- Internet Connectivity
- Various back end servers, configured by the service providers (PDS, MNREGA) that work reliably.
Every one of the above are fallible, and can fail. So, across a wide population, the chances of failure are a sum of (1-5). The UIDAI has already admitted that (1) above is 1.87% at controlled conditions. So it is only natural by the iron law of probability addition, again, one that is taught in primary schools, service denials are embedded in the design and architecture of the project.
The numerous affidavits filed by petitioners are testimonials to the mathematical certainty that form the basis of the challenge. But the way in which the majority dealt with mathematical certainties backed by data is stunning. It pronounced an undying faith on technological improvements, backed without evidence or data.
We understand and appreciate that execution of the Aadhaar scheme, which has otherwise a laudable objective, is a ‘work in progress’. There have been substantial improvements in the system over a period of time from the date of its launch.It was stated by the learned Attorney General as well as Mr. Rakesh Dwivedi, at the Bar, that whenever difficulties in implementation are brought to the notice of the respondents, remedial measures are taken with promptness. Cases of denial of services are specifically looked into which is very much needed in a welfare State and there can be a genuine hope that with the fine tuning of technology, i.e. the mode of advancement at rapid pace, such problems and concerns shall also be completely taken care of.
It is important to understand the implication of the above paragraph. When asked to make difficult decisions on the fallibility of a technological solution, the Majority first refused to engage with the unyielding iron laws of mathematics. It then made up a factually inaccurate claim that there have been improvements since the date of launch of the project, and then finally hoped that the technology will further improve over time, to address this issue, when the UIDAI’s CEO’s presentation itself showed authentication failures on government schemes increasing over time:
By doing so, the Majority avoided facing questions of fallibility, that were raised strenuously by the petitioners, through hope (and perhaps a prayer).
Witch Doctors and Bleeding
A very significant contribution of modern medical trials is the idea of Randomized Controlled Trial (RCT). A medical intervention that is expected to cure an ailing patient is put through a rigorous evaluation, to ascertain the effectiveness of the medical intervention, and also to identify unwanted side effects. This is important, to distinguish various competing interventions that attempt to cure the patient with the same problem.
While RCT is complex, it is also considered as the gold standard for evidence based policy making. For instance, corruption and leakage in welfare delivery can be thought of as a disease, one that plagues the ailing nation, and various interventions, including Aadhaar, can be thought of as state interventions that aim to cure the problem.
But how many of these interventions are effective? For instance, medieval witch doctors also had a noble intent, the curing of diseases through constant bleeding of the patient. While no one can question the state’s noble intent to remove corruption, the question of effectiveness of various measures still remains.
The state’s preferred method of showcasing the effectiveness of an intervention is “Savings to the Exchequer”. It cited fantastic figures, that always keep increasing over time, but which were refuted in depth by the petitioners, as noted by the Majority, in Para 316, Page 385. The Majority recorded the Petitioners’ following submissions:
First, it [i.e., the State] has failed to discharge its burden of showing that the purported leakages were exclusively caused due to identity fraud, and that those leakages would not exist if Aadhaar is implemented. The state has not given any empirical data. Leakages exist due to eligibility frauds, quantity frauds and identity frauds. Studies filed in Petitioner’s affidavits show that eligibility and quantity frauds are the substantial cause for leakages. Assuming that the Aadhaar Act prevents leakages, the biometric identification system can, at best, only cure leakages related to identity fraud. The government’s claims of savings inter alia of Rs. 14,000 crores in the PDS system, due to the deletion of 2.33 crore ration cards is incorrect, inflated, and based on wrong assumptions for the following reasons:
1. It admittedly does not have estimates of leakages in PDS, nor has any study been done to see if POS machines are effective in removing PDS irregularities;
2. It conflates issue of “bogus /ineligible ration cards” (eligibility fraud) with identity fraud
3. The figure of 2.33 crore includes West Bengal, where ration cards are issued to each person, as opposed to each household
4. A large number of these 2.33 crore cards were deleted even before Aadhaar-integration and seeding came into effect;
5. The savings figure includes even those eligible beneficiaries who have been removed from the list due to failure to link Aadhaar properly; and
6. It does not value the cost of loss of privacy. Most importantly, the basis for reaching such savings figure has not been disclosed.
Similarly, incorrect averments have been made in the context of LPG savings, using Aadhaar-enabled Direct Benefit Transfer (‘DBT’) scheme known as PAHAL.
How did the Majority deal with such detailed factual rebuttals? In Page 350, Paragraph 279, it points out that:
There have been cases of duplicate and bogus ration cards, BPL cards, LPG connections etc. Some persons with multiple identities getting those benefits manifold. Aadhaar scheme has been successful, to a great extent, in curbing the aforesaid malpractices.
That is about it. There is no engagement with facts, no weighing contradictory evidence, no engagement with methodology or the lack of it, behind the state’s assertions.
One of the stunning achievements of the human race, that distinguishes us from other organisms, is the fact that we can engage with the physical and material world through complex reasoning Technological progress emerged from such reasoning, and mathematical theory and laws form the bedrock of technology.
Methodological correctness, therefore, is also a side effect of technological progress. For instance, can one claim that one is a billionaire without any assets, jobs or wealth and only with a number written on a piece of paper?
A person who makes such a claim would be laughed out in seconds as delusional, yet the highest constitutional court, allowed the state to make the fantastic claim that Aadhaar helped in curbing malpractices, with no evidence, methodological backing or proof, and get away with it.
Worse still, though, it also endorsed it, and declared it as the truth.
By doing so, it relapsed to argumentum ad verecundiam, where statements made by authority figures, even if they are totally without basis, are held as the ultimate truth.
The Aadhaar case is fundamentally about the intersection of power, technology and freedom as noted by Chandrachud J. in his dissent. In understanding the interface between governance, technology and freedom, this case will set the course for the future.
A factual engagement with technology, mathematics, scientific approach and methodology was essential for the court to understand the issues involved. That it chose to depend on a ‘slideshow’ presentation and that too, only selectively (it ignored the increasing authentication failures in government schemes) to arrive at its conclusions, is disturbing, for more than one reason.
Part 3 of this series will highlight how it incidentally tripped up not only the petitioners, but also caused nightmares to the UIDAI and the state accidentally, because of its technological illiteracy.