諾維納教授 在【神經—邏輯 部落格】上討論「人工智慧」目前遇到的困境和展望未來研究的方向。摘要譯述於下。

或許我離開學校太久；或許琶琶撒格婁博士對電腦程式的性質有些外行；總之，我覺得他在《藝術與思想研究學會報導》這篇文章有點杞人憂天。至少，可以說是言過其實或聳人聽聞。

程式的「演算/決策程序」是設計師或軟體工程師寫的。她/他們自有預先制定的一套「演算/決策」準則，才能確保「演算/決策」結果(或「(電腦)輸出」)的一致性。缺乏一致性的軟體程式或「演算/決策程序」不過是一紙廢話或天書。

如果「人工智慧」(電腦)的「演算/決策」結果不符合常理，我至少可以想到以下三個可能導致這個情況發生的原因：

a. 軟體設計師或軟體工程師當初的設計不夠周全，或她/他沒有考慮到所有輸入數據的各種可能性；
b. 軟體設計師或軟體工程師當初的設計不夠完備，或她/他沒有考慮到所有可能發生狀況的各種排列組合；
c. 電腦的硬體設備有限，或它在預定時間內無法處理所有被輸入的資訊；同時，該程式的設計師或軟體工程師沒有寫進在這個情況下的「退場機制」。

我的結論：

1. 在2028以前，如果「人工智慧」的「演算/決策」結果不符合常理，這是設計團隊的疏忽和/或缺失。(以我的科技知識，我只有預測五年以內發展的信心。)
2. 「人工智慧」(或「電腦軟體程式」)跟「理性」的概念無關。


AI and the end of reason

Seeing through black boxes

Alexis Papazoglou, 05/24/23

Life changing decisions are increasingly being outsourced to Artificial Intelligence.  The problem is, AI systems are often black boxes, unable to offer explanations for these decisions. Unless regulators insist that AI needs to be explainable and interpretable, we are about to enter an era of the absurd, writes Alexis Papazoglou.

One of the greatest risks that AI poses to humanity is already here.  It’s an aspect of the technology that is affecting human lives today and, if left unchecked, will forever change the shape of our social reality. This is not about AI triggering an event that might end human history, or about the next version of Chat GPT putting millions of people out of a job, or the welter of deep fakes and misinformation war that’s coming. It’s about a feature of current AI that its own designers openly admit to yet remains astonishing when put into words: no one really understands it.

Of course, AI designers understand at an abstract level what products like Chat GPT do: they are pattern recognizers; they predict the next word, image, or sound in a series; they are trained on large data sets; they adjust their own algorithms as they go along, etc. But take any one result, any output, and even the very people who designed it are unable to explain why an AI program produced the results it did. This is why many advanced AI models, particularly deep learning models, are often described as “black boxes”: we know what goes in, the data they are trained on and the prompts, and we know what comes out but have no idea what really goes on inside them.

Often referred to as the issue of interpretability, this is a significant challenge in the field of AI because it means that the system is unable to explain or make clear the reasons behind its decisions, predictions, or actions. This may seem like an innocuous detail when asking Chat GPT to write a love letter in the style of your favorite poet, but less so when AI systems are used to make decisions that have real-world impacts such as whether you get shortlisted for a job, whether you get a bank loan, or whether you go to prison rather than given bail, all of which are decisions already being outsourced to AI. When there’s no possibility of an explanation behind life-changing decisions, when the reasoning of machines (if any) is as opaque to the humans who make them as to the humans whose lives they alter, we are left with a bare “Computer says no” answer. When we don’t know the reason something’s happened, we can’t argue back, we can’t challenge, and so we enter the Kafkaesque realm of the absurd in which reason and rationality are entirely absent. This is the world we are sleepwalking towards.

If you’re tuned into philosophy debates of the 20^th century involving post-modern thinkers like Foucault and Derrida, or even Critical Theory philosophers like Adorno and Horkheimer, perhaps you think the age of “Reason” is already over, or that it never really existed – it was simply another myth of the Enlightenment. The concept of a universal human faculty that dictates rational, logical, upright thought has been criticised and deconstructed many times over. The very idea that Immanuel Kant, a white 18^th-century German philosopher, could come up with the rules of universal thought simply through introspection rings all kinds of alarm bells today. Accusations vary from lazy, arm-chair philosophy (though Kant was very aware of that problem), to racism.  But we are currently entering an era that will make even the harshest critics of reason nostalgic for the good old days.

It’s one thing to talk about Reason as a universal, monolithic, philosophical ideal. But small ‘r’ reason, and small ‘r’ rationality is intrinsic to almost all human interaction. Being able to offer justifications for our beliefs and our actions is key to who we are as a social species. It’s something we are taught as children. We have an ability to explain to others why we think what we think and why we do what we do. In other words, we are not black boxes, if asked we can show our reasoning process to others.

Of course, that’s also not completely true. Humans aren’t entirely transparent, even to themselves if we believe Nietzsche and Freud. The real reasons behind our thoughts and actions might be very different from the ones we tell ourselves and give to others. This discrepancy can have deep roots in our own personal history, something that psychoanalysis might attempt to uncover, or it might be due to social reasons, as implied by a concept such as unconscious bias. If that’s the case, one could argue that humans are in fact worse than black boxes – they can offer misleading answers as to why they behave the way they do.

But despite the fact that our reasoning can sometimes be biased, faulty, or misleading, its very existence allows others to engage with it, pick holes in it, challenge us, and ultimately demonstrate why we might be wrong. What is more, being rational means that we can and should adjust our position when given good reason to do so. That’s something black-box AI systems can’t do.

It’s often argued that this opaqueness is an intrinsic feature of AI and can’t be helped. But that’s not the case. Recognizing the issues that arise from outsourcing important decisions to machines without being able to explain how those decisions were arrived at, has led to an effort to produce so-called Explainable AI: AI that is capable of explaining the rationale and results of what would otherwise be opaque algorithms.

The currently available versions of Explainable AI, however, are not without their problems. To begin with, the kind of explanations they offer are post-hoc. Explainable AI comes in the form of a second algorithm that attempts to make sense of the results of the black-box algorithm we are interested in the first place. So even though we can be given an account of how the black-box algorithm arrived at the results it did, this is not in fact the way the actual results were reached. This amounts more to an audit of algorithms, making sure their results are not compatible with problematic bias. This is closer to what is often referred to as interpretable AI: we can make sense of their results and check that they fulfil certain criteria, even if that’s not how they actually arrived at them.

Another problem with Explainable AI is that it is widely believed that the more explainable an algorithm, the less accurate it is. The argument is that more accurate algorithms tend to be more complex, and hence, by definition, harder to explain. This pitches the virtue of explainability against that of accuracy, and it’s not clear that explainability wins such a clash. It’s important to be able to explain why an algorithm predicts a criminal is likely to reoffend, but it’s arguably more important that such an algorithm doesn’t make mistakes in the first place.

However, computer scientist Cynthia Rudin argues that it’s a myth that accuracy and explainability are competing values when it comes to designing algorithms and has demonstrated that the results of black box algorithms can be replicated by much simpler models. Rudin instead suggests that the argument for the epistemological advantage of black boxes hides the fact that the advantage is in fact a monetary one. There is a financial incentive in developing black-box algorithms. The more opaque an algorithm, the easier it is to profit from it and prevent competition from developing something similar. Rudin argues that complexity and opaqueness exist merely as a means to profit from the algorithm that has such features, since their predictions can be replicated by a much simpler, interpretable algorithm that would have been harder to sell given its simplicity.  Furthermore, the cost of developing opaque algorithms might in fact be much lower than developing interpretable ones, since the constraint of making an algorithm transparent to its user can make things harder for the algorithm’s designer.

But beyond the crude financial incentives for developing opaque algorithms lies something deeper: the mystique around black box algorithms is itself part of their allure. The idea that only algorithms we can’t understand have the power to reveal hidden patterns in the data – patterns that mere mortals can’t detect – has a powerful pull, almost theological in nature: the idea of a higher intelligence than ours, one we can’t even begin to understand. Even if this is true, the veil of mystique surrounding AI serves to inadvertently stifle the idea that regulation is even possible.

One of the best examples of our reverence of such artificial intelligence, but also of the absurdity that results from outsourcing important questions to machine processes we don’t fully understand, is found in Douglas Adams’ The Hitchhikers’ Guide to the Galaxy. When a supercomputer named Deep Thought is tasked with finding the answer to “the ultimate question of life, the universe and everything” it takes it 7.5 million years to complete the task. When it finally reveals the answer, it is simply “42”.

If we can’t see properly inside the AI black boxes, we should stop asking them important questions and outsourcing high stake decisions to them. Regulators need to emphasize to developers the need to produce Explainable AI systems, ones whose reasoning we can make sense of. The alternative is to live in a social world without any rhyme or reason, of absurd answers we can’t question.

Suggested Viewing： The AI hoax