5 AI Safety books I recommend

1. Superintelligence: Paths, Dangers, Strategies

Why read it?

This 2014 cult classic in the field of AI Safety has been one of the catalysts for the popular concern around building superintelligent AI in the last few years.

Written in a time when the deep learning revolution had just started to take off, Nick Bostrom's book focuses on the risks of developing machines that are 'vastly smarter' than humans. Beginning with an exploration of 2 possible options: a 'slow takeoff' and a 'fast takeoff', the book rigorously discusses and analyses the risks associated with building superhuman AI.

Why is it relevant to AI?

This book gained quite a bit of media coverage due to the likes of Elon Musk, Bill Gates, and Stuart Russell, which then helped kickstart AI safety as a serious research field with proper funding, such as Nick Bostrom's own institute at the University of Oxford, The Future of Humanity Institute.

Memorable quote

Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an "intelligence explosion," and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.

2. Life 3.0

Why read it?

Although its topic is similar to Superintelligence, I found Max Tegmark's 2017 book to be a more balanced view of the future of AI, albeit slightly on the optimistic side.

Despite the fact that both focus on medium to long term societal implications of superintelligent AI, Tegmark tries to look at how we can enforce and maximise a positive outcome, rather than focusing on the potential risks of AI, as described in Bostrom's Superintelligence.

Life 3.0, in Tegmark's view, is a technological stage, where its lifeform designs both its hardware and software (here software is the mechanism through which we process information and make decisions and hardware is the body that stores the information and/or takes action). Life 1.0 is the biological stage, where its hardware and software are entirely a product of evolution, while Life 2.0 is the cultural stage with an evolved hardware and partly designed software (e.g. humans can learn complex skills).

Why is it relevant to AI?

Apart from being a more readable, updated, introduction into the recent developments in AI and AI Safety, Life 3.0 offers a range of scenarios and possibilities of how AI will impact different aspects of humanity that go from the next few years to far into the distant future, which is where his physics background comes most handy.

I also enjoyed his ventures into more metaphysical topics, such as consciousness or what humanity's purpose might be in the presence of superintelligent AI. Ultimately, I think it hits the right chord; it both terrifies and excites the reader in equal measures about the impact this technology will have.

Memorable quote

Without technology, our human extinction is imminent in the cosmic context of tens of billions of years, rendering the entire drama of life in our Universe merely a brief and transient flash of beauty, passion and meaning in a near eternity of meaninglessness experienced by nobody.

3. The Structure of Scientific Revolutions

Why read it?

Although this book was written in 1962, its take on the progress of science is still very relevant today. Thomas Kuhn introduced the idea that science doesn't develop by accumulating ideas and theories overtime, but rather by periods of slow development, interspersed with periods of revolutionary ideas that partially or entirely replace the established paradigms, a phenomenon which he called 'paradigm shifts'.

I found this a valuable read in order to better understand how scientific knowledge evolves and how novel ideas initially get suppressed or ignored by the scientific community.

This particularly amusing review compares the way scientists hold onto traditional ideas to the way people behave in romantic relationships.

Why is it relevant to AI?

Given that we are in the midst of an AI revolution at the moment, this book might offer insight into how this revolution happened and how it will proceed. It might help to understand why Deep learning has taken off and why it might take awhile until alternative, more novel approaches replace it.

Memorable quote

What man sees depends both upon what he looks at and also upon what his previous visual-conception experience has taught him to see.

4. Homo Deus: A Brief History of Tomorrow

Why read it?

Yuval Noah Harari, a history professor at the Hebrew University in Jerusalem, became famous with his first novel, Sapiens: A Brief History of Humankind. In his 2nd book, Harari focuses on the future of humanity and our increasingly tight relationship with technology, which could result in our evolution from homo sapiens to 'homo deus'. In his view, our quest to re-design our minds and bodies to eventually defy death is comparable to a quest to achieve divinity.

I particularly liked its focus on biotechnology, which, from what I have seen, is rarely part of the recent conversation around AI safety.

Why is it relevant to AI?

I have found Yuval Noah Harari's books to be quite polarising among AI practitioners. Coming from a history/humanitarian background he does seem to hold quite strong speculative opinions that are almost reductive at points, however, they are usually at least great conversation starters.

Whether you agree with him or not, he does raise important concerns about our identity and our sense of agency in an age of dataism, as highly intelligent AI could end up knowing us better than we know ourselves.

Are we only going to lose more and more control over our own decisions, becoming merely data points to be fed in ever increasingly intelligent algorithms? Are we making ourselves obsolete in the process of building AGI?

Memorable quote

In fact, as time goes by, it becomes easier and easier to replace humans with computer algorithms, not merely because the algorithms are getting smarter, but also because humans are professionalising. Ancient hunter-gatherers mastered a very wide variety of skills in order to survive, which is why it would be immensely difficult to design a robotic hunter-gatherer. Such a robot would have to know how to prepare spear points from flint stones, how to find edible mushrooms in a forest, how to use medicinal herbs to bandage a wound, how to track down a mammoth and how to coordinate a charge with a dozen other hunters. However, over the last few thousand years we humans have been specialising. A taxi driver or a cardiologist specialises in a much narrower niche than a hunter-gatherer, which makes it easier to replace them with AI.

5. Thinking, Fast and Slow

Why read it?

In this book, Nobel Prize laureate, Daniel Kahneman, explores the way we think and the multiple kinds of unconscious biases we have when making decisions. His theories are a product of decades of research into behavioural economics and give insight into when we should and shouldn't trust our intuitions.

Kahneman introduces the "dual-process" model of the brain, namely 2 very different ways of thinking, which he calls System 1 and System 2. System 1 is our fast way of thinking that works by pattern recognition, optimised by millions of years of evolution. System 2, on the other hand, is our slow, rational way of thinking.

This might be one of the most informative books I have read and it definitely changed the way I look at my own decisions.

Why is it relevant to AI?

When building AI systems or even AGI, we are likely to incorporate our own biases into it. The more we know about them and realise they are there, the more we can build a safe and fair AI.

If the algorithm seems biased, the issue is likely to be upstream with the data the algorithm was trained on or the model design specified by engineers or researchers building the AI model.

Memorable quote

A reliable way to make people believe in falsehoods is frequent repetition, because familiarity is not easily distinguished from truth. Authoritarian institutions and marketers have always known this fact.