- The Creativity Code, by Marcus du Sautoy
This is the first of a series of reviews of books I have been reading in order to provide at least a veneer of familiarity with the material on AI that I am subjecting to critical analysis. Marcus du Sautoy’s book should be the easiest to review, because du Sautoy is a mathematician and his book recounts a mathematician’s attempt to come to grips with the implications of recent developments in AI, some of them dramatic, for the future of his own profession, which is also mine. The title announces that du Sautoy wants to understand, and to help his readers understand, what AI may mean for the future of human creativity more generally, but it is clear that mathematics is very much on his mind, as it is on mine.…
At one point du Sautoy defines his existential crisis as concern “whether the job of being a mathematician would continue to be a human one.” This is where the ways he and I understand “existential” subtly diverge. I have written on this newsletter, and I will be writing again, that mathematics is “one of the innumerable ways that humans have found to infuse our lives with meaning.” That is not a job description. From beginning to end du Sautoy’s book makes it clear that he doesn’t really think of mathematics or any creative activity as a “job,” but when the word appears in sentences about machines replacing human mathematicians, as it does consistently, it’s a hint that someone is looking at the wrong existential crisis.…
2. Human Compatible, by Stuart Russell
Comment here on this week’s entry on the Substack newsletter.
Mathematics without Apologies, by Michael Harris 
I think your explanation of the term “deep learning” is off from how it is used in the field of machine learning. Deep learning refers to the use of deep neural networks, i.e. those containing a large number of layers. “The machine mainly trains itself” could instead be used to describe “supervised learning” or “self-supervised learning”. (When you fill out ReCaptchas you are, in machine learning parlance, “supervising” some network running at Google, which is almost certainly deep).
For me the biggest determiner of the creativity of machine learning models, or other software, is not the difficulty of explaining where it came from (which seems like a rather dismal goal to shoot for) but rather the extent to which humans can learn interesting ideas from it. For example, I understand AlphaZero taught human chess players ideas, such as certain ways of attacking pawns, that they have gone on to apply in their games.
In fact some of these ideas were incorporated into Stockfish, which until recently was a good old-fashioned AI built on years of manually inputted human knowledge, based on searching through positions in a human-comprehensible way (if not replicable by humans in anything approaching the same amount of time) and then evaluating them by a human-chosen function. Using this they were able to defeat LeelaChess Zero, an open-source program built along the same lines as AlphaZero, at several computer Chess competitions until the maintainers of Stockfish too turned to machine learning methods once a way was found to make them compatible with Fishtest (the collaborative, community-based source of Stockfish’s strength).
LikeLike
I’m sure you’re right about this, and I’ll be more precise in the future when I review the more technical books that I read last spring.
You and du Sautoy seem to have similar criteria for creativity — see the paragraph in which I use the expression “genuine creativity.” Note that the goals against which the creativity is measured are those established by humans. I don’t think du Sautoy’s book mentioned any instances in which the AI established clearly-defined goals of its own.
LikeLike
It seems like there are fundamental difficulties with both (1) designing AI that come up with goals of their own, and (2) humans recognizing that AIs are coming up with such goals, if they in fact are doing that. For the second point, if the AI creates its own goal, and it’s not a goal that humans share, then the AI’s attempts to achieve this goal could look, to us, like failed attempts to achieve some other goal, or just like random behavior.
For example GPT-3 has the human-created goal of imitating human text, which it sometimes does quite well, and sometimes does poorly, and sometimes does with a mix of accuracy and inaccuracy that creates an interesting artistic effect. From my knowledge of how it works, I’m pretty confident that it doesn’t in a meaningful sense have the goal of creating a particular type of effect. But if another AI text generator did somehow develop the goal of producing a text that creates a particular effect, we would probably just think it was imitating human text well (if it’s a similar goal to one we often seek in our own writing) or poorly (if it’s dissimilar).
I guess in fact we have little idea what it means to make a new goal (unless we want to take a maximalist position – if I generate at random a function f, and then search for x that maximizes f(x), have I created and achieved the goal of enlarging f?). The easiest kind of goal to understand may be those that are created to fulfill some other goal, e.g., presumably the goal of “win at chess” was created at around the same time as the game of chess, and both were likely created to fulfill the pre-existing goal of “have fun”, although nowadays people may adopt the goal of “win at chess” for many other reasons, for example to win the respect of others. Some contemporary AI methods do a version of this. For example, AlphaZero develops “goals” of the form “I want to get my chess pieces into a configuration with such-and-such properties”, based on experience where getting the pieces into those configurations previously resulted in victory, and then will search for sequences of moves that achieve this. However, these examples are clearly on a short-term level, consisting of a few moves in a chess game, because they are subservient to the unchanging human-provided overall goal of winning the chess game.
LikeLike
I used the word “goal” loosely but I am aware that there is an extensive philosophical literature on goals, under different names. I have not yet begun to explore how goals are understood in contemporary psychology, but I have no doubt that there are profound disagreements about definitions. Probably the AI literature is also concerned with different ways to define goals. I will have to familiarize myself with the literature in all of these areas before I can say anything more precise.
LikeLike