Writing in the Artificial Intelligence Review in 1989, Jordan Pollack got the
big thing right and the important thing wrong. He said the connectionist
revival would keep growing, and it did, beyond anything he set out. What he
expected it to grow into was a field he called
connectionist fractal semantics,
in which the distributed pattern would become a new kind of symbol, one with
internal structure you could reason about, capable of pointing back to the
larger structure it came from. He had a name ready for the object: a
supersymbol. Nothing of that sort was ever built. The prediction failed in the
strangest possible direction, by being fulfilled in volume and refused in kind.
Between 1988 and 1992 artificial intelligence occupied a strange interval
between promise and afterlife. One system was fading and the other returning,
and for a few years both were audible at once. Symbolic AI still carried the
authority of the old dream: intelligence made explicit, filed, named,
formalised, replayed. A bureaucracy of mind, in which every concept has its
proper label and every conclusion can be traced back along a chain of reason
to where it came from. The machine would not learn the world so much as be
instructed in it, and for a while this seemed less like one option among
several than like the only description that could possibly be true.
By the end of the decade the dream had gone hollow. Expert systems, sold as
the industrial future of the field, turned out to be brittle contraptions,
impressive inside narrow corridors of competence and
expensive to keep upright
outside them. They worked when the world behaved like the system's map of the
world, and ordinary reality refused that containment, going on producing
exceptions, ambiguities, tacit meanings and half-known contexts that no rule
base absorbed.
Symbolic AI didn't disappear, though. It lingered the way an institution
lingers after its purpose has become uncertain, and the great
knowledge-engineering projects of the period have a melancholy grandeur about
them. If machines lacked common sense, then common sense could be entered by
hand, one assertion at a time, as though the entire background of human life
were a document awaiting transcription. Douglas Lenat began Cyc in July 1984
at MCC in Austin on roughly that premise. By the end of its first six years
the project had entered over a million assertions, and
Lenat's own estimate
of what remained was about two person-centuries of further work to reach the
hundred million he thought necessary before the system could begin learning
usefully on its own. Two centuries of clerical labour, budgeted, to arrive at
the starting line. The more the project encoded, the more clearly it showed
the abyss underneath knowledge: the residue of habit, embodiment, memory and
practical familiarity that people draw on constantly without knowing they are
doing it.
Connectionism was coming back at the same time, out of an earlier obscurity.
The two Parallel Distributed Processing volumes landed in 1986 and offered a
different picture, cognition as pattern emerging across many small
adjustments rather than the manipulation of clean symbols. Meaning spread
across weights instead of filed in a drawer. The claim that mattered for what
followed was not that this worked better. It was that the distributed pattern
was supposed to remain readable: microfeatures standing for something, a
representation you could open.
The revival had a spectral quality, because none of it was new. Pollack
described it plainly as the rebirth of a programme that thrived from the
forties through the sixties and was severely retrenched in the seventies. The
ideas had been there near the beginning and were pushed aside by the prestige
of symbolic reasoning; the key training algorithm had
already been written down in 1974
and left to sit. What surfaced in the late eighties was a path not taken,
resurfacing precisely as the official future began to decay.
The symbolic order got one more authoritative statement, and it was a good
one. Jerry Fodor and Zenon Pylyshyn published their critique in Cognition in
1988, opening with the observation that connectionist models were catching on,
that there were conferences and new books nearly every day, and that
the fan club
included the most unlikely collection of people. Their argument was
systematicity: anyone who understands "John loves the girl" understands "the
girl loves John", and a classical architecture explains that symmetry for
free. What looks like a concession in their paper isn't one. You can reconcile
the two, they write, all that's required is that you use your network to
implement a Turing machine, which is to say a network only gets systematicity
by becoming the thing it claimed to replace.
David Waltz, a year earlier, had doubted you could take a large randomly wired
network, show it enough raw sensory input and desired output, and get
intelligence out the other end. The learning space
for vision and audio was astronomically large, he said, and learning to
perceive by feedback seemed cognitively and technically unrealistic. On the
narrow question of whether the method scales, he was wrong, and the last
fifteen years are the refutation.
The asymmetry between those two objections is the thing I'd point at. Waltz
was answered. Fodor and Pylyshyn were not answered; they were outrun. Nobody
demonstrated that distributed representations achieve systematicity without
implementing a classical architecture underneath. The models simply got large
enough that the question stopped being asked, which is a different outcome
from being settled, and it left the philosophical objection intact and
unattended somewhere behind the industry.
So the lost future of that interval isn't symbolic AI. Symbolic AI failed
legibly, and a legible failure can at least be mourned on schedule. The loss
that goes unmarked is the connectionism imagined in 1989: the readable
microfeature, the supersymbol with inspectable internal structure, the
representation you could open and reason about. That research line didn't die
so much as get overtaken by systems whose representations nobody can read. We
have interpretability now as a field, staffed and funded, which is itself the
admission. It exists because the thing Pollack expected to come built in has
to be excavated after the fact, from the outside, with uncertain results.
Cyc, meanwhile, never stopped. The transcription continued for decades, and by
2017 the knowledge base held something like
24.5 million assertions, roughly a
quarter of the way to the line Lenat had named as the beginning. The project
outlived its own future, which is a quieter fate than collapse and much harder
to put a date on.
Sources:
-
Connectionism: Past, Present, and Future — Jordan B. Pollack, Artificial Intelligence Review, vol. 3, 1989
-
Connectionism and Cognitive Architecture: A Critical Analysis — Jerry Fodor and Zenon Pylyshyn, Cognition 28(1–2), 1988
-
Connectionist Models: Not Just a Notational Variant, Not a Panacea — David L. Waltz, TINLAP-3, Association for Computational Linguistics, 1987
-
What is AI? Cyc — Jack Copeland, AlanTuring.net
-
Cyc — Wikipedia
-
Parallel Distributed Processing, Volume 2 — Rumelhart, McClelland and the PDP Research Group, MIT Press, 1986