P.S., GPT-3, whose accomplishments have been much ballyhooed in popular media [footnote], belongs to the class of transformer nets, many-layered encoder/decoder networks that have proven extremely useful for tasks such as language translation and language production. However, it is extremely important not to confuse language production, or even translation, for language comprehension [once again, see the footnote]. GPT-3 was trained on a corpus of texts it would take a human being over 3,000 years to read. It's only representation of linguistic meaning is in the co-occurrence statistics of words in sentences across corpora. For those who know what this means, GPT-3 is the very embodiment of the most extreme version of the Whorf-Sapir hypothesis that thought comes from language. (A moment's reflection will tell you how absurd this hypothesis is: It states that, instead of talking about things we think, we think only because we can talk.)

Footnote: A Google employee was recently reprimanded for declaring that GPT-3 had achieved "sentience" based on its ability to "converse". He has hired it a lawyer to represent it as a sentient being. This case is going to be heard by SCOTUS, a body of people decidedly unqualified to opine on the question of machine intelligence. More recently still, GPT-3 was asked to write a scientific paper about itself. The only (barely mentioned) flaw in its ability to "write" said paper is that its creators had to coax it into the process by asking it a series of detailed questions. In other words, it could not craft the paper on its own. This is no surprise: Doing something as complex as writing a paper entails having an understanding of the structure of an argument--an ability that is well beyond GPT-3, which is basically only capable of responding to simple questions/requests. (More formally, it is capable of computing mappings from inputs, including questions, to outputs, i.e., answers.) By contrast, composing a whole paper requires a cognitive architecture that can form goals and organize its knowledge hierarchically--tasks that are well beyond the abilities of any simple input-output mapping engine. For students of the Philosophy of Mind, the strengths and limitations of GPT-3 are a wonderful demonstration of the limitations of the Turing test of machine intelligence.

Sounds like you had a good meeting!

Why do people think you need language to think? Where does that come from? It seems like an entirely shallow interpretation of intelligence.
"If you can't explain it, you don't really understand it" is another related statement I abhor. There are things I can't explain because I don't have the words to describe them, and other things I could explain, but depends on the audience's understanding. Neither are related to my understanding.

It was a great meeting, @Joerg Fliege, thanks. It always is. We in the photo are among the few in the world who have not been swept away in the deep net excitement. It is disturbingly difficult to get our papers, which show the limits of these systems, published. Like Trumpism, deep nets have become a religion one tries to oppose at one's own peril. Fortunately for me, I'm long since tenured. The same cannot be said for most of the folks in the photo.

You're right, @Clara Listensprechen and @Sam Smith: The Sapir-Whorf/Whorf-Sapir hypothesis is absolutely absurd. I have no explanation for why people believe it beyond (1) some cherry-picked cross-linguistic demonstrations (which don't replicate; a hot-shot young professor at Stanford failed to get tenure due to their non-replication) on the "effects" of spatial metaphors on our comprehension of time and related "effects", and (2) more importantly I suspect, peoples' desire for easy answers to hard problems: The question of where our ideas come from is incredibly difficult; to say that "they come from language" is an easy (if absurd) answer.

“If you can’t explain it, you don’t really understand it” is another related statement I abhor.

I failed to comment on this earlier.

I'm gonna go out on a limb and say that If you can’t explain it, you don’t really understand it, with apologies to @Sam Smith.

As a scientist, I view a model (that is, a theory) as an explanation. Take Newtonian mechanics. The equation f = ma says that the force required to accelerate a body is equal to the mass of the body times the acceleration you wish to achieve. This model of acceleration is useful only because we can understand (i.e., explain) it: We know what the terms of the equation mean and how they translate to actions on our part.

One of the problems with deep nets as "models" of cognition is that we don't understand them any better than we understand the natural phenomena they are claimed to "explain". To me, this fact makes them useless as models. If I had a "model" of something... say, chemical reactions... and my "model" was completely opaque (e.g., "if 14 fairies dance on the head of a pin, then boom!, but if 16 fairies dance, then nirvana!"), then it would not be a model at all.

An "explanation" no one can understand is not an explanation at all. And a "model" no one can understand isn't a model. It's an exercise in engineering at best, and more likely an exercise in mental masturbation. GPT-3 is an exercise is mass group masturbation.

As a scientist, I stand squarely behind the statement that If you can’t explain it, then you don’t really understand it. Or to put it another way: If you claim to understand it but you can't explain it, then you're a fucking charlatan selling snake oil.

One of the problems with deep nets as “models” of cognition is that we don’t understand them any better than we understand the natural phenomena they are claimed to “explain”. To me, this fact makes them useless as models.

The computer's obscure sub-logic is not valuable if it does not rise to a level we can comprehend. Unless we keep the faith, perhaps.

I understand the computer sub-logic. (That's easy.) What makes deep nets incomprehensible is the fact they depend entirely on emergent phenomena. Many of these models depend on literally billions of free parameters. The learning algorithm does all the work. When it's done, the ~~model~~ system computes the input-output mapping you have trained it to, but you (the programmer) have no idea how it does so.

That's not a model. It's a fucking deck of Tarot cards.

"Entitlement" @jec? How so?

What makes deep nets incomprehensible is the fact they depend entirely on emergent phenomena. Many of these models depend on literally billions of free parameters.

I think I misspoke - or mis-termed. By “sublogic” I meant that which is under or beyond the execution of the code. I’m trying to think of a metaphor… like that which is subconscious in humans - strange and largley inaccessible, not made-up of the same stuff as ordinary conscious thought.

billions of free parameters

Nice. Now that is certainly beyond my ken.

That’s not a model. It’s a fucking deck of Tarot cards.

Hence my reference to Faith. We have a rigid faith in the results of what we are doing! ;-}

Alas we do, @tom grzyb.

@John Hummel

Thinking about thinking - this is where careful reading and proper choice of words makes all the difference. I was using the term "sublogic", while you are talking about "emergent" phenomena, which conceptually are very different notions (or images anyway). I wonder now if there is any way to compare what the computer is doing with what we humans (or other animals) are doing. Our brain is comprised of an exceedingly complex system, which is far from monolithic - which some people have characterized as a balance of chaotic functions. What ML AI is composed-of is very different - even in essence.

The Church-Turing thesis (named for Alfonzo Church and his grad student, the Alan Turing) states that any function that can be computed at all can be computed by some sort of machine (like Turing machine, a modern digital computer, or a brain). In this sense, nothing that our brain can do is out of reach for a computer, at least in principle. We just need to figure out what algorithms the brain is running -- again, at least in principle.

That said, I believe we are still far from human-like AI. And I personally believe current machine learning algorithms (such as CPT-3) are -- as promising as they appear to be -- a dead-end.

But what if what we are doing is not "computing", as such, but analog? I suppose a computer can model analog functions, but that would not deeply resemble the original.

@John Hummel for what its worth, I went to two conferences in the last two months, and at both times the scepticism against DNN was palpable.

The first one was a meeting of (mostly German) wind turbine engineers and reps from the wind energy industry. Given how energy production contracts are structured in Europe, nobody ever would sign a maintenance schedule for a field of turbines based on DNN. You would be liable for energy production in a situation where billions of € are at stake.

The second conference was on next generation autonomous systems. The defense guys present likewise said that nobody in the military would sign an order based on some inexplainable output from a net. Thats just how you get in front of a war crimes tribunal. The healthcare guys were thrilled about the vision of an automated Alfred the butler for elderly care, and saw great potential there. And they likewise stressed that without explainability and transparency, nobody would hire Alfred the butler-and-potential-killbot for their own parents.

I agree with you, @Clara Listensprechen, except that I would replace your phrase "... Asimov’s Three Laws would be part of the design" [emphasis mine] with "... Asimov’s Three Laws had better be part of the design."

Unfortunately, there is nothing in logic or computer science that demands that Asimov's laws would necessarily be part of the design of AI. Putting those laws in is up to there programmers. And so far, they are doing a shit job. For example, it is now possible to use AI to create video and audio "evidence" of things that never happened and things people never said.

Under the Church-Turing Thesis, the definition of "computing" is very broad, @tom grzyb, and encompasses anything and everything that can be construed as providing an "answer" (or response) to a "question (or stimulus/input). This definition covers everything from playing tic-tac-toe to performing surgery to deciding whom to marry to consummating said marriage. Literally everything. In other words, the phrase "is computable" basically translates to "can be done". (This definition applies only to things that can be done in a logical sense and does not apply to things that cannot be done in a physical sense, such as exceeding the speed of light: This can be done logically (so is computable) but cannot be done physically.)

Under the Church-Turing Thesis, the definition of “computing” is very broad, @tom grzyb, and encompasses anything and everything that can be construed as providing an “answer” (or response)

Well then we have nothing to worry about, since under such a definition we are comparing apples with apples and getting apples.

Thank you very, @Joerg Fliege! I'm glad to hear that when the rubber hits the road, the truth comes out. I hope these facts make their way into the scientific community as well.

Agreed, @Clara Listensprechen! One of the earliest computing devices ever discovered. When I was an undergrad, the kind of machine (computer program) on which I most now work would have been called an "inference engine". I still often think of it that way.

Speaking of inference engines, have you seen this one: https://www.proquest.com/openview/6e296fb11e81ff593a73821a70e5532e/1?pq-origsite=gscholar&cbl=18750&diss=y

... it looks to be useful ...

Yes, that looks extremely relevant. Thanks, @tom grzyb

I'm having trouble compiling the thing I referenced. But isn't that always the case?

... the point is - development software, in this day and age, is so far from usable by the run-of-the-mill hobbyist or practitioner as to be inaccessible.