History of AI and Fundamentals

The broadest frame is artificial intelligence; one approach within it is machine learning; and within that, the approach using neural networks with many stacked layers is deep learning. In the syllabus, all three terms appear together as keywords under "1. Definition of Artificial Intelligence." An explanation that places machine learning or deep learning as the broadest frame has the direction of containment reversed. Saying the three stand independently side by side also does not match the fact that deep learning is one approach within machine learning.

The four classifications of AI level are listed in the goals of syllabus section "1. Definition of Artificial Intelligence" as "simple control programs," "classical artificial intelligence," "machine learning," and "deep learning." Operating purely on a predetermined mapping between conditions and actions is a simple control program, exemplified by something like a thermostat that switches on and off at a set temperature. Classical artificial intelligence chooses a response through branching logic, search, or stored knowledge; machine learning learns regularities from data; and deep learning is the approach among these that uses a network with deeply stacked layers — all of these go beyond the stage where a person writes out every response by hand.

The AI effect is the phenomenon in which, once the mechanism behind an artificial intelligence becomes clear and its inner workings are understood, it comes to be seen as "mere automated processing, not intelligence," and its evaluation drops. The syllabus explicitly states, under the goals of "1. Definition of Artificial Intelligence," that one should "be able to explain the AI effect." The debate over the tipping point at which AI surpasses human intelligence is the singularity, which is a keyword under "2. Issues Debated in the Field of Artificial Intelligence." The tendency for performance to improve with data volume, and the inability to present reasons for a decision, are both separate topics from the AI effect.

An agent is a way of viewing something that observes its environment and chooses its own next action toward a goal. It is a term for capturing artificial intelligence in terms of "how it behaves" rather than "what mechanism it is built from," and in the syllabus it appears as a keyword under "1. Definition of Artificial Intelligence" alongside artificial intelligence, machine learning, deep learning, and the AI effect. An on-screen entry point for operation, a person who organizes data, and a mechanism that lends out computing resources are none of them what this term refers to.

ImageNet is a large-scale dataset of a huge number of images with classification labels attached, and ILSVRC is a competition that used it to compete on the accuracy of image recognition. In the syllabus, both appear as keywords under "6. Deep Learning," serving as material for explaining how deep learning has developed. They are not two names for the same model, nor is their relationship one of a labeling standard and a judging venue. What relates to Go is AlphaGo, which is also a keyword in the same subsection but is a separate thing.

LeNet is a neural network using convolution, built to recognize handwritten digits, and it is placed as a keyword under "6. Deep Learning" in the syllabus. What beat top human Go players is AlphaGo, the large-scale dataset with labels attached to a huge number of images is ImageNet, and what generates new text or images is generative AI — all separate keywords listed in the same subsection. Rather than memorizing just the names, it helps to keep in mind what each one targets.

AlphaGo is a program that beat top human Go players, and in the syllabus it is placed as a keyword under "6. Deep Learning" alongside ImageNet, ILSVRC, LeNet, the neocognitron, and generative AI. The goal of this subsection is to "be able to explain the history of how deep learning has developed," so it helps to memorize each name together with which scene of that development it represents. Transcribing spoken language, reading handwritten digits, and summarizing text are none of them achievements AlphaGo demonstrated.

A robot refers to the machine itself that has a body, and artificial intelligence is the part within it responsible for judgment. The two can overlap but are not the same thing. Many artificial intelligences have no body and run purely as software, and conversely there are robots that merely repeat fixed motions and contain no artificial intelligence. An explanation that swaps the judgment-bearing side and the body side has the direction backward, and it is neither a distinction based on whether something learns nor merely a difference in naming between fields.

The neocognitron is a model that took inspiration from how the human neural circuit works, aiming to handle visual information in stages by stacking layers. The syllabus's pairing of "the human neural circuit" with the neocognitron under "6. Deep Learning" can be read as making sure learners grasp where the idea of stacking layers came from. It is not that the wiring of the neural circuit was directly turned into computer circuitry, nor that research into the neural circuit has been elucidated. What relates to Go is AlphaGo, in the same subsection.

Search and Reasoning is the stage where a person builds the solution procedure; Knowledge Representation and Expert Systems is the stage where a person writes out expert knowledge; and Machine Learning and Deep Learning are the stage where regularities are learned from data. In other words, it is a flow in which the center of gravity shifted from a person writing things out to letting data do the learning. Note that the way of counting this flow as an ordinal "generation" never appears in the syllabus, so this lab refers to it simply by the subsection names. Shifts in the form of data, the location of computation, or who carries out the research are not what this ordering represents.

The Dartmouth Conference was a research conference held in 1956, spoken of as the venue where the term "artificial intelligence" was first used publicly. The test to distinguish humans from machines through text-only exchange was proposed in Turing's 1950 paper, a separate event that predates the Dartmouth Conference. Linking the two together is a classic error, so remember the year and the event separately. Research on expert systems and training methods for deeply stacked neural networks are both topics that came later than this conference.

The Turing test is a judging method in which, if a judge cannot distinguish a human from a machine through text-only exchange, the machine is deemed to have intelligence. The key point is that it does not provide a definition of intelligence itself, but rather proposes judging by externally observable behavior. The Loebner Prize is the actual contest that carries out this test in competition form, and it is also a keyword in the same subsection. It is neither a procedure that examines internal mechanisms nor a method of estimating a timeframe.

The Chinese Room is a thought experiment that argues, from the setup of a person merely rearranging symbols according to a manual without understanding Chinese, that even if behavior is indistinguishable from a human's, it does not necessarily mean understanding is present. In other words, it is a critique of the Turing test, brought forward as an objection to strong AI, the position that grants the existence of mind or understanding. Reading it as evidence supporting strong AI has the direction backward, and this is where the mix-up commonly occurs. It is not about the speed of symbol manipulation, nor about the relative merit of machine translation methods.

Strong AI and weak AI are distinguished by whether a machine has mind or understanding. Strong AI is the position that the machine truly understands, while weak AI is the position that it is merely a tool that behaves as if it understands. In contrast, general versus narrow is a distinction of applicable scope — how broad a range of tasks something can be used for — an entirely different axis. So having a broad applicable scope does not itself mean having a mind. An explanation that swaps the two has the direction reversed, and data volume, computing resources, processing speed, and cost are none of them the criterion for either distinction.

The frame problem is the problem that, when a certain action is taken, one cannot narrow down what changes and what does not change to only what is relevant, and handle it in finite time. In a word, it is a problem of narrowing down relevance. Connecting symbols to their real-world meaning is the symbol grounding problem; the work of writing out knowledge becoming massive is the knowledge acquisition bottleneck; and only being able to solve problems with limited conditions is the toy problem — all separate keywords listed in the same subsection.

The symbol grounding problem is the problem of how a system connects symbols with the real-world meaning they refer to. It is described in terms such as: even if a system has the word "zebra" and the knowledge that it means "a horse with stripes," that does not guarantee it is connected to the real animal in front of it. Not being able to narrow things down to only what is relevant is the frame problem, and the two are often asked about as a pair. Embodiment is the position that interaction with the environment through a body may be necessary, and it is one way of answering this problem. Assembling a sequence of actions, the echoing of information, and the number of features are none of them what this term refers to.

Embodiment refers to the position that the experience of interacting with the environment through a body may be necessary for intelligence to be established. It is discussed as one answer to the symbol grounding problem, which holds that symbols do not connect to real-world meaning — the idea being that having a body and touching the world is what creates that connection. It is not a claim that artificial intelligence must take the form of a robot to be practical, nor is it about being determined by part performance, nor about a procedure of turning bodily motion into formulas.

A toy problem refers to a limited problem with clearly fixed rules and states, stripped of real-world complexity. It is used in contexts describing how, while problems that fit this mold — such as mazes or puzzles — could be solved, real-world problems whose conditions are not clearly fixed remained out of reach. Note that it does not mean an "easy problem" but rather a setting that simplifies reality. It is neither teaching material for children, an unsolved difficult problem, nor a problem whose computational load is too large.

The knowledge acquisition bottleneck is the problem that the volume and difficulty of the work of manually writing out an expert's knowledge in the form of rules became a wall, preventing the approach of accumulating knowledge from spreading. The key point is that the cause lies not in a shortage of computing resources, data, or communication, but in the sheer effort of writing out knowledge itself. One attempt to reduce this effort is the interview system listed under "4. Knowledge Representation and Expert Systems" in the syllabus.

Rule-based machine translation is an approach in which a person writes grammar rules and a dictionary to assemble a translation, while statistical machine translation is an approach that selects the most plausible translation from statistics over a large volume of bilingual data. The flow from a person writing things out to letting data do the learning appears here within the single subject of translation. An explanation that swaps the two has the direction reversed. It is not a difference of unit handled, nor a difference between spoken and written language. Note that statistical machine translation is also listed, besides this subsection, under "27. Natural Language Processing."

Breadth-first search proceeds layer by layer from the one closest to the root, examining every node at the same depth before advancing to the next depth. It can find a shallow answer first, but tends to require remembering more nodes. Depth-first search descends as far as it can go in one go, and when it hits a dead end, backtracks and tries again. It needs to remember less, but the first answer it finds is not necessarily the shallowest one. An explanation that swaps the two has the direction reversed; choosing branches at random is the idea behind Monte Carlo methods, and traversing the tree from leaves to root does not apply to either.

The Minimax algorithm is a method for two-player games with alternating turns, in which the move most favorable to you is assumed to be chosen on your turn, and the move most unfavorable to you is assumed to be chosen on the opponent's turn, propagating the value of future positions back to the current one. Alpha-beta pruning is a refinement made during that search that cuts off branches once it is known that examining them further cannot change the outcome, reducing the amount to examine — it is not a separate evaluation concept from Minimax. It is neither a procedure for building the search tree nor one for double-checking the result afterward.

Brute force is the approach of examining every possible move one by one from the start. It is guaranteed to find an answer if one exists, but as the number of moves grows, the combinations increase sharply and it fails to finish in realistic time. This is exactly why refinements to reduce the range examined, such as alpha-beta pruning, become necessary. Examining promising branches in order of score, the Monte Carlo idea of trying random moves, and recording and reusing intermediate results are none of them brute force itself. A subject with clearly fixed states and rules, such as the Tower of Hanoi, is listed in the syllabus as practice material for this kind of search.

SHRDLU is research that let a world of blocks on a screen be manipulated through language instructions, demonstrating that language instructions could work within a world with limited conditions. STRIPS is an approach that expresses an action in terms of its "preconditions for execution" and "what changes after execution," assembling a sequence of actions that reaches a goal state. Swapping the two reverses the direction. Repeating trials with random numbers is Monte Carlo methods, and cutting off branches is alpha-beta pruning. Estimating molecular structure from mass spectrometry data is DENDRAL, and diagnosing bacterial infections is MYCIN, both placed under "4. Knowledge Representation and Expert Systems."

ELIZA is a conversational program that assembles a reply to match the pattern of the input sentence. It is cited as an example showing that something can appear human-like without actually understanding its content. Estimating molecular structure from mass spectrometry data is DENDRAL, diagnosing bacterial infections and recommending antibiotics is MYCIN, and the long-running effort to write down common sense is the Cyc project — all separate keywords listed in the same subsection. Be careful not to confuse something that diagnoses with something that converses.

MYCIN is an expert system that diagnosed bacterial infections and recommended a matching antibiotic, addressing the medical field. The chemistry-field system that estimated organic compound molecular structure from mass spectrometry data is DENDRAL, the one that took on university entrance exams is Todai Robot, and the one that competed with humans in question-answering on a quiz show is Watson. Since the pairing of name and field is asked about directly, memorize them together.

DENDRAL is an early expert system that estimated the molecular structure of an unknown organic compound from mass spectrometry data. Its field is chemistry, and it is often asked about paired with MYCIN, which addressed the medical field. The Cyc project is a long-running effort to keep writing in human common sense, Watson is a question-answering system, and Todai Robot is a project that took on university entrance exams — all separate items listed in the same subsection.

is-a is a superordinate-subordinate relation, expressing that a subordinate concept is included in a superordinate one, as in "a dog is an animal." has-a is a relation of having a part, viewed from the whole toward the part, as in "a car has tires." part-of is a relation of being a part, viewed from the part toward the whole, as in "a tire is part of a car." has-a and part-of describe the same connection viewed from opposite directions, so be careful not to confuse the direction. Expressing the same meaning, the opposite meaning, or order are none of them the relations these three represent.

The curse of dimensionality is the phenomenon in which, as the number of features handled — that is, the dimensionality — increases, data becomes sparser within that space, and the amount of data needed to maintain the same density grows sharply. Differences in distance between points also become harder to discern. An explanation that says reducing features increases the data needed has the direction reversed, and fit to training data, or the relationship between computing resources and accuracy, are separate topics. Note that in the syllabus this term is placed not under "7. Supervised Learning" but under "5. Machine Learning."

In a rule-based method, a person writes out what conclusion follows under what condition. Because it operates exactly as written, it is easy to show the reason for a decision, but it becomes impossible to write out as exceptions increase. Machine learning finds regularities from data, so it can handle a volume of case distinctions too large to write out, but in exchange it is harder to show the reason for a decision, and it requires data of sufficient quality and quantity. An explanation that swaps the two has the direction reversed — it is the rule-based side that makes it easy to show the reason. Spam filters, recommendation engines, and statistical natural language processing are examples of machine learning applications listed in this subsection.