Machine Learning Methods and Evaluation

Although logistic regression has "regression" in its name, it is a classification method that converts the result of a linear computation into a probability form to determine a class. Predicting a continuous value directly is the job of methods for regression problems, such as linear regression, and being drawn to that answer by the name is a classic mistake. Finding clusters without training data is clustering, a topic in unsupervised learning, and improving action choices guided by reward is reinforcement learning — both placed in separate subsections.

A regression problem predicts a continuous value, such as sales or temperature, and a classification problem predicts a discrete class, such as whether an email is spam. Swapping these two is the most common mistake. Supervised learning requires a pair of features and a training label for either type of problem, so it is not the case that only one of them needs just features. Learning guided only by reward is reinforcement learning, and does not describe supervised learning.

Simple regression analysis has one explanatory variable, and multiple regression analysis has two or more. Since the characters for "simple" and "multiple" directly indicate the number of explanatory variables, an explanation that swaps them is clearly wrong. Both are regression methods that predict a continuous value, so it is also incorrect to say one of them is for classification. Furthermore, regression analysis is supervised learning, requiring training labels regardless of the number of variables.

SVM performs margin maximization, drawing the boundary at the position that maximizes the margin between it and the nearest data point. These closest points are called support vectors. Repeatedly adding learners while increasing the weight of misclassified data is the idea behind boosting, and finding the direction of maximum variance is the idea behind principal component analysis (PCA) — neither describes SVM. Drawing the boundary through the midpoint between centroids does not consider margin at all.

The kernel trick makes nonlinear separation possible without actually computing the coordinates after mapping to a higher-dimensional space, by directly computing the inner product after that mapping using a kernel function. The reason it is called a "trick" is precisely that the coordinates do not need to be computed one by one, so an explanation saying it computes them first and then separates has the direction reversed. Splitting data without training labels is a clustering topic, and it is not a mechanism for reducing the number of training records.

Bagging builds learners in parallel and independently of one another from training data created by bootstrap sampling, then takes a majority vote or average. Boosting adds learners sequentially, with each new one weighting more heavily the examples the previous one got wrong, so it depends on order and cannot be built in parallel. Swapping parallel and sequential is the most common mistake here. Both use training labels as supervised learning, and the types of weak learner that can be used are not limited to decision trees or linear regression.

A random forest is a representative example of bagging: it creates slightly different training data through bootstrap sampling, which is sampling with replacement, and grows many decision trees in parallel and independently from that data, then takes a majority vote or average. Adding trees one by one while weighting more heavily the previous tree's errors is the description of the boosting side, such as gradient boosting, and swapping this is a classic mistake. It uses not a single tree but a large number of them, and what is combined is decision trees, not linear regressions.

A decision tree is a method that arranges branching conditions on features into a tree shape, and its strength is that a person can read which condition led down which branch. On the other hand, the deeper the tree is grown, the more it memorizes fine idiosyncrasies of the training data, making a single tree prone to overfitting. Growing it deeper actually improves the fit to the training data, and the problem is that performance drops on new data anyway, so saying the fit necessarily worsens is incorrect. A decision tree is supervised learning and can be used for both classification and regression.

An AR model explains the current value of a single time series using that series's own past values. A VAR model extends this to multiple time series, having them explain each other using each other's past values. The difference is one time series versus multiple, so an explanation that swaps this is wrong. Both are models for handling time series, not tools for extracting local image features or audio frequency components, nor is the distinction one of needing training labels or not.

Multiclass classification is a classification problem that determines which of three or more classes something belongs to, and assigning a handwritten digit to one of 0 through 9 is a typical example. Predicting a continuous value, such as a high temperature value, is a regression problem, not classification. Finding clusters of similar customers is clustering, a form of unsupervised learning, and choosing a move based on reward is reinforcement learning — neither is a classification problem in supervised learning to begin with.

The syllabus lists as a goal understanding that unsupervised learning requires only features. Without correct-answer labels, unsupervised learning finds the structure inherent in the data itself through clustering or dimensionality reduction. Presupposing pairs of features and training labels is supervised learning, and being guided by reward is reinforcement learning — each placed in a separate subsection. Discarding remaining data is also not a description of unsupervised learning.

The k-means algorithm is a non-hierarchical clustering method that fixes the number of clusters k in advance and repeats the operations of assigning each point to the nearest centroid and recomputing the centroids. Repeatedly merging the closest items without deciding the number of clusters is hierarchical clustering, such as Ward's method, which can produce a dendrogram (tree diagram). Maximizing the margin is SVM, a form of supervised learning, and estimating topics from how words appear is a topic model such as latent Dirichlet allocation (LDA).

Ward's method is hierarchical clustering that repeatedly merges the closest items in order, and the process of merging can be drawn as a dendrogram (tree diagram). Its advantage is that the number of clusters can be decided afterward by looking at where to cut. It is k-means that is non-hierarchical and fixes the number of clusters in advance, and it does not produce a dendrogram. Ward's method is neither a dimensionality reduction method nor a supervised learning method, and it does not use correct-answer labels.

PCA is linear dimensionality reduction that re-orients the axes toward the direction of maximum variance, while t-SNE is a nonlinear method that maps to a lower dimension while preserving the closeness of nearby points, and is often used for visualization in two or three dimensions. Swapping linear and nonlinear here is a classic mistake. Estimating the topics of a document is a topic model such as latent Dirichlet allocation (LDA), and decomposing a matrix is singular value decomposition (SVD). Both PCA and t-SNE are unsupervised learning, so neither requires training labels.

Because collaborative filtering recommends based on the similarity of ratings between users or between products, a new user or new product with no rating history yet provides no clue to go on, making it hard to produce recommendations. This is the cold start problem. Content-based filtering, which relies on a product's description text or attributes themselves, is discussed alongside it as an approach that compensates for this weakness. It is not about computation time or inventory.

Reinforcement learning interacts with an environment through trial and error, learning a policy that maximizes the cumulative reward received. The guide is reward, not a correct-answer label for each move, so it is not a form of supervised learning. Using only closeness of distance as a guide is the idea behind clustering and other unsupervised learning, and word occurrence counts are a feature-representation topic in natural language processing — neither is the learning signal of reinforcement learning. Evaluating the future with a discount rate, in case reward is not returned immediately, is also a characteristic of this framework.

The syllabus lists as a goal understanding the two representative approaches of learning a value function and learning a policy. The former estimates a state-value function or an action-value function and then chooses an action based on it, with Q-learning and SARSA as representative examples. The latter represents the policy itself with parameters and updates it directly, with policy gradient methods and REINFORCE as representative examples. Actor-Critic combines an Actor responsible for the policy and a Critic responsible for the value, having both together. An explanation saying only one of the two exists, or that neither can be a target of learning, is incorrect.

Both are reinforcement learning methods that update an action-value function, differing in what is used for the update. Q-learning uses whichever available action at the next state has the maximum value, while SARSA updates using the value of the next action actually chosen. Swapping the two is a classic mistake. Q-learning also updates an action-value function, so saying only one of them uses one does not hold. Both are frameworks that learn from reward, so it is also wrong to say training labels are required.

The arm with the highest estimated value is chosen in two ways: through the greedy choice, or by chance when the random choice happens to land on that same arm. The former is 1 minus 0.1, giving 0.900, and the latter is 0.1 divided by 4, giving 0.025, for a total of 0.925. 0.900 is the value obtained by overlooking the chance of landing on it via the random choice; 0.250 is the probability of choosing uniformly from the 4 arms; and 0.100 is just ε itself. The larger ε is, the more exploration occurs, and this probability falls.

Multiplying 4 by 0.5 gives 2, multiplying 12 by 0.5 squared, i.e., 0.25, gives 3, and multiplying 8 by 0.5 cubed, i.e., 0.125, gives 1, for a total of 6.0. 24.0 is the value obtained by adding 4, 12, and 8 with no discounting applied at all; 12.0 is that 24 multiplied by 0.5 just once; and 7.5 is obtained by reversing the order of multiplication, applying 0.125 to 4, 0.25 to 12, and 0.5 to 8 — all common mix-ups. Since the role of the discount rate is to value more distant future rewards less, the nearer ones carry the larger weight.

Precision takes as its denominator the items predicted positive, so dividing the 40 true positives by 50, the sum of 40 true positives and 10 false positives, gives 0.80. 0.67 is recall, which takes as its denominator the items that are actually positive, dividing 40 by 60, the sum of 40 true positives and 20 false negatives. 0.85 is accuracy, the proportion of the 170 correctly predicted out of all 200 cases, and 0.73 is the F-score, the harmonic mean of the precision 0.80 and the recall 0.67 — all values obtained by computing a different metric.

Precision is "of those predicted positive, the proportion that were actually positive," so its denominator is on the prediction side. Recall is "of those actually positive, the proportion that were caught," so its denominator is on the actual side. Swapping the two is the most common mistake, so it helps to recall the difference as prediction versus actual. Taking the total data count as the denominator is accuracy, and a metric that takes only true negatives or only false negatives as its denominator is neither of these two. When you want to reduce false detections, weight precision more; when you want to reduce missed detections, weight recall more.

The F-score is the harmonic mean of precision and recall, so multiplying 2 by 0.54, the product of 0.6 and 0.9, gives 1.08, and dividing that by 1.5, the sum of 0.6 and 0.9, gives 0.72. 0.75 is the value obtained by computing an arithmetic mean instead of a harmonic mean; 0.54 is simply the product of the two; and 1.50 is simply their sum. Worth confirming too is that a harmonic mean is pulled toward the smaller value, so it comes out smaller than the arithmetic mean of 0.75.

Since only 1% is positive, simply answering negative for everything still gets 99% right, so accuracy is 99%. However, because not a single actual positive is caught, recall is 0%, making this model useless. Looking only at accuracy when class counts are imbalanced leads to a misjudgment, so recall, F-score, and the ROC curve and AUC also need to be checked together. In this setting, accuracy never comes out to 1% or 50%.

The ROC curve is drawn by varying the decision threshold and plotting the true positive rate on the vertical axis against the false positive rate on the horizontal axis, and the area beneath it is the AUC. The closer the AUC is to 1, the better, while 0.5 means a level equivalent to random judging. Putting precision on the vertical axis and recall on the horizontal axis produces a different curve altogether, which is where the classic mix-up occurs. Accuracy paired with data count, or error paired with iteration count, are different graphs for looking at how training is progressing.

Both AIC and BIC are metrics that add together how well a model fits the data and a penalty for having more parameters, and a smaller value is considered a better model. Treating a larger value as better is wrong, and this is the point most often targeted. Choosing based only on fit to training data would end up always selecting models with more parameters, tending toward overfitting, which is why a penalty is included to balance it out. This is aligned with the idea of Occam's razor: do not make something more complex than necessary.

K-fold cross-validation splits the data into k parts, and repeats the procedure of using one part for validation and the rest for training, k times, averaging the results. Splitting 200 records into 5 gives 40 records per block, so validation uses 40 records each time, training uses the remaining 160, and this is repeated 5 times. Splitting only once and finishing is holdout validation, which is computationally lighter but more sensitive to how the split happens to fall. Using 160 records for validation confuses training and validation.

The squared errors are 4, 16, 16, and 0, summing to 36. Dividing this by the count of 4 gives 9, the mean squared error (MSE), and its square root, 3.0, is the RMSE. 9.0 is the answer left as MSE without taking the square root; 2.5 is the mean absolute error (MAE), the average of the absolute errors 2, 4, 4, and 0; and 6.0 is obtained by forgetting to divide by the count and taking the square root of the sum of squares, 36, directly. RMSE is easy to read because it shares the same unit as the original error.

The sum, 84, divided by the count, 7, gives a mean of 12. The 4th value when sorted in ascending order is 6, so the median is 6, and 4, which appears twice, is the mode. The key point is that only the mean gets pulled substantially by the outlier 50, while the median and mode barely move — an example illustrating why you should not judge based on a single summary statistic alone. Swapping mean and median, or confusing median and mode, are common mistakes.

Spurious correlation is the phenomenon in which correlation appears between two things that have no direct causal relationship, because of a common underlying factor — as when ice cream sales and the number of drowning accidents move together because of a shared factor, temperature. The lesson here is that correlation does not necessarily imply causation, so a positive correlation coefficient does not mean one thing is the cause of the other. Also, even a correlation coefficient of 0 can hide a non-linear relationship, so it cannot be concluded with certainty that no relationship exists. How outliers are handled is a separate issue from this.