Species (conc, heik)Jnt1: first jointJnt2: second jointconc, 18 heik)
Hyperplane in \(p\) dimensions: the points \(X = (X_1, \ldots, X_p)^\top\) with
\[ \beta_0 + \beta_1 X_1 + \beta_2 X_2 + \cdots + \beta_p X_p = 0, \]
For any point \(x = (x_1, \ldots, x_p)^\top\), write
\[ f(x) = \beta_0 + \beta_1 x_1 + \cdots + \beta_p x_p = \beta_0 + \beta^\top x, \]

\(\beta_0 = -70\), \(\beta_1 = 1\), \(\beta_2 = -1\)
\[ t = \frac{f(x)}{\|\beta\|} = \frac{\beta_0 + \beta^\top x}{\|\beta\|}, \]
\[ y_i = \begin{cases} 1 & \text{if observation } i \text{ is in one class,}\\ -1 & \text{if observation } i \text{ is in the other class,}\end{cases} \]
heik, \(y_i = -1\) for conc; \(n = 36\), \(p = 2\)\[ y_i\left(\beta_0 + \beta_1 x_{i1} + \cdots + \beta_p x_{ip}\right) > 0 \quad \text{for all } i = 1, \ldots, n. \]

Legend: direction of \(\beta = (\cos\theta, \sin\theta)^\top\), by its angle \(\theta\) from the Jnt1 axis

Margin \(= \min_i y_i f(x_i)\), with a unit normal vector
Maximal margin hyperplane: the separating hyperplane with the largest margin
\[ \begin{aligned} \underset{\beta_0, \beta_1, \ldots, \beta_p, M}{\text{maximize}} \quad & M \\ \text{subject to} \quad & \sum_{j=1}^p \beta_j^2 = 1, \\ & y_i\left(\beta_0 + \beta_1 x_{i1} + \cdots + \beta_p x_{ip}\right) \ge M \quad \text{for all } i = 1, \ldots, n, \end{aligned} \]
Fixed unit normal vector \(\beta\):
\[ 2M \le \Delta(\beta) = \min_{i: y_i = 1}\beta^\top x_i - \max_{i: y_i = -1}\beta^\top x_i, \]
\[ \beta_0 = -\frac{1}{2}\left(\min_{i: y_i = 1}\beta^\top x_i + \max_{i: y_i = -1}\beta^\top x_i\right). \]
Best hyperplane: midway between the classes, margin \(\Delta(\beta)/2\)

Call:
svm(formula = Species ~ Jnt1 + Jnt2, data = beetles, kernel = "linear",
cost = 1e+05, tolerance = 1e-08, scale = FALSE)
Parameters:
SVM-Type: C-classification
SVM-Kernel: linear
cost: 1e+05
Number of Support Vectors: 3
\[ \widehat\beta_0 = \frac{\widehat w_0}{\|\widehat w\|}, \qquad \widehat\beta = \frac{\widehat w}{\|\widehat w\|}, \qquad \widehat M = \frac{1}{\|\widehat w\|}. \]
rho; \(-\widehat w\) is the weighted sum of the support vectors (rows of SV, weights coefs)# w and w0 with heik on the positive side (svm() makes the class of the
# first beetle, conc, positive).
w <- -drop(t(mmc_fit$coefs) %*% mmc_fit$SV)
w0 <- mmc_fit$rho
# Unit-norm estimates and the margin.
norm_w <- sqrt(sum(w^2))
beta_hat <- c(beta0 = w0, w) / norm_w
M_hat <- 1 / norm_w
round(c(beta_hat, M = M_hat), 4) beta0 Jnt1 Jnt2 M
45.4025 0.3714 -0.9285 0.6499
# Signed distance of each beetle, and its distance on its own side.
x_beetles <- as.matrix(beetles[c("Jnt1", "Jnt2")])
beetles <- beetles |>
mutate(f_hat = drop(beta_hat["beta0"] + x_beetles %*% beta_hat[-1]),
distance = y * f_hat)
# Beetles on the margin: distance M_hat, up to the solver's accuracy.
beetles |>
filter(distance - M_hat < 1e-6) |>
select(Specimen, Species, Jnt1, Jnt2, f_hat) Specimen Species Jnt1 Jnt2 f_hat
1 5 conc 171 118 -0.6499337
2 13 conc 201 130 -0.6499337
3 35 heik 187 123 0.6499337
| Beetles | \(n\) | \(\widehat\beta_0\) | \(\widehat\beta_1\) | \(\widehat\beta_2\) | \(\widehat M\) | Support vectors |
|---|---|---|---|---|---|---|
| All beetles | 36 | 45.4025 | 0.3714 | -0.9285 | 0.6499 | 5, 13, 35 |
| Without beetle 11 | 35 | 45.4025 | 0.3714 | -0.9285 | 0.6499 | 5, 13, 35 |
| Support vectors only | 3 | 45.4025 | 0.3714 | -0.9285 | 0.6499 | 5, 13, 35 |
| Without beetle 13 | 35 | 3.4795 | 0.5450 | -0.8384 | 2.2638 | 1, 5, 35 |

New beetle \(x^* = (x^*_1, \ldots, x^*_p)^\top\), classified by the sign of
\[ \widehat f(x^*) = \widehat\beta_0 + \widehat\beta_1 x^*_1 + \cdots + \widehat\beta_p x^*_p, \]
concheik
DS 3030