1 Overview
TipLearning objectives
After this lecture you should be able to
- describe the stages common to data science and ETL pipelines, and locate where statistical learning sits within them,
- distinguish the roles of data analyst, data scientist, and data engineer,
- classify a learning problem as supervised, unsupervised, or semi-supervised according to whether the response is observed,
- distinguish regression from classification by the type of response, and identify the assessment measure used for each, and
- describe the bias-variance tradeoff and explain how bias, variance, and irreducible error combine to determine expected test error as flexibility increases.
1.1 Outline
- Data Pipelines
- Data Roles
- Statistical Learning
1.2 Tidyverse - Data Science Pipeline
1.3 Altexsoft - ETL Pipeline
1.4 MonteCarlo.AI - Data Pipeline
1.5 Geeks for Geeks - Data Science Pipeline
1.6 Data Analyst vs Scientist vs Engineer
1.7 BLS - Data Scientist Outlook
1.8 Statistical Learning
| Learning | Response |
|---|---|
| Supervised | Observed |
| Unsupervised | Not observed |
| Semi-supervised | Observed for some data |
1.9 Supervised learning
| Regression | Classification | |
|---|---|---|
| Response | Quantitative | Qualitative |
| Prediction | Number | Category |
| Assessment | Mean squared error | Misclassification rate |