4.2.1 Scikit-learn Basics

Why scikit-learn?

In Pandas, we turned a data file into a feature table X and a label column y. Scikit-learn takes the next step: it trains a model on those examples and uses the model to make predictions.

The library contains most of the classical machine-learning tools used in this guide: linear and logistic regression, K-nearest neighbours, decision trees, random forests, K-means, preprocessing, metrics and cross-validation.

The best part is that they follow the same small pattern:

choose a model -> fit it on training data -> predict on new data -> measure the result

Learn that pattern once and a new model becomes much easier to try. The class name and settings change; the workflow usually does not.

You should already be comfortable with Python Fundamentals, NumPy Basics and Pandas. You do not need to know how every model works yet. The following modules explain the models one by one.

What you need to know

By the end of this module, you should be comfortable with these ideas:

That is enough to begin every classical machine-learning module in this guide.

Pick one main course

As with Python and pandas, do not collect several courses and half-finish all of them. Pick one, type the examples yourself and complete the exercises.

Our pick: Kaggle Learn Intro to Machine Learning

Kaggle Learn Intro to Machine Learning is a short, practical course that runs in your browser. It starts with a pandas table, trains scikit-learn models and checks each exercise immediately.

Do the lessons in order:

Continue with Intermediate Machine Learning

After the intro course, use Kaggle Learn Intermediate Machine Learning for missing values, categorical variables, pipelines, cross-validation and data leakage. These topics matter, but they make more sense after you can train and evaluate one simple model without help.

If you want more depth

• A complete course: Machine Learning in Python with scikit-learn is a free course made by scikit-learn contributors. Its first module covers tabular data, preprocessing and pipelines carefully.
• A short official tour: Getting Started is useful once you know the basic words and want to see the whole interface on one page.
• A reference: use the API Reference to look up one class or function. It is not meant to be read from beginning to end.

A quick tour

We will use one small dataset from start to finish. Iris contains 150 flowers. Each flower has four measurements, and the label says which of three species it belongs to. The dataset comes with scikit-learn, so the code works without downloading a file.

Load the data

from sklearn.datasets import load_iris

iris = load_iris(as_frame=True)
X = iris.data
y = iris.target

X.shape, y.shape

((150, 4), (150,))

X has 150 rows and 4 feature columns. y has one answer for each row. This is the same X and y convention introduced in Pandas.

Before modelling, inspect the names and the class counts:

print(X.columns.tolist())
print(y.value_counts().sort_index().tolist())

['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']
[50, 50, 50]

The three classes are balanced: each appears 50 times.

Split before training

If we train and test on the same flowers, the score tells us how well the model remembers its training data. We want to know how it handles flowers it has not seen, so we hold out a test set first.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    X,
    y,
    test_size=0.2,
    stratify=y,
    random_state=1,
)

X_train.shape, X_test.shape

((120, 4), (30, 4))

• test_size=0.2 keeps 20% of the rows for testing.
• stratify=y keeps the class proportions similar in both parts.
• random_state=1 makes the same split every time the cell runs.

The exact seed is not important. Reusing it is.

Train a model

A scikit-learn model is an object, just like the objects in Object-Oriented Programming. Creating it chooses the settings; fit learns from the training data; predict uses what it learned.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=200)
model.fit(X_train, y_train)

predictions = model.predict(X_test)
predictions[:5]

array([2, 0, 1, 0, 0])

Read the important lines aloud:

1. LogisticRegression(...) creates the model.
2. fit(X_train, y_train) learns from known examples.
3. predict(X_test) predicts the labels of unseen examples.

Almost every supervised estimator follows this pattern.

Measure it

For classification, accuracy is the fraction of predictions that are correct:

from sklearn.metrics import accuracy_score

accuracy_score(y_test, predictions)

0.9666666666666667

The model classified 29 of the 30 test flowers correctly. model.score(X_test, y_test) gives the same number because accuracy is the default score for this classifier.

Do not assume that score always means accuracy. A regressor usually returns $R^2$, and a competition may use F1, ROC AUC, mean squared error or another metric. Use the metric named in the task; Model Evaluation Metrics explains the choices.

Parameters and learned attributes

The value max_iter=200 is a parameter: a setting chosen before training. get_params shows the current settings.

Values learned during fit are attributes, and scikit-learn gives their names a final underscore:

print(model.get_params()["C"])
print(model.classes_)
print(model.coef_.shape)

1.0
[0 1 2]
(3, 4)

C is a setting. classes_ and coef_ appeared only after fitting. Calling predict before fit therefore raises NotFittedError.

Probabilities

Many classifiers can return a probability for each class:

model.predict_proba(X_test.iloc[:3]).round(2)

array([[0.  , 0.02, 0.98],
       [0.97, 0.03, 0.  ],
       [0.29, 0.71, 0.  ]])

Each row belongs to one flower. The columns follow model.classes_, so the first row says that class 2 has the largest estimated probability. Not every classifier provides predict_proba; check the class's Methods section in the API reference.

Transform data safely

Some models work better when features use a similar scale. A transformer learns how to change data. StandardScaler learns the mean and standard deviation of each training column, then uses those same values everywhere else.

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

print(X_train_scaled.mean(axis=0).round(2))
print(scaler.mean_.round(2))

[ 0. -0. -0. -0.]
[5.87 3.06 3.79 1.2 ]

The two verbs matter:

• fit_transform(X_train) learns from the training set and changes it.
• transform(X_test) only applies the already learned change.

Never fit preprocessing again on the test set. That would let the test data influence the model's preparation and can make evaluation misleading. This mistake is called data leakage.

Keep the steps in a pipeline

A pipeline joins preprocessing and a model into one estimator. Calling fit fits each step on the training data in order; calling predict applies those fitted steps to new data.

from sklearn.neighbors import KNeighborsClassifier
from sklearn.pipeline import make_pipeline

pipe = make_pipeline(
    StandardScaler(),
    KNeighborsClassifier(n_neighbors=5),
)

pipe.fit(X_train, y_train)
round(pipe.score(X_test, y_test), 3)

0.967

The scaler is fitted only on X_train, and every later prediction automatically goes through that same scaler. This is safer than keeping separate variables and hoping you remember every step.

Pipeline parameters use two underscores between the step name and the setting:

pipe.get_params()["kneighborsclassifier__n_neighbors"]

5

You will use names like this later with GridSearchCV in Hyperparameter Tuning.

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder

numeric = make_pipeline(
    SimpleImputer(strategy="median"),
    StandardScaler(),
)
categorical = make_pipeline(
    SimpleImputer(strategy="most_frequent"),
    OneHotEncoder(handle_unknown="ignore"),
)

preprocess = ColumnTransformer([
    ("num", numeric, ["age", "income"]),
    ("cat", categorical, ["city", "device"]),
])

model = make_pipeline(
    preprocess,
    LogisticRegression(max_iter=200),
)
model.fit(train_df, y_train)
predictions = model.predict(test_df)

Where things live

You do not need to memorise the whole library. Learn the pattern, then look up the class you need.

Imports tell you where an object lives:

from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.cluster import KMeans

When you meet an unfamiliar class, read its API page in this order: Parameters (what you choose), Attributes (what fit learns), then Methods (what you can do).

Common mistakes

When an error mentions dimensions or feature counts, inspect shapes first:

print(X_train.shape)
print(y_train.shape)
print(X_test.iloc[0].shape)    # one row collapsed to 1-D
print(X_test.iloc[[0]].shape)  # one-row 2-D table

(120, 4)
(120,)
(4,)
(1, 4)

How to practise

• Retype the whole workflow. Load, split, fit, predict and measure from a blank notebook.
• Change one thing. Swap LogisticRegression for DecisionTreeClassifier while keeping the rest of the workflow.
• Inspect shapes. Say what every row and column represents before fitting.
• Keep a baseline. Compare a real model with DummyClassifier; a complicated model should beat a simple guess.
• Use the docs actively. Look up one class and find a parameter, a learned attribute and a method.
• Break the code on purpose. Predict before fitting, pass one row the wrong way, or reorder columns; then read the final line of the error.

Check yourself

Try each exercise before opening its answer.

If these exercises felt comfortable, you know enough of the interface to start learning the models themselves. You do not need to memorise every estimator or preprocessing class first.

More practice

Finish Kaggle Learn Intro to Machine Learning, then repeat its workflow on a different built-in dataset such as load_wine or load_breast_cancer. Keep the code structure the same and inspect what changes: the number of rows, the feature names, the classes and the score.

When you are ready for mixed tables and safer preprocessing, continue with Kaggle Learn Intermediate Machine Learning. For a slower and deeper route, work through the first module of the scikit-learn MOOC.

What comes next

• Linear Regression: the first model, for predicting numbers.
• Logistic Regression: a linear model for classification.
• K-Nearest Neighbours: a model where scaling and pipelines immediately matter.
• Model Evaluation Metrics: choosing a measure that matches the task.