This is not a page to read passively. It is a small contest problem to solve in order. At each checkpoint, stop, answer the question in your own notebook, and only then open the suggested answer. Each answer includes one short insight explaining why that step matters.
The goal is not to squeeze every possible leaderboard point from Titanic. The goal is to practise the routine a careful AI olympiad participant should be able to repeat under time pressure, using only ideas already introduced in this guide.
Get the data
Open the official Titanic competition data page , join the competition if Kaggle asks you to, and download:
train.csv labelled passengers used for training and validation;test.csv unlabelled passengers for the final predictions; andgender_submission.csv an example showing the required submission format.
Put all three files beside a new notebook. Do not use gender_submission.csv as hidden test labels: it is only a sample submission.
You should already be comfortable with Pandas , Scikit-learn Basics and Approaching a Typical Classical Machine Learning Problem . Every model and technique below has appeared earlier in the guide.
Tip: Contest mode
Run the notebook from top to bottom, keep one fixed validation procedure, and change one meaningful thing at a time. A slightly weaker model that runs reliably and writes the correct file is better than a complicated unfinished experiment.
1. Read the task before touching the data
A contestant should extract four facts immediately: the target, problem type, metric and submission format.
Exercise: Question 1 · What exactly is the task?
Read the competition overview and evaluation pages. What must the model predict, what kind of machine-learning problem is this, how is it scored, and which two columns must the submission contain?
Show answer and insight
Predict Survived: 1 if the passenger survived and 0 otherwise.
This is binary classification because there are two possible labels.
Kaggle scores the percentage of passengers classified correctly, which is accuracy .
The submission must contain PassengerId and Survived.
Insight: the metric decides how you compare models, while the submission specification decides whether Kaggle can score the result at all. Record both before modelling.
2. Load the files
Exercise: Question 2 · What shapes do you expect?
Load train.csv and test.csv, then print their shapes. Which file should have one extra column, and why?
Show answer and insight
INPUT Python import pandas as pd
= pd. read_csv( "train.csv" )
test = pd. read_csv( "test.csv" )
print ( "train:" , train. shape)
print ( "test: " , test. shape)
[OUTPUT] train: (891, 12)
test: (418, 11)
The training file has one extra column because it includes Survived, the label used to train the model.
Insight: shape is the fastest first check. It confirms that loading worked and immediately tells you how many labelled examples, test examples and columns you have.
Exercise: Question 3 · Which column is missing from test.csv?
Use the column names to find what appears in train but not in test. Do not answer by manually reading the headers.
Show answer and insight
INPUT Python print ( train. columns. tolist( ) )
print ( set ( train. columns) - set ( test. columns) )
[OUTPUT] ['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age',
'SibSp', 'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked']
{'Survived'}
Insight: comparing schemas with code is safer than scanning them by eye, especially when a real task contains hundreds of columns.
3. Inspect before deciding
Exercise: Question 4 · What kinds of columns are present?
Display the first few rows and the data types. Sort the columns into likely numerical features, categorical features, identifiers and columns that may be awkward to use directly.
Show answer and insight
INPUT Python display( train. head( ) )
print ( train. dtypes)
A sensible first grouping is:
Pclass is stored as an integer, but its values represent three passenger classes rather than a measured quantity, so treating it as categorical is reasonable.
Insight: a column's data type is only a clue. Its meaning decides how it should be processed.
Exercise: Question 5 · Where are the missing values?
Count missing values in both files. Which columns require a plan before most scikit-learn models can use them?
Show answer and insight
INPUT Python print ( "Training missing values:" )
print ( train. isna( ) . sum ( ) . loc[ lambda s: s > 0 ] . sort_values( ascending= False ) )
print ( "\nTest missing values:" )
print ( test. isna( ) . sum ( ) . loc[ lambda s: s > 0 ] . sort_values( ascending= False ) )
[OUTPUT] Training missing values:
Cabin 687
Age 177
Embarked 2
Age, Embarked and Fare need missing-value handling if they are used. Cabin is missing for most passengers, so the first clean solution can leave it out.
Insight: inspect training and test data separately. Fare looks complete in the training file but is missing once in the test file; a pipeline that handles missing numerical values protects the final prediction step too.
Exercise: Question 6 · Is the target balanced?
Count both target classes and calculate the survival rate. What accuracy would a model get by predicting the majority class for everyone?
Show answer and insight
INPUT Python print ( train[ "Survived" ] . value_counts( ) )
print ( train[ "Survived" ] . value_counts( normalize= True ) . round ( 3 ) )
[OUTPUT] Survived
0 549
1 342
Always predicting 0 would score about 0.616 accuracy.
Insight: this is the minimum useful baseline. A trained model that cannot beat it has not earned its complexity.
4. Ask small questions of the data
Exploration should answer a modelling question. Do not produce dozens of plots without deciding what each one is meant to reveal.
Exercise: Question 7 · Which simple groups differ most?
Before running code, predict whether survival rate will differ by Sex and Pclass. Then calculate the group means. Why does the mean of Survived equal a survival rate?
Show answer and insight
INPUT Python print ( train. groupby( "Sex" ) [ "Survived" ] . mean( ) . round ( 3 ) )
print ( )
print ( train. groupby( "Pclass" ) [ "Survived" ] . mean( ) . round ( 3 ) )
[OUTPUT] Sex
female 0.742
male 0.189
Because Survived is either 0 or 1, its group mean is the fraction of rows with value 1.
Insight: Sex and Pclass appear informative. This is association, not proof that either variable by itself caused survival.
Exercise: Question 8 · Can a small derived feature help?
SibSp counts siblings and spouses aboard, while Parch counts parents and children. Create FamilySize for both files, including the passenger themself. What would go wrong if you created it only in train?
Show answer and insight
INPUT Python for df in [ train, test] :
df[ "FamilySize" ] = df[ "SibSp" ] + df[ "Parch" ] + 1
print ( train[ "FamilySize" ] . describe( ) )
The + 1 includes the passenger. If the feature existed only in train, the model would later receive a different set of columns for test and prediction would fail.
Insight: deterministic feature engineering must be applied identically to every split. A loop here keeps the two transformations visibly the same.
5. Set up honest validation
The Kaggle test labels are hidden, so use part of train.csv as a local validation set. Keep Kaggle submissions for occasional confirmation, not for deciding every small change.
Exercise: Question 9 · Which columns should enter the first model?
Choose a small feature set from the columns already inspected. Should PassengerId be used as a feature? What should happen to Name, Ticket and Cabin in the first solution?
Show answer and insight
INPUT Python numeric_features = [ "Age" , "SibSp" , "Parch" , "Fare" , "FamilySize" ]
categorical_features = [ "Pclass" , "Sex" , "Embarked" ]
features = numeric_features + categorical_features
= train[ features]
y = train[ "Survived" ]
X_test = test[ features]
Do not use PassengerId: it identifies rows but has no intended predictive meaning. Leave Name, Ticket and Cabin out of the first model because using raw high-cardinality text well would require additional choices. They can be revisited after a reliable baseline exists.
Insight: begin with features you can justify and preprocess correctly. More columns do not automatically mean more information.
Exercise: Question 10 · How should the split be made?
Create an 80/20 training-validation split. Which two arguments make the comparison fair and repeatable?
Show answer and insight
INPUT Python from sklearn. model_selection import train_test_split
, X_val, y_train, y_val = train_test_split(
X,
y,
test_size= 0.2 ,
stratify= y,
random_state= 42 ,
)
print ( X_train. shape, X_val. shape)
print ( y_train. mean( ) . round ( 3 ) , y_val. mean( ) . round ( 3 ) )
stratify=y keeps the class proportions similar, and random_state=42 makes the same split appear every time.
Insight: if the split changes between experiments, you cannot tell whether a score changed because of the model or because of easier validation rows.
Exercise: Question 11 · What is the local baseline?
Fit a classifier that always predicts the most frequent training label. Measure validation accuracy before trying a real model.
Show answer and insight
INPUT Python from sklearn. dummy import DummyClassifier
from sklearn. metrics import accuracy_score
= DummyClassifier( strategy= "most_frequent" )
baseline. fit( X_train, y_train)
= baseline. predict( X_val)
baseline_score = accuracy_score( y_val, baseline_pred)
print ( round ( baseline_score, 3 ) )
The score should be close to 0.616, the overall majority-class rate.
Insight: record this number. Every later model is competing first against this trivial strategy, not against a perfect score.
6. Build preprocessing into the model
The first real model needs numbers with no missing values. Numerical and categorical columns need different treatment, so keep both paths inside one pipeline.
Exercise: Question 12 · How should each column type be prepared?
Choose a missing-value strategy for the numerical and categorical columns. Which additional transformation does logistic regression need for categories?
Show answer and insight
Fill missing numerical values with the training median.
Standardise numerical columns for logistic regression.
Fill missing categorical values with the most frequent training value.
One-hot encode categories into numerical indicator columns.
Ignore an unseen category at prediction time instead of crashing.
INPUT Python from sklearn. compose import ColumnTransformer
from sklearn. impute import SimpleImputer
from sklearn. pipeline import make_pipeline
from sklearn. preprocessing import OneHotEncoder, StandardScaler
def make_model ( model) :
numeric_steps = make_pipeline(
SimpleImputer( strategy= "median" ) ,
StandardScaler( ) ,
)
categorical_steps = make_pipeline(
SimpleImputer( strategy= "most_frequent" ) ,
OneHotEncoder( handle_unknown= "ignore" ) ,
)
preprocess = ColumnTransformer( [
( "numeric" , numeric_steps, numeric_features) ,
( "categorical" , categorical_steps, categorical_features) ,
] )
return make_pipeline( preprocess, model)
Insight: the imputer, scaler and encoder learn only from whichever data is passed to fit. Keeping them inside the pipeline prevents validation information from leaking into training.
Exercise: Question 13 · Where is the leakage?
Suppose someone fills missing ages with train["Age"].median() before creating the training-validation split. Why is that not the cleanest validation procedure, and what does the pipeline do differently?
Show answer and insight
The median was calculated using every labelled row, including rows that later enter the validation set. That lets a small amount of validation information influence training preprocessing.
Inside the pipeline, SimpleImputer is fitted only when the model is fitted. During validation it learns a median from X_train and reuses that value for X_val.
Insight: leakage is not limited to the target column. Any value learned from data—medians, means, category lists or scaling values—must be learned from the training portion only.
7. Train the first useful model
Exercise: Question 14 · Does logistic regression beat the baseline?
Build a logistic-regression pipeline, fit it on the training split and measure validation accuracy. Compare the result with baseline_score rather than judging it in isolation.
Show answer and insight
INPUT Python from sklearn. linear_model import LogisticRegression
= make_model( LogisticRegression( max_iter= 1000 ) )
logistic. fit( X_train, y_train)
= logistic. predict( X_val)
logistic_score = accuracy_score( y_val, logistic_pred)
print ( "baseline:" , round ( baseline_score, 3 ) )
print ( "logistic:" , round ( logistic_score, 3 ) )
The exact score can vary with software versions, but it should comfortably beat the majority baseline and usually land around the low 0.8s on this split.
Insight: this is already a complete solution: reproducible split, leakage-safe preprocessing, a fitted classifier and the correct metric. Improvements now have something trustworthy to compete against.
Exercise: Question 15 · What mistakes is the model making?
Accuracy gives one number. Build a confusion matrix to count the four kinds of prediction. Which entries are mistakes?
Show answer and insight
INPUT Python from sklearn. metrics import ConfusionMatrixDisplay, confusion_matrix
= confusion_matrix( y_val, logistic_pred)
print ( cm)
. from_predictions( y_val, logistic_pred)
Scikit-learn arranges the matrix as:
The off-diagonal entries are mistakes.
Insight: two models with similar accuracy can fail on different passengers. Inspecting error types is more informative than looking only at a leaderboard-style score.
8. Compare familiar models fairly
Keep the same features, split and metric. Only the estimator changes.
Exercise: Question 16 · Which model generalises best?
Compare logistic regression, a shallow decision tree and a random forest on the same validation rows. Predict which one will win before running the cell.
Show answer and insight
INPUT Python from sklearn. ensemble import RandomForestClassifier
from sklearn. tree import DecisionTreeClassifier
= {
"logistic regression" : make_model(
LogisticRegression( max_iter= 1000 )
) ,
"decision tree" : make_model(
DecisionTreeClassifier( max_depth= 4 , random_state= 42 )
) ,
"random forest" : make_model(
RandomForestClassifier(
n_estimators= 300 ,
max_depth= 6 ,
min_samples_leaf= 2 ,
random_state= 42 ,
n_jobs= - 1 ,
)
) ,
}
= { }
for name, model in candidates. items( ) :
model. fit( X_train, y_train)
pred = model. predict( X_val)
validation_scores[ name] = accuracy_score( y_val, pred)
print ( pd. Series( validation_scores) . sort_values( ascending= False ) . round ( 3 ) )
Your exact ordering may vary. The important comparison is that every candidate saw the same training rows and was measured on the same untouched validation rows.
Insight: do not choose the most sophisticated model by reputation. Choose from evidence produced by a fair local experiment.
Exercise: Question 17 · Why not trust one split completely?
One validation split may favour a model by chance. Use five-fold stratified cross-validation to estimate each candidate's average accuracy and its variation across folds.
Show answer and insight
INPUT Python from sklearn. model_selection import StratifiedKFold, cross_val_score
= StratifiedKFold( n_splits= 5 , shuffle= True , random_state= 42 )
cv_results = { }
for name, model in candidates. items( ) :
scores = cross_val_score( model, X, y, cv= cv, scoring= "accuracy" )
cv_results[ name] = scores. mean( )
print (
f" { name: 20s } mean= { scores. mean( ) : .3f } " f"std= { scores. std( ) : .3f } folds= { scores. round ( 3 ) } " )
The mean estimates typical performance, while the standard deviation shows how sensitive the result is to the particular held-out rows.
Insight: the pipeline is essential here. Cross-validation refits its preprocessing separately inside every fold, so each held-out fold remains unseen.
Exercise: Question 18 · Is the apparent improvement convincing?
Suppose one candidate's mean accuracy is 0.812 and another's is 0.817, while their fold-to-fold standard deviations are around 0.02. Can you confidently claim that the second model is better?
Show answer and insight
No. The difference of 0.005 is much smaller than the variation between folds. Treat the models as effectively tied unless repeated evidence shows a stable advantage.
When scores are close, prefer the model that is simpler, faster and less likely to fail. You can also inspect the individual fold scores and repeat the comparison with another fixed cross-validation seed if time permits.
Insight: a leaderboard or validation score is a noisy measurement, not an exact ranking of model quality.
9. Make one controlled improvement
Do not start a large blind search. First identify one model worth tuning and a small number of meaningful settings.
Exercise: Question 19 · Which settings would you test?
For the random forest, test a small grid of tree depths and minimum leaf sizes with cross-validation. Why should the preprocessing remain inside the searched pipeline?
Show answer and insight
INPUT Python from sklearn. model_selection import GridSearchCV
= make_model(
RandomForestClassifier(
n_estimators= 300 ,
random_state= 42 ,
n_jobs= - 1 ,
)
)
= GridSearchCV(
estimator= forest,
param_grid= {
"randomforestclassifier__max_depth" : [ 4 , 6 , None ] ,
"randomforestclassifier__min_samples_leaf" : [ 1 , 2 , 5 ] ,
} ,
scoring= "accuracy" ,
cv= cv,
n_jobs= - 1 ,
)
search. fit( X, y)
print ( search. best_params_)
print ( round ( search. best_score_, 3 ) )
The double-underscore names address settings inside the pipeline. Preprocessing must remain inside the search so that every cross-validation fold learns its own imputing and encoding rules.
Insight: this grid contains only nine combinations and tests settings that directly control tree complexity. It is a focused experiment, not an expensive attempt to search everything.
Warning: Do not tune against the public leaderboard
Submitting every small variation and keeping whichever public score is highest gradually overfits your decisions to that leaderboard subset. Use local validation and cross-validation for model selection, then submit occasionally as an external check.
10. Choose and refit the final model
Use the cross-validation evidence rather than assuming the tuned forest must win. The code below selects the strongest untuned candidate by mean cross-validation accuracy; you may instead use search.best_estimator_ if its cross-validation score is clearly better.
Exercise: Question 20 · Why fit again on all labelled rows?
Select the best candidate from cv_results, fit it using all 891 labelled passengers, and predict the 418 test passengers. Why is the old validation split no longer held back?
Show answer and insight
INPUT Python best_name = max ( cv_results, key= cv_results. get)
final_model = candidates[ best_name]
. fit( X, y)
test_pred = final_model. predict( X_test) . astype( int )
print ( "chosen model:" , best_name)
print ( "predictions:" , test_pred. shape)
print ( pd. Series( test_pred) . value_counts( ) . sort_index( ) )
Validation was needed to make modelling decisions. Once those decisions are finished, refitting on every labelled row gives the final model as much information as possible.
Insight: do not evaluate this refitted model on the same rows and report that training score as evidence. Its honest performance estimate came from validation and cross-validation.
11. Build the submission file
The last stage is part of the problem, not paperwork. A perfect prediction array in the wrong format still earns no score.
Exercise: Question 21 · What does Kaggle expect?
Load gender_submission.csv and inspect only its shape and column names. What does it tell you, and what does it not tell you?
Show answer and insight
INPUT Python sample_submission = pd. read_csv( "gender_submission.csv" )
print ( sample_submission. shape)
print ( sample_submission. columns. tolist( ) )
display( sample_submission. head( ) )
It shows that the file needs 418 rows and the columns PassengerId and Survived. Its prediction values are an example, not the hidden answers and not something the model should learn from.
Insight: sample submissions are schema documentation. Use their headers, row count and order as checks, never as ground truth.
Exercise: Question 22 · Create solution.csv
Create solution.csv with the exact required columns and row order. Then read it back and write assertions that would catch the most common submission mistakes.
Show answer and insight
INPUT Python solution = pd. DataFrame( {
"PassengerId" : test[ "PassengerId" ] ,
"Survived" : test_pred,
} )
solution. to_csv( "solution.csv" , index= False )
= pd. read_csv( "solution.csv" )
assert list ( checked. columns) == [ "PassengerId" , "Survived" ]
assert len ( checked) == len ( test) == len ( sample_submission)
assert checked[ "PassengerId" ] . equals( test[ "PassengerId" ] )
assert checked[ "Survived" ] . isin( [ 0 , 1 ] ) . all ( )
assert checked. isna( ) . sum ( ) . sum ( ) == 0
( checked. head( ) )
print ( checked. shape)
print ( checked[ "Survived" ] . value_counts( ) . sort_index( ) )
The file should begin with the header:
[OUTPUT] PassengerId,Survived
It should have 418 data rows, contain only integer predictions 0 or 1, preserve the test-file passenger order and contain no extra index column.
Insight: reading the file back tests what was actually written to disk, not merely what the in-memory DataFrame looked like.
12. Final contest check
Before uploading solution.csv, restart the notebook kernel and run every cell from the top. Then answer each question with yes:
Did the notebook load only the files that will be available during the task?
Is the target absent from every feature matrix?
Were PassengerId and the sample predictions excluded from training?
Is every learned preprocessing step inside the pipeline?
Were models compared using the same metric and validation procedure?
Are all randomised steps given a fixed random_state?
Was the chosen model refitted on all labelled rows?
Does solution.csv have exactly the required columns, row count and order?
Does the notebook recreate the file without relying on hidden state?
If every answer is yes, upload solution.csv on the Titanic competition page . Record the public score beside your local cross-validation result, but keep the local procedure as the basis for future changes.
Note: What a strong attempt looks like
A strong first contest attempt is not the notebook with the most features or the largest model. It is the one that reaches a valid submission early, measures every change honestly, keeps preprocessing reproducible and leaves enough time to rerun and verify the final file.
Where to go after the first submission
Only after the complete pipeline works should you try one change at a time. With the topics covered so far, sensible experiments include removing FamilySize, changing the logistic-regression regularisation, adjusting tree depth, or comparing a small number of random-forest settings. Keep a short table of the change and its cross-validation score so that every decision has evidence behind it.
Comments
Stuck on something, spotted a mistake, or found a resource that helped you? Tell us below.
Loading comments