Machine Learning Interview Questions & Answers

Answer: Machine Learning is a subset of artificial intelligence that enables computers to learn and make decisions from data without being explicitly programmed for every task. Instead of following pre-written instructions, ML algorithms identify patterns in data and use these patterns to make predictions or decisions on new, unseen data.

Key Points to Mention: Pattern recognition, learning from data, making predictions, three main types (supervised, unsupervised, reinforcement learning).

Answer:

• Supervised Learning: Uses labeled training data where both input features and correct outputs are provided. The algorithm learns to map inputs to outputs. Examples: email spam detection, house price prediction.
• Unsupervised Learning: Works with unlabeled data to discover hidden patterns or structures. No correct answers are provided during training. Examples: customer segmentation, anomaly detection.

Follow-up: Be ready to explain semi-supervised learning and provide real-world examples of each type.

Answer: Overfitting occurs when a model learns the training data too well, including noise and random fluctuations, making it perform poorly on new, unseen data. The model has high accuracy on training data but low accuracy on validation/test data.

Prevention techniques:

• Cross-validation to assess model generalization
• Regularization (L1/L2) to penalize complex models
• Early stopping during training
• Reducing model complexity (fewer features/parameters)
• Increasing training data size
• Dropout in neural networks

Key Insight: Always mention the bias-variance tradeoff and how overfitting relates to high variance.

Answer: The bias-variance tradeoff is a fundamental concept that describes the relationship between model complexity and prediction error:

• Bias: Error from oversimplifying assumptions. High bias leads to underfitting (missing relevant patterns).
• Variance: Error from sensitivity to small fluctuations in training data. High variance leads to overfitting.
• Tradeoff: As model complexity increases, bias decreases but variance increases. The goal is finding the sweet spot that minimizes total error.

Practical Example: Linear regression (high bias, low variance) vs. Decision trees (low bias, high variance).

Answer: Cross-validation is a technique to assess how well a model generalizes to unseen data by partitioning the dataset multiple times and training/testing on different subsets.

Common types:

• K-Fold CV: Dataset split into k equal parts; model trained on k-1 parts, tested on remaining part, repeated k times
• Stratified CV: Maintains class distribution in each fold
• Leave-One-Out CV: Special case where k equals dataset size

Importance: Provides reliable estimate of model performance, helps detect overfitting, guides hyperparameter tuning.

How it works: Decision trees create a tree-like model of decisions by recursively splitting data based on feature values that best separate classes or reduce variance. Each internal node represents a decision based on a feature, branches represent outcomes, and leaf nodes represent predictions.

Follow-up: Explain splitting criteria (Gini impurity, entropy, information gain) and pruning techniques.

Random Forest: An ensemble method that combines multiple decision trees, where each tree is trained on a random subset of data (bootstrap sampling) and random subset of features. Final prediction is made by averaging (regression) or majority voting (classification).

Why it's better:

• Reduces overfitting: Averaging multiple trees reduces variance
• Handles missing values: Uses proximity measures for imputation
• Feature importance: Provides measures of variable importance
• Robust: Less sensitive to outliers and noise
• No hyperparameter tuning needed: Works well with default parameters
• Parallel training: Trees can be trained independently

Key Concept: Explain the difference between bagging and boosting, and mention out-of-bag (OOB) error estimation.

When to use: Bagging when reducing variance is priority; Boosting when reducing bias is priority.

SVM: A classification algorithm that finds the optimal hyperplane (decision boundary) that maximizes the margin between different classes. The hyperplane is positioned to maximize distance to the nearest training examples (support vectors).

Kernel Trick: A technique that allows SVM to handle non-linearly separable data by mapping it to higher-dimensional space where it becomes linearly separable, without explicitly computing the transformation.

Common kernels:

• Linear: For linearly separable data
• RBF (Radial Basis Function): Most popular, handles circular boundaries
• Polynomial: For polynomial decision boundaries
• Sigmoid: Similar to neural networks

Key Points: Mention C parameter (regularization), gamma parameter (kernel coefficient), and that SVM works well with high-dimensional data.

K-Means: An unsupervised clustering algorithm that partitions data into k clusters by minimizing within-cluster sum of squared distances. It iteratively assigns points to nearest centroid and updates centroids until convergence.

Algorithm steps:

1. Initialize k cluster centroids randomly
2. Assign each point to nearest centroid
3. Update centroids to mean of assigned points
4. Repeat steps 2-3 until convergence

Limitations:

• Must specify k in advance
• Sensitive to initialization (local minima)
• Assumes spherical clusters
• Sensitive to outliers
• Struggles with varying cluster sizes and densities
• Only works well with numerical data

Solutions: Mention elbow method for choosing k, K-means++, and alternatives like DBSCAN for non-spherical clusters.

Neural Networks: Inspired by biological neurons, they consist of interconnected nodes (neurons) organized in layers. Each connection has a weight, and neurons apply activation functions to weighted inputs.

Backpropagation: The learning algorithm that trains neural networks by:

1. Forward Pass: Input data flows forward through network to produce output
2. Loss Calculation: Compare predicted output with actual output
3. Backward Pass: Calculate gradients of loss with respect to weights using chain rule
4. Weight Update: Adjust weights to minimize loss using gradient descent

The algorithm "propagates" error backward through the network, hence the name.

Key Points: Mention activation functions (ReLU, sigmoid, tanh), gradient descent variants (SGD, Adam), and vanishing gradient problem.

Vanishing Gradient Problem: During backpropagation in deep networks, gradients become exponentially smaller as they propagate backward through layers. This causes early layers to learn very slowly or stop learning entirely.

Solutions:

• Better activation functions: ReLU instead of sigmoid/tanh (avoids saturation)
• Better initialization: Xavier/He initialization to maintain gradient magnitude
• Batch Normalization: Normalizes inputs to each layer
• Residual connections (ResNet): Skip connections allow gradients to flow directly
• LSTM/GRU: For RNNs, use gating mechanisms to control information flow
• Gradient clipping: Prevents exploding gradients

Related: Be prepared to explain exploding gradients and why deep networks were historically difficult to train.

Based on confusion matrix (TP, TN, FP, FN):

• Precision = TP/(TP+FP): Of predicted positives, how many are actually positive? (Quality of positive predictions)
• Recall = TP/(TP+FN): Of actual positives, how many did we correctly identify? (Completeness of positive predictions)
• F1-Score = 2×(Precision×Recall)/(Precision+Recall): Harmonic mean balancing precision and recall

When to use:

• High Precision needed: Email spam detection (avoid marking good emails as spam)
• High Recall needed: Medical diagnosis (don't miss any diseases)
• F1-Score: When you need balance between precision and recall

Follow-up: Be ready to explain ROC curves, AUC, and precision-recall curves for imbalanced datasets.

Imbalanced datasets have unequal class distributions, making models biased toward majority class.

Solutions:

Evaluation: Use precision, recall, F1-score, and AUC instead of accuracy.