When Deep Learning Is NOT the Right Choice
Deep learning is not universally better than classical ML. Choose gradient boosting (XGBoost, LightGBM) over deep learning when: the data is tabular and structured (gradient boosting consistently outperforms neural networks on tabular data with fewer samples); the dataset is small (<10,000 labelled examples deep networks overfit on small datasets where gradient boosting generalises better); interpretability is required (gradient boosting + SHAP is more interpretable than a deep network); or inference latency and computational cost are constraints (gradient boosting inference is orders of magnitude cheaper than a neural network on the same hardware). Choose deep learning when: the input is unstructured (images, text, audio, video); the dataset is large (100,000+ labelled examples); the prediction task requires learned representations that feature engineering cannot capture; or transfer learning from a pre-trained model provides a shortcut that makes the dataset size problem tractable.