Interview questions for an AI engineer.

The questions.

1. 01

   culture medium · owning-failure

   Tell me about a model you shipped that didn't work. What did you learn?

   Strong answer

   Specific model. Specific failure mode. What you'd do differently. Not just 'we iterated.'

   Weak answer

   'Everything worked out.' Or vague 'we learned a lot.'
2. 02

   technical medium · experimentation

   Design an A/B test for a new recommendation algorithm.

   Strong answer

   Power analysis first — what effect size, what n? User-level randomization. Guardrail metrics (not just the primary). Ramp-up. Duration that covers behavior cycles.

   Weak answer

   50/50 split, runs 'until we have enough data.' No power analysis. One metric.
3. 03

   technical medium · data-quality

   How do you assess data quality before training on a new dataset?

   Strong answer

   Missing-value patterns. Outliers by group. Label noise estimation. Distribution shift vs holdout. Duplicates. Temporal consistency.

   Weak answer

   Runs describe() and moves on.
4. 04

   situational hard · model-deployment

   Your model is live. A bug report: one customer is consistently getting recommendations that hurt their experience. What do you do in 24 hours?

   Strong answer

   Fallback rule for this customer immediately. Investigate why — segment, feature anomaly, cohort effect. Fix at the segment level if possible. Not a blanket rollback.

   Weak answer

   Rolls back the model entirely. Or denies it's a model issue.
5. 05

   experience baseline · resume-probe

   Your resume says you built a recommendation system. What was the baseline, and how much better was yours?

   Strong answer

   Specific baseline (most-popular, random, collaborative filtering). Specific uplift metric. Explains what trade-offs they made.

   Weak answer

   'Users liked it more.' No baseline. No metric.
6. 06

   technical medium · feature-engineering

   Walk me through a non-obvious feature you engineered that moved a model meaningfully.

   Strong answer

   Specific problem. Specific insight about the business context. Before/after numbers. Honest about how many features you tried that didn't work.

   Weak answer

   Generic 'one-hot encoding' or 'standardization.' Not domain-specific.
7. 07

   technical hard · causality

   Why isn't correlation causation, and when does it matter in practice?

   Strong answer

   Gives a concrete confounder example. Knows RCT is the gold standard. Discusses observational causal techniques (propensity, IV, DiD) at least by name.

   Weak answer

   Recites the phrase without being able to distinguish when each matters.
8. 08

   communication medium · exec

   Explain to a CEO why a model with 95% accuracy might be a bad model.

   Strong answer

   Class imbalance — if 95% of events are negative, 95% accuracy is trivial. Precision vs recall. Business cost of each error type. Uses a concrete example.

   Weak answer

   'Because ML is complicated.' No concrete framing.
9. 09

   technical medium · statistics

   A product manager tells you 'variant A converts 12%, B converts 14%, ship B.' What do you say?

   Strong answer

   Asks about sample size, time window, statistical significance. Is 2 percentage points meaningful given sample? Novelty effect? Cross-variant leakage?

   Weak answer

   'Ship B, 2pp is huge.' Or rejects outright without asking questions.
10. 10

    technical hard · model-failure

    Your churn prediction model had 92% accuracy in training, 60% in production. Where do you look?

    Strong answer

    Data drift first — distribution shift between train and prod. Label leakage. Feature availability at inference time. Temporal patterns missed. Base rate bias.

    Weak answer

    'We'll add more features.' Or 'retrain more often.' No diagnostic framework.