Planning under Goal Uncertainty

When the goal is hidden, how do people decide what to do next?

Summary

How do humans plan when their own goal is itself uncertain? This project builds and tests a two-phase MDP framework that separates goal inference from goal-directed planning, then benchmarks it against heuristic and mixed-strategy baselines on human behavioral data.

Motivation

Most real-world decisions are not played on a known objective. Reading a colleague’s intent, predicting a stranger’s next move, choosing what to ask next in an unfamiliar conversation — the goal is hidden and has to be inferred from sparse observations. Classical MDPs assume the goal is given; purely Bayesian accounts usually stop short of asking how people commit to a plan once a goal becomes plausible. This work sits in that gap.

Approach

  • Phase 1 — goal inference. A POMDP-style update over a discrete candidate-goal space, conditioned on observed state sequences.
  • Phase 2 — goal-conditioned planning. Given an inferred posterior over goals, solve for the action sequence that maximizes expected reward under the optimal policy.
  • Heuristic baseline. Linear regression over hand-crafted state and action features as predictors — captures most of the behavior, isolating where goal-inference is doing real work.
  • Mixed-strategy model. Logistic regression over weighted combinations of optimal-policy and heuristic features — absorbs residual variance that neither component explains alone.
  • Multi-goal extension. Vectorize the single-goal heuristic features so a shared predictor handles the multi-goal setting without retraining.

Highlights

  • Designed a two-phase MDP that separates inferring the goal from planning under it — recovers the optimal strategy when observations are informative.
  • Built a feature-based heuristic model (linear regression) that captures most human behavior and isolates the contribution of pure goal inference.
  • Introduced a mixed-strategy model (logistic regression over weighted combinations) that explains residual variance beyond the optimal strategy alone.
  • Vectorized the single-goal heuristic features to extend to the multi-goal setting without retraining from scratch.

My role

Conceptualization, computational modeling, and result analysis. Co-developed the modeling framework with the team; ran the parameter-fitting and model-comparison pipelines; led the analysis.

Stack

Python · scikit-learn · NumPy · PyMC (Bayesian fits)

Status

Ongoing (Aug 2024 – Apr 2026). Manuscript in preparation.

Collaborators

To be added.

  • Methodology: two-phase POMDP planning under uncertainty.
  • Application: human goal-inference in social-cognition experiments.