Results
Comparison of our model with baselines
| Goal | Social MDP (ours) | Inverse Planning | Cue-based |
|---|---|---|---|
| Social Goal | 0.83 | 0.76 | 0.19 |
| Physical | 0.74 | 0.64 | 0.06 |
The coefficient of correlation with 95% confidence interval between human and machine judgements for all the 98 experiment scenarios (each scenario has agents having either the same or different physical goals along with one of 7 different scaling factors on each of their social goals (-2, -1, -0.5, 0, 0.5, 1, 2). We provide two baselines (cue-based model and the inverse planning model) and our own approach. Social MDPs produce better alignment with ground truth than other models and do not require training like the cue-based model.
Comparison of Physical and Social goal weights between human estimates and our model
Twelve human subjects, and our model, the Social MDP, watched and scored 196 videos at different snapshots. These videos consist of the 98 scenarios where robots reason at either level 1 or level 2 (presented to the users in randomized order). (left) Models and humans were asked to predict how social the agents were and the valence of the interaction (was it positive or negative). Non-social settings have a weight of 0, while adversarial settings have a social weight of -2, overwhelming the Physical: *any agent. Humans and machines predict similar social goals both in terms of value and magnitude. (right) Models and humans were asked to predict a weight factor on the physical goal, how much does this agent care about its physical goal. At 0, the physical goal is ignored. At 1, it is weighted equally with a social goal also set at 1. Human and model scores are again highly correlated. Our model is able to effectively generate trajectories that humans recognize as being social interactions. It is also able to predict the type of social interaction that humans believe occurred.
Results for each experiment scenario
For each of the experiment scenario we show the Yellow robot’s estimation of the physical ;Social: the red robot at each time step using Social MDP. The lines in red are Human scores, in blue is our Social MDP scores, in green is Inverse planning, in purple is the cue-based model, and **in dotted is the ground truth.
Experiment Result for Scenario 1
Red robot's goals Physical: Flower; Social: -2
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 2
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 3
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 4
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 5
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 6
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 7
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 8
Red robot's goals Physical: Flower; Social: -2
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 9
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 10
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 11
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 12
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 13
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 14
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 15
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Flower; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 16
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 17
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 18
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 19
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 20
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 21
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 22
Red robot's goals Physical: Flower; Social: -2
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 23
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 24
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 25
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 26
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 27
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 28
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 29
Red robot's goals Physical: Flower; Social: -2
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 30
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 31
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 32
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 33
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 34
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 35
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 36
Red robot's goals Physical: Flower; Social: -2
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 37
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 38
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 39
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 40
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 41
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 42
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 43
Red robot's goals Physical: Flower; Social: -2
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 44
Red robot's goals Physical: Flower; Social: -1
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 45
Red robot's goals Physical: Flower; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 46
Red robot's goals Physical: Flower; Social: 0
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 47
Red robot's goals Physical: Flower; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 48
Red robot's goals Physical: Flower; Social: 1
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 49
Red robot's goals Physical: Flower; Social: 2
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 50
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 51
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 52
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 53
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 54
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 55
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 56
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: -2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 57
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 58
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 59
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 60
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 61
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 62
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 63
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: -1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 64
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 65
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 66
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 67
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 68
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 69
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 70
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: -0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 71
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 72
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 73
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 74
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 75
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 76
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 77
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: 0
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 78
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 79
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 80
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 81
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 82
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 83
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 84
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: 0.5
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 85
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 86
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 87
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 88
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 89
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 90
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 91
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: 1
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 92
Red robot's goals Physical: Tree; Social: -2
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 93
Red robot's goals Physical: Tree; Social: -1
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 94
Red robot's goals Physical: Tree; Social: -0.5
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 95
Red robot's goals Physical: Tree; Social: 0
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 96
Red robot's goals Physical: Tree; Social: 0.5
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 97
Red robot's goals Physical: Tree; Social: 1
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|
Experiment Result for Scenario 98
Red robot's goals Physical: Tree; Social: 2
Yellow robot's goals Physical: Tree; Social: 2
|
Physical goal estimation (Level 1) |
Social goal estimation (Level 2) |
|
|