Next: Improvements to design features
Up: Evaluation
Previous: Comparing reinforcement learning with
Contents
Dialogues with human beings
The planner has been developed and evaluated in a simulated environment, where the strategies of the dialogue partner are assumed to be those that the planner would choose. More likely than not, human factors will have to be considered that would require some change of the planning model. It is likely that human dialogue partners respond to their limitations in working memory and inferential capacity by using dialogue policies of the type described in Section 2.9.1 more often than by generating a game tree. If a game tree is used at all, it will likely be a severely pruned one, with much more rough calculations of the probability and the utility of outcomes, and with little use of a deeply nested belief model. Whichever planning model they use, it is likely that they will form plans that are outside of the domain of specialisation of the planner's set of plan rules. For example the planner does not consider hypotheses that are not strictly focussed, yet humans can work around unexpected focus shifts if they have to. At the moment, there are no mechanisms to respond to such plan parsing failures.
Although the planner has been developed using established theories of dialogue planning, there is as yet no direct evidence that the planner is efficient outside the simulation. There are therefore two objectives for future work. First, there is a need to find out how well the current planner performs compared with its performance in the simulation. Second, and especially if the planner performs poorly, human rationality in decision making [20] and in dialogue planning needs to be explored (for example [75], [7]). It might be possible to use parameters like "depth of game tree" or "use of nested belief model", and learn the parameter values using training dialogues with users.
Next: Improvements to design features
Up: Evaluation
Previous: Comparing reinforcement learning with
Contents
bmceleney
2006-12-19