Optimal teaching to infer the nature of the human learner and knowledge organisation

The best way to communicate knowledge to a learning agent may depend on its cognitive limitations. Likewise, which knowledge structures are easiest to acquire may depend on the agent. Previous work finds that basic decisions people make, such as judging a person as a friend or foe, are guided by a small set of examples that are retrieved from memory at decision time. This limited and stochastic retrieval places limits on human performance for probabilistic classification decisions. In light of this capacity limitation, recent work finds that idealising training items, such that the saliency of ambiguous cases is reduced, improves human performance on novel test items. Here, we use an optimal teaching approach to determine the best set of training examples to maximise performance for people given their cognitive limitations. We find that human performance is best when the model of the learner used incorporates hypothesised cognitive limits, resulting in idealised training sets that share commonalities with previous ad hoc idealisation approaches. This optimisation process can be further abstracted to include the learning problem itself. These analyses find a preference for hierarchical organisation of concepts even when the learning agent does not represent concepts hierarchically. Optimal teaching methods can provide a useful method for testing proposed limitations in human learners, can identify how people prefer knowledge to be organised, and can inform best instruction practices. Real-world examples involving forecasting the outcomes of sports contests and classifying mammograms will be discussed.