Experimental Results

The reader might question whether redundant sensing is as common as we suggest, or wonder whether the cost of utilizing the LCW machinery outweighs the benefit due from pruning XII 's search space. To address such concerns, and to empirically evaluate our LCW implementation, we plugged XII into a UNIX agent, providing XII with operator descriptions of standard UNIX commands, and enabling it to actually execute the commands by sending (and receiving) strings from the UNIX shell. We gave the agent a sequence of goals and measured its performance with and without LCW. Table 1 quantifies the impact of the LCW mechanism on the agent's behavior. We found that our LCW machinery yielded a significant performance gain for the agent.

In this experiment, the agent's goals consisted of simple file searches (e.g., find a file with word count greater than 5000, containing the string ``theorem,'' etc.) and relocations. The actions executed in the tests include mv (which can destroy ), observational actions such as ls, wc and grep, and more. Each experiment was started with and initialized empty, but they were not purged between problems; so for each problem the agent benefits from the information gained in solving the previous problems.

Maintaining introduced less than 15%overhead per plan explored, and reduced the number of plans explored substantially. In addition, the plans produced are often considerably shorter, since redundant sensing steps are eliminated. Without LCW, the agent performed 16 redundant ls operations, and 6 redundant pwds in a ``typical'' file search. With LCW, on the other hand, the agent performed no redundant sensing. Furthermore, when faced with unachievable goals, the agent with LCW inference was able to fail quickly; however, without LCW it conducted a massive search, executing many redundant sensing operations in a forlorn hope of observing something that would satisfy the goal. While much more experimentation is necessary, these experiments suggest that closed world reasoning, as implemented in XII, has the potential to substantially improve performance in a real-world domain.