While for some databases the current implementation of
decision trees performs relatively well (with accuracies over 90%) it is also
true that for some other databases presented decreased performance. Comparing
the results with my teammate and with other students in the class, it was
obvious that for some databases like (credit.dat) the performance of multi-way
branching on continuous attributes was lower than binary branching. It appears
that the Gain Ratio splitting criterion alleviates the situation, but still
does not compensate completely.
Additionally, pruning has less of an impact in the current implementation
because of the use of multi-way branching. A result of multi-way branching,
trees tend to be wide and shallow, with few nodes. Therefore, pruning a node
implies removing many branches, especially in the case of continuous
attributes. Therefore pruning a node implies removing many splits and merging
their examples. In a binary split architecture, pruning only means merging two
of the possible values into a single node. Also, in a binary architecture
splits on an attribute with multiple values, might be distributed all over the
three. However, in multi-way branching, attributes tend to be split only once
in a single place of the tree. This is even more notorious on continuous
attributes than on discrete.
Although the decision tree effectively learns different
concepts, it appears that has relevant drawbacks. An interesting improvement on
the current implementation would be to support binary splits, such that it
could be evaluated the results of pruning, using binary splitting compared to
multi-way splitting.