Maximum Entropy Distribution Subject to Subset Constraints with Application

After introducing some concepts and computations of entropy, a new type of estimation of distribution algorithms (EDAs) is developed by using principle of maximum entropy. This type of algorithms replaces the crossover and mutation operators used by genetic algorithm (GA) with the estimation of the maximum entropy distribution of schema in the population and sampling from maximum entropy distribution to generate new population. Among this type of algorithms, only contiguous schemata are used in order-2 contiguous schemata algorithm. Therefore, order-2 contiguous schemata algorithm may work better than GA when interactions between variables tend to be between variables that are located close to each other on the string.