An effect size is calculated to indicate the impact of a program in standard units. So a larger effect size means the program had a greater impact on child outcomes than one with a smaller effect size, and the use of standard units means that scores can be compared across a number of different evaluations or programs.
Effect sizes differ from probability (or p values). P-values only tell you how likely it is that your hypothesis is true. They do not tell you anything about the strength of a relationship or effect. In the case of evaluating a program aimed at reducing depression for example, effect size calculations can tell you to what degree depression was reduced.
In the case of evaluating programs, it is often suggested that an effect size (or d) of 0.2 is a small effect, 0.5 a moderate effect and 0.8 a large effect. They are based on standard units derived from mean and standard deviations. An effect size of 0.33 denotes that a treatment led to a one-third of a standard deviation improvement in outcome. Similarly, an effect size of 0.5 denotes a one-half of a standard deviation increase in outcome. Because effect sizes are based upon these mean and standard deviation scores it allows direct comparisons across studies.
However, a small effect size (of say d = 0.1) does not necessarily mean an unimportant effect. Indeed, most ‘proven’ prevention and early intervention programs demonstrate only small or moderate effects. As Kathleen McCartney and Robert Rosenthal (2000; pg. 175) point out, ‘just as children are best understood in context, so are effect sizes’. Issues of cost-benefit often come into play. For example, if a program is relatively inexpensive to provide (in terms of financial, provider and time investment) and results in small effect sizes, this may still be far more favourable compared to another program with far greater costs yet only slightly larger effects.
See: McCartney, K and Rosenthal, R (2000). "Effect Size, Practical Importance, and Social Policy for Children". Child Development, 71, (1), 173 – 180.