INVERSE SQUARE LAW

More is Not Necessarily Better

10-31-96

Where this paradigm is most immediately demonstrable, is our STANDARD ERROR of PERCENT calculator (see S.E. CALCULATOR).

As playing with this CALCULATOR will show, increasing sample size can only take you so far with respect to improving SURVEY PRECISION. Once the sample size is large enough to insure the minimum level of precision you are willing to live with, MORE is not necessarily better. Doubling the sample size will only improve your precision by one fourth.

What this also implies (in spite of what Mae West may have said, i.e. 'Too much of a good thing is WONDERFUL!') is that diminishing returns are a fact of life. Other phenomena like AWARENESS as a function of MEDIA EXPENDITURES work in a similar fashion. A more sophisticated variant of this is the S-CURVE (see CUMULATIVE NORMAL CURVE).

11-05-96

Since the elections will have probably come and gone before you see this, it won't matter if we comment on the types of ERROR that typically show up in SURVEY DATA. Sample size (which dertermines RANDOM ERROR) is actually a minor factor in the PRECISION of POLITICAL SURVEYS.

When a political survey is reported, it will usually give an error range expressed as a plus/minus percent. This refers to RANDOM ERROR, and is the kind of error that we compute in our STANDARD ERROR of PERCENT CALCULATOR. It's essentially a function of sample size and the closeness of the target percent to 50%.

In REAL WORLD surveys, this type of error is the least likely culprit in terms of misrepresenting the REAL underlying state of affairs. Two other more pernicious types of error are seldom mentioned. They are: SAMPLING ERROR and MEASUREMENT ERROR.

SAMPLING ERROR and MEASUREMENT ERROR can be SYSTEMATICALLY MANIPULATED, given the proper ideological motivation (see INFORMATION THEORY).

SAMPLING ERROR
This type of error results from talking to the wrong people (i.e. not talking to those most likely to vote in an election, or talking to people that are not representative of the voting population). In addition to deliberate sampling decisions made by the survey sponsors, it is also affected by resondents' willingness to co-operate.

Potential resondents may REFUSE to participate in the survey, thus affecting the representativeness of the sample (after all, REFUSERS also vote).
A classic example of this type of error is a PHONE survey of how people planned to vote in the DEWEY vs. TRUMAN election. The phone sample clearly picked DEWEY. However, telephone owners were NOT represetative of the TOTAL voting population at the time, and thus the survey was WRONG.
MEASUREMENT ERROR
This is essentially a function of how willing the respondents are to give useful information and how you ask the questions.
Respondents may participate but LIE. An example is the Nicaragua Syndrome. Surveys taken prior to the first elections in Nicaragua, indicated a sure win for the Sandinistas. However, respondents deliberately lied in the surveys for fear of reprisals by the Sandinistas. When it came to voting however, the Sandinistas were soundly defeated. This type of behavior is largely a function of how much a respondent TRUSTS the sponsor of the survey.
It also makes a lot of difference how you ask the questions. This is true for both QUESTIONNAIRES and REFRENDUMS. For instance, the kind of response you'll get if you ask:

"Do you favor cutting MEDICARE?"
may be far different from what you get if you ask:

"Do you favor increasing MEDICARE payments from $4800 per capita to $7200 per capita over the next five years?"
Is this Son of SPIN?

If the question(s) are not asked properly in the first wave of a survey, the error is often PERPETUATED even if it's recognized by the survey's sponsors. The reason is that the measuring instrument in subsequent waves has to be the same as for the first wave so that trends can be tracked.

Because of these factors, the odds are that the TOTAL ERROR in a MEDIA REPORTED SURVEY is 80% due to SAMPLING ERROR and MEASUREMENT ERROR and 20% due to RANDOM ERROR. If the reported error is + or - 5%, (an estimate of RANDOM ERROR), the TOTAL ERROR is more likely to be + or - 30%. It will also skew in a direction consistent with the political sympathies of the survey's sponsor.

You can trust a MEDIA SURVEY approximately as much as you trust the MEDIUM that reports the results.

This is of course different from PROFESSIONAL surveys, which are used to inform decisions for those who sponsor the surveys. These types of surveys will try to MINIMIZE all three of the sources of ERROR. Unfortunately, you will not get a chance to see the results of such surveys.

11-20-96

An interesting article appeared on the op-ed page of the Wall Street Journal on Tuesday, November 19. It was by EVERETT CARLL LADD and was entitled The Pollsters' Waterloo. It contained a table which is reproduced here:

  SURVEY ORGANIZATION  CLINTON    DOLE   PEROT  MARGIN of VICTORY
  _______________________________________________________________

  CBS/NY Times           53%       35%    9%          18
  Pew Research           49%       36%    8%          13
  ABC News               51%       39%    7%          12
  Harris                 51%       39%    7%          12
  NBC/WSJ                49%       37%    9%          12
  USAToday/CNN/Gallup    52%       41%    7%          11
  HOTLINE/Battleground   45%       36%    8%           9
  Reuters/Zogby          49%       41%    8%           8
  _______________________________________________________________

  ACTUAL RESULTS         49%       41%    8%           8

While none of the surveys got the winner wrong, the fact that the first (an allegedly mainstream source) was predicting a landslide comparable to the elections of LBJ or Reagan, is significant. The 'Heisenberg' effect that surveys seem to have on elections, and the dismal turnout of under 50%, may not be entirely independent of each other.

01-22-97

OVER THE HUMP

While the example of DIMINISHING RETURNS involving sample size postulates a model where MORE is still good, but the costs to attain MORE beyond a cerain point are not necessarily justifiable, there is another variant of the model where MORE beyond a certain point is actually BAD. This is the classical QUADRATIC UTILITY MODEL. It looks something like this:

Some simple examples of how this model works:

The amount of SALT or SUGAR that you would put on certain foods. Too LITTLE is BAD, somewhat MORE is GOOD, but in excess of that is BAD.
PRICING also works this way. If your price is TOO LOW, you may gain VOLUME but your PROFIT MARGIN will be too low and you'll lose money. If it's JUST RIGHT, you'll have both an adequate VOLUME and a good PROFIT MARGIN. You'll make money. If it's TOO HIGH, VOLUME will suffer and even with a good PROFIT MARGIN, you'll lose money. (When we refer to losing money, we mean relative to the OPTIMUM level that you could be making, rather than in an ABSOLUTE sense. In extreme cases it means an ABSOLUTE LOSS in addition to a relative loss).

This model is in contrast to the LINEAR model, where MORE of something is BETTER, on and on into INFINITY. Let's assume that the NON-LINEAR model is more likely to be appropriate for a lot of REAL WORLD phenomena. For proof, check it out against your own experience and reality.

If you accept this as a reasonable MODEL (remember to check it against your own experience), keep in mind that people with VESTED or ENTRENCHED interests just LOVE its alternative the LINEAR model. It says in effect that whatever they espouse or do should increase forever, regardless of consequences. In any event, here's a thought exercise:

Where on this curve (TOO LITTLE; JUST RIGHT; TOO MUCH) would you place policy issues like TAXES; CRIMINAL RIGHTS; VICTIM's RIGHTS; CIVIL RIGHTS; WELFARE SPENDING; FREEDOM of SPEECH; SIZE of GOVERNMENT; AMOUNT of LITIGATION; SPENDING on EDUCATION; DEFENSE SPENDING; REGULATIONS; ENVIRONMENTAL SPENDING etc. (add your own).