There is a concept called service time value. It refers to the use of an accessible and tangible measurement for outcomes not necessarily because it is the best way to measure the outcomes, but because it is one of the easiest and most convenient measurements available.
Confusing? Take the following example. Your clothes dryer suddenly stops working. The clothes will not dry. You go through the basics for a dryer, which include cleaning out the lint filter and making sure the gas/electricity is intact. Beyond that, you're thinking you have to buy a new $400 dryer. You call the repair service and the guy comes out to look at the dryer. He does the basics as well, of course, and turns the dryer on and off, playing with the knobs and dials. Then he takes off the front panel of the dryer, replaces the thermal fuse, and gets the dryer working again. You get a $120 bill for the fifteen minutes it took him to change a $2 thermal fuse.
Many people would be upset to at least some degree in that situation. $118 for fifteen minutes of work? How many people make that much an hour? Neurosurgeons? This isn't brain surgery, folks. It did not take seven years of postgraduate schooling to learn how to change a thermal fuse. It doesn't even take four years of college to learn that. If you watched the guy do it you would have learned right then how it's done.
On the flip side, not everyone knows how to fix a dryer. Or rather, not everyone wants to have to know how to fix a dryer. I have other things that I care more about other than the inner workings of a dryer. That's why there are warranties for these kinds of things--so I don't have to spend my Saturday looking up dryer specifications on Google.
But that is the concept of service time value. It is the use of the easiest and most tangible factor to measure the quality of the outcome. In the example of the malfunctioning dryer, it was easier to time how long the repair took rather than to qualify the difficulty or personal time it would have taken you to repair the dryer yourself. The same can be said for assembly, construction, and development jobs. It might take a computer programmer one day to put together a website, but that does not mean it is a simple task.
What are the broader implications of this concept? Time does not have to be easiest and most tangible factor. Take the No Child Left Behind act. Without reviewing the intricacies of the legislation, which can found at the department of education website, NCLB was supposed to improve the education in primary and secondary schools, and thus translate into improved vocational opportunities in the future for these children. The problem is that it is difficult to measure the success or impact of changes in the primary and secondary school systems on the future of the target children. So they used the easiest and most tangible factor they could measure--results of math and reading tests of the schoolchildren—the NAEP. Unfortunately, the results did not ultimately support improved math skills and literacy despite having an increase in average test scores over time.
What ended up happening was that were both rewards and punishments instituted for achieving and missing goal average test scores on the NAEP. Teachers and schools of students who averaged a higher test score received more grant money, which was taken away from those who did not meet to goal. So what is a school to do? Focus the majority of the education on the math and reading topics being tested. You can see how this may lead to a worse education rather than an improved education. Of course, schools that still teach everything else still provide excellent educations and also reach the NCLB goals. Thus the convenient measurement of test score averages has brought the NCLB, a bill with so much good intention, under tremendous scrutiny and criticism.
Health care is another area where we will see this concept. With health care costs rising every year and with more and more people requiring health care due to the aging baby boomers, politicians and insurance companies have devised a way to reduce costs. The idea is simple--they will reimburse physicians who practice bad medicine less, and physicians who practice good medicine more. But how do you tell the difference between a good physician and a bad one?
This seems like an easy task. There are many guidelines out there pertaining to all areas of medicine. Take preventative medicine--colonoscopy, mammogram, cholesterol, diet, exercise, immunizations, etc. Or diabetes--blood sugars, urine protein, cholesterol, blood pressure, eye exams, foot exams, etc. We can do this for just about any disease or syndrome out there. On top of these factors, many programs throw in "timely access to health care" and "doctor-patient communication". Some insurers also have IT related qualifications. On the surface, this seems to make perfect sense. Clearly, physicians that don't get their patients to have colonoscopies after the age of 50 are bad doctors. And the ones that have patients with perfect blood pressures are good doctors.
Think of the underlying message in that format. It insinuates that all patients do everything that is necessary for their health and that if a patient does not meet the recommended guidelines the physician is at fault. It also makes a very dangerous application of averaging. You could argue that the insurers are not looking for 100% of the patients to be at target, only 80%. Or 75%. But from where does that number come? It is an arbitrary number selected to fit what the insurers believe is "acceptable". And that's where the misuse of averages appears. No one actually thinks that inner city patients are the same as middle class retirees. Or that low income patients are the same as wealthy patients. Socioeconomic factors greatly influence healthcare delivery. It is well established that lower socioeconomic patients have poorer health for a variety of reasons. Their ability to afford medications and tests are hampered. They have less understanding of their diseases. The healthcare facilities in their area are less than state of the art. Reaching targets for these patients is more difficult. Will insurers now go to each medical practice and assess socioeconomic status to decide on appropriate percentage goals? That would be a near impossible task with the number of practices out there, the clear privacy violations it requires, and the need to update the determination every year as cities change.
An extreme example would be a physician that had a great deal of underserved patients who cannot afford many medicines and for whatever reason find their health to be a secondary priority. This physician would have a tough time controlling his hypertensive patients, ensuring that screening tests were done, and even getting adequate follow up appointments. His reimbursement would be low, and he would also incur a penalty for failing to meet standard of care. Now take another physician who has a practice near a newly built retirement development. His patients are well off and care about their health. They see him frequently, and can afford all their medicines and tests. This physician has excellent "performance" by the insurers measurements and will not only receive more money because his patients have higher paying insurances, but also a bonus for hitting all targets for standard of care. Without any other information about the two physicians, can you really tell if either are good or bad doctors?
There may not be a better alternative to the current proposed measurement techniques for improvement in education and healthcare, but at the very least we should acknowledge that there are two parties responsible for the outcome--the educators and the students and their families, and the physician and the patients and their families. The latter groups need to also have a stake in the bonuses and penalties if the models can even have a chance of working out.