Wednesday, October 16, 2013

Measuring Intelligence: Pitfalls and Fallacies

“Top performing IT professionals outperform their peers not by a factor of 2 times or 10 times or even 100 times, but by 10,000 times.”
– Nathan Myhrvold, former Chief Technology Officer at Microsoft [1]


The currently accepted IQ scale does not seem reasonable if we look a little bit more closely

The scale measures intelligence in terms of age. An IQ of 150 indicates a 10 year old boy’s general intelligence is equal to the general intelligence of a 15 year old boy. People have a life expectancy of around 70. So there is an upper limit to how much intelligent a person can be! Beyond that, we are sorry, intelligence is not measurable! If we continue with this line of argument, then the whole process of going through the works of geniuses and labeling them with some IQ score is questionable.  

Here is an excerpt I found in one of my books:
Good programmers are up to 28 times better than mediocre programmers, according to individual differences research. Given that their pay is never commensurate, they are the biggest bargains in the software field.—Robert Glass (Fact 2 of Facts and Fallacies of Software Engineering [2002])

Is this claim reasonable? Let’s think.

A genius can't be just 1.5 times or 2 times better in performance (and worse yet, only compared to his / her age!) as the IQ scale shows. Reading speed can vary 3, 4 (or more) times. Learning capability can vary several times. Someone who knows how to organize the newly learned knowledge and store it as a model by modifying and augmenting his previous knowledge (More background knowledge leads to more understanding which leads to more effective learning.) can learn several times quicker. Intellectuals are persons who by definition enjoy intellectually satisfying tasks. So they naturally spend a lot more time on learning and applying knowledge than others. They have more background knowledge which makes it easier for them to learn new concepts faster and go deeper. 

If we want to measure someone’s general intelligence, numeric measures representing skills should be multiplied. Someone with 2 times the reading speed and 2 times the learning abilities of me, who spends 2 times more time on studying and who has 5 times more background knowledge than me which makes him 2 times more effective at learning newer concepts, should be able to learn 2*2*2*2 = 16 times more than me in a time frame of, say, several days. 

Next comes the question of applying the learned knowledge. Problem solving capability varies greatly, though there is no standard way of measuring problem solving capability. We can look at different problem solving competitions held regularly. If one problem at a certain competition can be solved by 200 contestants  a second problem by 20 and a third problem by only 2, then we can reasonably conclude that the third problem is at least 10 times or more (maybe 100 times) as hard as the first problem. In this case those 2 solvers are at least 10 times better than those who could only solve the easiest one. Whoa! Now, who doesn't want to become a genius given it’s true that one can become a genius if he / she is willing to put the required efforts?

Sometimes we underestimate geniuses. A genius can make mistakes in a field in which he / she doesn't have much interest and / or knowledge. He / she might perform poorly under mental pressure / tension / inattentiveness as mental pressure and tension can take up parts of his / her working memory (which acts as the temporary memory for holding intermediate steps during problem solving) which is probably one of the most crucial factors in abilities that are tested in IQ. Among many other faulty judgments we make, one is what I call superset-subset faulty judgement: we believe that if X is more knowledgeable than Y in subject S, then knowledge of X in subject S is a superset of Y, which is rare as the breadth and depth of knowledge in almost any subject area is so huge that its impossible for a single person to know everything in a subject. So knowledge of one person is rarely the superset of knowledge of another person in a particular subject.

There are other factors like Emotional Intelligence, Personality traits (e.g., perseverance) that play equal or more important role in success. Someone who is 10 times more intelligent than another person, and 2 times better at handling stress, plus 2 times more goal directed, focused and well planned, overall might be 10*2*2 = 40 times more effective.

So it seems reasonable to conclude that an intellectually gifted person might probably be 10 to 50 (or maybe more) times better performer in intellectual tasks combined than the average. So next time you meet an intellectually gifted person, don’t forget to show some respect! 


References

  1. The Human Side of IT

No comments:

Post a Comment