distribution of new ideas

I'm continuing to think about a data-centric world--in particular about what kind of world data would build for itself in order to have maximum control.

There has to be a way to get the system (of people) to take on new ideas quickly, quicker than rational or careful thought would otherwise deem necessary. At the same time, the system would have to act rationally and carefully in other situations, especially when information harmful to the data is available.

Here's a relevant quote from a book I'm reading, The Electric Meme: A New Theory of How We Think:

When some practice, technology, or idea is novel, it nearly always spreads through a population in the same way. If you graph the proportion of the population in exhibiting the new trait in terms of time, this proportion always begins at very low frequency (because it is an innovation, after all). The rate at which people adopt the trait starts to increase slowly, then faster, until a point of relative satiation is observed, after which time the number of new adopters dwindles, causing the adoption curve to bend over. At some point, no new cases occur (perhaps because everyone is already doing it), and the population curve flattens out or even goes back down a bit (because sometimes a number of people go back to the "old way" of doing things). This "S-shaped" cumulative adoption curve is a very robust finding from several thousand studies on the diffusion of innovations. The types of traits following this pattern can be quite different and include the adoption of hybrid corn among Iowa farmers, bottle-feeding practices among impoverished Third Worlders, new governance practices among Fortune 500 companies, chemical fertilizers among small-scale farmers, novel approaches to mathematics training (the "new math") in secondary schools, and the practice of not smoking among Americans.

Recent formal models of the psychology underlying this diffusion process have proven an important point: Reproducing the S-shaped pattern requires that people not base their adoption of novel traits strictly on trial-and-error experience of their own. Direct learning by individuals from the environment, based on cost-benefit analysis, does not generally produce the characteristic S-shaped curves that dominate the empirical literature. Just observing one's neighbor engaging in the relevant cultural practice and copying it if it makes sense (a purely rational form of adoption) cannot explain how new traits spread over time through a population. Instead and S-shaped curve is created only if individuals prefer to copy their neighbors ("keep up with the Jonses"). Only if people are not just making judgemnts for themselves about the trait will the adoption process have a social dynamic--speeding up once enough adopters are around to serve as models for others. Only then will changes in behavior or belief have the correct temporal dynamics.

This, to me, seems like a great back-door for data to feed us ideas when it needs to upgrade the system, or fix a bug. After it's complete, we all blink our eyes and wonder what came over us, why we all had to buy wireless cards and install wireless lans in our houses, why we all had to get a weblog, why we all had to watch Survivor and go see the Matrix. There's something about these things that the data wants us to know, wants everyone to know.