I wasn't happy with
The shape of a complex thing in my head is a lot like a spider web that someone has picked up in their fingers by the center. The web hangs in a cone-like pattern from their fingers and, at the tips, is still rooted to branches and handrails and stucco walls. Each concentric circle represents a level which serves as a "meta-level" for the circles inside it. If I want to clean houses more carefully with my broom, I can either do two rounds of sweeping with my broom instead of one, or I can buy a vacuum, or I can go to school and learn about house cleaning strategies. Those would be three concentric circles of the web, with the two rounds of sweeping on the innermost circle.
Part of what I've noticed about building complex things is that, in order to work on them, we have to put large chunks of the structure into our mind. When I work on a book, I need to have a certain subset of the entire previous book available to me in my working memory, so that I can tie things together, foreshadow, etc. Same with big website projects, I need to have a certain understanding of how the modules are working together in order to continue working on my part. Really complex projects which have many people working on them are usually broken up so that people working on one part of the complex project only need to understand their little section, plus the interface to the other sections. It's an object-oriented way to approach large projects. So, this ends up building a whole set of complex things which have relatively simple interfaces to one another that don't change very often. Sometimes this structure is explicit... coming to work for a new company you'll probably be told about how the pieces all work together... but sometimes it's not explicit. When you write a book you don't necessarily lay out how the character objects will interact with the plot objects to create tension objects, etc.
I think whenever something becomes too large to fit in your working memory, you should explicitly build these interfaces between pieces so that any one part of it will fit in your memory. When I began writing a novel right out of college I got about 150 pages though it before deciding to take a 3 month break from it. When I came back, I had lost the structure in my mind that was holding the novel together and I had no idea how to begin working on it again. This is where the "rebuild from scratch" impulse comes from, I think. When something is too big to fit in your working memory, the structure of it no longer makes sense. If you want to move a bit of it from one section to another it's like doing surgery blind. You have no idea what the repurcussions of that move will do. And so you need to either just do it and see what happens (which is dangerous, especially if the only way to test it is by re-reading the entire book, or releasing it and seeing if people are interested in it), or you have to slowly try to break it apart into smaller bits (which is also diffiicult because you might be missing the structure that was actually there before), or you can start from scratch. Every software developer I know loves to think about rebuilding from scratch. The cost of trying to keep an old structure in your head that is slowly falling apart due to lazy maintenance of the interfaces and the accumulation of moves that don't fit the structure is very high. We all hope that by building something new, the structure will shine through the object and nobody will dare to break the structure ever again. Or at least for as long as we're on the project.
That's another thing... most of this structure lives in our heads... it's very difficult to articulate these shapes in documentation. They're three-dimensional shapes. How well can words describe three-dimensional shapes? And so they stay in our heads.
When Steve Jobs rejoined Apple, he reduced their line of computers from something like 16 to 4. The structure was basically a grid with Professional and Consumer along the top and Desktop and Laptop along the side. Not only Jobs but the customers could easily load this structure into their heads and zero in on a product that they wanted. How do you build on a structure like this? They've since introduced a couple different price points within each box. Sometimes the price points are differentiated by size: 12" laptop, 15" laptop, 17" laptop. But then the 15" has two price points, depending on speed. Slowly, this shape is evolving into something more complex. Slowly, customers are having a more and more difficult time understanding the structure and may hesitate a little bit longer before buying. But as long as they do end up buying, each of these choices to complicate will appear to be a good choice. People like having choices. People also like things to be simple. How does a company reconcile these battling desires? Usually by starting simple, slowly building in more complexity, and every once in a while collapsing back down into a simple structure. It's like a neverending cycle.
I do the same thing with my life. I tend to take on more and more projects until I can no longer hold them all in my memory at once. Then I slash and burn projects until there's only one left. Then I think, "hey, wouldn't it be fun to ____?" Yes it would. So the cycle continues anew. My book and CD collections are also good metaphors for this phenomenon. Buy, buy, buy, buy, buy, sell everything! Buy, buy, buy, buy, buy, sell everything! Build, build, build, build, build, destroy!
Is this the right way to live? Is this cycle a healthy one or is there some way to build and destroy simultaneously in such a way that we still have the sensation of forward progress? For both making something more complex and making something simpler seem like forward progress. One is measurable by results, the other is measurable by the feeling of relief. To do both at once, however, would remove both the measurable aspect and the relief because they would cancel each other out: you would not get a measurable improvement since the complexity is not increasing, and you would not feel relief because the simplicity is not increasing. And yet the net result is the same. It's like having one guy work 8 hours a day by building a brick wall while another guy words 8 hours a day taking it down. I just work both jobs at different times.
We wouldn't need to do this if we were able to actually build complex things efficiently. If we had a better way to store these complex structures in a way that they could be easily shared by people, and easily remembered when necessary. What tools could be built for this purpose?