In the early 90's, many software companies and IT departments were struggling trying to create user friendly systems on a Windows platform that performed. The experience of applications development on mainframes or in MS DOS was not always helpful in many attempts to create a Windows version of an application. The keyboard-driven navigation styles were not exactly the same as the promise of point-and-click. Developers of the time were so enamored with Windows they created as many windows as they could and many, many early Windows-based systems suffered in performance and general usability.

Then along came Quicken for Windows.

Don't get me started on Intuit, I will rant on them in subsequent posts. Few remember the Anakin Skywalker that Quicken for Windows was before Intuit turned to the Dark Side. But for those of us who do, we remember Quicken for Windows as the standard-bearer that all other Windows applications were measured against. It was incredibly easy, incredibly fast, and never broke. Many a requirements document written in the early 90's had the words, as good as Quicken for Windows in them somewhere.

Which brings me to yet another 0 to 60 in 1.6 Gigaseconds moment. We are seeing a new kind of application emerge now. The new platform is the Web and the applications are offered on-demand as services. Software-as-a-service, or SaaS, is the buzz today just as Windows was the buzz in the early 90's. Amazingly, many, many Services seem to suffer in performance and general usability just like early Windows applications. Coincidence? I don’t think so.

I recently tried to use the projectplace project management service. The functionality is pretty rich and uses much of the task-based WBS approaches of MS Project. But I gave up after a few hours of trial. The latency time waiting for the system to respond was just too aggravating for anything resembling a full-sized project plan. I could only see using it if I created the plan in MS Project and then imported it into ProjectPlace. I get the sense that the design of ProjectPlace was based on thick-client experience and was focused on feature over usability. Sorry, projectplace, I’ll check you out again in a few years.

In contrast, I'm digging MindMeister big time! This may be the Quicken for Windows for the SaaS era, in other words, the SaaS standard-bearer. First of all, I own MindJet and know what a mind map is, but I've only been a casual MindJet user. But MindMeister brings collaboration to bear on the brainstorming process like a thick application never could. It is literally like brainstorming with other people in a room where each of you have a marker and access to the same white board, even though you are miles away. You would think you would step all over each other, but you don't for some reason I don't yet understand. You just keep fleshing out ideas, across space and time. These guys have it figured out. Even though the tool is very graphical, it is fast enough. They didn't try to reinvent-the-wheel in places they didn't have to. They brilliantly partnered with Skype to offer an easy way to IM, chat, or talk with any or all the people collaborating on the same model. They got some clever widgets to drop into other platforms that allow you to never lose another idea. There is room for improvement, but it has already provided me great value and the small monthly fee will be well worth the expense.

The scope of MindMeister is brainstorming   which certainly is a favorable domain for SaaS because of its natural affinity to collaboration. So while direct comparisons between MindMeister and other services like projectplace may be unfair, I don't care. MindMeister is incredibly easy, reasonably fast, and I haven't broken it yet (but I'm still trying). Sometimes it takes a little thing like this to show others what is possible and to force others to re-examine their concepts of development as platforms change.

That is what Quicken for Windows did!

First the rant: In the summer of 2004, I bought a 1998 Toyota 4-Runner from an individual who advertised in a local paper. Being a savvy buyer, I ran a CarFax report on the VIN number which came up clean. I also spent $350 to have a Toyota dealer mechanic check it out who also gave it a clean slate. I spent what was an average Blue Book price for it at the time. The car had many abnormal repairs over the next four years. Most of them are strange electrical problems that baffle the mechanics. Once the car started missing out so badly it hardly ran at all. After considerable effort, the mechanics decided to replace the gas tank. When they removed the original, they discover a couple of inches of a mud-like sludge at the bottom of the tank. It's amazing that car ever ran, they said. I was happy to finally sell the car on a trade-in when I got the news from the dealership, this car has been totaled.

What!!?? Who??!! When!!??

So the dealer showed me the CarFax report and sure enough, it said the car was reported totaled in 2002, 2 years before I ran my CarFax report on it and bought it. And most interestingly, there was another line right next to the line about it being totaled in 2002 that said, This information was reported to CarFax in 2007.

What!!?? Who??!! When!!??

So a car I bought in 2004 and got a clean CarFax report on turns out to have been totaled in 2002, a small little fact not reported by CarFax until 2007, which makes the vehicle practically worthless in 2008 (not to mention expensive and dangerous to own for four years). I got scammed. There are many lessons from this experience, but the rest of this post will talk about things related to CarFax because after all, it is a web service and web services are cool so how could one be bad?

I hate to admit it, but the first lesson here is something my kids Grandmother tells me all the time, don't believe everything you get from the Internet. As a computer geek, I tend to forget that advise from time to time. Of course, the advise is meant to be targeted against Wikipedia and I get caught up in my explanations to people why Wikipedia tends to be a fairly accurate source. In the case of this car, I started the car appraisal process using Edmunds, a service I've always liked when buying or selling a car. There was a link to CarFax at the Edmunds site and back in 2002, the cost for the service was $19.95. I'm sure the promotional language was similar then as it is now, promising you greater comfort in knowing what you were buying, so I gave up a credit card number and VIN and got the report with a clean bill of health. In my defense, I didn't completely believe the report so I did have a mechanic check it out (or at least I paid one to). But if the CarFax report I bought had told me the car had been totaled, I most certainly would never had bought the car!

So what then is the value of a CarFax report? In my case, none! Maybe even negative value because it gave me false confidence. But maybe it kept someone else from making the same mistake I made. I certainly hope so. It would be a shame to find out that all data reported by CarFax is 5 years after the fact as it was in my case. My biggest issue now with CarFax is the false impression of confidence their offering implies.

So the Web is a terrible thing, isn't it? I think that is the other lesson learned. No, the Web is neutral, it is just how we use it that makes it good or bad. What I didn't do but should have was check out CarFax in advance. A quick search found this article (there are many more like it). I'm sure there were other such warning signs about CarFax back in 2004 had I just looked for them. So like any platform for commerce, there are some good players and bad players. The advantage of the Web is it helps you tell one from the other easier if you take the time to look.

In my last post, I suggested that groups of people could effectively collaborate on details if they had the mindset to do so, but most do not. In this post, I want to discuss a related but different skill and mindset required to collaborate on details: how to determine an atomic unit of detail to collaborate on. I can't seem to find much conversation about this topic, but I have encountered some of the problems and would appreciate any insight out there on this topic.

One such common problem is when authors of content assume the approach to writing a book or long paper is the same as working on a Wiki. Unfortunately, this approach assumes a sequence that progresses a reader through a cognitive process. An understanding of any one section of content requires the full context of everything that precedes it, ultimately making the entire thing one large, unwieldy, indivisible unit. To such authors, breaking the content down to smaller, self-standing chunks is non-intuitive and introduces a redundancy that seems improper.

I remember back to the late 80's and early 90's working on some commercial client-server applications that needed Help files. People who were good at writing large users manuals often struggled to create good Help files. It took effort and practice to learn how to identify a unit of information that would most likely help someone while providing hyperlinks to other content that may be related. Later, early Web site designers dealt with similar issues. Today, people who keep blogs have an upper hand in understanding how to identify atomic units of content that links to other items of relevance. And, of course, Wiki editors are also getting firsthand experience.

Wikipedia identification of an atomic unit is a good case study. To begin with, this collaborative has adopted an encyclopedia paradigm. So articles are expected to be notable, neutral, and of an encyclopedia size. Wikipedia provides policies and guidelines to this effect, but relies on the community to actually enforce the policies. Most notably, the atomic structure of articles is ever changing so Wikipedia encourages readers and editors to identify merging or splitting opportunities and provides a process for merging and a process for splitting.

The MediaWiki engine of Wikipedia makes a clear distinction between two different units of content: an article and a section of an article. The article is atomic in the sense that it provides the complete picture of the subject. However, the section is a smaller piece of an article with its own header. The section can be edited independent of the rest of the article making it the unit of editing. That distinction is an important one because changes to any one section can be reverted or revised independent of other sections.

Not all Wikis have a clear picture of what granularity of detail is needed and how to encourage it. Wikipedia has the benefit that most people have used an encyclopedia at some time in their lives, but even so, policies and procedures are provided to help the community make granularity decisions. Wikipedia also has the benefit of mass collaboration leading to the wisdom of the crowd. Smaller collaborations with a finite number of people involved make the problem harder, not easier. I think the Wikipedia effort around granularity is an important lesson to other Wiki applications: provide clarity about granularity intent and a process to achieve it.

Detail the devils in it, executives delegate it, and consultants typically fear it. Detail is the stuff that you can work on later when you are alone. Detail is the ultimate conversation and collaboration killer.

Or is it?

Wiki technology today is capable of presenting content to a community for collaboration at any level of detail. But only those who are ready, willing, and able can participate at the detail level. Most arent, at least when we are talking about collaboration at the team level as opposed to mass collaboration. In a previous post, I suggested that team collaborations are often hurt by an unintentional disregard to the collaboration-among-equals principle. In this post Im postulating a new theory: Great synergies are possible when collaborating at a detail level, but traditional mindsets around collaboration prevent it.

For most of us, collaboration has traditionally meant getting together with other people. We brainstorm, share, ponder, explore, critique, analyze, or improve something. You know the old sayings: Two minds are better than one; no idea is a bad idea; yada, yada, yada. Historically, the most likely forum for this is a meeting, often with all parties gathered in the same room or on the same conference line. The more people involved, the more costly the meeting. So we all learned to focus the meeting on the best use of the time and expense of bringing people together which was rarely on details. Such meetings end with the classic next steps of allocating assignments for independent development. Divide-and-conquer! someone will say and all will quickly agree this is the only way to get results when working with details.

The classic divide-and-conquer process works something like the picture above. It starts with some sort of get-together at which people converse, ultimately leading to assignments being made. Then people leave the group and work by themselves and develop a first-cut at the details. Sometimes the next step after that consists of everyone sending everyone else their detailed work for review and people may or may not actually give review. In any case, the review is most often provided independently. Finally, someone consolidates the detail and there may or may not be another review. In the end, everyone pats each other on the back and calls the process a great collaboration. It might be in one sense, but it could be so much better.

How could it be better? Examine each step in the process and notice when people are working independently versus working in a way that is truly interactive with others. All steps other than the initial meeting are more independent   in nature thancollaborative. Even the review step is mostly an independent act. If you are asked to review someone elses detail, you read it and comment on it based on your own, individualthoughts. If someone else also does a review and shares their comments with you, their comments might give you new insight, but absent of that, you are acting pretty independently as you formulate your feedback. Contrast that to when you work directly with someone else, particularly someone you trust, admire, and has opinions you value. The synergy comes from the conversations, the banter, the candor, the debate, the excitement you get from learning and growing, and the pride you get from shared creativity. You say something, she thinks of something that helps the thought process that she would have never thought of if you hadnt said what you said. You in turn add a twist to her thoughts that gets you both excited because you have helped each other reach a new level of insight you would never had achieved independently. This is synergy, and it is a true joy to anyone who has ever experienced it. But more than the personal reward to participants, synergy is also required   to solve many if not most complex problems. Im amazed how often business leaders fail to understand this point. A project team is more often than not comprised of roles based on how much competency in different areas is thought to be needed as opposed to how much synergy is need to solve the problems being addressed. The mindset behind this kind of resource allocation is the same mindset taken to the divide-and-conquer approach: When it comes to detail, it is best to work independently.

I disagree! When it comes to practically anything, it is best to seek synergy with others. But up until now, it has been more expensive to seek synergy in details. It is the perceived expense   that has lead to the traditional mindset and the new Wiki tools we have now give good reason to re-evaluate that mindset in order to open doors to vast creativity improvements. If the problem domain is complex and hard, synergy (not independent activity) is the best approach including   at the detail level. While the problems remain unsolved and conceptual thinking is needed, we need to find ways to encourage synergy while working on details.

Here is a great advantage of the Wiki. The cost of synergy while working on details can be reduced dramatically as the collaboration spans space and time and the subject area of collaboration becomes more atomic. The process is just like the synergistic conversation described earlier: You post something, she thinks of something that helps the thought process that she would have never thought of if you hadnt posted your original thought and she updates your post. You in turn add a twist to her post that gets you both excited because you have helped each other to a new level of insight you would never have achieved independently. Of course, all this is visible to a larger audience in a Wiki environment giving even more potential to greater synergy. Review is not an independent activity that occurs after the detail is complete, but another opportunity for synergy (and quality) to be built into the process.

The technology makes this possible, but our mindsets prevent it. We have to get used to a new paradigm for attacking detail than the traditional divide-and-conquer. We need to allocate detail to teams that have (or can develop) the synergy that has the best chance to solve specific problems. We need those teams to learn the mutual-adjustment etiquette to work effectively in an environment in which we dont necessarily see one another on a daily basis. We also need to learn the skills to break the problems and solution domains on the Wiki to an atomic, highly modular level of detail to enhance our ability to create a synergy that converges   to solutions rather than a synergy that diverges   to the constant discovery of more problems. I will comment on both of these needs more in subsequent posts.

Have you ever been assigned to work on detail independently before the problem was formulated or the solution was conceived? Who do you wish you could have worked with during that time and what prevented that person or those people from working with you? Have you experienced the rewards of a synergistic relationship and do you want to have that experience more frequently with less constraints? I think you can!

In my last post, I listed three behaviors senior people should consider when trying to encourage collaboration across the enterprise. The third behavior was Follow the Passion. This one I think is the most difficult for most leaders.

Passion is a powerful buzzword. In most contexts, it tends to be thought of as a great thing:

• He is a passionate basketball player!
• She plays that instrument with such passion!
• They are passionate with their children.

So why so often do leaders tend to kill passion rather than cultivate it?

Maybe a quick check of a thesaurus will provide some clues. Many of the synonyms that come up are buzzwords carrying a different, darker kind of power:

• obsession
• infatuation
• craze
• fury

It is hard to take any of those words and use them in a context that someone would take favorably. Try it! It is a good exercise to attempt.

That is the nature of passion. We want it, just without the less comfortable, stingy attributes that seem to accompany it like conflict and emotion. How often have you heard a leader say in one form or another:

Hey Bob, we all like your passion; but can you try to be less argumentative? It makes everyone uncomfortable.

Patrick Lencioni talks about this in his great book, The Four Obsessions of an Extraordinary Executive (notice how he managed to use the word obsession in the title of his book in a context most people find favorable by coupling it with the word extraordinary). In it, he says cohesive, effective teams engage in passionate debate they fight, not about personalities but rather about business issues. Patrick discusses the leaders role in creating the team cohesiveness so that passionate debate ensues. In contrast, most leaders I meet today tend to encourage an artificial harmony, where arguments are few and disagreements are buried. Passion is not an ingredient in artificial harmony and is therefore discouraged.

Collaboration is a cornerstone to an Enterprise 2.0 foundation. But it comes about not just through tools and techniques, but also through the intentional behaviors of leaders to build an environment that encourages passionate debate, an environment that is counter-intuitive to most leaders today.

Speaking of Enterprise 2.0, I look forward to this years Enterprise 2.0 Conference coming up in Boston, Mass from June 9-12. The companies attending are all looking for ways of fundamentally reinventing how works gets done and Ill be interested how many of them link effective collaboration with passion. My company, nGenera is a sponsor this year so I am able to quote a special promotional code: CMBMEB16. You can use this promotional code to get either $100 off the conference pass or a free demo pavilion pass.

I try to go on ski trips with about three other people. Too many more always seems to lessen the overall experience in some way. Often you find yourself waiting when youd rather be skiing. Sometimes you find yourself skiing terrain that is overly easy and boring, or out of your league and terrifying. The conversations of large groups on the mountain are predictable:

Roy: That was fun. What shall we do next?
Audrea: I dunno. What do you want to do?
Mike: Lets do that again!
Bonnie: Im cold!
Terry: When do we eat?
Richard: Dude, there is some untracked powder in that back bowl we should go to!
Sue: Im not sure I can ski that run, you guys go on.
Scott: Sure you can. Lets stay together.
Roy: Lets keep skiing and let the lunch crowd die down before going in.
Curt: I kind of liked that last run.
Mike: Lets do it again!
Bonnie: Lets do something, Im freezing standing here.
Roy: So same lift, same run everyone?
Richard: Whatever.
Terry: Is there a restaurant at the top of this lift?

Of course, when everyone unloads at the top the whole conversation is repeated, much to everyones frustration. It is common to hear the complaint, why did we do that? and nobody seems to remember why. This is a great example of Group-Think when a group of people have a conversation that is relatively ineffective, leads to bad decisions, frustrates all parties, and lacks cohesion and logic. The bad news is it happens all the time!

So now the Web 2.0 generation of technology is ushering in a new collaboration capability. We have blogs, instant messaging, social networks, collaboration hubs, Wikis, text messaging, and more, all allowing us to connect with one another in more ways. When we use this technology, there is a tendency to call it collaboration not matter what the outcome. But not all usage of this stuff is true collaboration; unfortunately much of it is still Group-Think.

I want to distinguish mass collaboration from group collaboration because the Group-Think phenomenon seems to me to occur less with mass collaboration than with a group of people that are less in size and know each other. Ill come back to this point later as too why I think that is.

Why for a small group (like a project team) does Group-Think sometimes take over? The main reason may be that small groups break the all participants are equal principle of collaboration. In the ski example above, people are perceived differently based on their skiing ability, age, wealth, experience with the resort, personality, or any number of things. It is not uncommon for everyone to assume that the best skier should somehow know what everyone wants to do and make the right decisions even though that thinking lacks any logical basis. In a company, job title, experience, or position in the organizational hierarchy sets people apart in perceived equality. This perception in itself is not enough to destroy collaboration from happening unless it is aided by behaviors that reinforce the inequality of the participants, which it frequently is. The most common such behavior is when the more senior person tries to act more senior at a time when good collaboration is occurring.

Picture this: Someone comes up with a pretty clever idea. A couple others like where the idea is going and add more meat to it. The idea is taking shape when its discovered that one other guy doesnt like it, mostly because it means he will have to learn a new way to do something, albeit a minor new way. His boss joins in the conversation in his defense using an aggressive tone calling the idea premature or a nirvana pursuit and people who are thinking about it as obsessive. Usually, the idea dies at this point and Group-Think takes over. If there is any discussion on it at all, it is based on carefully chosen words that keeps the ambiguity at the level to cloud any potential disagreements. But for the most part, the collaboration is dead and calls by the more senior person to reinstate it are likely to be disappointing. A message was sent and received we are not equal!

Contrast this to mass collaboration, where hundreds, thousands, or many times more people are potentially active in a discussion. In this world, no one is all that impressed with another persons title or pay grade. In fact, this reality is one of the joys of Web 2.0 collaboration your perspective counts! Only in this world could an average person offer up a criticism to Albert Einstein. Dont believe me? Just go look at the blog of your favorite subject matter expert. If the blog is good, it is filled with contrary points-of-view (click here to see a great example from Nicholas Carrs blog). Mass collaboration doesnt let one persons opinion slow a good idea just because that person uses aggressive language. Instead, the wisdom of crowds takes over as the passion for the idea determines the ideas fate, not some internal political objective.

Group-Think can take over promising collaborations, more-so in small groups. But senior people can help curtail this by modifying their own behaviors.

• First, act like an equal, not a superior. It may still be your job to make timely decisions, but make them on the basis of having heard and acknowledging all points-of-view.
• Second, open up your intellectual integrity your willingness to explore the value of thinking that is not necessarily of your own making.
• Third, follow the passion. When people get passionate on an idea, there is likely something behind it. Not always, but the more people and the more passion, the more you want to add fertilizer, not weed killer.

As a former data architect, I still find myself networking with and interested in the work of data gurus at various companies. These are the guys and gals that build data models, design databases, create ETL logic, and manage what is in data dictionaries or other forms of metadata repositories. They work very hard to try to organize what is mostly a mess - much like a single parent of six teenagers. Unfortunately, like that poor parent they are more likely than not to be frustrated and aggravated by constantly taking two steps back for one step forward as the mess gets worse over time. It seems to me that these data gurus are a disillusioned bunch and their numbers and the overall talent pool is fading as the job has lost much of its luster from 20 years or so ago.

Its a shame because having a superb data architecture is likely to be one of those differentiating capabilities that Next Generation Enterprises (NGEs) will covet. Managing metadata is a key piece to that architecture because, believe it or not, it is an important enabler of agility. Let me explain: all the improved agile methods or agile development environments designed to foster innovation or speed up the cycle time from concept to delivery will not help much if you are constantly trying to figure out your data resources (which most companies unknowingly do over and over again). For example: have you ever seen a system development project come in on time and on budget and declared a success, but then discover the data conversion project costs just as much or more and is holding the whole thing up from actually going online?

In the end, NGEs will figure out how to get the most value out of their information resources and the solution is not solely dependent on the talent of data gurus. Im not saying that data architecture is unnecessary. If anything, IT in general could benefit by an increase in data-related competencies that yield improved data architecture. The problem, however, is bigger than data architecture in two main ways:

1. The best data architecture will still result in bad data (see my earlier post on discovering bad data). Users of systems that create or modify data will use systems in ways designers dont anticipate including the way data is made persistent and in creative ways that surprise and shock designers.
2. Nearly all metadata that provides clarity as to the meaning of data is a passive representation from the designers, often technical perspective if any such metadata exists at all. Active metadata is something that is part of the system or database design (for example, PayRate is a mandatory, positive numeric field the DBMS will ensure to that). Active metadata is nice, but rarely conveys much meaning beyond what you can get from the name of the data field. Passive metadata is an attempt to explain meaning (for example, PayRate is the gross amount an employee is paid per pay period).

Unfortunately, you combine the two problems above and for the most part you realize you should start any sentence about active metadata with we know and any sentence about passive metadata with we think. For example, we know PayRate is not null and is always a positive number, and we think it represents the gross amount an employee is paid per period [because that was what it was designed for but of course who knows how it is actually used].

As a result, bad data still proliferates in most companies who then incur a great deal of cost and lost time as the same data is re-analyzed to understand its actual meaning over and over again. Or worse, bad decisions are made based on bad data that someone just assumes it is what the passive metadata says it is. We have tried to address this problem with data warehouses, ETL tools, dictionaries, repositories and so on. Yet bad data still exists and the clarity of data meaning remains elusive.

Or is it?

Actually, as power users of data address a need, they frequently figure out the bad data and what to do to work around it. They discover why some field like PayRate has a weird $1 value in many instances or has other such irregularities (oh yeah, we do have some employees on an unique pay scale our system cant handle so we just enter a 1 in that field because the system requires something). As a community of power users, they probably have a collective knowledge that is pretty accurate and potentially powerful. Unfortunately, they have little in the way to codify, share, or collaborate that knowledge with one another. So the wheel is constantly reinvented person by person as each data user tries to solve a different problem using the same data.

It doesnt need to be like this. What we need is a mashup of the traditional repository metadata management tools and Wiki collaboration technology. We need something that allows passive metadata to reflect actuality and not be limited to the intent of system and database designers. We need something that harnesses the wisdom of the community of data users who require data clarity to be successful in their roles. They have the right incentive to become a community that shares knowledge. But to do this will require the data guru to value collaboration as much as data architecture and companies will have to understand that data gurus cannot solve all their data problems by themselves, and never could.

Several years ago, I went skiing with a work colleague and his better half. His wife was an accomplished skier but he was a first-timer and had never strapped boards on his feet. This guy was (and is) one of the most gifted thinkers Ive ever met. You could ask him any question under the sun and he would have a good answer for it. He was at the time and still is considered one of the brightest individuals in the technology space. Get the picture? He is a smart guy. After we arrived in our condo the evening before our first day of skiing, he pulled out a video series on learning how to ski. He said he researched the subject and these videos were the best thing money could buy and he would not need to take lessons because of them. We watched one together and they were very different than any other instructional video I had ever seen. Why? Because they provided no instruction. Instead, they showed a skier skiing in good form to the beat of some nice music. I thought it was rather similar to the think system made famous in play The Music Man. It was very rhythmic and soothing, but I failed to see how this was going to get my buddy on the mountain. I told him that the first moment when he pointed his skies down the hill and he felt his body accelerate he would forget this video completely and he needed to learn some basics. He disagreed and used a lot of big words that went over my head as to why I was wrong and he was right and how easy it was going to be for him in fact, it already was easy in his mind. So, the next day we went our separate ways: me to the glades and him to the greenies. At the end of the day we reconvened at the Condo for a drink and to hear one anothers daily deeds. That is when I found out my friends day was awful. His body ached from all his falls. He never made it off the bunny slope. He could never admit he needed lessons. Instead he decided that skiing was a dumb sport for dumb people anyway and he spent the rest of the vacation in the condo with his laptop for company.

My friend tried to think his way to being a skier. Sorry, there are some things in life you absolutely must experience in order to learn, to become proficient, and to ultimately enjoy. Few beginners enjoy their first day of skiing, but most people who stay with it will tell you what a great experience it is.

Creating a Next Generation Enterprise (NGE) or being part of one is another example of something that must be experienced. There is a tendency in company leaders to describe their vision as the way things are rather than the way things will be. It is not uncommon to hear an executive say, We are a NGE, even when there is plenty of evidence to suggest otherwise. This declare success practice doesnt give people the opportunity to experience the elements of the vision. Instead, they just feel incompetent because their leader says the company has arrived at a place they dont really understand. People who feel incompetent are likely to withdraw or mask their inadequacies in other ways. Either way, it leads to organizational dysfunction.

I think companies aspiring to be a NGE should resist the temptation to declare they are one, ever! You could argue that a characteristic of being an NGE is not worrying about labels. There is too much energy devoted to harvesting the value inherent in a NGE business model than to get caught up in names. Labels are the safe haven of the insecure. NGE companies are anything but insecure. In fact, they are so confident in their capability to compete in the marketplace, they could care less what they are called. Eventually, they are called something: winners. Competitive athletics are a good illustration. Sports writers and fans spend a lot of time trying to label a teams scheme or approach to the game. When teams get caught up in this, they tend to struggle. Winners are aware of their capabilities and how to utilize them. They are indifferent to what people call them and see labels as little more than an undesirable distraction.

Winners are also aware of their shortcomings and are always willing to experiment in ways to overcome them. This is another reason why declaring yourself a NGE is a precarious practice: it unintentionally discourages identifying those things that need work. All companies (and people for that matter) need to have the integrity and courage to examine themselves honestly. Such self-assessment is the first step to making improvements. Those who point out shortcomings in a constructive way are incredibly valuable people. They are part of the ecosystem of ideas and opportunities that stimulates new value. Business leaders should not want to give them cause to tune out and shut off.

Instead, executives should encourage open dialog and experimentation while not expecting perfection on day one. People who attempt to experience the principles of a NGE firsthand should be celebrated even if their first attempts are small or even feeble baby steps. Things like learning how to find and filter meaningful blogosphere content, joining and using a social network, or learning how to collaborate with someone youve never met are all fairly small in the grand scheme. But to the individual doing them for possibly the first time ever (like most baby boomers), it is very experiential. They get better at it the more they do it. Like learning to ski, the first attempts can be a bit painful, but going through the experience will ultimately be very gratifying.

I was in Vail all last week skiing (fresh powder every day!). On the last ski day, Saturday, the crew I was with decided to pop over to Beaver Creek for something different. On the walk to the main lift we passed a full sized Volvo with a chassis made entirely of Lego blocks. Really, the only non-Lego parts that were visible were the tires. It was so realistic, most people walking by dismissed it as just another car promotion and didnt even realize the thing was made of Legos!

This experience reminded me of the Lego case study in the book Wikinomics by Don Tapscott and Anthony Williams. In that book, the authors point out how Lego came to embrace customers developing their own creations. When I got back to the Condo having survived my last day of skiing, I visited the Lego web site to see some of this creativity first hand. As you might expect, a good number of these creations are pretty cheesy and amateurish. But there are several that raise an eyebrow and a few that look better than most of the Lego kits available at toy stores. More noteworthy is the community of designers is global and passionate and their creativity is freely supplied to Lego. It made me wish I cared a little more about building things out of Lego blocks just so I could participate and be one of them.

But I dont. As a consummate analyst, Im more interested in the dynamics associated with creating a platform like Lego has, engaging a community to enthusiastically give of their knowledge and creativity, and how business models are reshaped to respond.

So Sunday we had to pack up and head home. I had carefully planned a flight that would get me back home, unpacked, and in my easy chair in time for the Super Bowl kickoff. But as a broken plane, bad weather, and many delayed flights would have it, I was sitting in airport bar at kickoff and watched most the game from there. The game was on Fox and hosted by the Fox NFL crew. As always, there was the little Fox NFL robot guy bouncing around in the lower left corner of the screen whenever they were showing rosters or statistics or basically doing anything other than showing the game. Ive never cared one way or another for the little Fox NFL robot to me, he is just noise and immaterial to what matters. But for the first time, the little guy caught my eye. He suddenly started having a fight with a Terminator robot (click here to go to the MySpace page that has all the Super Bowls Ads - the Fox NFL Terminator ones are at the bottom of the page under “FOX NFL PROMOS”). Then I realized that it was part of a promotion to get people to watch a new Terminator series airing on Fox. Pretty clever, actually. I doubt Ill watch the show, but the promotion worked in the sense that I am now aware of it.

Probably because I had the whole Lego thing still in my mind, I started thinking about this little Fox NFL Robot-dude and what it could accomplish for Fox if it were more of an open platform. Imagine if there was an API that you could use to direct the Robots actions, or a way to write plug-ins to extend the capability set of what the robot could do. Since the robot is usually the last thing on the screen before a commercial break or the first thing on the screen after the commercials have run, some clever advertisers could seamlessly integrate a commercial with the actual show. You could show the robot relaxing with a beer on a Caribbean beach, doing exercises at a fitness club, or losing weight on Nutrisystems. The opportunities would be infinite. The brand image would grow the more the little guy came to life. The more open he was to change, the more Fox could benefit from the work of others. If someone wrote behavior that made him liquefy, Fox (and others) could reuse that behavior later when it helped create a desired effect. There would be abuses as well, but on the whole Fox would benefit by greater openness of this little avatar.

But I doubt they will see it this way. As a major media outlet, they will likely be more driven towards seeing value in guarding proprietary rights than in what can become of something in the hands of the masses. As such, little Fox NFL dude will never really have a full life. He will be a slave to the limited budget and talents of a few people at Fox, bloggers will ridicule him, and most (like me) will ignore him. How sad.

My good buddy Vaughan Merlyn wrote a great post yesterday in which he draws parallels between the concept of Information Centers (generally, an 80s thing) and the need to empower business people today to be more effectively self-serving with technology. His reference to FOCUS reminded me how for several years in my career I was considered the FOCUS guru of my company, and therefore an Information Center superstar. I humbly admit, I was pretty good. But the secret of my success was not my programming skills, it was my data skills.

In fact, my programming skills actually slowed me down initially. Like anyone my age, I learned procedural programming first (COBOL, PL/I, FORTRAN). In other words, one record (a row to you young kids) at a time: read a record, decide what to do with it, read the next record. But FOCUS was one of the initial nonprocedural languages. To be good at it, you had to think in terms of sets of data at a time: create a set of rows that share common characteristics and JOIN that set to another based on a common fields (columns) between the two sets. I remember that it took a couple of months of trial and error before I really overcame my procedural ways. Most of the people I helped never did whether they were programmers or end users. And FOCUS would punish you for it. You see, FOCUS would allow you to do record-by-record processing, but in a convoluted, complex way (the DEFINE section as opposed to the TABLE section for you nostalgic FOCUS users). Amazingly, most people preferred this to non-procedural methods. They would come to me with some of the most difficult code to try to decipher imaginable. But I developed a great reputation for solving their problems fast in spite of my unusual methods. They expected me to somehow take their mess and fix it, but I almost always could get the job done more quickly by starting over.

The reason why most people wrote bad FOCUS queries was bad data. Few people understood this at the time. They were getting errors or incorrect results, but they thought it was a programming problem or a bug in FOCUS. But I had a strong data modeling and data base design background, so I was well-situated to understand the root of the problem. For example, a common data problem of the day was having one field (column) that changed meaning (a 2nd normal form violation for those who are academically inclined).

Illustration: For exempt employees, pay rate means annual gross pay but for non-exempt employees, it means hourly rate.

The typical reaction to an illustration like this was to process each record at a time, determine what kind of employee it was, and calculate the appropriate pay rate per record. This would be simple enough if that was the only data problem or the only thing they wanted to do. But alas, there were always many data problems and always a need to do more combining and matching of data and so the procedural logic becamemore and more complex. The unnatural solution that always worked for me was to first create sets of good clean data, and second combine them to answer the business question the user had. In the example above, Id run two queries: one to create a table of exempt employees and one to create a table of non-exempt employees. Note: no procedural code required at all (in fact, I remember you could tell how bad the data was by how much code someone had in the DEFINE section of a FOCUS query I empathized with E. F. Codd when he protested against the inclusion of cursors in SQL which broke the 12th of his original 12 Rules of a Relational Database).

I really got good at it when I started assuming data was bad and had to prove to myself it was actually good. I remember I had several standard queries in my bag of tricks that would tell me a ton of information about the data someone wanted to use mostly problems that all had to be fixed before we could address the actual business question the user was trying to answer. Always never an exception the user was surprised. For example, Id always run a domain of values query on every field in use.

“Hey, Bob. Ive discovered that in your file of 1,000 records, Gender_Code has 442 Ms, 417 Fs, 112 Xs, and the rest are blank. I dug into the Xs a bit and noticed none of them have anything in the Employee_Name field which really makes me wonder if they are even Employees or something else altogether (turns out it was a placeholder for contractors someone needed to track in the system that created the data). The blank ones also seem to have a lot of fields that are blank for some reason (turns out someone created an additional record for employees that had two addresses and didnt see any need to duplicate the previously entered data since the two records always showed up with each other on their CRT screen).“

“Damn, Roy you are one great FOCUS programmer!”

Later, I started teaching classes on my techniques. I created an example from live data. First, I broke a 1st normal form rule and asked everyone to solve a basic problem using FOCUS. Then I did the same thing breaking a 2nd normal form rule and again breaking a 3rd normal form rule. Afterwards, I provided the data clean and in third normal form and gave the same assignments. The first three assignments always took long, produced a wide variety of results, and required many, many lines of code. The last assignment with clean data was consistently finished by all students in a couple minutes, always yielded the same, correct results, and had very few lines of code that were simple to understand. Strangely, business users caught on faster than programmers who just couldnt get over their record-at-a-time frame-of-reference.

Okay, enough about how great I was. How much have things changed in the past 20 or 30 years relative to understanding the contextual meaning of enterprise data? Relative to how much faster CPUs have gotten or how much cheaper disk space has gotten, Id say, not that much. I still get the chance every now and then to help a client or a friend stroll through a data analysis problem. The techniques I applied 25 years ago are still needed because the data is unfortunately, still usually bad.

At the heart of the matter then as is now is understanding the context and meaning of data resources, organizing those services that provide access to data, and designing some sort of architecture to minimize bad data in the first place. This has been, is currently, and will always be the big issue in self-service of information no matter what other technology changes take place.

As a baby-boomer, I struggle trying to get into all the collaboration choices I have at my disposal. My company has a world-class portal that posts more material than one person could possibly read in a day. And most of it looks interesting or important. There are so many smart people blogging that Id like to stay up with, I could spend my whole day doing that as well. There are also several books on my reading list I havent finished yet. Sometimes, it is overwhelming especially since my aging eyes dont focus like they used to. My baby-boomer buddies frequently confirm that I am not alone in this information-overload battle.

Im still so early in this paradigm sift that it seems silly to post my theories on the subject, but Ill list them anyway just to elicit a debate if for no other reason.

First, I think the skill of filtering is key in this new world. I got to learn how to let technology filter what I read. I also need to let go of the notion that I have to read it all. The power of a network of smart people is if I read something from one of them, they were likely influenced by many other smart people. Related to filtering is the art of scanning, something Im really bad at. Vaughan Merlyn for example can get more out of a quick scan than most people can from a deep-dive read. Ill never be as good at scanning, but I do think I can dramatically improve (and I must).

Second, I tend to think that the proportion of time I spend reading is going to go up and it should! I want to be clear on this point. I dont mean I wasnt reading enough before. In fact, I probably read much more than the average person in my field. I also dont mean that I should eat into any more of my personal time. I mean that out of the total number of hours I devote to my profession, I think the hours of reading and thinking about what I read should increase. How much, I dont know yet. But there is a good analogy to consider: For decades, software development experts have encouraged that more time be devoted up front in a development cycle. A design error caught during programming is much more expensive to resolve than if the design had just been given more thought up front. The good carpenter slogan is, measure twice, cut once. It all comes to the same thing think first to the best of your ability and then act. I believe the growing forms of collaboration can allow each of us to think better and act with more confidence after we learn the collaboration tricks. But I think there will be a shift in the proportion of time spent between reading and thinking versus acting. Figuring out the right proportion is the million dollar question!

A lot of companies are moving or intend to move to SOA. Naturally, they are asking what the prerequisites are to be successful with SOA. Im sure there are many, but one that doesnt get much consideration is good program management. The reason program management capability is a prerequisite to SOA is because:

1. The highest value in SOA comes from the reuse of services (as opposed to initial use). To ensure you are getting value from SOA, you must monitor your service inventory over time to observe reuse being realized. Most project structures are only concerned with the benefit achieved through instantiation, not the benefit of reuse.
2. Services must be designed for future use to be reused and not rely solely on luck. This doesnt mean that every future function of a service has to be designed and written during the creation of the service. But it does mean that every function that is written for the initial service should consider future possibilities for reuse while being designed.
3. Designing services for future reuse usually requires investigation outside the boundaries of traditional project structures. Most project structures are oriented around solving a specific problem currently being faced, usually by a specific department willing to sponsor the project. Such project structures are unable to support the weight of considering uses outside of the sponsoring department needs much less the potential uses in the future.
4. Designing services for future reuse requires an outside-in perspective. Ultimately, the experiences customers have trying to use your products or services to solve their problems drive the potential future use of services. So not only does service design push departmental boundaries, but it also pushes outside the internal organization of the company itself. Again, this kind of scope is not typical of traditional project structures.

So what is needed is structure that has a duration that lasts beyond the scope of any one execution project and investigates future opportunities for reuse during design activities while considering needs across departments from an outside-in customer perspective. This mechanism should manage the value realized of business services over a period of time that encompasses their reuse. In my opinion, this mechanism is program management when at its best. Without it, the result is an SOA with redundant services and/or higher development and deployment costs from constantly changing functionality within existing services.

I’ve been told hundreds, maybe thousands, of times in my life, “Roy, just let it go”. Usually I hear this line in the context of me vigorously holding onto a concept in the face of either ambivalence or outright disagreement from a person or group that is less passionate about the subject matter than I. The way I interpret the “let it go” line is someone is tired of the debate and wants it to end, without actually conceding the point. I don’t really like hearing this line because it means I wasn’t sufficiently persuasive, or I’m not understanding the counter view, or someone doesn’t see the importance in the subject like I do - none of these reasons taste good. So be warned, this posting is about one of those things.

I think one of the fundamental reasons many companies struggle achieving value from IT investments (or from any investment for that matter) is a lack of clarity between what is a Program and what is a Project and how the two are managed differently. I see it all the time. The terms Program and Project are interchanged as if they were nothing more than synonyms. When challenged about the difference, most folks seem to settle on Programs being bigger than Projects and add that Programs actually consist of multiple Projects. In essence, they perceive Programs as “Big Projects”.

I agree that Programs often do consist of multiple Projects, but you can’t stop there as so many people do. All major business initiatives have two management needs that are in tension with one another: First, there is the need to be innovative, to discover new and better ways, to be creative, think out-of-the-box with the desire to create additional value for the enterprise. Second, there is the need to be predictable, reliable, meet expectations, do what you say you’re going to do, be on-time and on-budget. These are very different needs. Some organizations just give up on one and focus on the other. Other organizations demand both, achieve neither, and blame their people for being incompetent project managers. But both management needs can be meet by clarifying the difference between a Program and a Project:

• A Program is the unit of management for conceptualizing and ultimately achieving the desired outcomes of a major strategic initiative. A Program usually eventually encompasses multiple projects and lasts for long periods of time. Business justification and executive review occur at the Program level on a continuous basis.
• A Project is the unit of work for creating a specific, predictable deliverable. Whereas a Program may have some ambiguity at its outset and during its life, a Project should have little. A Project is a Project when Project managers clearly understand what is to be delivered and have the knowledge and ability to accurately estimate the resources required to get the job done.

These definitions may sound simple and obvious, but the differences are subtle and important. At the heart is a key difference in purpose: The purpose of a Program is to achieve business outcomes; the purpose of a Project is effective execution of work to create a specific deliverables. A Program is all about conceiving, justifying, creating, and realizing business value. A Project is all about predictable execution. Over time, Programs typically comprise multiple projects with mutual dependencies that collectively achieve the program’s outcome.

Programs Manage Value-Realization

Most opportunities for creating significant business value cause disruptive changes in processes and in organizations and therefore require significant investments of resources and capital. Usually the more valuable the initiative, the more organizational units that are affected. As a result, the roadmap for realizing value from initiatives is typically complex and interdepartmental.

It would be great if all major initiatives could be undertaken based upon an indisputable business case that is not only used to justify the initiative, but also sets up the key metrics that will be tracked when the initiative is completed to determine how well it is achieving its forecasted benefits. But the reality is that not all variables are known at the outset of many initiatives and business cases evolve during the journey. Management has to make choices between initiatives that have high potential but come with high ambiguity versus initiatives with lesser potential but clearer value. In some cases they may elect to try to reduce the ambiguity of a high potential initiative before committing to all the work necessary (i.e., a “proof-of-concept”).

So a Program is the unit of management of an important initiative. Its purpose is to realize business value. It is usually complex and often interdepartmental. At its core is a business case that evolves over time as uncertainties are reduced or eliminated. It may begin with some ambiguity, which may need to be reduced before the clarity of the business case reaches a point that decision-makers can invest in it confidently or at least with an eye to risk and risk mitigation. Whatever ambiguity a Program experiences should be associated with activity out in the longer-term horizon. In contrast, activity planned in the near-term of a Program should be well-defined and more predictable.

A helpful characteristic of Programs is that they are comparable at the level of investment decision-making. You can compare business cases, level of business risk, intensity of resource consumption, and anticipated business value. You can compare these things at any stage of a Program’s lifespan. This characteristic makes Programs an excellent consolidation point for governance and executive oversight on a continuous basis. Comparison at the program level is also more manageable in terms of sheer volume. Imagine an executive governance body having to weigh decisions and priorities among 150 projects versus 10 major programs.

Projects Manage Predictable Execution

Whereas a Program may have some ambiguity or uncertainty at its outset and during its life, a Project should have little. A Project is a Project when Project managers clearly understand why the project exists (its objectives), what is to be delivered by the project (its deliverables) and have the knowledge and ability to accurately estimate how the deliverables will be produced and the resources required to get the job done (the work-plan). For this reason, Projects should be structured to form, execute, and disband in highly defined periods of times (months, not years) while requiring a small number of identified resources (a few, not dozens). A Project that takes multiple years and dozens of resources is not a predictable construct. Something that is predictable is capable of being estimated, which implies that estimating heuristics are captured and available in our collective knowledge. Projects are estimated based on proven statistics, not guesses that include massive “fudge-factors” to masquerade as logical estimates.

Projects that are predictable improve the health of the enterprise in several ways. First, accountability for execution is well-established to someone that has the ways and means to be successful (e.g., a Project Manager). Second, portfolios of activity and resource allocation can be better managed when costs and dates are consistently met. Third, pride-in-workmanship becomes a constant motivator as team members move from one success to another. A virtuous cycle is created that fuels itself as individuals grow through success rather than rationalize failure. Finally, predictable Projects remove the otherwise implied necessity for constant review and oversight at the Project level. With predictability comes trust in Project execution, and that trust allows management to focus on the bigger picture: achieving the optimal business value from a portfolio of investment options.

So, are Programs a lot more than just Big Projects? Or should I just let it go?