Transverse section shows multiscale detail

So with the new Work Sea­son start­ing on April 1, I’ve spent some time revis­ing my active project port­fo­lio. A few shifts, a few start-​​of-​​the-​​month and tax-​​time digres­sions, but on track again.

One delayed item came to fruition this week, as I spent a lovely two-​​day trip in Cleve­land vis­it­ing Lean­dog, an agile soft­ware stu­dio I’ve heard a lot about. They have a nice open door pol­icy, and wel­comed my pry­ing “let me watch you do it” vis­its both morn­ings with grace. Had a nice (but brief) chat and lunch with Zee Spencer, Angela Harms, and Jason Felice, gave a “light­ning talk” (I think I yam­mered on for an hour) at CleRB about genetic pro­gram­ming, and spent Fri­day morn­ing chat­ting with Lean­dog founder Jeff Mor­gan, from whom I will even­tu­ally learn where to buy a pizza with egg on it in Cleve­land. And quick meet­ings with a num­ber of other folks, notably Matt Bar­comb and Michael Nor­ton, to be con­tin­ued next time.

There were three main rea­sons to go to Leandog.

First, because I know at least four inter­est­ing peo­ple who have gone to work with Leandog—sought them out, I think—and that’s inter­est­ing. Since I want to make places peo­ple seek out, it’s worth­while merely grow­ing that social net­work one more step out, adding more peo­ple who can have such allure. Rule one: Always add inter­est­ing peo­ple to your network.

Sec­ond, because after eight years I still con­sider myself as work­ing on a long-​​term project that’s equal parts com­plex sys­tems stuff, com­mu­nity design, and agile project man­age­ment. I love oppor­tu­ni­ties to see healthy, pro­duc­tive, community-​​driven work­lives like the Lean­dog folks seem to have. Workan­tile is a step towards what I have in mind, but it’s not the thing. Mix­ing in some Leandog-​​style struc­ture will be helpful.

And third: because peo­ple complain.

I hear a cho­rus of dis­may and dis­ap­point­ment com­ing from the founders and early adopters of Agile (from way back before it was called “Agile”). They’re dis­mayed or dis­ap­pointed in the way the move­ment seems to be going, the way things shift as more peo­ple start “doing” it in a cor­po­rate cul­tural set­ting. Or say­ing they’re doing it…. And in turn there are peo­ple tout­ing “post-​​agile” and “anti-​​fundamentalist” approaches to the same stuff.

It’s like a car­toon exam­ple from Andrew Abbott’s The Sys­tem of Pro­fes­sions, frankly.

Cross­ing the Chasm was always, in my dot­com days, con­sid­ered some­thing to be sought, and you’d think any­body who founded or pro­moted a worldview-​​changing approach would be pleased when all of a sud­den a bunch of hith­erto deaf and blind main­stream adopters showed up ask­ing earnest questions.

But that’s not always the way it feels. I suf­fer from this myself some­times, when run­ning into some­body from a totally dif­fer­ent back­ground, who appears as though he’s just using our words to describe his old bor­ing bad harm­ful worldview.

And that’s inter­est­ing. A trend in my life.

As I’ve said before, I remem­ber when Chaos The­ory started to be invoked to math up all kinds of mar­ket­ing bull­shit, and how the actual math­e­mati­cians and physi­cists said their work was being sub­verted. About how the com­plex sys­tems research com­mu­nity still has this bad taste in its mouth from new uses and abuses of “emer­gence” and “edge of chaos” in mun­dane worka­day set­tings like con­sult­ing and pol­i­tics. About how those of us in the early 90s who wanted to do com­pu­ta­tional biol­ogy got all pissy when the bioin­for­mat­ics peo­ple took the mind­share away and made it mean “data­bases and string han­dling algo­rithms”, not actu­ally mak­ing dig­i­tal organ­isms like we had planned. And the noises old col­leagues are mak­ing right now to what they per­ceive as “com­modi­ti­za­tion” of genetic pro­gram­ming (as if that were a thing). And lately the ten­sion cowork­ing peo­ple have voiced over the “cor­po­ra­ti­za­tion” of cowork­ing, and threats they per­ceive about it being trans­formed into a tool for com­pa­nies to make quirky “inno­va­tion fac­to­ries” (as if that were a thing).

A trend, as I said. Enough data just from my per­sonal expe­ri­ence to call that social dynamic inter­est­ing. It feels like some kind of coun­ter­point to the innovator’s dilemma: a sense that when an inno­va­tion starts to take off, it’s out of con­trol of even the founders and core philoso­phers, those with a deep per­sonal inter­est in see­ing it suc­ceed on their terms. “No, trust me, I invented it: you’re doing it wrong.”

So I went to Cleve­land to think a bit, and see one of these chasm-​​crossing transitions—the agile one—as it’s happening.

From what I saw, Lean­dog attracts peo­ple who have strong tech­ni­cal skills, but who also work well with peo­ple. They believe that some companies—the ones falling on the scale some­where between floun­der­ing and try­ing to improve—can actu­ally learn and adapt and become more agile. And that “becom­ing more agile” means some­thing real and use­ful, about chang­ing cul­ture, not adopt­ing diluted ritual.

All my evi­dence argues that Lean­dog is good peo­ple, and they have good cus­tomers that are being helped and changed. Not because (as a com­menter said recently) Cleve­land is some kind of back­wa­ter where they’re just now catch­ing up, but because things are dif­fer­ent every­where.

That’s my point.

I’m not con­cerned with evan­ge­liz­ing agile busi­ness prac­tices. That will sort itself out. I’m much more con­cerned about some macro-​​scale cul­tural prob­lems com­ing down the road.

There’s an omi­nous sense in the air that many folks express—the one that Umair Haque in par­tic­u­lar seems to riff and run with so much. The sense that things have all gone to hell and peo­ple aren’t pay­ing atten­tion to the big pic­ture and the new ideas.

I think that por­ten­tous sense has per­fused Amer­i­can cul­ture before: at the end of the Gilded Age, and maybe also before the Civil War. And it drove us dan­ger­ously close to a kind of pop­ulist fascism.

The Cas­san­dra schtick about how Nobody is Doing it Right, and we need a Rev­o­lu­tion, and Change Agents Need to Change Stuff, it sounds to me an awful lot like the peo­ple talk­ing that talk have never really been any­where, or trav­eled much. “What these peo­ple need,” and “They ought to do some­thing,” are often fol­lowed by, “…if they don’t we will.”

There is no sin­gle “they”. There is no sin­gle “we”. Peo­ple who tell you there are—you watch out for them.

So I drove across Ohio last Fri­day. And that’s the third reason.

Cleve­land is an inter­est­ing town, not least because I grew up in the sub­urbs. Reduced in a sense to an inter­me­di­ate place, it was once a core indus­trial pow­er­house, and one that didn’t fall as far as Detroit; it was the first and best of the Mid­west Maker Towns; it seems rel­a­tively untouched by sprawl, and was already pre-​​stressed eco­nom­i­cally before the Great Reces­sion started.

I drove across north­west Ohio, the slow road: Route 6, all the way from Pub­lic Square in Cleve­land to Bowl­ing Green, with a side-​​trip or two to San­dusky and Fre­mont and Grand Rapids.

And it reminded me that the future is never evenly dis­trib­uted. To con­sider a sin­gle cen­tral­ized solu­tion to the prob­lems I saw across that cross-​​section, to imag­ine that the strengths and skills and beliefs and desires and inten­tions of all the peo­ple along that route as if they were iden­ti­cal or typ­i­cal or average—or even com­men­su­rate—is fool­hardy.

I drove past farm work­ers’ hov­els out­side Fre­mont, and sprawl­ing mid-​​century colo­nial “cot­tages” along Lake Erie with lovely gar­dens tended by the own­ers, and empty shop­ping cen­ters and lively down­towns. Antiquing towns (Grand Rapids, OH), recov­er­ing mill towns (Huron and Lorain), unchang­ing but sub­tle sub­urbs like Avon Lake, ram­shackle lost towns like Sandusky.

Through the years I’ve rid­den the roller coaster of early-​​adopter-​​enthusiasm five or six times, and every time that enthu­si­asm is sup­planted by hip-​​rats-​​leaving-​​the-​​sinking-​​ship you don’t under­stand what we were try­ing to do grip­ing. I do it. Peo­ple around me do it. And it’s finally start­ing to sink in that there’s noth­ing quite so dan­ger­ous as the glib map­ping of one’s per­sonal world­view onto the actual world.

Not news, sure. But I think it’s worth an effort to be reminded.

I ended my mean­der­ing mus­ing drive at Seed Cowork­ing in Toledo, who had an open house to show off their loca­tion and thank/​solicit Kick­starter sup­port­ers (which I am one of).

I stood around yam­mer­ing like an idiot with some Toledo folks I over­heard talk­ing about “get­ting an agile group together”, includ­ing a nice man with a startup, and some Seed folks, and a fel­low from BGSU.

And the Toledo folks made Ann Arbor sound so cool. I wish I could visit that Ann Arbor some day.

It’s funny how folks think of our provin­cial lit­tle mill town of Ann Arbor as if we were an edgy rev­o­lu­tion­ary thought leader, and not a city wannabe that’s never felt any real eco­nomic pain. Instead of a town run on the prin­ci­ple that Town­ies are early adopters and founders of New Polite Lib­er­al­ism, and that these new peo­ple and kids who come in here try­ing to build build­ings and rent apart­ments are just igno­rant new­com­ers dilut­ing the pure vision of our lit­tle Town of the Mind.

It’d be funny if it weren’t sad. It’s what dri­ves peo­ple away, that sentiment.

In a word: “con­ser­vatism”. Not the ridicu­lous pas­tiche float­ing around in polit­i­cal cir­cles these days, but the Burkean kind that’s well-​​meaning and fru­gal at its best, that hon­ors diver­sity and let­ting peo­ple have their own reins. But it’s run through with the anger and melan­choly you feel when you real­ize your kid isn’t just act­ing out, she’s mak­ing bad deci­sions, and also a dose of the eye-​​rolling you do when an earnest touchy-​​feely per­son says they want to start a com­pany that’s the next Google, as soon as they find a tech­ni­cal cofounder to work with.

It’s the same sen­ti­ment that makes you dis­miss the small suc­cesses of peo­ple around you, as you are busy striv­ing to pre­serve or revive Fun­da­men­tal Truths.

Because you really did have a good idea, once. You had the best inten­tions in the world, but after a few years you got tired of being the lone voice in the wilder­ness. And now these new­com­ers, these mun­dane folk who say they’ve heard some­thing that sounds an awful lot like what you were try­ing to say back when you still cared?

Well, they’re too late, as far as you’re con­cerned. It’s not the same, they’re miss­ing the point, they’re dilut­ing your crys­talline ideas.

But I am reminded: It’s never the same.

Go visit some­body new. Things are dif­fer­ent some­where else. And when you get back home, maybe things will be dif­fer­ent there, too. Espe­cially if you feel strongly that a rev­o­lu­tion is called for: per­haps there is one going on there, or will be here when you get back.

Or maybe it already came and went, and you just missed its threads out there on the face of the world. Maybe it’s been hap­pen­ing, here and there, all along.

More on modeling Requirements/​Product Design

Had a very nice and ram­bling con­ver­sa­tion with Ron Jef­fries at the Brighton Agile Round Table yes­ter­day, where we talked all around the two straw men mod­els of “Strict Water­fall Project Man­age­ment” and “Absolute Agile Project Man­age­ment”, in light of the com­plex­o­log­i­cal model I’ve been sketch­ing.

Mostly there were inter­est­ing reac­tions and clar­i­fi­ca­tions to be made on the “analy­sis” end of the model, so I’m going to focus there for the moment. Recall that the point of this exer­cise is to craft a com­plex systems-​​y model of “project man­age­ment” that’s flex­i­ble enough to include the “Water­fall” sto­ry­line and the “Agile” sto­ry­line, keep­ing track of costs, time and fea­tures and… what’s the other one? Oh yeah, qual­ity. [joke!]

So as you’ll recall, I started (as any heir to Stu Kauff­man would) with a Ran­dom Boolean Net­work to rep­re­sent the “ground truth” of what peo­ple want. Orig­i­nally, I’d sketched this as the dynam­ics of an RBN, with the “prod­uct” being built try­ing to pre­dict and match the dynam­ics of the market’s whims. Seems that’s not a very com­fort­able anal­ogy to draw: where I was see­ing the time-​​series of inter­con­nected dynam­ics of many fea­tures “as” a cor­re­lated fea­ture, the lan­guage we use to talk about soft­ware projects very strongly empha­sizes fea­tures as fixed struc­ture, and user expe­ri­ence as the dynamic thing. So the sense of Prod­uct try­ing to match the twin­kling lights of Mar­ket is tricky.

On the other hand, I have to set my heels at the idea of features-​​as-​​atomic-​​traits, since after all I’m ulti­mately aim­ing to model devel­op­ment and team learn­ing as fea­tures are built and revised, and pat­terns are learned and re-​​used. So I’m not will­ing to drop the RBN’s com­plex­ity all the way back to a more “tra­di­tional” Kauff­man­ian sta­tic Nk model of a fit­ness land­scape. Not that there isn’t a lot of nice tun­able ground to be cov­ered in fit­ness land­scapes and stuff, but because soft­ware is used, it doesn’t just have attributes.

So here’s what we’ll try next:

Sup­pose that the ground truth of what the Mar­ket wants is deter­mined by a Ran­dom Boolean Net­work. But not in the sense I described before, where the desired fea­tures are only evi­dent via dynam­ics, and the goal is to build a prod­uct that mim­ics those dynam­ics. So no more “twin­kling lights”.

Instead, con­sider the RBN to be the (secret) map of Mar­ket expec­ta­tions. The Mar­ket will pay (or cost, in some cases) a cer­tain amount for every Prod­uct fea­ture which works in the same way as the ground truth expec­ta­tion. But in this new ver­sion, the Mar­ket is not actively gen­er­at­ing some kind of light-​​twinkling trace of data for the Team now, it’s just will­ing (for a price) to answer ques­tions about its pre­ferred outcomes.

Think of every pos­si­ble tran­si­tion of N input bits to N out­put bits as a use case over the N input fea­tures. “When the browser is open to the home page, AND the user field is empty, AND I click ‘log in’, AND I do not have a cookie set, THEN this other stuff should happen….”

A per­fect Prod­uct will pro­duce all the same out­put strings as the explicit Mar­ket func­tion does, for all 2^N inputs. Any given Prod­uct earns money in pro­por­tion to how well the released fea­tures match the Market’s tar­get desires.

But the Team only has access to aggre­gate income infor­ma­tion, unless they pay for mar­ket­ing.

There’s a lot of lee­way here. Sup­pose a Team play­ing the low-​​hanging-​​fruit game comes along, and they release a Prod­uct that is just a con­stant out­put, say 1111100000, the ulti­mate in “you can have any color car, as long as it’s black.” They might just make some money, since there’s a chance the under­ly­ing Mar­ket desire is biased that way.

Although I don’t have any inter­est to do it in this model, I have to note there’s an open­ing here for explor­ing “mar­ket dynam­ics” in such a world. A slightly smarter Team (in the same Mar­ket set­ting) might come along and release some­thing that is just as dynam­i­cally dumb as the 1111100000 folks, but bet­ter matches the aver­age desired out­put for each fea­ture. They’ll make more. Some inno­va­tor may well come along and release a prod­uct that pays atten­tion to some inputs, maybe first-​​order inter­ac­tions. They’ll make a killing.

But today I want to focus on the process within a Team, not the dynam­ics of a full mar­ket pop­u­lated by Teams with dif­fer­ent strate­gies and capacities.

Now I’m reminded that the mod­els of project man­age­ment (at least the ones we’re used to) will include some big block process labeled “analy­sis” or “mar­ket­ing” or “project man­age­ment”, and since Agile project man­age­ment implies holis­tic adap­ta­tion that should include mar­ket research on the Team when­ever mar­ket research gen­er­ates busi­ness value, we’d bet­ter include that block in our model.

Call it Require­ments Gath­er­ing (or maybe Refine­ment, if you want to imply the inclu­sion of the tra­di­tional “Main­te­nance” func­tion­al­ity, as we should in an Agile Team). Let’s give them some tools.

How about inter­views? Say each inter­view costs a cer­tain amount of money and time, and the Team’s ana­lysts can (with 100% con­fi­dence) elicit one input-​​output com­bi­na­tion from the Mar­ket. “If you’re 1011100011, what will you do next?” To me, this has the metaphoric capac­ity to han­dle peo­ple not know­ing until they’re asked, and also rev­e­la­tion of only indi­rect infor­ma­tion.

An inter­view like that should be expen­sive, I think. How about A/​B test­ing, or more gen­er­ally polling? “If you’re 1001101111, which of the fol­low­ing do you pre­fer as the out­come, 1011100011 or 1100000011?

Then of course there’s sales data, for sit­u­a­tions in which there is in fact a Prod­uct already released: As I men­tioned, even a very dumb prod­uct that fills some roles slightly bet­ter than the ones it messes up has the capac­ity to pro­vide his­tor­i­cal infor­ma­tion (and rev­enue to pay for improvements!).

Now this brings up one more change from the pre­vi­ous iter­a­tion of this model: Revenue.

In the first ver­sion, I pro­posed a model in which each fea­ture had an intrin­sic pos­i­tive value, and a released Prod­uct col­lected that value dur­ing every time period where it matched the intrin­sic Mar­ket dynam­ics. We don’t have Mar­ket dynam­ics now, we have a Mar­ket that’s a black box.

The base­line in this old one was “you make noth­ing”. I’ve rethought that, too; let’s throw a mon­key wrench into our Team’s world, and allow fea­tures to be neg­a­tives as well as pos­i­tive desires. In other words, when­ever a fea­ture with a neg­a­tive value is matched, the Team loses money.

So where are we now?

Let’s say the Prod­uct is some kind of func­tion that takes N gen­er­al­ized Boolean inputs and pro­duces (coin­ci­den­tally) N gen­er­al­ized Boolean out­puts. The space of inputs is all N–bit binary strings, gen­er­al­ized to include # or “don’t care” inputs where an input is ignored, and the out­put space is all N–bit binary strings gen­er­al­ized to include ? or “unset” out­puts, when an out­put bit is not explic­itly set to either 0 or 1.

Take a sec­ond here. We can there­fore say a Team “starts” (hav­ing done no work think­ing about or devel­op­ing any Prod­uct at all) with a “default” Prod­uct that ignores all inputs and pro­duces no out­puts: ###…### -> ???…???. They aren’t earn­ing or los­ing any money because their Prod­uct has no vis­i­ble fea­tures to cap­ture (pos­i­tive or neg­a­tive) Mar­ket share.

Sup­pose after some devel­op­ment time a Team releases a (rather stu­pid) Prod­uct that imple­ments ###…### -> 111…111; in other words, it always returns 1 for every fea­ture. How much money do they earn?

We deter­mine this either by doing the math or by sam­pling. We know the ground truth Mar­ket func­tion, a large and rather com­pli­cated truth table. If we want to do the math, we can con­vert it into a cum­ber­some prob­a­bil­ity func­tion, and just assume that the diver­sity of peo­ple in the mar­ket is absolutely uni­form: that is, that peo­ple will “use it” by apply­ing every pos­si­ble input. Alter­nately, we can sam­ple this space (using the same assump­tion of uni­for­mity) by tak­ing 1000 or so ran­dom bit-​​strings as inputs, and count­ing up how much rev­enue the Prod­uct gen­er­ates (or loses) by match­ing those features.

Sup­pose in our exam­ple there are 21 fea­tures, and that the value of the first is -\$10, the sec­ond -\$9, and so on, with the 21st pro­vid­ing value of +\$10. The Prod­uct that returns 1 for every input will lose $10 for every 1 in the first feature’s out­put table, and earn $10 for every 1 in the 21st feature’s out­put table. So it’s just a mat­ter of count­ing to deter­mine how prof­itable it turns out to be.

Just to make it clear, sup­pose some other spe­cial­ized Team releases a Prod­uct that does a per­fect job at only one fea­ture whose intrin­sic value is $4. It will earn the Team $4 for every pos­si­ble input com­bi­na­tion, regard­less of the other fea­tures; no money is lost or gained for fea­tures that are not present, or do not match desired outputs.

More later; time for lunch!