Category Archives: agile

Requirements, Design, Implementation, Verification, Maintenance

Posted on by
9

Spent a few cycles yes­ter­day think­ing about Lau­rent Bossavit’s provo­ca­tion, and while I’m hav­ing fun get­tin’ all com­plex­o­log­i­cal after so many years away from the Santa Fe Style, I’m increas­ingly con­fi­dent it’s not quite what he was ask­ing for.

After all, com­plex sys­tems mod­els offer no pre­dic­tive value, except in the sense that they pro­vide insights into mod­els’ intrin­sic con­se­quences. As with Genetic Pro­gram­ming vs Sta­tis­tics, agent-​​based com­plex sys­tems mod­els are use­ful only inso­far as they sur­prise and inspire fur­ther mod­els, not as tools inter­act­ing with the “real world”.

At any rate, I’m find­ing it a use­ful exercise.

Here’s where I am now, as of this writ­ing (link to old commit).

Per­son­ally I’m con­fi­dent that the toolkit of com­plex­o­log­i­cal mod­el­ing is emi­nently prac­ti­cal; Nk mod­els, Boolean net­works, agent-​​based sim­u­la­tion, all the rest is in my blood, so the design pat­terns are so obvi­ous to me that I can slap some­thing together in a few min­utes. But it’s inter­est­ing as I talk with folks (Lau­rent and Ron Jef­fries, among oth­ers) about this lit­tle sketch, how often my cor­re­spon­dent wants to drop my mod­el­ing toolkit back down to the famil­iar level of aggrega­tive sta­tis­tics, or stock-​​and-​​flow continuous-​​valued sys­tems models.

Sub­jec­tively, when folks ask for some­thing so intrin­si­cally focused on story-​​telling to be “sim­pli­fied down to cleaner math”, it fees a lot like when old-​​school pro­gram­mers ask to use Inte­ger val­ues for Boolean val­ues (“-1 is false!”), or when I see sim­ple structs of prim­i­tives used instead of full-​​fledged objects in an object-​​oriented lan­guage. Some­thing about habits and famil­iar­ity, surely, but also an utterly dif­fer­ent sense of ele­gance and pur­pose, of craft, creep­ing in.

(Which is inter­est­ing, and worth watch­ing over time. Some­thing very sim­i­lar comes up when I read this.)

At any rate, a Boolean Net­work feels like a nice, tun­able model of a com­plex world with hid­den inter­nal struc­ture. In addi­tion to tun­able inter­nal com­plex­ity, one can always add exter­nal­i­ties, noise, struc­tural dynam­ics, all sorts of things that model the way we treat error and uncer­tainty in the real world much more authen­ti­cally than merely adding some normally-​​distributed epsilon to a number.

But there’s an inter­est­ing mod­el­ing choice before me, now, and I’m think­ing aloud about it. As you’ve read, I’ve sketched a world with use­ful tun­able para­me­ters; I’ve sketched a com­par­i­son (“water­fall” vs “agile” as vari­a­tions in tim­ing and order of staged work). But one does want one’s sim­u­lated “Team” to be able to do that work in a rea­son­able way.

So what is “require­ments gath­er­ing”? Or design, or cod­ing, or test­ing? As hap­pens now and then when build­ing com­plex sys­tems mod­els, the ques­tion is not “What do peo­ple really do?”, but rather, What is the sim­plest con­vinc­ing mimic of what is implied by a story in which we say, ‘Five require­ments have been gath­ered’? Or, Six­teen designs have been con­sid­ered, or Eleven tests have been cre­ated and run?

I have some notions, as you can see in my notes at GitHub. But there’s also this feel­ing that peo­ple brought up in the clas­si­cal math­e­mat­i­cal mod­el­ing world (the one that per­fuses com­puter sci­ence as well) will prob­a­bly raise their eye­brows. One is brought up to think of vari­ables as prim­i­tives, and of inter­ac­tions not as algo­rithms but as equa­tions. One is trained by decades of expen­sive com­put­ing and resource lim­i­ta­tion to assume that aggre­ga­tion takes place prop­erly within a model, using stocks and flows and aggrega­tive mea­sures and sta­tis­tics, deriv­a­tives and aver­ages all over—and not to see each instance or run of a model as being contingent.

This is a kind of pre­ma­ture opti­miza­tion that I haven’t heard remarked before. But I think it’s close to the core of what makes “com­plex­ol­ogy” abhor­rent unto the jour­nal­ist. And what’s inter­est­ing is that it’s com­ing from nerds who have almost cer­tainly dab­bled a bit with these alien, thrilling, com­plex­o­log­i­cal mod­els: imple­mented a Game of Life, for example.

And yet I think most haven’t real­ized that con­tin­gency and sub­jec­tiv­ity are baked right in. Pro­tip: On aver­age, over long time scales, all Game of Life runs are empty.

Maybe nobody’s pointed that out before. I think they prob­a­bly ought to have done so. :)

How would one compare “agile” to “other” software development usefully?

Posted on by
2

Lau­rent Bossavit says in passing

This is an inter­est­ing propo­si­tion, but hav­ing looked over the work of Abdel-​​Hamid a bit, I’m won­der­ing if we might con­sider a more agent-​​based approach.

The orga­ni­za­tional behav­ior and systems-​​dynamics frame­work we see in that cor­pus is admirable and inter­est­ing, but one has to admit it’s focused on aggre­gate behav­iors of “devel­op­ers” and “man­age­ment” observed from the stand­point of a cor­po­rate plan­ner stake­holder. I don’t feel obliged to have that con­ver­sa­tion any more.

What does one want to explore? The things that set apart agile projects from inag­ile ones, I sup­pose: Risks of the many modes of fail­ure; adapt­abil­ity; scal­ing; work load; busi­ness value deliv­ery rates and prob­a­bil­i­ties; sub­jec­tive expe­ri­ence; knowl­edge shar­ing; some­thing about the many stake­hold­ers (“man­age­ment”, “team”, “cus­tomer”, “users”)?

All pos­si­ble. I’ve had some inci­den­tal con­ver­sa­tions with Ron and Chet about this, too, over the years. Maybe it’s time to pur­sue it a lit­tle ways….

But what does one want to compare?

Who are you, and what do you want?

The Mirror Dojo: Genetic Programming for Agile Teams

Posted on by
Reply

This is the cur­rent iter­a­tion of the Genetic Pro­gram­ming work­shop née Agile Team Dojo I’ve been work­ing on over the last few years.

I’m look­ing at the Michigan/​Ohio/​Indiana region for an inter­est­ing place to run it. If you’re inter­ested in sched­ul­ing me for a two– or three-​​day work­shop, feel free to con­tact me online.

You know how to do that.

The Mir­ror Dojo: Genetic Pro­gram­ming for Agile Teams

Genetic Pro­gram­ming has been actively researched and pro­moted for more than a quar­ter cen­tury. It’s a broad col­lec­tion of design prac­tices and mod­el­ing tech­niques for the “auto­matic” dis­cov­ery of abstract pat­terns and struc­tures.

And that means full-​​fledged pat­terns and struc­tures: algo­rithms, pre­dic­tive mod­els, com­plete mechan­i­cal and opti­cal and elec­tronic designs, and even blue-​​sky arti­fi­cial intel­li­gence systems.

Some of the field’s big hits include:

Sexy stuff! Nerds like us love it.

Bet­ter yet: I can describe all the basic design prin­ci­ples of Genetic Pro­gram­ming in four sen­tences. It’s so sim­ple to describe that I’m com­pletely con­fi­dent that I can help you you—a com­pe­tent soft­ware devel­oper work­ing on an agile team—write a work­ing GP sys­tem in an hour or so!

And that’s just what we’ll do in this two-​​day workshop.

But there’s one more thing.

You’ve prob­a­bly noticed that I always put extra-​​scary scare-​​quotes around “auto­matic” when­ever it comes up.

Dur­ing this dojo we’ll be approach­ing this mate­r­ial as an agile team. We’ll build at least two full-​​featured genetic pro­gram­ming sys­tems, and we’ll bump very quickly into those scare-​​quotes.

And that’s what this work­shop is about.

See, I’ve been work­ing in this field for most of 20 years. It turns out that even after all that time, there’s a large and trou­bling gap between the tuto­ri­als and demos of genetic pro­gram­ming, and suc­cess­ful problem-​​solving with GP. You can mea­sure that gap in terms of time, or com­pu­ta­tional resources, or expected qual­ity of results.

Sound famil­iar?

Much of the advanced aca­d­e­mic research being done today in Genetic Pro­gram­ming focuses on ways to increase the com­pu­ta­tional power, to bring more proces­sors and faster code to bear so that “auto­matic” problem-​​solving has a bet­ter “chance of suc­cess” on a com­plex problem.

Ah, look; more scare quotes.

See, in this work­shop we’re not advanced aca­d­e­mic researchers in Genetic Pro­gram­ming. We’re much bet­ter pre­pared than they are: we’re an agile team.

In the work­shop we’re going to be explor­ing how to tell our lit­tle arti­fi­cial “team” of “auto­matic devel­op­ers” what it is we want, and how they should go about mak­ing it for us, and (because GP just works) they’ll be “releas­ing soft­ware” for us. What we’ll be doing is design­ing the rules by which they solve our prob­lem: espe­cially the ones that spell out how we want them to inter­act with one another.

Which should explain the name of this dojo. And maybe even why it will take lit­tle bit longer and a bit more effort than most oth­ers you’ll run into.

Scope: This is a two– or three-​​day work­shop, for three to eight soft­ware devel­op­ers, engi­neers, coaches, design­ers, sci­en­tists, and other nerds.

The major­ity of par­tic­i­pants should be famil­iar with com­mon platform-​​agnostic pro­gram­ming lan­guages (Ruby, Python, Java, Smalltalk). They should be com­fort­able work­ing in an agile team: we’ll col­lec­tively work in one shared pro­gram­ming lan­guage, and rely on auto­mated unit and accep­tance tests, rapid release sched­ules, agile plan­ning, and pair pro­gram­ming. There should be enough lap­tops for every pair to code, and net­work con­nec­tiv­ity enough to use github for ver­sion con­trol and coordination.

On the first day of the work­shop (6 hours plus lunch) we’ll estab­lish the social infra­struc­ture, and imple­ment a sim­ple but full-​​scale genetic pro­gram­ming sys­tem for sym­bolic regres­sion. At the end of the day, we’ll choose an advanced project for the next day.

Because we’re all nerds, you and I both know you can’t “stop work­ing” after just five or six hours—and that’s fine. But the work day for the project is six hours plus lunch. So no com­mits overnight!

On the sec­ond day (6 hours plus lunch) , we’ll use genetic pro­gram­ming to address a tech­ni­cal prob­lem where results are obvi­ously practical—and prob­a­bly pub­lish­able.

In three-​​day work­shops, the final day can be used (at the team’s dis­cre­tion) for either refine­ment and pub­lic release of tan­gi­ble prod­uct from the prior days, or for a third project using dif­fer­ent GP design patterns.

Why?: The dojo is just what it says: an expo­sure for agile soft­ware devel­op­ers to a sexy but poorly-​​understood tech­ni­cal prac­tice with great eco­nomic poten­tial in the com­ing years. At the same time they’re learn­ing about the tech, they’ll be sur­fac­ing aspects of their own work, and the way agile prac­tices mold project man­age­ment in the “real world”: require­ments, goal-​​settings, information-​​sharing, met­rics, col­lab­o­ra­tion pat­terns, infra­struc­ture, deliv­ery sched­ules, and even the juris­dic­tion of man­age­ment vs. developers.

Cost: The most impor­tant cost for this exer­cise is the par­tic­i­pants’ inter­est and atten­tion. If those have been made avail­able, the only finan­cial costs are for the venue, travel, food and board (where needed) for the participants.