Category Archives: complexology

Down is just the most common way out

Posted on by

I recently spent a week in a tower look­ing down on Philadel­phia, rid­ing up and down to talks and bacon-​​filled break­fasts and warn­ing the other res­i­dents away from the fabled Ele­va­tor that Gets Stuck, divid­ing my day among the nine par­al­lel “tracks”—as if they were dis­con­nected and unre­lated in any way from one another—of a tech­ni­cal con­fer­ence in a field I “work in”.

I spent the week watch­ing peo­ple nego­ti­ate the var­i­ous fields they say they “work in”. Watched them talk­ing and argu­ing, enlight­en­ing and redefin­ing one another through their descrip­tions of their own work, their geog­ra­phy and fam­ily trees (the Ger­mans made a big show­ing; the Vir­gini­ans not so much), their social strata (stu­dents, post-​​docs, pro­fes­sors, cor­po­rates, and then the strange inex­plic­a­ble escapees like myself). Some of us acknowl­edged and hon­ored the 20th anniver­sary of the most influ­en­tial work in the sub­ject, John Koza’s Genetic Pro­gram­ming: On the Pro­gram­ming of Com­put­ers by Means of Nat­ural Selec­tion, and I watched as we lined our­selves up (this “field” we are) along the expected lines and ranks all over again.

I sat for an hour or so after the poster ses­sion until the her­nia pain I was hav­ing sub­sided, and a lit­tle crowd of enthu­si­as­tic Com­puter Sci­en­tists got caught in the eddy I made and sat down to chat about Arti­fi­cial Intel­li­gence and sci­ence fic­tion and what makes Genetic Pro­gram­ming the last best hope for the future of Strong AI, and so on. Now and then I’d open my mouth and say some­thing about how the notion of AI has become a frag­ile social shred of Cold War hubris, how even the idea of design­ing soft­ware is sub­ject to inter­pre­ta­tions these friends and col­leagues have never really under­stood, about the ways that sta­tis­ti­cians and soft­ware devel­op­ers and soci­ol­o­gists do their work with­out step­ping into these onto­log­i­cal traps… but those went over like a lead balloon.

I was watch­ing a “field” folded up into itself, address­ing itself. A closed field. And that’s OK. Fields are ubiq­ui­tous and ephemeral. They’re what we make of them, and the use we derive from being able to tell sim­pler sto­ries is more than enough to com­pen­sate for the obsta­cles they can occa­sion­ally create.

It was thrilling to watch my conference’s “field” being born, twenty years back. There is no less thrill in see­ing the lit­tle cracks and folds, the seams split­ting and the periph­ery falling away, as it falls apart.

Soon all sorts of raw mate­ri­als will be exposed and made avail­able again. All sorts of pos­si­bil­i­ties are stored already there in Res Poten­tia (as an old friend would say)—in the echoes of what was said along the way but ignored and for­got­ten in the rush for­ward, and the glimpses folks have had (but kept to them­selves) of other fields’ sto­ry­lines. Com­po­nents, parts and pas­sages, the stuff nobody has ever done a close read­ing of, the unrecorded his­to­ries and the things nobody even both­ers to say “I thought of that first!” about.

Turnover. Progress. A net­work unstitched and rewritten.

Some­thing like fif­teen years before the tower con­fer­ence, I’d been asked to leave my posi­tion as a grad­u­ate can­di­date in Biol­ogy at the Uni­ver­sity down the road from where I was now loom­ing. I went to the con­fer­ence think­ing I might find time to be melan­choly or bit­ter with that shadow down ’tother end of Wal­nut Street, like the other sur­vivors of Grad School Culls I’ve met. Grad­u­ate School and the aca­d­e­mic life are so important-​​feeling when you’re that age.

But there were no pangs or twinges. Grad­u­ate School—and the Acad­emy more broadly—are no less sto­ries than my “field” is a story. We use them as excuses for the embar­rass­ing mad thoughts and triv­ial affec­ta­tions we enter­tain while we fill our days with life. More broadly, they’re mem­o­ries of the Cold War and its resource lim­i­ta­tions, and lit­tle mir­rors of the states in which we house them: impe­r­ial, famil­ial, col­le­gial, or ruth­less. We speak of them as though they’re tools, and in a sense they are. But their util­ity comes not from what they do directly, but the boost they give the scan­sion of our lives after the fact.

It all reads bet­ter when you’ve done what’s expected, don’t you think? Con­tra com­mon usage, you don’t do Grad­u­ate School. It’s a thing that explains what you’ve been doing, why you look that way or act that way, why your enthu­si­asms and naiveté are so refresh­ing or enrag­ing. Grad­u­ate School is itself a “field”.

I real­ize I never did tell any­body at the con­fer­ence what it is I really do. What my “field” “really” is.

They tend to just assume, when you’re at a tech­ni­cal con­fer­ence. A few old friends and col­leagues know a bit bet­ter, but they still can’t quite con­nect the dots. Beyond some jokes about me being a spy of some sort because I was so cagey about my plans and scope and affil­i­a­tions, I don’t think many folk really noticed. We talked about the things I’ve done of course, but that’s how this sort of thing works, and it’s the com­mon ground for any conversation.

I con­fess that I look for­ward to the day we all meet at some other future con­fer­ence and com­pare notes, and end up frown­ing and smil­ing about the dif­fer­ent sto­ries we’ve told about the same stuff. I’ll be there, smil­ing and frown­ing and shrug­ging right along with them. Pick a plane or a cave wall to project the shadow of the Real World onto, and tell a story about the out­lines it makes. The trick is to shrug and smile and pick another plane and do it all again to get a com­pletely dif­fer­ent shadow, until you find the one most use­ful for the day. It’s a magic trick for most folks. Now and then I try to share the secrets, but when I start to explain the habits and prac­tices and assump­tions that make this stuff fea­si­ble and inter­est­ing… those go over like a lead balloon.

That story—itself about stories—isn’t sim­ple yet.

At the Wal­nut Street Uni­ver­sity fif­teen years ago, I was asked to quit mak­ing a scene by beg­ging my the­sis com­mit­tee to treat my com­pu­ta­tional whatever-​​it-​​was (“research”?) as Biol­ogy proper. The story I can tell now is that I was freed to do the work in the proper “field”, the one that brought me back to this con­fer­ence in the tower fif­teen years later. But in the con­text of the day it was a blow. Look at the young enthu­si­asts in schools around the world, learn­ing and eager and lis­ten­ing to the sto­ries we build tow­ers out of. Ivory tow­ers, con­fer­ence towers—all kinds.

It’s good to fit. To have a sim­ple story every­body knows, and use our sto­ries of sim­i­lar work and sim­i­lar life to shore up the walls of the place we all work together. Our “field”, our “Uni­ver­sity”, our “discipline”.

The story I tried to tell when I was in Grad­u­ate School down ’tother end of Wal­nut Street, before I knew how to do this sort of thing, was about Bio­log­i­cal Engi­neer­ing, and Maker Cul­ture, and explain­ing things by chang­ing the world. It was all the start of some­thing hap­pen­ing in some other tower, I real­ize. The story I end up telling now is how doing that same work has nearly bro­ken my old friend and advi­sor (or at least made him sound like a crack­pot to our peers), how the world has caught up and it pleases me to see peo­ple in places besides the other end of Wal­nut Street doing the very work we wanted, and how much plea­sure I take in know­ing peo­ple who knit DNA and cre­ate jel­ly­fish from rat cells and threaten to cure not dis­ease but a worldview.

Dif­fer­ent plane and a dif­fer­ent pro­jec­tion of the same real stuff. The world doesn’t give a damn what we say about it, so we’re free to make new sto­ries on demand. There are always new tow­ers being built, and raw mate­ri­als get­ting freed up as older ones are disassembled.

I wave down Wal­nut Street, and never really think about it again except with a smile. I won­der where those peo­ple live now, and what it’s like in the world for them. Is it the same? Is it trans­formed yet?

Always be will­ing to wait for one of the Ele­va­tors that Doesn’t Get Stuck.

What I do is edit. I’m an Edi­tor. It will be a lit­tle while before folks real­ize what that means, is all. And I’ll be shrug­ging right there along­side them as we find the words to use when we explain it, and tell that story more use­fully, more sim­ply. And maybe a cure will start to come along with these new sto­ries, as more peo­ple real­ize they have trou­ble telling unfaceted tales, lin­ear tales, sta­ble tales of one thing lead­ing to another.

Being an Edi­tor has a lot to do with sal­vage, with sur­fac­ing and sug­gest­ing uses for the raw mate­ri­als freed up when our sto­ries change them­selves. You site your­self at the edges of sev­eral shad­ows, and you squint up at the sun to see what’s really up there, and over time you learn to make some shad­ows of your own. Every story changes itself in the telling. That’s not merely our work as Edi­tors, but our lives.

Nobody would believe me if I came right out and said that I cre­ate the field to suit the work I want to do. On the fly; not from whole cloth, but from the chunks of other fields as needed. Nor will they believe you, when you are cured of your pro­fes­sion and start to merely do what’s called for to make your­self useful.

At least that’s the story I tell myself. It does the job.

Later: Lau­rent Bossavit has reminded me of Venkatash Rao’s sim­i­lar essay from a few months back, far less ellip­ti­cal than mine. Go and enjoy. It’s good to fit. :)

If systems were fishes we’d all something something

Posted on by

Peo­ple want sys­tem­atic advice. Uni­ver­sal advice is always best. Lack­ing uni­ver­sals, then some sort of sys­tem­atic advice will do, along with a com­pre­hen­sive but attain­able onto­log­i­cal key that one can use to quickly iden­tify the reader’s posi­tion in their sys­tem so as to get at the advice part faster.

Then there is the lesser sort of advice, framed as a sort of arti­fi­cial appren­tice­ship: pre­sent­ing a series of lessons in the form of a long (and cum­ber­some!) path or jour­ney which—if con­ducted dili­gently and mindfully—will instill in the sup­pli­cant the min­i­mum set of skills that might pos­si­bly help them when “real sit­u­a­tions” arise in their sub­se­quent journeymen’s lives.

And finally, I sup­pose, there is the sort of advice which is no advice at all: a pile of tools, or user’s man­ual, maybe with some shod­dily cat­a­logued “here’s a pretty one” anec­dotes shuf­fled in to ground the vast uncon­nected diver­sity of stuff in some sto­ries. But of course the reader is left hav­ing to map their prob­lems of inter­est onto that ran­dom pile of anec­dotes, and reminded when­ever they ask where to start that it depends, really.…

So I’m writ­ing this book about Genetic Pro­gram­ming. In the seven or eight weeks I’ve been pok­ing at it, it’s been a fas­ci­nat­ing tra­ver­sal of this hier­ar­chy of advice-​​giving. And I think there’s some­thing in the pain I’m feel­ing, a thing I need to get at.

All the other books I’ve ever seen are tool cat­a­logs. Mine is—as far as I’m aware—the first of the long (and cum­ber­some!) jour­ney kind. There are no exam­ples of either if-​​this-​​then-​​that keys, nor of uni­ver­sal principles.

Yet the field is twenty-​​mumble years old. Isn’t that weird? What’s our problem?

Talk­ing about talk­ing about

I’ve just (nine hours ago) fin­ished read­ing John Levi Martin’s excel­lent The Expla­na­tion of Social Action. Not because I’m writ­ing about a social sys­tem, which depends inti­mately on the sub­jec­tive and con­tin­gent fram­ing nar­ra­tives that mod­ern nerds report using when they under­take to solve com­plex prob­lems of a know-​​it-​​when-​​they-​​see-​​it category.

Oh no—heaven for­fend! I’m writ­ing about com­put­ers and arti­fi­cial intel­li­gence, of course. I was read­ing it just because, you know, I needed a break from writ­ing about com­put­ers and tech­ni­cal stuff. Yeah.

Any­way, dur­ing this brief “break” from “tech­ni­cal mat­ters”, I found myself read­ing this book in which Mar­tin essen­tially tears the social sci­ences a new ass­hole. In a pro­fes­sional and well-​​cited and fun­da­men­tally help­ful way, of course. Because his­tor­i­cally, the main stream of the sci­en­ci­est brands in the social sci­ences have been premised on mod­els of peo­ple doing things as if there were exter­nal­i­ties in their lives. As if their goals were inbuilt or con­cocted by exter­nal ratio­nal plan­ners; as if their beliefs were genetic or self-​​contained rules spelled out in some extra-​​social rule­book; as if their inten­tions arose from some rar­i­fied “ratio­nal” think-​​space, or as mat­ters of ingrained habit. Social expla­na­tion of that sort is—to para­phrase and elide a lot of Martin’s won­der­fully snarky overview—trying to make all that messy peo­ple stuff go away.

There are a lot of ways to make all the messy peo­ple stuff go away in social sci­ence, as it hap­pens; that’s at least one rea­son Martin’s book is so enjoy­ably long. You can con­found cor­re­la­tion with cau­sa­tion, and use regres­sion equa­tions to sug­gest that devi­a­tions from expected behav­ior are “errors” on the part of will­ful rebels. You can impose a good old-​​fashioned Great Chain of Racial Being on folks you quiz in a desert, and ignore the incon­ve­nient facts that folks in a desert have not only never seen line dia­grams of cubes, but have no social frame for “how to answer right when being quizzed”. You can cat­e­go­rize the stuff peo­ple do into a few handy cat­e­gories they them­selves don’t use, and then ignore the lit­tle tense knots of con­tra­dic­tion inevitably tucked in at the cor­ners. And—this is my favorite, for rea­sons you may already be suss­ing out—you can assume that human beings carry around for­mu­laic com­pu­ta­tional and algo­rith­mic rule-​​books in their heads, and as they per­ceive “sit­u­a­tions” in their “envi­ron­ments” they refer to these just like com­put­ers do to decide among “plans”.

This is not an asper­sion cast on indi­vid­ual soci­ol­o­gists (very much), but rather the iden­ti­fi­ca­tion of a sys­tem­atic fail­ure to think of peo­ple liv­ing in the world as being any­thing like peo­ple liv­ing in a world.

As you can tell, I enjoyed the… break it pro­vided. Because of course I’m writ­ing about how you can use com­put­ers to bet­ter per­ceive “sit­u­a­tions” in your “envi­ron­ment” to decide among many “plans”.… Com­pletely dif­fer­ent thing.

Just to fin­ish this arc, Mar­tin is not point­ing and laugh­ing at his col­leagues’ intel­lec­tual short­com­ings here, he’s mak­ing the case for a rather dif­fer­ent approach to the social sciences—a field approach. Not “the field” as in “out there with a pith hel­met and note­book mak­ing obser­va­tions”, but fields as in Maxwell and mag­net­ism (or Pep­per and aes­thet­ics). Fields as com­posed by objects and their rela­tion to one another, and more impor­tantly as per­ceived by the vec­tors imparted on objects by their rela­tions to one another. Very few things are new under the sun, and Mar­tin does a good job build­ing this notion up with an arm-​​long list of paths-​​not-​​taken in the social sci­ences, and ties it all up in a pleas­ant lit­tle bow at the end labeled “What else are you going to do?”

But the inter­est­ing thing, to me, is that in stir­ring the pot of soci­ol­ogy, Mar­tin has wielded the same spoon I’ve been wav­ing for years in… well, what­ever the hell my “field” is. “Genetic pro­gram­ming” maybe (have I told you how much I dis­like the phrase yet today?), or machine learn­ing, or arti­fi­cial intelligence.

That is: the Spoon of Re-​​drawing Sys­tem Bound­aries to Include the User as an Inte­gral Part of the Sys­tem. [Or, as I like to call it, SpoRSyBoIUIPotS]

So it begins

I hang out with well-​​meaning peo­ple who are sad much of the time, because they give good advice and peo­ple for one rea­son or another don’t fol­low it. Includ­ing me1. They are artists dis­ap­pointed with how peo­ple fail to appre­ci­ate art, and crafts­men dis­ap­pointed in how few peo­ple are will­ing to take a crafts­manly approach to their work-​​lives, and var­i­ous other train­ers of cre­ative skills whose focus is not on skills so much as on stance.

And you know, it’s funny. The peo­ple I hang out with who seem least sad about peo­ple not fol­low­ing their advice seem to all be GP peo­ple and Arti­fi­cial Life peo­ple, and the other ones who work in the var­i­ous sub­dis­ci­plines of com­plex­ol­ogy. This is not to say that they don’t try to train peo­ple how to do things, or that the peo­ple they have trained imme­di­ately do them cor­rectly every time, but rather (I think) that they are still—twenty-mumble years into their col­lec­tive adventure—pretty much OK with peo­ple who fail to fol­low in their footsteps.

And, thanks to John Levi Martin’s big blue hard­cover, I may have real­ized what one dif­fer­ence is between the sads and the sad-​​nots. The sad peo­ple have all framed their work in broad sys­tems of expla­na­tion: man­i­festos and First Rules of Fight Club and guild­ish guide­lines. My com­plex­o­log­i­cal peo­ple are still in that heady low-​​hanging fruit stage where one can make a lot of head­way by just mod­el­ing things and see­ing what hap­pens. “Look! emer­gence!” and “the agent is your model of the world; run them inter­act­ing to explore the con­se­quences of your model” and so forth.

That’s chang­ing, I expect. Smart peo­ple are nib­bling at the philo­soph­i­cal edges of “emer­gence”, try­ing to sys­tem­atize it away. Wol­framism is grow­ing in power, col­lect­ing together under one big Math­e­mat­i­cal umbrella all the work other peo­ple have done in their enthu­si­asm. Peo­ple have filled enough large vol­umes of col­lected papers on Why My Genetic Pro­gram­ming Approach is Bet­ter Than Yours, and they’re start­ing to notice that X writes one of these about Y one year, and Y writes one about X the next, and so they’re start­ing to frown a bit. And (as I’ve pointed out else­where), com­plex­ol­ogy is enter­ing a stage of pro­fes­sion­al­iza­tion war, hav­ing its hand­i­est tools coöpted (sym­bolic regres­sion comes to mind), at the same time it’s den­i­grated into nonex­is­tence wher­ever it threat­ens some other field’s sinecure (the Sta­tis­tics folks already have their frowny faces at the ready).

So yeah; my GP peo­ple are going to start being mopey a lot more. And soon. Which I’ve told them so, but hey. I think it may be ben­e­fi­cial to look at what’s hap­pened in a closely related field, in prepa­ra­tion. Maybe an ances­tral predecessor.

How to write a com­puter program

Some of the sad peo­ple I spend time with are founders of the agile soft­ware devel­op­ment move­ment. I’ve hung out with them for a long time (more than a decade now), lurk­ing in the back­ground since back in heady low-​​hanging fruit days.

They are sad, I think, because they are forced to work with peo­ple whose job it is to think pro­gram­ming is not a social endeavor. Go ahead and parse that; it took me a while to get the words right.

The Agile Man­i­festo starts off talk­ing about peo­ple, not process. And right there, they are snap­ping their fin­gers in the face of an awful lot of peo­ple who want their work to be sim­pler than that. Peo­ple who want aver­ages to be more impor­tant than devi­a­tions from the con­trol chart. Who want pro­gram­ming to be an infi­nitely divis­i­ble pro­duc­tion plant, that can be allo­cated among projects with­out ref­er­ence to geog­ra­phy or soci­ol­ogy. Who want a clean sep­a­ra­tion between the body of work­ers from the mind of plan­ners and managers.

Note, these aren’t all just “man­age­ment” (what­ever that is); it sounds to me as though there are plenty of pro­gram­mers in the world who want to go sit in their cubi­cles for eight hours exactly, Be Very Smart for exactly the nec­es­sary time, and avoid as much as pos­si­ble talk­ing to any­body or let­ting them change “their” code in future.2

Go look at how they write instruc­tional mate­r­ial about their work. Com­pare an agile “How to write a com­puter pro­gram” pas­sage to an inag­ile old-​​fashioned one. What’s different?

As I see it, it’s a mat­ter of SpoRSyBoIUIPotS.

Since the early days of the “agile coa­les­cence”, when Design Pat­terns started pop­ping up, there has been an accel­er­at­ing shift from for­mal­ism and the­o­ret­i­cal abstrac­tion to what I can only call a more con­tin­gent and social view of pro­gram­ming. Look at the tools of the agile pro­fes­sional: they’re social (includ­ing those for self-​​control, and for man­ag­ing the inter­face between you-​​the-​​typist and you-​​the-​​reader-​​of-​​typed-​​code), not any­thing to do with mak­ing code run faster or use less CPU time. They re-​​draw the bound­ary around “writ­ing com­puter pro­grams” to include “how peo­ple write” and “com­puter pro­grams for peo­ple to use”. In doing so, they step away—ready for this?—from the very sort of soci­o­log­i­cal model John Levi Mar­tin glee­fully punc­tures, the one with Gauss­ian dis­tri­b­u­tions of “pro­duc­tion”, and some mag­i­cal exter­nally cre­ated “design” and Pla­tonic “archi­tec­ture”, and lin­early decom­pos­able require­ments (“rules”) that must be com­plete enough to include every­thing that might pos­si­bly hap­pen before any­body actu­ally does any­thing.

All that stuff is, not to put too fine a point on it (and again elid­ing and rephras­ing much of Martin’s excel­lent book), a load of bull­shit as soon as you deign to include human beings in the sto­ries you tell about what human beings actu­ally do.

Sto­ries of how soft­ware gets writ­ten by agile folks, whether instruc­tional or anec­do­tal, are about peo­ple talk­ing. About sto­ries, and plans that change, and forehead-​​slapping real­iza­tions of what they’ve been miss­ing. About involv­ing the cus­tomer and the man­ager as part of the team.

Sto­ries of how soft­ware gets writ­ten by inag­ile folks are about stocks and flows and feed­back loops, and pre­dict­ing the num­ber of days until dis­crete large-​​scale events hap­pen, and clean divi­sions of “human resources” into nonover­lap­ping abstract cat­e­gories (Qual­ity Assur­ance, Data­base Admin­is­tra­tion, Archi­tec­ture, Pro­gram­mer).3

The tools and design pat­terns offered up by agile folks are about sit­u­a­tions defined as rela­tions between peo­ple and objects. They aim to con­trol the flow of infor­ma­tion and knowl­edge, to min­i­mize risks of mis­com­mu­ni­ca­tion and faulty plan­ning and bad cod­ing, to help steer a course rather than fol­low a plan. The tools of inag­ile folks are about ways to bring the observed behav­ior of a project back in line with its foun­da­tional expec­ta­tions: back on sched­ule, back under con­trol, min­i­mize “change orders”, how to stay on bud­get, how to expect it to scale with “resources”.

I don’t think there are any seri­ous (and thought­ful) books on the gen­eral sub­ject of “How to Write a Com­puter Pro­gram”. If there were, I imag­ine the two groups would approach the sub­ject very dif­fer­ently. I can look at screen­casts and blogs and think back to the ridicu­lous classes I’ve moaned my way through, and I sense a dif­fer­ence. The more “agile” ones involve prac­tices, stances for the pro­gram­mer to take in rela­tion to other peo­ple and her own sen­si­bil­i­ties and skills. The “inag­ile” ones involve some kind of cat­a­log of tools—the the­o­ret­i­cal and poten­tial kind, like iter­a­tion and recur­sion, object-​​oriented and func­tional lan­guage struc­tures, com­pu­ta­tional com­plex­ity and parallelization.

When you show the agile ones (inso­far as they exist) to inag­ile folks, they will point out how it sounds more like a frickin’ eti­quette book or some sort of mar­tial arts man­ual, all about puri­fy­ing your mind and show­ing the proper respect to some imag­i­nary “user” or “cus­tomer” instead of learn­ing the really cool fast way to make your pro­gram cool and fast; and by the way, it doesn’t even say how to write pro­grams, it says how to write tests for pro­grams. When you show the inag­ile ones (inso­far as they exist) to agile folks, they will point out how every lit­tle cun­ning tool smacks of pre­ma­ture opti­miza­tion, and how there is no state­ment any­where of why you would want to “unroll the loop” instead of just using a human-​​readable loop instead, or that if there is such an argu­ment it invokes some weird-​​ass abstrac­tion like “effi­ciency” as though every pro­gram had to be carved out of a block of wood one punch-​​card at a time.

The one describes the world as a field cre­ated by and among peo­ple. The other describes the world as dri­ven by the­o­ret­i­cal objec­tives fully exter­nal to people.

Which sounds kind of famil­iar, if you think about it.

And so?

I have no use­ful idea about “and so—”.

There’s some­thing in there, and it revolves around how we tell sto­ries to one another about how to suc­ceed. It involves the dynam­ics of pro­fes­sion­al­iza­tion, the unrest and rebel­lion and messy frowny mad-​​making mut­ter­ing that goes on when peo­ple don’t talk about the “same stuff” using the “same mod­els”. It involves the trou­ble artists I know have had with “art fund­ing” in our lit­tle dumb town, and the trou­ble train­ers I know have had with stu­dents who have been sent to their “how to pro­gram” classes (both agile and inag­ile), and ulti­mately about the effec­tive agen­das of “sta­tis­tics” and “bioin­for­mat­ics” and “arti­fi­cial intel­li­gence” and some of the other junk that I do.

What I sense, and the noise I’ve been mak­ing for some time now, is that “my” “field” (Genetic Pro­gram­ming) is com­ing quickly up to that tran­si­tion in its life, where it will start to fly apart into a vari­ety of pro­fes­sions, either sub­verted and con­sumed or divided from within.

And maybe this is just me say­ing “This might sting a lit­tle bit” before I start cut­ting. I am sharp­en­ing the saw even now, and eye­ing sta­tis­tics, dis­crete math­e­mat­ics, genet­ics and bioin­for­mat­ics, finance and pub­lic pol­icy all with a keen glint in my eye.

Because it’s time for those to start being seen as social struc­tures that include the prac­ti­tioner in addi­tion to the “sub­jects”. Not because that approach is inher­ently “bet­ter” by some abstract exter­nal scale, of course. But because inso­far as peo­ple want to use them to do things, pur­po­sive things, then they are social struc­tures. The model of Genetic Pro­gram­ming as a sort of mag­i­cal black box in which one pours “gen­eral problem-​​solving abil­ity” pre­sup­poses that any­body knows what “problem-​​solving abil­ity” is.

I have good friends and old col­leagues (some are the same peo­ple!) in the field of Genetic Pro­gram­ming, and they can­not teach peo­ple what works—only some things to do. Their books are lists of things peo­ple did, with­out much expla­na­tion of why (even in a tech­ni­cal sense). They’ve eaten all the low-​​hanging fruit you can get with­out mak­ing the work a social con­struct, in the abstracted inag­ile stock-​​and-​​flow Cold War way that made com­puter pro­gram­ming degen­er­ate into con­trol chart man­age­ment in the 1980s and 90s.

Some­body has to start teach­ing peo­ple how to stand in rela­tion to one another. Yet I have no idea about “and so—” Just some­thing some­thing. Nonethe­less, at least “some­thing some­thing” makes the line read cor­rectly, in terms of meter…. Some­times you have to fill in the words later.

The Prob­lem of Genetic Pro­gram­ming that’s most press­ing, as I see it, is the one that goes some­thing like, “Why can’t we use the same sys­tems we’ve had suc­cess with so far, which have auto­mat­i­cally dis­cov­ered such amaz­ingly cool stuff (as reported in The Press!), and just scale it up a bit until it auto­mat­i­cally invents com­pli­cated soft­ware like a word proces­sor or even a lit­tle cal­cu­la­tor? Per­haps if we estab­lish bench­marks, more dif­fi­cult test cases for researchers in the field to approach, then that will spark the tech­ni­cal and the­o­ret­i­cal inno­va­tions that are needed to over­come this qual­i­ta­tive barrier?”

And so—

  1. I “hang out” with myself a lot of the time… you know, typ­ing and stuff. What? ↩

  2. I’d like to know what the social dynam­ics of cathe­dral stone­ma­sons was like. Were there “cow­boy cut­ters” who made lovely lit­tle key­stones nobody could use because they weren’t made to match their neigh­bors? ↩

  3. I think there are per­fectly rea­son­able his­tor­i­cal rea­sons for this; they involve the Cold War, and the resource lim­i­ta­tions of the period when com­put­ing as such was invented, and how hard it has always been to do long divi­sion and square roots and junk by hand. But that’s all another rant, with a lot more foot­notes. ↩

More on modeling Requirements/​Product Design

Posted on by

Had a very nice and ram­bling con­ver­sa­tion with Ron Jef­fries at the Brighton Agile Round Table yes­ter­day, where we talked all around the two straw men mod­els of “Strict Water­fall Project Man­age­ment” and “Absolute Agile Project Man­age­ment”, in light of the com­plex­o­log­i­cal model I’ve been sketch­ing.

Mostly there were inter­est­ing reac­tions and clar­i­fi­ca­tions to be made on the “analy­sis” end of the model, so I’m going to focus there for the moment. Recall that the point of this exer­cise is to craft a com­plex systems-​​y model of “project man­age­ment” that’s flex­i­ble enough to include the “Water­fall” sto­ry­line and the “Agile” sto­ry­line, keep­ing track of costs, time and fea­tures and… what’s the other one? Oh yeah, qual­ity. [joke!]

So as you’ll recall, I started (as any heir to Stu Kauff­man would) with a Ran­dom Boolean Net­work to rep­re­sent the “ground truth” of what peo­ple want. Orig­i­nally, I’d sketched this as the dynam­ics of an RBN, with the “prod­uct” being built try­ing to pre­dict and match the dynam­ics of the market’s whims. Seems that’s not a very com­fort­able anal­ogy to draw: where I was see­ing the time-​​series of inter­con­nected dynam­ics of many fea­tures “as” a cor­re­lated fea­ture, the lan­guage we use to talk about soft­ware projects very strongly empha­sizes fea­tures as fixed struc­ture, and user expe­ri­ence as the dynamic thing. So the sense of Prod­uct try­ing to match the twin­kling lights of Mar­ket is tricky.

On the other hand, I have to set my heels at the idea of features-​​as-​​atomic-​​traits, since after all I’m ulti­mately aim­ing to model devel­op­ment and team learn­ing as fea­tures are built and revised, and pat­terns are learned and re-​​used. So I’m not will­ing to drop the RBN’s com­plex­ity all the way back to a more “tra­di­tional” Kauff­man­ian sta­tic Nk model of a fit­ness land­scape. Not that there isn’t a lot of nice tun­able ground to be cov­ered in fit­ness land­scapes and stuff, but because soft­ware is used, it doesn’t just have attributes.

So here’s what we’ll try next:

Sup­pose that the ground truth of what the Mar­ket wants is deter­mined by a Ran­dom Boolean Net­work. But not in the sense I described before, where the desired fea­tures are only evi­dent via dynam­ics, and the goal is to build a prod­uct that mim­ics those dynam­ics. So no more “twin­kling lights”.

Instead, con­sider the RBN to be the (secret) map of Mar­ket expec­ta­tions. The Mar­ket will pay (or cost, in some cases) a cer­tain amount for every Prod­uct fea­ture which works in the same way as the ground truth expec­ta­tion. But in this new ver­sion, the Mar­ket is not actively gen­er­at­ing some kind of light-​​twinkling trace of data for the Team now, it’s just will­ing (for a price) to answer ques­tions about its pre­ferred outcomes.

Think of every pos­si­ble tran­si­tion of N input bits to N out­put bits as a use case over the N input fea­tures. “When the browser is open to the home page, AND the user field is empty, AND I click ‘log in’, AND I do not have a cookie set, THEN this other stuff should happen….”

A per­fect Prod­uct will pro­duce all the same out­put strings as the explicit Mar­ket func­tion does, for all 2^N inputs. Any given Prod­uct earns money in pro­por­tion to how well the released fea­tures match the Market’s tar­get desires.

But the Team only has access to aggre­gate income infor­ma­tion, unless they pay for mar­ket­ing.

There’s a lot of lee­way here. Sup­pose a Team play­ing the low-​​hanging-​​fruit game comes along, and they release a Prod­uct that is just a con­stant out­put, say 1111100000, the ulti­mate in “you can have any color car, as long as it’s black.” They might just make some money, since there’s a chance the under­ly­ing Mar­ket desire is biased that way.

Although I don’t have any inter­est to do it in this model, I have to note there’s an open­ing here for explor­ing “mar­ket dynam­ics” in such a world. A slightly smarter Team (in the same Mar­ket set­ting) might come along and release some­thing that is just as dynam­i­cally dumb as the 1111100000 folks, but bet­ter matches the aver­age desired out­put for each fea­ture. They’ll make more. Some inno­va­tor may well come along and release a prod­uct that pays atten­tion to some inputs, maybe first-​​order inter­ac­tions. They’ll make a killing.

But today I want to focus on the process within a Team, not the dynam­ics of a full mar­ket pop­u­lated by Teams with dif­fer­ent strate­gies and capacities.

Now I’m reminded that the mod­els of project man­age­ment (at least the ones we’re used to) will include some big block process labeled “analy­sis” or “mar­ket­ing” or “project man­age­ment”, and since Agile project man­age­ment implies holis­tic adap­ta­tion that should include mar­ket research on the Team when­ever mar­ket research gen­er­ates busi­ness value, we’d bet­ter include that block in our model.

Call it Require­ments Gath­er­ing (or maybe Refine­ment, if you want to imply the inclu­sion of the tra­di­tional “Main­te­nance” func­tion­al­ity, as we should in an Agile Team). Let’s give them some tools.

How about inter­views? Say each inter­view costs a cer­tain amount of money and time, and the Team’s ana­lysts can (with 100% con­fi­dence) elicit one input-​​output com­bi­na­tion from the Mar­ket. “If you’re 1011100011, what will you do next?” To me, this has the metaphoric capac­ity to han­dle peo­ple not know­ing until they’re asked, and also rev­e­la­tion of only indi­rect infor­ma­tion.

An inter­view like that should be expen­sive, I think. How about A/​B test­ing, or more gen­er­ally polling? “If you’re 1001101111, which of the fol­low­ing do you pre­fer as the out­come, 1011100011 or 1100000011?

Then of course there’s sales data, for sit­u­a­tions in which there is in fact a Prod­uct already released: As I men­tioned, even a very dumb prod­uct that fills some roles slightly bet­ter than the ones it messes up has the capac­ity to pro­vide his­tor­i­cal infor­ma­tion (and rev­enue to pay for improvements!).

Now this brings up one more change from the pre­vi­ous iter­a­tion of this model: Revenue.

In the first ver­sion, I pro­posed a model in which each fea­ture had an intrin­sic pos­i­tive value, and a released Prod­uct col­lected that value dur­ing every time period where it matched the intrin­sic Mar­ket dynam­ics. We don’t have Mar­ket dynam­ics now, we have a Mar­ket that’s a black box.

The base­line in this old one was “you make noth­ing”. I’ve rethought that, too; let’s throw a mon­key wrench into our Team’s world, and allow fea­tures to be neg­a­tives as well as pos­i­tive desires. In other words, when­ever a fea­ture with a neg­a­tive value is matched, the Team loses money.

So where are we now?

Let’s say the Prod­uct is some kind of func­tion that takes N gen­er­al­ized Boolean inputs and pro­duces (coin­ci­den­tally) N gen­er­al­ized Boolean out­puts. The space of inputs is all N–bit binary strings, gen­er­al­ized to include # or “don’t care” inputs where an input is ignored, and the out­put space is all N–bit binary strings gen­er­al­ized to include ? or “unset” out­puts, when an out­put bit is not explic­itly set to either 0 or 1.

Take a sec­ond here. We can there­fore say a Team “starts” (hav­ing done no work think­ing about or devel­op­ing any Prod­uct at all) with a “default” Prod­uct that ignores all inputs and pro­duces no out­puts: ###…### -> ???…???. They aren’t earn­ing or los­ing any money because their Prod­uct has no vis­i­ble fea­tures to cap­ture (pos­i­tive or neg­a­tive) Mar­ket share.

Sup­pose after some devel­op­ment time a Team releases a (rather stu­pid) Prod­uct that imple­ments ###…### -> 111…111; in other words, it always returns 1 for every fea­ture. How much money do they earn?

We deter­mine this either by doing the math or by sam­pling. We know the ground truth Mar­ket func­tion, a large and rather com­pli­cated truth table. If we want to do the math, we can con­vert it into a cum­ber­some prob­a­bil­ity func­tion, and just assume that the diver­sity of peo­ple in the mar­ket is absolutely uni­form: that is, that peo­ple will “use it” by apply­ing every pos­si­ble input. Alter­nately, we can sam­ple this space (using the same assump­tion of uni­for­mity) by tak­ing 1000 or so ran­dom bit-​​strings as inputs, and count­ing up how much rev­enue the Prod­uct gen­er­ates (or loses) by match­ing those features.

Sup­pose in our exam­ple there are 21 fea­tures, and that the value of the first is -\$10, the sec­ond -\$9, and so on, with the 21st pro­vid­ing value of +\$10. The Prod­uct that returns 1 for every input will lose $10 for every 1 in the first feature’s out­put table, and earn $10 for every 1 in the 21st feature’s out­put table. So it’s just a mat­ter of count­ing to deter­mine how prof­itable it turns out to be.

Just to make it clear, sup­pose some other spe­cial­ized Team releases a Prod­uct that does a per­fect job at only one fea­ture whose intrin­sic value is $4. It will earn the Team $4 for every pos­si­ble input com­bi­na­tion, regard­less of the other fea­tures; no money is lost or gained for fea­tures that are not present, or do not match desired outputs.

More later; time for lunch!

Requirements, Design, Implementation, Verification, Maintenance

Posted on by

Spent a few cycles yes­ter­day think­ing about Lau­rent Bossavit’s provo­ca­tion, and while I’m hav­ing fun get­tin’ all com­plex­o­log­i­cal after so many years away from the Santa Fe Style, I’m increas­ingly con­fi­dent it’s not quite what he was ask­ing for.

After all, com­plex sys­tems mod­els offer no pre­dic­tive value, except in the sense that they pro­vide insights into mod­els’ intrin­sic con­se­quences. As with Genetic Pro­gram­ming vs Sta­tis­tics, agent-​​based com­plex sys­tems mod­els are use­ful only inso­far as they sur­prise and inspire fur­ther mod­els, not as tools inter­act­ing with the “real world”.

At any rate, I’m find­ing it a use­ful exercise.

Here’s where I am now, as of this writ­ing (link to old commit).

Per­son­ally I’m con­fi­dent that the toolkit of com­plex­o­log­i­cal mod­el­ing is emi­nently prac­ti­cal; Nk mod­els, Boolean net­works, agent-​​based sim­u­la­tion, all the rest is in my blood, so the design pat­terns are so obvi­ous to me that I can slap some­thing together in a few min­utes. But it’s inter­est­ing as I talk with folks (Lau­rent and Ron Jef­fries, among oth­ers) about this lit­tle sketch, how often my cor­re­spon­dent wants to drop my mod­el­ing toolkit back down to the famil­iar level of aggrega­tive sta­tis­tics, or stock-​​and-​​flow continuous-​​valued sys­tems models.

Sub­jec­tively, when folks ask for some­thing so intrin­si­cally focused on story-​​telling to be “sim­pli­fied down to cleaner math”, it fees a lot like when old-​​school pro­gram­mers ask to use Inte­ger val­ues for Boolean val­ues (“-1 is false!”), or when I see sim­ple structs of prim­i­tives used instead of full-​​fledged objects in an object-​​oriented lan­guage. Some­thing about habits and famil­iar­ity, surely, but also an utterly dif­fer­ent sense of ele­gance and pur­pose, of craft, creep­ing in.

(Which is inter­est­ing, and worth watch­ing over time. Some­thing very sim­i­lar comes up when I read this.)

At any rate, a Boolean Net­work feels like a nice, tun­able model of a com­plex world with hid­den inter­nal struc­ture. In addi­tion to tun­able inter­nal com­plex­ity, one can always add exter­nal­i­ties, noise, struc­tural dynam­ics, all sorts of things that model the way we treat error and uncer­tainty in the real world much more authen­ti­cally than merely adding some normally-​​distributed epsilon to a number.

But there’s an inter­est­ing mod­el­ing choice before me, now, and I’m think­ing aloud about it. As you’ve read, I’ve sketched a world with use­ful tun­able para­me­ters; I’ve sketched a com­par­i­son (“water­fall” vs “agile” as vari­a­tions in tim­ing and order of staged work). But one does want one’s sim­u­lated “Team” to be able to do that work in a rea­son­able way.

So what is “require­ments gath­er­ing”? Or design, or cod­ing, or test­ing? As hap­pens now and then when build­ing com­plex sys­tems mod­els, the ques­tion is not “What do peo­ple really do?”, but rather, What is the sim­plest con­vinc­ing mimic of what is implied by a story in which we say, ‘Five require­ments have been gath­ered’? Or, Six­teen designs have been con­sid­ered, or Eleven tests have been cre­ated and run?

I have some notions, as you can see in my notes at GitHub. But there’s also this feel­ing that peo­ple brought up in the clas­si­cal math­e­mat­i­cal mod­el­ing world (the one that per­fuses com­puter sci­ence as well) will prob­a­bly raise their eye­brows. One is brought up to think of vari­ables as prim­i­tives, and of inter­ac­tions not as algo­rithms but as equa­tions. One is trained by decades of expen­sive com­put­ing and resource lim­i­ta­tion to assume that aggre­ga­tion takes place prop­erly within a model, using stocks and flows and aggrega­tive mea­sures and sta­tis­tics, deriv­a­tives and aver­ages all over—and not to see each instance or run of a model as being contingent.

This is a kind of pre­ma­ture opti­miza­tion that I haven’t heard remarked before. But I think it’s close to the core of what makes “com­plex­ol­ogy” abhor­rent unto the jour­nal­ist. And what’s inter­est­ing is that it’s com­ing from nerds who have almost cer­tainly dab­bled a bit with these alien, thrilling, com­plex­o­log­i­cal mod­els: imple­mented a Game of Life, for example.

And yet I think most haven’t real­ized that con­tin­gency and sub­jec­tiv­ity are baked right in. Pro­tip: On aver­age, over long time scales, all Game of Life runs are empty.

Maybe nobody’s pointed that out before. I think they prob­a­bly ought to have done so. :)

I’m fostering diversity of coverage in social networks, Nerdy

Posted on by

Yes­ter­day I spent a while jot­ting down a prover­bial Idea-​​from-​​the-​​Shower. Briefly, it’s a social mech­a­nism (scor­ing sys­tem) for dri­ving mem­bers of a com­mu­nity of peo­ple to pay enough atten­tion to a col­lec­tion of objects broadly—that is, with­out copy­ing one another too closely.

In the end, you’ll recall I real­ized that maybe what I’m inter­ested in is a mul­ti­ob­jec­tive dynamic, not some sin­gle met­ric that makes all kinds of side-​​effects hap­pen. So I’m revis­ing my goals and think­ing a bit, here.

My arbi­trary snap­shot exam­ple of a lit­tle net­work of Ada, Byron and Charles look­ing at three items looked like this:

net_metrics.jpg

The scores, by my all-​​in-​​one met­ric, were (Ada: 3), (Byron: 1), (Charles: 1).

But think­ing about what that all-​​in-​​one-​​score mea­sures, I’d rather return to the sep­a­rate goals I’ve inad­ver­tently (and in my long expe­ri­ence, unwisely) col­lapsed and con­flated here.

Let’s look at stake­hold­ers. As admin­is­tra­tor of this col­lec­tion of peo­ple and items, I have par­tic­u­lar goals I’m try­ing to elicit.

As admin­is­tra­tor, as the set of “mem­bers” take turns adding new links to items, I want:

  • to min­i­mize the num­ber of items which haven’t had enough peo­ple look at them, because I want to have a sense of progress in our col­lec­tive work; “enough” might be 1, but more likely it’s 3 or 4 peo­ple, espe­cially in a com­plex prob­lem where we really want to have unre­li­able mem­bers check­ing one another
  • to min­i­mize the num­ber of items that have been looked at by the same sub­set of peo­ple, in order to pro­mote diver­sity of expe­ri­ence and exper­tise among the membership

In graph the­ory terms, I sup­pose what I’m think­ing here is that I want to avoid mak­ing con­nected com­po­nents of the graph: cliques (in the social sense) of peo­ple who all fol­low along together and look at the same stuff. Because when that hap­pens, I’m con­cerned that they play off one another’s weaknesses.

So to get the diverse mem­ber­ship to do what I want, with­out nec­es­sar­ily try­ing to slap up a big chart on the prover­bial wall and say, “Hey, every­body, can we raise Met­ric X please?”, I think I should be design­ing objec­tives for the mem­bers them­selves to pur­sue. Indi­vid­u­ally, and com­pet­i­tively and collaboratively.

If you’re a mem­ber, I think you should try to:

  • “col­lab­o­rate”: max­i­mize the num­ber of items you’ve linked to (which indi­rectly leads to coverage)
  • “diver­sify”: max­i­mize the num­ber of dif­fer­ent sets of peo­ple who have looked at the same items as you (which pushes towards my sec­ond goal)

Now as a rule of thumb in mul­ti­ob­jec­tive search, I always rec­om­mend one pick a direc­tion and stick with it. I usu­ally min­i­mize (because the graphs are eas­ier to draw) so let’s trans­form these into more mathematical-​​ish terms like:

  • “col­lab­o­rate”: min­i­mize your neg­a­tive outdegree
  • “diver­sify”: min­i­mize the neg­a­tive count of dif­fer­ent sets of peo­ple who have looked at the same items as you

(Alter­nately, we might con­sider “min­i­mize the num­ber of dupli­cated sets of peo­ple who have looked at the same items as you”, but that may not be the same thing exactly; we’ll look in a bit at that variation.)

How do Ada, Byron, and Charles do on those objec­tives, and what does this trans­late into in our sta­tic snap­shot of a dynamic process?

                Ada Byron Charles
  collaborate:   -3   -1    -1
    diversify:   -2   -1    -1

If Byron next links to item #3, then we get

net_metrics2.jpg
                Ada Byron Charles
  collaborate:   -3   -2    -1
    diversify:   -3   -2    -1

If in the next step Charles links to item #1, we get

net_metrics3.jpg
                Ada Byron Charles
  collaborate:   -3   -2    -2
    diversify:   -3   -2    -2

Assum­ing there aren’t any more items (for the moment), then Ada has no moves, and inevitably Byron links to item #1, so we get

net_metrics4.jpg
                Ada Byron Charles
  collaborate:   -3   -3    -2
    diversify:   -2   -2    -1

Notice what’s hap­pen­ing to the “diver­sify” scores? The rel­a­tive posi­tions of the “play­ers” on this score got bet­ter, then got worse and is now flat­ten­ing out. Byron’s move elim­i­nates all of Ada’s advan­tages. The final move by Charles elim­i­nates all dif­fer­ences between every player, on both objectives.

This is a very lim­ited exam­ple, but I’m still just think­ing out loud here so bear with me. What’s hap­pened in the mean­time to my Administrator’s goals?

I have two goals. I want “progress”, and in this exam­ple say I’ve decided an item is “done” when at least 2 peo­ple link to it. So I want to min­i­mize the num­ber of items with fewer than 2 links. My other goal is some­thing like “promis­cu­ity”, which I’ll mea­sure as the num­ber of dupli­cated inde­gree label sets over all items.

  time         t     t+1    t+2    t+3     t+4 (Charles finishes it)
  progress     1      2      3      2       1
  promiscuity  2      3      3      3       3

This is inter­est­ing, frankly. I’m going to think a bit about that. But it seems to be work­ing, in this toy example.

So, open ques­tions I’m con­sid­er­ing: What will hap­pen if there are a lot of peo­ple and items? We started look­ing in this exam­ple at a stage where most of the pos­si­ble links were already present; what hap­pens as new items are added, or if old (“fin­ished”) items are removed from the sys­tem? If the play­ers greed­ily try to dom­i­nate one another on these two scores, what hap­pens? If play­ers act ran­domly, what happens?