Notional Slurry Logo

Archive for January, 2007

You’ve got to stuff all that nonlinearity somewhere…

Over on the XP list, Bob Hathaway points to a recent column on pair programming by Matt Stephens, self-styled “agile iconoclast”.

I posted a draft of this on the list, but think it might go a bit farther in the context of some other work I’m doing….

First, the innocent reader should realize that what’s being discussed here is “pair programming”, a core practice in the Extreme Programming software development methodology, and in many other agile development approaches. But clearly Stephens has an issue not merely with the single practice, but with the underlying management philosophy.

The content and tone of the article reminds me of a lot of the teen-blogging I’ve encountered on Xanga or MySpace. It provides a lot more psychological and sociological insight about the author — and his affected audience — than I think he intended. Dig down through the BO-radiating garlic-eating programmer prejudices. If you grit your teeth and think about it, it could be seen a very interesting cultural artifact.

My first reaction? “Never, ever hire this guy or his buddies.” Ever.

But it does make you think.

I wonder if I’m hearing a deep assumption in this piece: that software projects are only for making products. The customer only benefits from being given what they ask for; a team is hired only on the basis of the speed and accuracy with which they program; the developers are expected to develop themselves only on their own time. In other words: learning is not business value. The customer shouldn’t be bothered to learn or adapt or participate in the development process, as they must in all XP projects. And the “programmers” shouldn’t slow down their typing to learn and chat and discuss anything.

Stephens is clearly not somebody you want to chat or discuss anything with. But that’s OK. I don’t have to. What I do have to do, as a project manager, is obtain value. He’s demonstrated, among other things, that the only value he can provide is from being used as an example.

So Stephens assumes value only comes from “smart people” typing, and that any amount of talking slows down those good typists. Unless you’re transcribing books, this proves to be really stupid fallacy… but a philosophically interesting one. In a way, it implies that learning behavior shouldn’t happen because it’s not adding value to the project, but shunting value to the team. They’re talking, they’re learning, they’re slowing down their “productivity”.

Call it “stealing” value from the customer.

Now by applying such an assumption to the XP process, Stephens and his ilk demonstrate a fundamental lack of understanding. Not just of XP (which he’s clearly never done), but also of complex project-based software development work, project management… and frankly even programming.

One of the fundamental notions in XP projects is the practice of breaking the customer’s goals into linearly decomposable chunks, small enough so that everybody (customer and team) can understand and evaluate them independently, but large enough so that each one provides business value when delivered.

In a way, this is also a myth of sorts. In dividing complex and inevitably nonlinear projects into small stories we treat as linearly decomposable bits of business value, we simultaneously shunt all the linking nonlinearity and complexity into the social structure and dynamics of the team. Working with the customer, we clarify what’s wanted and needed, but in doing so we manage the dependencies and complexity on the social side.

You talk about it.

That’s why XP’s practices and values of pair programming, shared code ownership, universal test coverage, and all the rest are so crucial. They forge the collective learning system in the team, which supports the miraculous decomposition of the customer’s difficult project into a pile of apparently unconnected stories. The process allows the customer to make immediate progress — and to maximize the impact of the first small step — exactly by masking all the difficult wiring inside the development team’s heads, the code they write, and test base. And most of all, in the ongoing conversations.

I’m willing to agree with the author that, “Not everyone is naturally extroverted or a social butterfly that will wither away if he doesn’t get to chatter to his colleagues for eight hours a day.” He’s clearly one of those non-butterfly folks (not least because he’d rather write about his BO-stinking colleagues in a public forum than chat with them directly).

But see… if he managed to actually start working on an XP team, and stopped talking and pairing and nattering away they day, and instead picked out work he thought was useful to the customer without consulting them, and went off alone and typed shit in, and in the end everybody had to deal with the mess he’s made… in that case he is the person stealing value. He has reintroduced all that complexity; he’s undermined its storage in the team; he’s making it harder for the client to choose incremental value; he’s keeping secrets and making himself an obstacle, with his own self-centered uncommunicative blockish head.

And thus he threatens that project’s success.

He’s off the team. Implicitly, when he fails to collaborate and obstructs collective work. Explicitly, when it becomes apparent.

links for 2007-01-31

links for 2007-01-29

Ten ideas about Ideas

Ten ideas about Ideas, from Doc Searls. Absolutely crucial advice for life in Modern Times, in domains quite distant from mere software development.

Sad I haven’t seen them before today. But now we all have.

Schumpeterian Gales of Derisive Laughter, Bruce

Dorothea at Caveat Lector translates from the AAP’s response to the Nature exposé:

And hey, we may be stumblebum slimepuppies, but we’ve been stumblebum slimepuppies for decades! Nay, centuries! You can’t want to get rid of us now! After all, nobody protected the pennyfarthing bicycle, and just look what happened! You don’t want that on your conscience, do you?

The broken mirror of the Mind[map]

I don’t get Mind Maps. Really. I’ve worked with people who used them all the time. I know several folks who draw them in real-time, while they’re brainstorming, as a way of connecting things and spitting out an… well, an outline. There a innumerable books and guides and workshops and software packages. Sliced bread’s popularity seems to be waning in the light of the advancing Mind Map Revolution.

And I look at the results, and I see: A rooted tree. An outline. A strict hierarchy. A homogeneous classification diagram. A bundle of undifferentiated hard constraints. A simple Venn Diagram made difficult and misleading by the addition of geographical information.

Me, I even find acyclic directed graphs uncomfortably constraining, let alone trees. Me, I’ve spent 20 years [crap!? old now!] doodling fuzzy directed hypergraph visualization techniques (A fuzzy graph is a mathematical object in which the “attachedness” of nodes is qualitative. A hypergraph is a mathematical object, like a graph, but where sets of nodes are connected to sets of nodes — not single nodes to single nodes — or possibly nodes are connected by sets of edges to one another. Or both…), and I use some of those techniques in my more detailed professional work. Admittedly I can’t show them to you, because they’re too dense. But they’re better for me than Mind Maps.

And they say I’m a visual thinker. Just not a planar one, I spose.

So if you’re going to communicate something to me that wants to be described by a Venn Diagram, then sketch an outline with a root at the top and the sub-nodes indented, and use your face and your hands and your spoken and written words to explain it to me.

But if you want to describe anything where you’d have the word “also” in the description, then ferchrissakes take the time to connect every thing as needed — in a web — with all the qualia and ontological assumptions spelled out right there on the nodes and edges.

Besides, they look like the Shadows invented them. Not that there’s anything wrong with that. I always sided with the Shadows, anyway. Frickin’ nosey Vorlons….

Schering-Plough classified ads in today’s Ann Arbor News

The same week that Pfizer announced the closure of its research facility here, and the loss of 2000+ jobs from Ann Arbor, Barbara (reading the paper just the other side of the room) points out that there’s a quarter-page of Schering-Plough job ads in the Sunday paper. Of course we don’t know for sure if they were there last weekend as well.

Doubt it.

Social blogging ideas for the taking: Penumbra plugin

Most bloggers maintain a blogroll of feeds they read regularly.

I suspect there’s a tendency to load up quickly on feeds related to one another, and then shift to a comfortable steady state in which few if any feeds are added or removed. I can see how this dynamic might tend to crystallize local networks of bloggers, linked to one another, very soon after they arrive on the scene. At the same time, the general insularity and inertia of blogrolls might act as a barrier to entry to useful new voices, who otherwise would be a great fit in the community. Trackbacks from new bloggers to established ones serve to some extent to erode this barrier, but trackbacks don’t propagate, and they also need to overcome the linked-to blogger’s limited attention.

John Holland would say something smart here about the trade-offs between exploration (adding new and better links) and exploitation (building ongoing conversations). I’m worried that exploitation is inbuilt in the socio-technical system.

So here’s an idea for a service, whether it be a website or a plugin.

Take my blogroll. Now spider each site, and compile a database of the blogrolls of all the sites thereon. Filter out all entries that already appear on my blogroll.

Sound like a referral service so far? Not quite….

First, I don’t want to cinch my little echo chamber tighter. So: Sort the database by the number of inbound links, and remove the top quartile. In other words, remove the most-linked-to blogs.

Second, I’m sure there’s a wide range of special interests I don’t share; I don’t knit, I don’t pray, I have no cat; I do read 19th-century magazines, I do care about automated mechanical design, and I long to understand TextMate enough to make it the basis of my laptop user experience… so let’s avoid the rarest of the rare. Very few care about all my quirks, and vice versa. Throw out the least-linked-to quartile, as well.

Call the remaining blogs the “penumbra” of my blogroll: Sites that have caught the attention of a few people on my blogroll, but not just one, and which I have not yet seen. The Goldilocks fraction.

And, thinking I should explore a bit more and exploit a bit less, I want to go see those sites. Hear what they have to say. And then maybe I’ll add them to my blogroll, and thus bump up their centrality.

If link-counting doesn’t work as expected, here’s one notion that might help: Give each database entry — each blog in the compiled list — a score calculated on the basis of a Technorati ranking. So big blogs and eeny weeny blogs linking to feeds would disappear, leaving the Long Tail middle to have its say.

I would like that. No time now. Make it work as a plugin in Wordpress, or make it a site I can send an OPML file to, and we’ll all benefit.

links for 2007-01-28

Worrying about collectibility

Sitting here in the recliner, drinking my requisite morning liter of Lime Diet Coke (yeah, I know, bad for me), catching up on blogs via Shrook, planning out the boxes of stuff I need to photograph later today to sell on eBay, and the NPR station is talking about the presidential candidates that are now so prolifically plopping out of the ass of the American-lack-of-consensus.

And I hear the commentators talking about the latest offerings’ worries about collectibility. “He’s going to need to work on his collectibility, if he’s going to have any chance in the run-off election season.”

As my attention was yanked from my laptop to the radio by these arcane admonishments, I thought for a second about how that might work. Collectible trading card games? Is there an online marketplace? Is it those pin-back buttons? We just came across a Nixon button in my mother in law’s basement.

And then I turned off the radio, and went back to planning eBay auctions.

Left as an exercise for the student: The Simple Guess-the-dots Problem

Suppose I have a secret list of N objects, with some attributes. Think of them as a set of N points in the Cartesian plane, with point i located at coordinates (xi,yi).

I tell you how many points there are, and we agree that (a) point #1 is located at coordinates (0,0), (b) exactly one of the other points lies at coordinates (1,0), and (c) all the points are in or on a unit circle of radius 1 centered at (0,0). I’m not going to tell you where the points are. You need to derive that from a little guessing game; you ask me a question, and I give you some information. Eventually, you can figure out where all the points are.

You name any subset of three points from the set of N, and I will tell you which pair in those three is closer together, or if they are all three equidistant. We repeat this process as long as you want.

Too hard? OK. I will even tell you how you derive the distances between the points: You keep a little matrix D of relative distances in your head; an N×N matrix in which element dij is the estimated distance between point i and point j. For every triple of points (A, B, C) you ask me about, if I tell you “A and B are closer”, you decrease the values of dAB and dBA by 10%, and increase the values of dAC, dBC, dCA and dCB by equally distributing the “shrinkage” of AB/BA among them.

Or, more formally,

AB ← 0.9 dAB

BA ← 0.9 dBA

AC ← 0.05 dAB + dAC

BC ← 0.05 dAB + dBC

CA ← 0.05 dAB + dCA

CB ← 0.05 dAB + dCB

You may then do whatever math you like on the matrix. Now this process implies you have selected some matrix D0 as an initial guess, which obeys certain tacit assumptions (all the values are symmetric across the diagonal, all points are at different locations, maybe some others). And the process also implies that you use the math phase to renormalize the values somehow to produce a matrix of guesses that fit my unit circle constraint.

If you want. Whatever. You just tell me that distance matrix, when you’re done fiddling with it in whatever ways best float your boat.

Of course, knowing where all the points really are, I can easily construct a matrix A, where each element aij is defined as the actual distance between points i and j.

So you’ll have some D0 in mind, and start spitting out a time-series of subsets of points. And I’ll give you back information about which pair is closer (if any) in that triplet, and you will update your matrix as I’ve described, and then do whatever arithmetic on it you like to get a matrix Dt after t guesses.

And given your Dt, I can evaluate how far off your distance array is from mine by:

  • the maximum difference |aij–dij| for any i,j
  • the summed squared differences (aij–dij) over all i,j
  • the total absolute difference |aij–dij| summed over all i,j

Maybe I’ll use all of them. I think I will! If you have the positions of all the points correct, then your score will be a point Z*=(0,0,0). If you have any of the positions of any point wrong, your score Z will be some point in the positive orthant.

Challenge 1: I will select N=33 points. Describe a heuristic that (a) selects a specific initial guess D0, and (b) generates a series of three-element subsets of {1,2,…,33}, so that the time-series of scores Zt converges to Z*=(0,0,0). Your heuristic must include a simple method for choosing the order of triplets.

Challenge 2: Describe the heuristic such that the time-series of Zt approaches Z* as fast as possible.

Challenge 3: Suppose another student has proposed a heuristic in which D0 is filled with random numbers in the range [0,2], and for the first 1000 steps they choose triplets with equal probability, and do not renormalize their distance array at any stage. (This may be stupid, or it may not.) If after 1000 steps, you are given their distance array D1000 and forced to use it as your D0, can you expect any change in the performance of your heuristics from the other challenges?

Hint: Suppose instead of points in a plane, we restrict ourselves to points on a line. Would that help you select a series of triplets? Why?

Older entries »