More advice on the box-stacking algorithm that cropped up in the book.
Category Archives: agile
A quick little coding kata
Requirements, Design, Implementation, Verification, Maintenance
Spent a few cycles yesterday thinking about Laurent Bossavit’s provocation, and while I’m having fun gettin’ all complexological after so many years away from the Santa Fe Style, I’m increasingly confident it’s not quite what he was asking for.
After all, complex systems models offer no predictive value, except in the sense that they provide insights into models’ intrinsic consequences. As with Genetic Programming vs Statistics, agent-based complex systems models are useful only insofar as they surprise and inspire further models, not as tools interacting with the “real world”.
At any rate, I’m finding it a useful exercise.
Here’s where I am now, as of this writing (link to old commit).
Personally I’m confident that the toolkit of complexological modeling is eminently practical; Nk models, Boolean networks, agent-based simulation, all the rest is in my blood, so the design patterns are so obvious to me that I can slap something together in a few minutes. But it’s interesting as I talk with folks (Laurent and Ron Jeffries, among others) about this little sketch, how often my correspondent wants to drop my modeling toolkit back down to the familiar level of aggregative statistics, or stock-and-flow continuous-valued systems models.
Subjectively, when folks ask for something so intrinsically focused on story-telling to be “simplified down to cleaner math”, it fees a lot like when old-school programmers ask to use Integer values for Boolean values (“-1 is false!”), or when I see simple structs of primitives used instead of full-fledged objects in an object-oriented language. Something about habits and familiarity, surely, but also an utterly different sense of elegance and purpose, of craft, creeping in.
(Which is interesting, and worth watching over time. Something very similar comes up when I read this.)
At any rate, a Boolean Network feels like a nice, tunable model of a complex world with hidden internal structure. In addition to tunable internal complexity, one can always add externalities, noise, structural dynamics, all sorts of things that model the way we treat error and uncertainty in the real world much more authentically than merely adding some normally-distributed epsilon to a number.
But there’s an interesting modeling choice before me, now, and I’m thinking aloud about it. As you’ve read, I’ve sketched a world with useful tunable parameters; I’ve sketched a comparison (“waterfall” vs “agile” as variations in timing and order of staged work). But one does want one’s simulated “Team” to be able to do that work in a reasonable way.
So what is “requirements gathering”? Or design, or coding, or testing? As happens now and then when building complex systems models, the question is not “What do people really do?”, but rather, What is the simplest convincing mimic of what is implied by a story in which we say, ‘Five requirements have been gathered’? Or, Sixteen designs have been considered, or Eleven tests have been created and run?
I have some notions, as you can see in my notes at GitHub. But there’s also this feeling that people brought up in the classical mathematical modeling world (the one that perfuses computer science as well) will probably raise their eyebrows. One is brought up to think of variables as primitives, and of interactions not as algorithms but as equations. One is trained by decades of expensive computing and resource limitation to assume that aggregation takes place properly within a model, using stocks and flows and aggregative measures and statistics, derivatives and averages all over—and not to see each instance or run of a model as being contingent.
This is a kind of premature optimization that I haven’t heard remarked before. But I think it’s close to the core of what makes “complexology” abhorrent unto the journalist. And what’s interesting is that it’s coming from nerds who have almost certainly dabbled a bit with these alien, thrilling, complexological models: implemented a Game of Life, for example.
And yet I think most haven’t realized that contingency and subjectivity are baked right in. Protip: On average, over long time scales, all Game of Life runs are empty.
Maybe nobody’s pointed that out before. I think they probably ought to have done so.
How would one compare “agile” to “other” software development usefully?
Laurent Bossavit says in passing
@Vaguery I’ve been toying with the idea of building simulations of sw projects for years now, & looked at Abdel Hamid’s work, but it’s hard.
— Laurent Bossavit (@Morendil) March 3, 2012
This is an interesting proposition, but having looked over the work of Abdel-Hamid a bit, I’m wondering if we might consider a more agent-based approach.
The organizational behavior and systems-dynamics framework we see in that corpus is admirable and interesting, but one has to admit it’s focused on aggregate behaviors of “developers” and “management” observed from the standpoint of a corporate planner stakeholder. I don’t feel obliged to have that conversation any more.
What does one want to explore? The things that set apart agile projects from inagile ones, I suppose: Risks of the many modes of failure; adaptability; scaling; work load; business value delivery rates and probabilities; subjective experience; knowledge sharing; something about the many stakeholders (“management”, “team”, “customer”, “users”)?
All possible. I’ve had some incidental conversations with Ron and Chet about this, too, over the years. Maybe it’s time to pursue it a little ways….
But what does one want to compare?
Who are you, and what do you want?
The Mirror Dojo: Genetic Programming for Agile Teams
This is the current iteration of the Genetic Programming workshop née Agile Team Dojo I’ve been working on over the last few years.
I’m looking at the Michigan/Ohio/Indiana region for an interesting place to run it. If you’re interested in scheduling me for a two– or three-day workshop, feel free to contact me online.
You know how to do that.
The Mirror Dojo: Genetic Programming for Agile Teams
Genetic Programming has been actively researched and promoted for more than a quarter century. It’s a broad collection of design practices and modeling techniques for the “automatic” discovery of abstract patterns and structures.
And that means full-fledged patterns and structures: algorithms, predictive models, complete mechanical and optical and electronic designs, and even blue-sky artificial intelligence systems.
Some of the field’s big hits include:
- the “automatic” rediscovery of the basic rules of physics [PDF] from digitized video of experimental systems,
- the complete and “automatic” redesign of an antenna for a space probe,
- systems that let robots “introspect” to find ways of walking, even when seriously damaged,
- “automated” stock trading systems,
- patentable commercial analog circuit and optical designs,
- “automatic” design of backgammon players [PDF], and
- “automatic” discovery of code patches for real, working software [PDF].
Sexy stuff! Nerds like us love it.
Better yet: I can describe all the basic design principles of Genetic Programming in four sentences. It’s so simple to describe that I’m completely confident that I can help you you—a competent software developer working on an agile team—write a working GP system in an hour or so!
And that’s just what we’ll do in this two-day workshop.
But there’s one more thing.
You’ve probably noticed that I always put extra-scary scare-quotes around “automatic” whenever it comes up.
During this dojo we’ll be approaching this material as an agile team. We’ll build at least two full-featured genetic programming systems, and we’ll bump very quickly into those scare-quotes.
And that’s what this workshop is about.
See, I’ve been working in this field for most of 20 years. It turns out that even after all that time, there’s a large and troubling gap between the tutorials and demos of genetic programming, and successful problem-solving with GP. You can measure that gap in terms of time, or computational resources, or expected quality of results.
Sound familiar?
Much of the advanced academic research being done today in Genetic Programming focuses on ways to increase the computational power, to bring more processors and faster code to bear so that “automatic” problem-solving has a better “chance of success” on a complex problem.
Ah, look; more scare quotes.
See, in this workshop we’re not advanced academic researchers in Genetic Programming. We’re much better prepared than they are: we’re an agile team.
In the workshop we’re going to be exploring how to tell our little artificial “team” of “automatic developers” what it is we want, and how they should go about making it for us, and (because GP just works) they’ll be “releasing software” for us. What we’ll be doing is designing the rules by which they solve our problem: especially the ones that spell out how we want them to interact with one another.
Which should explain the name of this dojo. And maybe even why it will take little bit longer and a bit more effort than most others you’ll run into.
Scope: This is a two– or three-day workshop, for three to eight software developers, engineers, coaches, designers, scientists, and other nerds.
The majority of participants should be familiar with common platform-agnostic programming languages (Ruby, Python, Java, Smalltalk). They should be comfortable working in an agile team: we’ll collectively work in one shared programming language, and rely on automated unit and acceptance tests, rapid release schedules, agile planning, and pair programming. There should be enough laptops for every pair to code, and network connectivity enough to use github for version control and coordination.
On the first day of the workshop (6 hours plus lunch) we’ll establish the social infrastructure, and implement a simple but full-scale genetic programming system for symbolic regression. At the end of the day, we’ll choose an advanced project for the next day.
Because we’re all nerds, you and I both know you can’t “stop working” after just five or six hours—and that’s fine. But the work day for the project is six hours plus lunch. So no commits overnight!
On the second day (6 hours plus lunch) , we’ll use genetic programming to address a technical problem where results are obviously practical—and probably publishable.
In three-day workshops, the final day can be used (at the team’s discretion) for either refinement and public release of tangible product from the prior days, or for a third project using different GP design patterns.
Why?: The dojo is just what it says: an exposure for agile software developers to a sexy but poorly-understood technical practice with great economic potential in the coming years. At the same time they’re learning about the tech, they’ll be surfacing aspects of their own work, and the way agile practices mold project management in the “real world”: requirements, goal-settings, information-sharing, metrics, collaboration patterns, infrastructure, delivery schedules, and even the jurisdiction of management vs. developers.
Cost: The most important cost for this exercise is the participants’ interest and attention. If those have been made available, the only financial costs are for the venue, travel, food and board (where needed) for the participants.