Sta­tis­ti­cal Mod­el­ing, Causal Infer­ence, and Social Sci­ence: Visu­al­iza­tion of large datasets:

smooth­Scat­ter from gene­plot­ter R pack­age can help a lot in pro­duc­ing more infor­ma­tive and eye candy graphs.

See also the excel­lent cited paper from Gen­tle­man et al.

This is a step towards an ongo­ing effort to Clean My Desk. And also to cut back on the friv­o­lous increase in the world’s heat I gen­er­ate by indis­crim­i­nately pick­ing books up at the library and haul­ing them back here (only to be told they’re over­due with­out being read). I’m not review­ing; rather, not­ing inter­est­ing arti­cles in the books.

I’ve already had some words, months back, about the ridicu­lous prices charged by cer­tain North­ern Euro­pean Pub­lish­ing Clans, and I don’t want you to con­sider buy­ing these books for an instant. The links to the left are offered more in the spirit of a pub­lic shame through lit­eral fact, not an appro­ba­tion. You could go to Ama­zon and buy some other crap—some use­ful crap, if you want. I wouldn’t mind that. Because Ama­zon would pay me, so I would be less mind­ing that, for the pay­ing. I am not at all about mind­ing paying.

But in gen­eral these are books that are for the bib­li­o­graphic padding of the con­trib­u­tors, not to be read, not even to be ref­er­enced phys­i­cally, but merely to be socked away on some shelf in a base­ment High Den­sity Stor­age, and to see the light of day only when some idiot (like me) gets a dose of undi­rected curiosity.

So: Expen­sive aca­d­e­mic press van­ity doorstops = dumb. But: Authors in said books = some­times very inter­est­ing. To kill π birds with one stone, I’ll call out some of the arti­cles and chap­ters and equa­tions that catch my eye, and briefly dis­cuss them. And offer links to free preprints online, as available.

I’m start­ing with the fat­test. Clear that space off quick.

[I’ll add the tele­graphic notices as I have time today. These books are due, after all.]

AI 2004: Advances in Arti­fi­cial Intel­li­gence is one of those catch-​​all pro­ceed­ings vol­umes that is full of this and that. As local Spe­cial­ist in This and That, I like it. Not all. Here are some con­tri­bu­tions that at least caught my eye:

• “Crit­i­cal dam­age report­ing in intel­li­gent sen­sor net­works” by Jiaming Li, Ying Guo, and Geoff Poul­ton. [not avail­able online!?] Wrap a space­craft in a “skin” of locally-​​connected sen­sor agents. When a lit­tle meteor or a way­ward space bolt strikes it, they yell at each other. How do you arrange them so that the col­lec­tive net­work struc­ture can under­stand (and com­mu­ni­cate) the dif­fer­ence between ran­dom fail­ure, minor dam­age and crit­i­cal dam­age? Espe­cially when you don’t know where the dam­age will be, and if it will affect the cru­cial “por­tal” com­mu­ni­ca­tor agents. You evolve a pheremone-​​directed sig­nal­ing route on the fly.
• “Com­bin­ing Bayesian net­works, k near­est neigh­bours algo­rithm and attribute selec­tion for gene expres­sion data analy­sis”, B. Sierra, E. Lazkano, J. M. Martínez-​​Otzeta, and A. Asti­gar­raga. [also not online!? sheesh.] Biol­ogy used to be so sim­ple, so ele­gant, so obser­va­tional. Now it’s bur­dened with data lack­ing knowl­edge, and all those years of com­plaint that “Math is hard; let’s do biol­ogy!” have wrought a fear­some slack, being taken up by folks in other dis­ci­plines. Like these. The prob­lem here: Gene expres­sion chips (AffyMetrix and oth­ers) result in thou­sands of data points for every exper­i­ment. Each of those 2000+ num­bers is (arguably) the expres­sion level of a cer­tain RNA species in vivo. How do you take a 2000–dimen­sional time­series, and recon­struct a genetic reg­u­la­tory net­work from it? The authors’ response (roughly) is an iter­a­tive variable-​​selection and learn­ing cycle: iden­tify a small set of salient (influ­en­tial explana­tory) genes from the mess, and add them to a data­base; build naive Bayes mod­els of the data­based gene dynam­ics using the com­plete dataset, in order to iden­tify new genes to add to the mix. Iterate.

Tell me a story:

Charles Stross’s Accelerando:

The lob­sters are not the sleek, strongly super­hu­man intel­li­gences of pre sin­gu­lar­ity mythol­ogy: They’re a dim-​​witted col­lec­tive of hud­dling crus­taceans. Before their dis­car­na­tion, before they were uploaded one neu­ron at a time and injected into cyber­space, they swal­lowed their food whole, then chewed it in a chitin-​​lined stom­ach. This is lousy prepa­ra­tion for deal­ing with a world full of future-​​shocked talk­ing anthro­poids, a world where you are per­pet­u­ally assailed by self-​​modifying spam­lets that infil­trate past your fire­wall and emit a bliz­zard of cat-​​food ani­ma­tions star­ring var­i­ous allur­ingly edi­ble small ani­mals. It’s con­fus­ing enough to the cats the ads are aimed at, never mind a crusty that’s unclear on the idea of dry land. (Although the con­cept of a can opener is intu­itively obvi­ous to an uploaded Pan­ulirus.)

(Via The author.)

…which I have at one time or another under­taken, or which have oth­er­wise been afflicted upon me by cer­tain forces.

Intro­duc­tion

For most of the last dozen days—and many of their nights—I have been engaged in untan­gling the skeins and feed­ing out the strands of a num­ber of long-​​postponed assign­ments. All are admit­tedly of lit­tle innate impor­tance, except for the increas­ingly shrill insis­tence of cer­tain patrons who point­edly remind me of their com­ple­tion at every chance, and whose notion of tardiness—by mutual assent—trumps my notion of importance.

As is often felt in times of such a rig­or­ous atten­tion, whether it is built by choice or need, that part of my mind respon­si­ble for the struc­ture of the job has been dri­ven into that state called “flow”. In the mid­dle of a night (like this one, for I write these words at the dark­est hour, in the qui­etest part of the house), it is as if I have spent two weeks dri­ven by the pilot of some river craft, who han­dles the tiller with a sub­tle engaged surety while treach­er­ous rapids are tra­versed. In hind­sight, it is no sur­prise that the crew and passengers—at least those who do not find them­selves retch­ing over the sides—should feel relief at once again reach­ing smooth water. But they should have rested all the length of those rapids (bar­ring retch­ing), on the the min­i­mal surety that the pilot had in his hand dur­ing every moment of the trip a way to grant them a con­trolled and exhil­a­rat­ing doom. If that had been nec­es­sary. Which it was not.

It is no mat­ter. Here we are.

As a consequence—or per­haps a symptom—of this greatly increased atten­tion being paid over these last weeks by “the pilot” in the “rapids” of my recent nec­es­sary tasks, cer­tain mem­o­ries have been shaken loose, and brought to light, and held up in com­par­i­son to more recent events. These mem­o­ries are the par­tic­u­lars of some trav­els I under­took many years ago, moti­vated by a com­bi­na­tion of adven­tur­ous­ness and necessity.

There were sev­eral trips. I am on one now, as it hap­pens, and I shall come to that as well. In each case I left for dis­tant strange lands armed with a mot­ley mix of a tourist’s sen­si­bil­ity, a missionary’s zeal, a folklorist’s mis­ap­pre­hen­sions and an unwar­ranted trust in oth­ers’ tales. As is often the case when one spends long times away from home, the exotic sen­si­bil­i­ties of the natives of those lands in all cases began to chafe, and sure as dawn I even­tu­ally regret­ted my sur­round­ings, and pined to depart them.

Thus does lengthy travel end for every trav­eler, whether tourist or emi­gré, men­di­cant or mis­sion­ary. (That this char­ac­ter­is­tic dis­en­chant­ment might affects the vis­i­tors to the traveler’s own home, is of van­ish­ing con­se­quence, and never need be considered.)

But as I say, my late immer­sion in dili­gence has brought back cer­tain mem­o­ries of those times past and far away. A con­comi­tant illu­mi­na­tion has struck their facets, and ignited in me a sort of thought­ful wonder—much like one feels when the sud­den recon­sid­er­a­tion of long-​​ago con­ver­sa­tions with an ancient and inti­mate friend, which after years pass can be re-​​staged, shows that one was being made an ass of, though at the time one felt the fond­est sort of camaraderie.

One scarcely ever reads the abstract of one’s life from within the work itself. Rather it is like the well-​​known girl with the bears, had she only real­ized the tri­une pat­tern of her adven­tures when telling the tale to her own grand­chil­dren, and until then sim­ply been focused on the tem­per­a­ture of var­i­ous food­stuffs, and the con­di­tion of her rump upon sitting.

I have made copi­ous notes through the years, and today hav­ing con­sulted many of my old note­books and diaries I see that my old travel expe­ri­ences and concerns—the tem­per­a­ture of var­i­ous food­stuffs, and the con­di­tion of my rump—have all along fallen down in a well-​​partitioned pat­tern as suc­cinct as that girl’s per­sis­tent trin­ity. More­over, I have not trav­eled alone, and even today as I write these words am part-​​way down a new long road, pas­sen­ger among young folk of many sorts. When I hear snips and cuts of their tales, I see the same lines there as well.

Yet few of the tourists I have met—nor surely any of the natives of the lands I’ve visited—ever had a notion of what they had con­spired to set them­selves against, or for: with few excep­tions they know only roughly where they are, by virtue of their own unques­tioned pres­ence, but never by dis­tinc­tion with what else­where may be dif­fer­ent. Said “else­where” being hard enough to see, hav­ing been there; pity the igno­rant and untraveled.

I know I must take care. As with any uncon­firmable traveler’s tale, my own webs of inter­pre­ta­tion are just my own, and any move to pass them on must make them sound like pedantry, or the worst mean­der­ings of a fan­ta­sist. I will eschew all ped­a­goguery and pedantry, and set down the sim­plest facts and mem­o­ries uninterpreted.

My mem­o­ries of trav­els in those dis­tant lands will be thus a guide, like a pilot’s log, and per­haps will help point out some shoals and reefs along the way.

As my notes are set in order, I will fur­nish accounts of the fol­low­ing voy­ages, which have been made (by com­bi­na­tion of choice and acci­dent) in my last three decades:

• My long visit to the Keep­ers of the Unin­ter­preted Record, and what was found there (in par­tic­u­lar, the Vasty Chasm)
• Among their neigh­bors and ene­mies, the Cal­lig­ra­phers of The One True Story
• My brief con­ver­sion to the inex­plic­a­ble faith of the Jump­ing Zealots of the West­ern Val­ley, explained
• My time among the Nomadic Wázirs of Nego­tiable Virtue

As ever, my goal is to be brief, clear, and fac­tual in all things.