Monthly Archives: April 2006

What it about not being the

Posted on by
10

I wonder some days what we think we’re testing in academia. This is not merely the usual harping about giving or getting grades, nor merely another thing about whether testing instruments are valid, and nor what sorts of generalization they can inform. Not even revisiting my hare-brained ideas on multi-objective grading.

Just what it is we actually test, and what we think we’re testing.

I’ve had an important lesson slammed painfully home this week. I’m one year into a entirely new discipline (industrial engineering), after 20+ years’ comfortable success in another couple (theoretical biology, and what in hindsight was apparently “not-industrial” engineering): all my planning estimates are off by a factor of 4x or 5x. I suck at deadlines.

But I’ve sucked at deadlines, like, forever. A lot of people in the University do. Look at the academic blogs, and much of what you’ll read is complaints about “Oh, my, I have so many meetings!” or, “Can you believe my students can’t do everything I tell them to?!” I’m still convinced that the demographic who survive to become professors are exactly those who’re best at postponing things to the knife-edge of the last minute, without pushing them too often or too far over the edge into regular tardiness. And like those folks, I’ve never been very tardy. Well, as such. Sure there are some papers I’ve wished I could give another edit, some thoughts left for followup papers, some deeply abbreviated discussions. But generally I get the job done.

Because I have a strong sense of the effort required. I know the stuff: domain knowledge takes little or no time for me to dredge up. Regardless of the domain. I was, after all, employed as a Know-it-all. And along with “all”, I also know how to write. So I’ve gotten pretty good through the years at weighing these strengths against the relative complexity of the tasks I’m assigned. Yeah sure, that’s a kind of laziness. This is the point where I can remind you that all the best engineers are intrinsically lazy, that it’s the driving force in the field.

But this year I suck. All my estimates are off, and all my work suffers because of it. Not because of this new “yes, really mathematics means industrial” engineering domain knowledge I need to learn; maybe it takes me twice the time to dredge up some long-ignored fact about linear algebra or matrix multiplication, but what’s an extra ten minutes? No, what’s off is my ability to communicate.

I spend all my time editing. I spend extra time,not my own, editing. And re-editing. Not just “Did that sentence make sense?” or “Do I understand that word?” but also “If I use this commonplace term, will the reader know what I’m talking about?” or “To me this is the first obvious step. If I were them, would it be?”

The words make sense, but maybe not to the reader: my use of “fitness landscape” and “autocorrelation structure” come to mind as recent examples that have proven I’m from Mars. Or I understand the word, but not in the same way as the reader: “complex systems” and “data mining” come to mind. Obvious rules of thumb and tools I always use, unthinkingly, in every case—like random sampling the search set of an optimization problem, or saving a record of every solution tested so I can watch progress of an ongoing search…. well, I may as well have invoked Marx or done a little interpretive dance. And at the same time, the consumers of my products are watching for certain significant passwords, and they’re just not coming. Because I’m not seeing the cues, the raised eyebrows, the head tilted, the muttered “…and—?”

So everything I do needs to be edited. Four times, five times. I’ve learned the hard way that it may as well be blank, as handed in uninterpretable. Indeed, it’s because I’ve so deeply misunderestimated the amount of social editing required, that yesterday I was forced to turn in a paper—one that I was originally very proud of—mostly blank.

Crippled. What am I going to do, facing a strict deadline and already having spent two weeks and 30 hours straight working on it, leave a skeleton of headers for all the beautiful things I have in my head but left myself no time to communicate? Even though I’d easily pass them along in a face-to-face conversation, or writing to an old colleague? All it would take is a few moments and a few choice turns of familiar phrase. Which I can’t use; when I do use them, when I write a complete and cogent account of some interesting things I’ve done, still the cues are not there, and what I get is another “…and–?”

So my final report for this class was a piece of embarrassing crap. If I were given it, I’d throw it away half-way through. But then, I didn’t even get halfway through writing it; who knows, maybe that could play out in my favor.

As I heard Dr. Dan Gottlieb say on the radio, there’s nothing more painful than seeing yourself helpless to aid somebody you love. I know all about that. It’s also true, though to a lesser extent, about something you love doing.

Because, I suppose, the stuff is you.

Which-all sends me down the dispiriting spiral, and starts me thinking about the nature of graduate pedagogy.

When a student is being tested, to succeed they need to (1) recognize the explicit and tacit expectations of the questioner, (2) obtain (reconstruct or recall) the domain knowledge and raw facts necessary to answer the explicit question, (3) construct a plan for a cogent response, and then (4) execute and convey that response with the appropriate formalism and idioms. This is true in education from the elementary level to the professorial job talk: Somebody poses a challenge to you, and your response will be judged not merely on its factual content but also on the degree to which it communicates your membership in a joint community of practice.

In my new domain with its weird idioms and cultural assumptions, my first response to hearing many pedagogic utterances is typically a mix of

  1. “Oh, cool, I’ve never thought of that!”
  2. “Uh huh, sure. Get to the point.”
  3. “Why would anybody in their right minds consider that reasonable?”

Thinking back to my days as an instructor, and talking after the fact with my professors here, surely they have the same set of responses to my (and other students’) work product. What differs from case to case is the proportion of the time spent in the first and last ones. We’re all brimming over with domain knowledge; what differentiates us is disciplinary framework, culture, confidence.

On the subject of confidence, briefly: it gets far too much respect in academia. Unlike knowledge or experience, confidence has a dangerous Goldilocks-curve type of nonlinearity. Neither too little nor too much is a good thing. Treat confidence warily.

The goal of higher-level education is to build confidence in the instructor that the student is spending increasingly more of the time in the second phase of “Uh huh, sure. Get to the point,” and less in “Oh, cool, I’ve never thought of that!” The last one is always undesirable, both in the student and in the instructor, except in the rare occasions when somebody’s outside their discipline.

Indeed, I suspect the proper way of thinking of a discipline is like a biological species, which is differentiated by reproductive barriers. In academia, it’s not about the fucking, but about thinking the other people are fools. If they’re time-wasting fools, they’re in another discipline. (And if they’re in another discipline….)

Yet I find I often say that bit about, “Why would anybody in their right minds consider that reasonable?” My instructors report they say it a lot about me. Not surprisingly, this leads to increased pressure in testing situations.

The point being: We often seem to forget how important issues of pragmatics and culture are in the pedagogic cycle. Instructors know the damned answer. The older the student is, the more confidence the instructor should have that the student also knows the damned answer. What an advancedinstructor should be doing is looking for cues that somewhere in that stranger’s head is a cultural framework and body of knowledge that together will suffice in the future. Maybe (for some weird reason we shouldn’t dwell upon) that graduate student won’t end up being a professor… I suppose… but in the meantime, a mentor should be seeing numerous cues that the student will soon be able to talk the talk, even outside this protected, nurturing creche that is graduate school.

So, two objectives: knowledge and communication.

I’m thinking back to years spent grading intro biology stuff—and for that matter some of the manuscripts I’m sent for peer review—and reflecting on the “memory dump” pattern that we all know and love. The canonical “memory dump” appears on the face of it as a blind listing of facts, and we’re tempted to see it as a clear demonstration of failure to synthesize knowledge. I wonder, though, if the author of a “memory dump” might be feeling challenged in the communicative aspects of the process. In conversations, where they can read cues and adjust the flow of talk on the fly, they never get a chance to spew random facts. [Well, most don't.] They come over as “smarter”. Thus it is confusing when we encounter a stupid-sounding list, an essay that dances around the subject and never gets to the point.

When we test knowledge, we are also testing acculturation. When we embark on an assigned task, to succeed we need not merely know, but demonstrate the knowledge, and for that matter show we have the ability to do so with facility.

And when we are given a test, an essay, a report, a manuscript to review, and it’s full of gaps and holes, what then should we infer?

What do we test, exactly?

You sick people

Posted on by
Reply
  1. My Mom has sleep apnea. I do too. My Dad did, too, worse than anybody I’ve ever heard. A lot of people do, it seems.

    The University Health Services have a fancy clinic now, just down the street from our house, where they will fit you with a fancy CPAP machine that you will wear every night for the rest of your life and it will possibly make you better, so your risk of heart disease is lower because you don’t wake up so much in the night.

    The sleep apnea thing is a huge growth industry, clearly. It’s a simply-described condition with a broad range of possibly serious detrimental medical effects, and it’s often treatable. Made for windfall profits: Soon it will be clear that you’re being selfish and hogging medical effort if you don’t get treated. Think of the savings that could be passed on to treat real diseases, if only we mouth-breathers didn’t clog up the healthcare profession with our trivial problems.

  2. An over-produced glossy six-color flyer shoved in every grad student’s mailbox (what? maybe 5000 of them?) the other day pointed out that over 30% of graduate students seek help for depression or stress-related syndromes. That makes sense to me; seems a little scant, frankly.

    When I was a Biology grad student at U Penn, there came a point where my cohort suddenly looked across the lunchtime bitch-table at one another, and something in folks’ eyes made me call out, “Anybody not taking antidepressants here, now, raise your hand.” And we all sat there raising our eyebrows at each other. Not our hands.

    I’m not depressed but merely stressed these days. I find I don’t get depressed much any more, perhaps because now I am old and like many old people I have much more rational first-hand experience of how much worse it can be. The inimical evil of the world, the relentless march towards the grave, &c &c. Sure there’s some stress, but… stress, well, that’s just an extra dose of cortisol and some insomnia. Everybody knows older people sleep less.

    At any rate, I suspect the rate of pharmaceutically treated depression among the populace is relatively high. Every week get between three and five emails announcing “Take Time Off From Your Ph.D.!” and “Winding Down the Stress Levels” and “No, Really, Delayed Gratification is On Its Way To You Someday Real Soon Now, We Promise” meetings from the grad school HQ.

    I am led to conclude that smart people probably get depressed more than dumb ones. Which makes a kind of sense, in hindsight. Right? Think of the savings that would result if we all sought treatment, and the accompanying productivity gains if smarties just stopped being depressed and stressed, stopped malingering. It would foster a new era of… well, something.

    Medical breakthroughs, surely. Because smart people, they helped make medicine the revolutionary necessity it is today.

  3. Obesity is rampant in America (including Canada and some US Protectorates, I ‘spect). We’re all gonna get vascular disease and diabetes. You’re a bad person if you don’t diligently exercise, eat reasonable amounts of bran and pomegranate and spelt wheat, and end up using valuable resources that could better be applied to help people who need it.

    I am led to believe it’s a worse social sin for smart people to be overweight. Which makes a kind of sense. Because poor people are fat, and smart people aren’t poor. Right? (I’m still working on this one). But think of the savings if we all just took three hours a day and walked around in bright white shoes and officewear, and ate salad at the cafeteria—savings that would benefit the really sick.

    That’s the feeling I got from my recent Health Maintenance Exam, at least. “When are you going to get in shape?” Because, my young man, we’re not going to wait for you much longer.

  4. I am told that my generation (“13ers”, also known as Early-onset GenX Syndrome Sufferers) and those afterwards are prone to getting allergies and asthma and autoimmune diseases because we didn’t eat enough dirt when we were kids, and now it’s too late to eat dirt because our immune systems have learned incorrectly that (a) many important molecules in our bodies look something like the surfaces of Bad Microbes, and (b) the surfaces of many ubiquitous molecules on the surface of Regular Household Stuff (dogs, books, flowers) also bear a striking similarity to the surfaces of Bad Microbes, so (c) our noses are always running and we complain frequently of serious headaches, and therefore (d) we’ll need to take prophylactic antihistamines for the rest of our lives.

    I bet the smart kids are more prone to allergies. Think of the stereotype: asthmatic, myopic dweebs. No dirt-eating at all as a kid = sickly and sniffly. Stereotypes, as it happens, don’t actually lie that often. Right? So as a parent you’re shirking if you don’t take your kids outside now and then and force-feed them some dirt.

    And then spray them quickly with antibacterial soap.

    Think of the savings &c &c.

  5. And there’s statins. Wow. Biggest-selling pharmaceuticals in the world. We’re all much better for taking them. Think of the savings in the medical community that can be passed on to treat other, more serious pathological conditions. ‘Nuff said.
  6. New-generation NSAIDs. Aspirin is dangerous; it might give you tummy troubles. Never take aspirin! We should all switch to newer NSAIDs, that are much better because they’re newer. Savings. Passed on. Think!

Am I saying I think we’re a bunch of whiners? No. These are all real problems, especially the just plain constant pain one—my Mom suffers from terrible osteoarthritis, for instance, and NSAIDs give her the ability to walk around a bit. Depression sucks, is dangerous, is more terrible in a way than all the others, because you watch yourself being sick and unable to do anything about it. Heart disease, diabetes, sleeplessness, are all actual Bad Things. I’ve watched several people die badly from diabetes and its complications.

Nobody should suffer any of these problems or risks. Nobody anywhere. We are not—the sick ones—whiners.

Alas, we all do suffer these problems. If not these, then some other ones. Don’t get me started on cancer.

An underreported statistic I have not read lately, presented here as a reminder: nearly 100% of patients seeking medical treatment eventually die.

How can we stomach that? Why doesn’t anybody do something about it?!

What I’m prodding with my Short Stick here is something about our collective attitudes. Culture. Tacit assumptions, implicit understandings. I don’t know whose in particular. Doctors? Sure. They are, in general, ignorant overworked habitualists who have so little time to listen to details or think about consequences that they often as not end up mistreating their patients. That is, prescribing anything just to get the patients off their backs.

And the patients, well, they’re stuffed so full of risk aversion that they immediately seek advice and treatment and solace for any complaint, including (as noted in (2) above) over-complaining and Generalized Life Difficulty. They want surety, and feel betrayed when something goes wrong that isn’t immediately replaced or refunded under their implicit Warranty Service Agreement.

And then it hits me: This is how religions get started.

Look at it. Think about it.

We have the overwhelming desire for simple, effective solutions to ubiquitous problems. We have a harried special class, who by dint of historical contingency and a few early successes have become the class explicitly charged with providing solutions for those problems. But inevitably they’re unable to cope with the world’s diversity, entropy’s insidious creativity. So these people, they focus on low-hanging fruit, and increasingly point out how much more work they could get done if people just went through a few simple ritual moves: statins, CPAP machines, salad not fat, just plain aspirin is now Bad For You. And the observant populace, and the majority of the specialist priesthood, don’t have the time or inclination to actually think about what happens in the end. What happens to every solace-seeker, no matter what they try. They shunt old and sick people off to special temples, for preparatory purification ceremonies involving respiratory support.

Those sick folks, they’ve gone to A Better Place. Some lucky few come back, briefly, and tell thankful tales of the wonders.

But they always go back, in the end. Back to the place we don’t see them.

Religion is not a symptom of irrationality. Irrationality takes hold of all of us, no matter what. We are not rational beings. Indeed, the modern scientific atheist’s tendency to decry irrational religiosity so flies in the face of facts as to be reduced to an artistic act of Grand Irony. Prescriptivist Philosophy as Performance Art.

Religion is a robust cultural system, a cultural attractor that sucks in societies wholesale. Look at how far it’s progressed, in just the last 100 years or so. In the case of Medicine, I mean.

So many converts. So many ads in all the magazines. “Ask Your Doctor.”

It thrives because any priesthood inevitably fails when asked to work on the Really Hard Stuff. So it adapts by playing to our inherent pain-avoidance, trotting out the old favorite solutions of sequestration and denunciation of problematic facts, elision of complexity, denial of diversity.

Religion thrives because we are built to classify, but made to over-generalize.

Of course, it’s more complicated than that. But we don’t have time to go into that now.