DadHacker

    Somethingary          AlmostThere          JustOneMoreMonth
 Su Mo Tu We Th Sa Fr  Su Mo Tu We Th Sa Fr  Su Mo Tu We Th Sa Fr
     1  2  3  4  5  6         1  2  3  4  5               1  2  3
  7  8  9 10 11 12 13   6  7  8  9 10 11 12   4  5  6  7  8  9 10
 14 15 16 17 18 19 20  13 14 15 16 17 18 19  11 12 13 14 15 16 17
 21 22 23 24 25 26 27  20 21 22 23 24 25 26  18 19 20 21 22 23 24
 28 29                 27 28 29 30           25 26 27 28 29 30

     HellWeek                Shitstorm
 
 Mo Mo Mo Mo Mo Mo Mo   Sa Su Sa Su Sa Su Su
     1  2  3  4  5  6                   X  X
  7  7  8  8  8  9  9    X  X  X  X  X  X  X
 10 10  9  9 10 10 11    X  X  X  X  X  X  X
 12 11 11 11 10  3  4    X  X  X  X  X  X  X
  5  6  7  8  9  9 12    X  X  X  X
 13 14 13 13 13 13 13
 13 13 13 13 13 13 13
 13 13 13 13 13 13 13        Patches
 13 13 13 13 13 13 13
 13 13 13 13 13 13 13    Su Su Su Su Su Su Su
 13 13 13 13 13 13 13                 !  @  ?
 13 14  0 15 15 15 15     /  +  *  #  ]  }  ^
 16 16 16 16 16 17 18     "  ;  :  . ?: &= ||
 19 20 21 22 23 24 25    -> ~! () += != "  ''
 26 27 28 29 30 31 32    %%  `  , // /* */
 33 34 35 36 37  0 38
 39 40 41 42 43 44 45
 46 47 48 49 31

Once upon a time I thought that makefiles were a cool idea. Okay, this was the early 80s, rocks were still young, and I didn’t have a version of make on any of the platforms I was using, so I wrote one. My own version of make wasn’t very good, but it was simple, did what I needed at the time, and I gave it away for free (you can probably still find it on the net. One of the reasons that Richard Stallman doesn’t like me much is that it’s close to my total contribution to Free Software. Trust me, it’s not worth the hunt).

Fast-forward a few decades and I’m wrasslin’ with makefiles large enough to have detectable gravitational pull, with dizzying levels of nested includes, wrapper programs bolting together metric buttloads of definitions from auxiliary files that were first cut in clay tablets Hammurabi’s scribes, macro systems from hell that wrap back on themselves through higher dimensions to form legal XML, and default rules that actually reach back in time and break builds that have already succeeded (which sure explains a lot, doesn’t it?).

And yet, with these hundreds of thousands of lines of intricate and fragile declarations accreted over uncountable hardscrabble engineer-years, with with the multi-hour-turnaround time, the only friend at my back is an ECHO statement that lets me go back and stick tracing statements where I think the problem might have been. I don’t need ECHO, I need a time machine.

What I’d actually like to have is a fapping debugger, but I suspect it’s easier to build a gizmo to tear apart and reconstruct the elementary fabric of the universe than it is to interrogate the infernal interiors of NMAKE after things have gone sour. (Yes, NMAKE. Don’t get all superior on me: I’ve used GNU make as well, and while GNU make is better, this is still like expressing a preference for a particular brand of cyanide in your coffee).

—-

A person from outside the culture of modern software engineering would respond with something like “Pull the other one,” or more likely, “Stop whining,” and that person would, sadly, be right on the money in both cases. We have no one to blame but ourselves. With all the fancy languages we employ, all the type-safety and exception safety and interface meta-languages and theorem proving, why are we jack-legging software together with rubber bands and duct tape?

Make was written in 1977, and it hasn’t fundamentally improved in decades. Instead of improvements, we got features: Powerful macro expander syntax, looping constructs, electric and non-electric variable definitions, some lame attempts at parallelism, but all of these extras just added complexity and didn’t address the everyday problem of figuring out why a build is failing two hours in.

If I were to tell someone “I’m going to design a programming language that’s going to be used by millions of programmers every day: It’s going to have tons of hidden state, no obvious control flow, obscure and terse syntax, and programs written in it are going to run for upwards of six hours before bombing with an error message like ‘File not found’ — oh, and I’m not going to write a debugger, and all the state will be hidden and completely lost when things go wrong” — I’d be strung up in the stairwell alongside the guy who invented trigraphs.

Don’t get me started on autoconf. (Someone else wrote a nice flame; there have been others). Tools like this just paper over what’s really wrong: We have too much crap and we have to build it all the time.

—-

Make is only part of the problem. Modern compilers are still rooted in the smelly primordial ooze of the paper tape era of computing.  Well, maybe magnetic tape.

Imagine I’m building a house; to achieve this, I will be nailing some boards together. Given that I am a relatively savvy and modern software engineer, what I do is:

1. Grow a tree

2. Cut it down and drag it out of the woods behind my ox

3. Extract a board, using a pit and a great big bloody saw

4. Dry the board out in a kiln

5. Cut and plane the board to proper dimensions

6. Repeat 1-5 with another tree, resulting in another board

(I’ve omitted steps involving mining iron ore, making coke, refining the ore, smelting same, making steel, and pounding out a nail)

7. Nail the stupid boards together. Oh, you wanted glue? I don’t do glue; there are good reasons for that.

… and in about forty thousand years that house is finally assembled (which is about par for how late a lot of software projects are). This is pretty much the life-cycle of a compiler: Suck in several megabytes of header files and/or precompiled headers, process a miserable handful of ten or fifteen functions and methods, spew out some object code and a fuck-ton of debugging info, then do the whole thing over again with the next set of sources. After all that’s done, you feed it to a linker. (Don’t get me started on linkers; I did a lot of work on linkers in the 80s and 90s, talk about a thankless job…).

Of course, modern build systems get rid of some of the duplication of effort here, since they will precompile headers for you and do some dependency analysis. But I dare you to change one common structure, or touch one common header file containing, say, a list of error codes. It’s time to recompile the world; see you in a few hours.

C and C++ need a module system so badly that we should pretty much stop adding features (yes, Mr. Freaky Template Thingy I’ll Never Use in a Responsible Real-World Project, I’m looking at you) and do nothing else to these languages until this is fixed. Architecturally. We need to ban #include (and no, precompiled headers are not the answer) and get a type definition and importing system that actually fucking works and that scales to tens of millions of lines of code. Once we have that, I’ll hunt down every single use of #include and #if/else/endif and club them to death.

—-

Something absolutely magical happens when you have turnaround time that is less than about five seconds. It almost doesn’t matter what language you are programming in. If you’ve got a system that gives you five seconds from source change to running code, it’s possible you’ll forget to eat and starve at the keyboard, even if you’re hacking away in assembly.

Build times sneak up on you. Pretty much every project I’ve worked on from scratch has gone from that magical “seconds” window to “minutes” (tolerable), to ten minutes (get coffee), and somehow reaches 45 minutes to an hour (go to lunch, surf the web, do email, write documentation, attend meetings, play video games). Around the two or three hour mark and you’re talking about doing SCA re-enactments in the hallways using parts from build servers as props.

Frankly I don’t see this problem being solved any time soon, at least for the kind of dead-bits “EXE” development that happens in embedded work and high-performance cores of video games or operating systems. While it may be possible for hardware to get to the point where we can JIT and message-pass ourselves to Nirvana and forget about cache line awareness and punt global optimization, weren’t we saying that ten years ago, too?

The essential core of makefiles and text-based includes were 70s-era hacks of convenience that went only so far. Speed of turnaround is a language feature, and you don’t have to be a dope-addled Smalltalk hacker to appreciate the beauty of dropping into the debugger, changing the structure of a structure, and continuing blithely along as if nothing extraordinary had happened. We’ll never be there with C (at least, a language that supports that probably doesn’t look very much like C), but it’s interesting to contemplate.

A couple years ago, Trader Joes performed one of its classic product disappearing tricks on one of our household’s staples, the TJ’s “Mojito” simmer sauce. This was one of the few sauces that did not contain stuff we were allergic to and that was not too spicy, and when we discovered it had been discontinued we bought all the remaining jars we could find. A year or so ago we used the last jar.

I’ve spent a bit of time on what I think is a faithful reconstruction of the sauce. It’s pretty simple.

2/3 cup good quality orange juice
3 T lime juice, or more, to taste
3 T rice wine vinegar
3 T olive oil
3-4 cloves garlic, diced very fine
1/3 medium onion diced very fine
1-2 t dried oregano
1-2 t ground cumin, or more, to taste
a good grinding of pepper
salt to taste
1/2 cup peas

Proportions are not terribly critical, but I think that the lime juice and cumin are important to balance. I use a food processor to chop the onion and garlic very fine indeed. Just put everything but the peas in a bowl and whisk. I usually sin by also whisking in a bit of corn starch, to help consistency.

This is great with chicken; cut a couple of chicken breasts into 1 inch cubes and marinate for a little while, then dump it all into a largish skillet and cook over medium heat, turning occasionally until done, generally less than ten minutes. Add the peas a few minutes before the chicken is fully cooked. Serve over rice.

[updated: fixed horrible spelling mistake.  I am very tired.]

“Kid,” we said to the new kid at lunch today, “We’re professional software engineers. We’ve got well over two centuries of collective experience writing software sitting at this table. We’ve been around the barn a few times. Trust us.”

The young kid nodded. It was his first day. He was still brimming with excitement, fresh from whatever they pumped into the air during orientation, and eager to get on with the job. He was too wound up to eat his sandwich. He was visibly vibrating.

“Ask us anything.”

“Uh…”

Phil said, “Can I take one of your chips?”

“Sure,” said the kid.

Phil reached over to the kid’s bag of corn crunchies.

“STOP!” I cried. Heads around us turned to look, then turned away. Engineers.

“First lesson in software engineering in the real world,” I continued, “You don’t let anyone take any of your chips. Not even symbolically.”

“Um, okay.”

“It’s a slippery slope. First, they cut your clock rate, next they’re carving away at memory, until finally they’re saying, ‘Hey, can’t you do all of this in software?’ and you’re standing bare-ass naked in a design review with something like a four-bit dishwasher controller, half a kilobyte of RAM, thirty milliamps, and a three week schedule. And they want a billion polygons a second through it because someone with a spreadsheet ran the numbers and thought it was possible, once.”

Phil and the Fat Engineer With a Beard nodded.

“And if you can’t do it, they’ll find some young naive and defenseless newbie who’ll kill himself proving them wrong.”

Light dawned in the kid’s eyes. They got wider, and he said, “So that’s why the director of hardware wants to meet with me this afternoon.”

Phil and the Fat Engineer With a Beard nodded again. Yes, we have spies.

“Now, can I take a chip?” said Phil, reaching.

“It’s just a corn chip …” started the kid.

I gave him a look.

“Fuck off.”

“Right,” said Phil.

—-

“The next lesson is called, ‘They Have Hot Pointy Things That Burnnn Usss, Yesss, and We Have Only Our Wits.’”

“Hardware engineers. Soldering irons. I know all about those.”

“No, I mean project managers with Agile.”

If a member of the Qualithian race [all-powerful, multi-dimensional critters capable of transcending space and time, and able to write programs in Visual Basic without losing their sanity] ever pops into existence by my elbow and offers me a single super power, I am prepared: Quite simply, the ability to delete obsolete versions of documents, regardless of location.

So we could end forever conversations like this:

“Did you read the spec?”

“Sure.”

“The one I sent out just before the meeting?”

“Uh . . . maybe. What version?”

“1.17b; I did a bunch of edits but I left the version the same so that the document server wouldn’t get cranky again, and you should use the time stamp instead.”

“What about the version that Figby marked-up after your changes?”

“I don’t know, did he get the bit about the automatic zothic bongo yadda-badinga processing on big-endian systems?”

“I thought that was out in version 1.15g.”

“Little endian, or he was lying.”

“He cannot lie. He just doesn’t know what the truth is. And these versions aren’t helping.”

“Well.”

“Version 1.15g, that’s like three days old. Fucking ancient.”

“And it went out to customers.”

“WHAT?!”

“I saw the pallets on the loading dock. Five thousand copies back from the printers, all bound up and shipped off to the four corners of the civilized world. Also, Washington D.C.”

—-

See? With a single wave of my super-power wand, all this confusion would vanish.

“Um, so what is the latest . . .”

(SWOOP! ZAPF!)

“Oh, thank you, Version Man! You’ve saved our meeting!”

“No thanks necessary, just keeping entropy at bay for another day.”

(SWOOP!)

I thought that the Europeans were self-panicking and over reacting to the volcanic eruption of <unpronounceable name> in Iceland, but the ever-practical Fins have flown in the dust, and have provided photos of what it did to their engines:

http://www.flightglobal.com/articles/2010/04/16/340727/pictures-finnish-f-18-engine-check-reveals-effects-of-volcanic.html

I trust they are being honest about any warranties they have voided.

A few books that I’ve enjoyed recently.

Sandman Slim. (Richard Kadrey). It’s been years since I’ve this much fun with a “bad-ass comes back from the dead to wreak retribution” novel. It’s a little predictable and somewhat repepepetititititiousify, but you’ll enjoy it.

The Gone-Away World. (Nick Harkaway). Don’t let the stupid fuzzy dust jacket on this book confuse you; there is madness and power in these pages. The old world is gone, replaced by terrible, mysterious menaces resulting from man’s quest for better and better weapons. There is humor, there is beauty, there are terrible secrets, there is love and war, and there are ninjas in a desparate battle with midgets.

K. J. Parker’s Engineer trilogy: Evil for Evil, Devices and Desires and The Escapement. From the opening chapter of E4E I was hooked; an engineer (quasi-medieval technology, at the level of catapults and spring-driven spear-throwers and so forth) is betrayed, condemned to death, and escapes his home city-state to take refuge in the “great unwashed” cities outside. How he gets his revenge takes up three books, and it’s worth it.

Wireless. (Charles Stross). Collects many of his recent short stories, including Missile Gap, one of my favorites.

Hacker’s Delight. The first chapter — everything you’ll ever need to know about bit twiddling — is worth the price of admission. Beyond that, just about every simple algorithm I’ve ever heard of for doing efficient low-level, wacky stuff is covered here. BCD arithmetic on modern 32-bit architectures? Covered. Population-counts? It’s all here, and tons more.

Programming Erlang: Software for a Concurrent World. (Joe Armstrong). Erlang is a refreshing programming language. While reading this book I kept having flashbacks to reading the original The C Programming Language (which I think is one of the best programming language books around). Erlang is crisp, expressive, and looks like lots of fun (wish I had time to do a project in it).

The Eerie Silence (Paul Davies). Good discussion of the Fermi Paradox (basically, where are all the ETs?).  More than just a re-hash of the tired old stuff (Drake equation, etc.), he’s got some interesting points about what the likelihood of and might mean to make contact with a million-year-old technological civilization.

Solaris Internals (Richard McDougall and Jim Mauro). Yet another goopy “here’s the innards of a modern(ish) real-world OS.” Reminds me of the Vax/VMS internals docs and the various books on the 4.2bsd kernel by McKusick. I don’t like Solaris much, but the details are fun.  (Heh, speaking of million-year-old technologies :-) … )

#pragma expiration(date)

Refuse to compile the following code after the specified date.

#pragma moran(email address)

Emails the specified address every time there is a compilation error in any code that follows.

#pragma tricky

Tells the optimizer to go easy on the next bit.

#pragma borken

Tells the compiler to do its worst; it hardly matters, the code is busted anyway.

#pragma stfu

Tells the compiler to shut up and compile. That’s your job, and that’s it. Your job is not to announce banners about compiler versions, or display company logos in blinking text, or declare with amazement (and a full line of “!!!!!!!!!” marks) that the compilation has finished with 0 errors, 0 warnings, 0 flatulent events, and 0 animals harmed.

#pragma blame(username)

If there are errors in the following code, arrange with the source code control system to blame it on some other checkin made by the specified user.

#pragma whew

Dear reader, take a break, you have earned it.

#pragma notme

This design was someone else’s idea, not mine. Don’t blame me.

#pragma myway

If I were to do it, I’d write it this way. I feel like a total moron doing it that other way, but nobody listens and nobody appears to care. (Implementation-dependent side effect: The code is enabled when the project is cancelled).

Somewhere, a voice is calling. You’re neck deep in iterators, destructors are going off all around you, and if the bloody river full of pirhana and man-eating allocators doesn’t get you, the bunker of locally scoped nuns armed with automatic variables on the far bank will ruin your day just as soon as your schnozz clears the mine field. These runtime missions always suck. “A walk in the park, Sarge. Sure.” A star shell goes off and illuminates the whole scene, and when your vision un-dazzles you get your first clear look at the far bank, which is simply crawling with the ugliest damned set of indented curly braces you’ve ever …

That voice again, faintly: “You ready for lunch?”

The haze clears. Things looked bleak there for a second. You blink. The source code in front of you still looks bleak. Lunch sounds like a great idea.

You tell a story at lunch. The one advantage of being an old fart is that you can tell some great stories. You don’t even have to make the crazy shit up; that would be harder.

—-

I’ve been on a number of projects where it’s gone like this:

“Okay folks, the ROM is full. Put down your keyboards, it’s time to stop typing, we’re all done now.”

Team: “Yay! Ship it!”

And so final ROMs are burnt and shipped to Asia for manufacturing. The ship party is glorious; we finished three months early (the ROM was full, yeah) and customers are very happy.

Urm, well no, not really. What usually happens is:

“Oh golly gosh jiminy jeepers [not really those exact words], we’re out of ROM space. Whatever in [someone's or something's name taken in vain] shall we do?”

“Roight-O. Bring out [music: duh-duh-duhhnnn!] . . . the Code Squasher!”

“Not [duh-duh-duhhnnn!] . . . the Code Squasher!”

“Yes, I’m afraid so. If we don’t, things will only get worse.”

The [duh-duh-duhhhnnnn!] Code Squasher is about three stories tall, with a hopper on top where you stuff in the programmers, and a little tray at the bottom where you collect the . . . final product. It uses three phase power. You lure the programmers by dangling a Mac Powerbook over the –

No, not really (thank goodness). What usually happens is:

“Out of ROM space again?”

“How did you tell?”

“You’re carrying that monster copy of Compression Algorithms around, just like last month. Look, that’s not going to save us this time.”

“Another five percent and we might be able to –”

“Nope. We need to [duh-duh-duhhnnn!] … delete code.”

[look of horror] “Delete code? Doesn’t that mean we have to get rid of features?” He’s expecting Marketing Ninjas to leap down from a skylight, scream something in MadisonAvenueSpeak and whack his head off, I think.

“Look, there are probably sixteen implementations of linked lists in the system, half a dozen hash table classs, two XML parsers and any number of ways to zero a miserable block of bytes, half of them ‘optimized’. Even if you only find half of the places where people have re-written strcpy and printf, you’ll find enough space to make the next release.”

“Okayyyy. But that’s still not going to help. Maybe we need overlays, or a p-code engine, or some kind of compressed paging manager that….”

“The real disease is that you’re practically paying engineers by the line of code. Re-examine all the mini-parsers, all the places where a routine is just re-shuffling its parameters and passing the buck to another useless layer of abstraction, all the code that is in there for a rainy day that you’re just not ever going to use. Remove that junk.”

“Then what?”

“Toss most of it away and start over.”

[heart attack]

Systems age, just like people do, and it takes exercise of diligence to keep the fat at bay.  I don’t have a silver bullet, but good organizations have people who are nosy about every single checkin.  Be one of those people; you’ll learn a lot, and you’ll catch bugs, too.

A degenerate old man lurches into a Home Despot.  His coat is thirty years out of fashion and threadbare where his hump rises up.  His mouth moves silently and his expression changes from moment to moment, as if he’s on the losing side of an internal argument.  One eye orbits, now looking at the ceiling, now at the floor, while the other eye ogles every visible female.  His shoes have the thick soles of medically prescribed special shoes, but it’s possible that he’s stolen them, maybe from a morgue; he looks like the kind of guy who might running an errand for his master, searching for a cheap used brain.

He selects a cart of sufficient squeakiness and limps to the Outdoors department.  He finds a store employee (a minor miracle, with all the downsizing), raps his steel-tip cane rudely on a propane cylinder, and asks in a voice thick with phlegm and Old World cobblestones:

“How much for the leetle grill?”

“What?”

“Zee grill.  The Weber there.  How much for her?”

“Uh, list is $599.”

“Gut.  What accessories do you recommend?”

“A propane tank, of course, and this adapter, and perhaps some cleaning supplies…”

“All gut.  Okay.  I take them.  Are you on commission?”

“No sir, but thank you for asking.”

“Something for your trouble, then.”  $20 appears in his hand.

The man limps off to the automated checkout line, where he swears at the touchscreen in a powerful and evil language that no one understands, but that everyone in earshot fully comprehends.  He finally pays and gives the kiosk a goodbye kick.  As he leaves through the security gate, the self-checkout station flashes bright panicky error messages, utters a terrified squeal and crashes hard.  The doors close with an audible clonk.  Even people in the far recesses of the store look at each other and breath a sigh of relief, and they don’t know why.

My question: What is that guy barbecuing?