I had a dream that I came home one day and found that my dad had bought a massive old server computer that was VAX-compatible. By VAX-compatible, I mean that it was built with the intention of running OpenVMS on it, because in my dream it was pointed out that it wasn’t fully VAX-compatible, and couldn’t run old VAX software. In fact, it came with the commercial version of SuSE Enterprise. But the machine must’ve had some weird hardware, because it was also pointed out in my dream that SuSE Enterprise was the only Linux that could run on it. Proprietary hardware drivers? Who knows. It was about the size of a college dorm refrigerator.
Anyway, my point is: Why did my dad buy a mainframe minicomputer in my dream? He explained it to me in my dream, but I forget now. Something about orchestrating a big supercomputing environment. He had called me for help.
Immediately, I pointed out that his VAX machine was very old and that any cheaper, smaller x86(_64) machine would do the job much better. But he was irritated that he couldn’t set the thing up himself and just wanted it to be done, even if this machine wasn’t optimal. The guy at Meijer said that this VAX was the thing he needed. I was like “You bought this mainframe computer at Meijer?” and he was like “No, I came in there looking for this stuff, but they didn’t have it. I told the guy in the computer department what I wanted to do, and he said that he’d heard that VAX machines were how people did that, and told me where to get one.” I was like “You went to Meijer for advice on how to set up a supercomputing cluster?”
I pointed out that in order to set up the supercomputing architecture he’d described, we’d need to buy a bunch of other computers (otherwise it wouldn’t be a cluster). He was disappointed – he thought he had everything he needed. I tried again to convince him to replace the VAX while we were at it, but he wouldn’t have it. The guy at Meijer told him that this was what he needed. Couldn’t I make it work? I made the mistake of saying that technically, I could make it work. That locked me in.
So I started thinking out loud about the physical and logical layout of the cluster. “Hm. We’ll have to move this VAX machine over there so it can serve as the director. Then I’ll have to install some of the new boxes here to be the cache.” My mom pointed out that it was very hard to move the VAX, and couldn’t it serve as the director AND the cache? (My dad had insisted that the VAX be the director, and I thought that was a fine use for it, since its load would be small compared with the actual working machines). I thought my mom’s advice was sensible. The same machine could serve as the director and the cache, and the VAX was already set up to be the first machine attached to the network – being the cache would work well for it.
After I’d drawn up the plans for the physical layout, I started writing the software that we would use on the system. I started thinking “Man, it seems like people try to do this sort of thing a lot. Shouldn’t there be some software that does this stuff out of the box so I don’t have to write so much? Oh well, I’m almost done…”
Well, when I woke up, I realized that the software I was working on was basically Apache Hadoop. It would’ve been a lot easier to just use that. Also, I only thought I was almost done because I hadn’t yet run into all the complexity.
Computer programming, people, amirite?