HENNESSY: Well, I have a different point of view. What I would point out is that, first of all, the only remaining CISC architecture per se, is the Intel X86 architecture. All the rest have sort of vaporized. That alone says something. I think that market presence is primarily what’s kept the X86 alive.

But even if you look at the modern implementations – Pentium Pro, Pentium II, Pentium III implementation of the X86 - the way they work is they take the X86 instruction set and translate it into micro-instructions, to these things they call micro-ops, which are basically RISC instructions - they do load, store, add, things like that – and then they pipeline that structure. So they don’t pipeline X86 instructions, they actually pipeline and detect all the interlocks among these things.

This actually gets back to an earlier idea we had. When we did MIPS, one of the reasons that we said "Well, we shouldn’t have micro-coding in there" is that I believed that microcode did work in runtime that could be done in compile time. What it really does is interpret the instruction set from one level to another level: to this micro level. Well, that’s stupid, because it’s stupid to do work in run time that you can do in compile time. So that’s really the way a modern X86 works.

Now what has happened, there’s no doubt about it, is that the RISC machines have become extremely complicated in their implementations. To some extent, yeah, there have been instrucions added and things like that. But it was never about instruction count; it was about designing the instruction set so that a high performance implementation was made easier. What’s happened is really interesting. We’ve designed these out-of-order, dynamically-speculative machines. Well, I think if you told people 10, 12, or 13 years ago, that this was the kind of machines they were going to be designing, they would have said, "You’re crazy."

In fact, the only machine that had ever been designed with that philosophy was the 360/91, chipped six processors – never fully debugged. They never got all of the bugs of the hardware, because at that time (this was in the 1970’s) the machine was so complicated for the state of the art technology they couldn’t get it right. I remember Mike Flynn telling me, "You know, nobody will ever build a machine like that again" because it was too complicated.

Well, several things happened. The RISC machines, at one level, made it easier to think about design in a piece of hardware that was that complicated because things are a lot simpler if the instructions only take one cycle and they only do simple formats, you can actually contemplate something that is logically that complicated and get it right. That’s the ongoing issue: the simplicity of the instruction set, or the care in the design of the instruction set makes it easier to do the implementation. And I think that’s an ongoing advantage that translates into how big the design team needs to be, how long it takes to do the design.

Whether or not that translates into a market advantage depends on how big the market is – how many people are buying your processor. The fact is that Intel can afford to spend ten times as much on a chip design than anybody else because they have ten times the volume. That’s certainly made it an interesting, intereseting game, you know? What will happen with the whole I864, and Merced? I don’t think that’s necessarily the right design plan, but I think it’s a smart idea for Intel to try get a new architecture. They took the X86 which was designed to be a 16-bit machine and they cobbled it to be a 32 bit machine. It’s another hack to make it be a 64-bit machine which is where they need to move to.

So there are good reasons for them to think about a new architecture. Whether or not this is the right choice, well, that’s another debate. It’s certainly a…let’s say they didn’t hesitate to add things to this new architecture. They added a lot of stuff. It’s a pretty big architecture with a lot of…and my view has always been: start small. Because if you have an architecture that’s going to live 15 or 20 years of the commercial MIPS architecture now 16 years old, you’re going to end up adding things along the way because new markets are going to develop. Graphics is going to become important – you’re going to want to put some support for graphics in there. Or security’s gong to become important…

So start small so that when you have to add those things you don’t end up with something that looks completely like a kitchen sink. And that’s hard to do. When you start out with an architecture that tries to do everything in day one, I guarantee that you won’t get it right and you’ll end up with something that looks terrible, especially after it evolves after time.

US: What kinds of bottlenecks are there today in processor technology?

HENNESSY: That’s a good question. I think there are a variety of things. If you want to fetch and dispatch multiple instructions per clock, then the first bottleneck is just how many instructions you have to keep fetching if you expect to issue four a clock, especially if you account for instruction cache misses and branch miss predictions and everything else. The biggest problem right now is that….there are two problems: one is that the peak to sustained ratio for these machines that are all trying to do four to six instructions per clock are terrible.

Now they’ve got better in this second generation of machines then they were in the first generation of machines – the engineers just got smarter. The first time you design this speculative, out of order machine you get a lot of performance bugs just because you didn’t understand as you did the design where the bottlenecks were. And what happens in any of these designs is you’re going along, you’re simulating it, you’re designing it, you figure "I can’t quite yet," the high-level architecture says do it this way, but it ain’t gonna work. I end up doing it this way, and then you end up introducing bottlenecking with the machine and if you could afford to do it all over again or extend the design cycle by another six months, then you can fix it. But by that time you’re already late, the schedule’s already behind, and you’re racing to try and get it done.

So, this peak-to-sustained ratio is a real problem and the way to think about this is – well, you watch what happens. They gobble a bunch of instructions in the front end (and occasionaly they get cache miss), so instead of their fetching (let’s say they’re fetching four instructions per clock and trying to dispatch four), you know occasionaly they get cache miss, so they end up with an average of, let’s say, three instructions per clock actually fetched. Then sometimes they get branch misdirections, so they lose a little more. And then the instructions aren’t aligned properly in someway, or there are three loads in a row (and they can’t dispatch more than two loads in a row), so a little more gets out. Then you get a, say, secondary cache miss, and all of the reservation stations and all of the buffers get filled up with things that can’t quite issue until that miss gets back, so you lose a little more, lose a little more. Start at one end: four instructions per clock, or six instructions per clock, and you end up with one and a half or two, just a kind of <chopping noise> along the way.

So trying to understand that, optimize that process, figure out what the right balance between compiler technology and automated dynamic hardware techniques is. We don’t yet have a graceful combination of those two. We have a set of dynamic techniques; we have a set of static techniques. Each of them works okay on their own, but each has something to contribute that the other side isn’t quite taking advantage of. How to bring those two together is, I think, a key question.

The other bottleneck is that as you try to boost these issue rates, and go from four to six to eight to sixteen eventually, it’s just incredibly complex. The amount of hardware you have to throw in, the amount of simultaneous decisions that you have to make in a single clock, goes up very rapidly. In these machines in a single clock cycle, you have to determine all of the interdepencies among those sets of instructions. So if each instructions has a result operand and each instruction reads two operands, you’ve got to do a comparison, in order, between all of the instructions in the issue packet to look for those, and then adjust the renaming and reorder buffer numbers so that you can issue that whole packed of instructions.

And that has to be done in, basically, a single clock cycle, effectively. Because that’s your issue bottleneck. So there was a thing Mike Flynn actually wrote a paper about this many years ago where he called it the Flynn Bottleneck where he sort of said said if you want to execute more instructions you can’t execute more than – well, at that time, he said more than one per clock – because you have to look at each instruction, get it through that little hole. Well, now that number is up to four or so, but the problem is each time you raise it, there’s a piece of that that’s quadratic, where the number of things you have to look at is the square of the number you want to raise it by. And that’s the real bottleneck that makes it difficult.

So, add to that the fact that you get diminishing returns. Let’s suppose now you’re at four issues per clock, let’s say you sustain on things that don’t have a lot of cache misses, you might sustain one and a half to two instructions per clock – so you’re slight less than 50%. When you go to eight, you’re not going to get four. And with the difference, you’ll probably be a little lower than four – maybe three. And now, the absolute difference, though, in terms of hardware and number of transistors that are sitting there doing nothing, has gone way up.

So these things are all becoming challenges. I gave this analogy in this talk I gave last year to the Federated Computer Conference. I said it’s like trying to push a giant rock up a hill. And we’ve been pushing this rock up the fill. First it was very flat, just putting the rock along flatland. And eventually you get near the top of that hill and that rock, with so much gravity trying to push back on you, occasionally the rock actually rolls backwards, squishing the design team. And you get these after spending three years and who knows, maybe fifty engineers a year, to conclude you can’t finish – you’ve missed the window. And all of the sudden you cancel a project on which at that point you’ve probably spent thirty million, forty million dollars on. And then you just <"poof" sound>. Well that’s really bad. So that should be avoided.

So I think we need to think long-term, my view is that we need to get a better handle on parallism at a higher level, we need to figure how to get the software systems there, and that’s a really long term transition. But if we don’t start kind of moving in that direction, that way all of the rocks will get too big to push up the hill and it won’t work anymore.

US: Are there any recent advances in RISC processing, or anything like that, that we should know about – any rocks that have gone all the way up the hill?

HENNESSY: Yeah, yeah, I think there are a few ideas that are floating around. I think the idea of doing speculative execution is now pretty well understood (five years ago people would have say "Well, you’re crazy," but now it’s pretty well understood). I think that some of this thread-level speculation, which is still up on the research side, is pretty interesting.

The simultaneous multi-threading idea is a very interesting idea: it enables you to, if you’re going to build a dynamically scheduled machine, to get a lot more throughput out the back end of the machine with very little extra hardware by basically running multiple threads and interweaving threads on the hardware.

And then there’s this idea being kicked around called "value prediction" where you predict the result of an instruction – the value that an instruction will return. It’s interesting for two different kinds of applications. One of the problems you have in these machines is that they can reorder loads and stores in the flow, and that’s important to do because if you have a store here and you have a load here, that load is often at the beginning of a calculation that you really want to get started early, because it has a bunch of things that depend on that load. So what you want to do is yank the load up across the store, but the problem is that the load and store might touch the same memory location. So the way that this problem is solved is that you break the loads and stores into two pieces. In the first piece you calculate the effective address – the memory address that’s touching. And then what you do is you build a little buffer which keeps the effective addresses in and then you look at the effective address that the load wants to touch and the effective address the store wants to touch and before you have the store results (you don’t actually need the value that you’re going to store yet) , you can move the load across the store.

The problem with this is if you have programs with a lot of pointer chasing, then what’s actually happening with the store, you’re storing with a pointer value, and with load you’re loading with a pointer value, so you may not know what the address is, early. So then what happens is you can’t get these two things to pass by one another.

But, if, suppose you run the program, and you went through this loop fifty times, and you found that in that fifty times that the load and store never touch the same threads, well then you guess that in fact they wouldn’t touch the same address again, pull the load above the store, and then use all of these hardware mechanisms that are in these speculative machines. And if the guess was wrong, you undo it. So that actually could work okay. That’s one use of it.

The other use of it is: suppose you have this load down here, and that load loads a memory value, and you notice that, in the last ten executions, it always loaded the value five. Then you can guess that the next load will be the value five, so you do the same thing. You get that it’s going to be five, and then you start using the code, and then again you can use the speculative mechanisms to help back it up. These are all ideas that try to increase the sustained to peak ratio so that you get it closer, so that you get some improvement in it. I like the SMP idea. Have you seen this? Simultaneous multi-threading? So you know what multi-threading is?

US: Um hm.

HENNESSY: So multi-threading is you just run multiple threads on the same background. So multi-threading has been around for a while and it’s an interesting idea. Somebody made the observation that, if you build a back-end machine that is dynamically scheduled, speculative, and all that kind of thing, that all it really is is an instruction structure that fetches instructions, throws it into a buffer, and then this buffer basically schedules the instructions in data flow order.

So, you can take those instructions from multiple threads and throw them into the buffer and let the back end of the hardware do all of the work about deciding which instruction will execute next. And the thing that’s advantageous about this is the same thing that’s advantageous about all of the threading in general. If you have a single thread, and let’s say it gets a cache miss, that cache miss may be long enough that it stalls the entire thread. And even though you have the next twenty or thirty or forty instructions buffered from that thread, eventually they will all depend on that cache miss, and you can’t go forward because there’s an add that depends on that, and there’s a load that depends on the add, and more on that. But by doing multi-threading, you break the problem that you may have dependencies or clustering of cache misses, and you pull the different threads and execute them. So it provides you a way to boost the throughput of these things.

The other thing I like about it is that it moves you gracefully from this notion of using just construction level parallelism to trying to use thread level parallelism. It’s just a graceful movement of that in that direction. So, I’m optimistic that it may help in trying to think about how to design the next-generation machine.

US: What do you think about Moore’s Law? Do you think that processors are going to continue to double every eighteen months or if we’re going to eventually reach a point where we can’t do that anymore?

HENNESSY: I think we are on the verge of reaching a point for integer code where we can’t do it, or it would be very hard to do it. Floating point guys are doing quite well, I think. If you could do it for floating point code. The problem is the floating point is not growing as fast as the rest of the market. The market that’s really growing is sort of the Internet explosion market and it’s not floating point intensive. So floating point and scientific calculations as a fraction of the overall market are shrinking. So it wouldn’t be a smart move to just concentrate on that part of the market. You’d find yourself in trouble.

So, I think Moore’s Law will continue for awhile. Whether or not that could be translated into processor speed is the tricky thing. The thing to remember about increasing the transistor density is that the number of transistors increases quadratically as you decrease the size. But the transistors only get faster linearly. So you want things which use more transistors to get speed, not just relying on faster transistors.

Now, that all said, there have been a number of introductions into the process capability which are providing at least one-time jumps: copper, silicon insultor technologies. New changes so they obtain the short channel effect – low capacitance technologies. So, they’re providing, I think, a very ambititous growth in clock rate. One of the reasons why you see so much hype on clock rate is that right now, clock rate is easier to get than a lower CPI, so everybody focuses on clock rate for a while. But I think it’s going to be hard to sustain that rate of growth on the integer side. We need a really killer idea.

And the other interesting thing that I’ve observed is that ten years ago there were lots of ideas which were sitting on the warehouse shelves on the university and research labs that hadn’t been deployed in machines. No one had built a speculative machine; no one had returned to dynamic scheduling in many years. Lots of things that had been explored by research teams, simulated, but nobody was really using them yet. Now, the gap between university research and industry has shrunk dramatically. The industry is just pulling ideas out, and they’re moving so quickly into use. I think the reason is that they’re starved for new ideas to try to boost performance.

US: We just have one more question. Can you point us in the direction of any resources or articles we might want to look at that would help us for this topic?

HENNESSY: Uh, yeah…Well, what do you want to know. Do you want to know where it’s going? Are you more interested in the historical development?

US: Well, it’s kind of both.

HENNESSY: I think there are a couple of things you might find interesting to read about from a historical perspective. One of them would be the extremely fascinating machine that was never brought to fruition at IBM, but played a tremendous role in feeding some of the ideas, was a machine called ACS – Advanced Computing System. It was never developed, but it’s a machine that John Coche was a lead architect on. It was the first machine to propose issuing more than one instruction per clock. It never quite got there; later on, they worked on a machine called the Americas, which became the prototype for eventually the IBM power series, but that’s an interesting machine to look at, and there are some references on the web if you look up ACS on Google you’ll find it. Mark Smotherman at SMU has also done some background on that machine.

But it’s interesting because not very much is known about the machine. That’s true of the early days of the eta-1, too. There are a bunch of other people around that have interesting perspectives. John Masseuris, who was one of the co-founders of MIPS and was at IBM before that can give some interesting history of what happened at IBM, and probably even tell you why they cancelled the eta-1 and decided instead to product-ize what was called the micro 370, which was a fiasco. But, he lives in Palo Alto, and you can get a very interesting perspective on what’s happened on the project from him.

What’s been happening lately, there are some slides on my website which are some slides from the talk I gave at SCRC. The first half of it is kind of a "What’s happening in processor achitecture" talk. There’s also a talk there that I did for the 25^th anniversy of the microprocessor that is kind of a historical development of the microprocessor – more focused on the microprocessor in general than on RISC, but interesting still from that perspective. Those are all good things.

There’s a guy who just retired from HP but you probably can still find him there, called Joe Burnbaum, who was the head of the IBM Watson research lab at the time that the Eta-1 was being done. And then came to HP and started the HP RISC efforts, so he has an interesting perspective. Of course, Patterson, the Sun guys, can give you an interesting perspective on their view, as well. He’s at Berkeley. Dave Ditzel, who worked on the first RISC paper with Dave Patterson, is the CEO now of Transmeta, which is a local company doing low-power stuff. He has an interesting perspective on some of those early developments as well and where things are going. Those are all good people. Good resources.

US: Um, we’re going to present next Wednesday, so…

HENNESSY: Oh! <laughs> Look on the ‘net! I think one of the interesting things about architectures in general is that there’s a very interesting balance between an idea, a concept, an approach, and the underlying technology. This is very much a discipline where the technology shapes the ideas that are happening at any time.

Gordon Bell at Watson said something really interesting to me: when memory was very expensive, which it was until DRAMs really took over, so in the core days if you were designing from the perspective of somebody building this core memory, memory was more expensive than logic. What you did was minimize the use of memory. But when DRAMs came over and took over, all of the sudden that flipped, and initially they were maybe the same price, but then memory plumetted in price compared to DRAM, partly because it’s design cost was so low. You designed one DRAM and just replicated it, right? Cache became very inexpensive. Gordon Bell said when that change occurred, moving from microcoded machines like the VACs to non-microcoded machines that relied on cache and had a larger memory footprint was exactly the right thing to do. So it’s interesting to watch that occur.

There’s a really excellent – you know one of the long battles that went on here, it’s really hard to imagine, but when Patterson and I first started writing papers, people were mad – they were really mad. I remember going to a panel one time and one guy got up and said this is fraud. Really, they did. They just said it was fraud. Really, this technology was university, in the real world it wouldn’t work. If you had to ship it in a product, it wouldn’t work. This was one of the reasons to start a company, to prove our point. But, you know, he said, this is fraud. Well, this guy just said, they started this company, they took money from investors, what should they do? A counselman said well if I were them, I’d take the money and move to Barbados.

One of the reasons was that we had these benchmarks that said it was faster, but we didn’t have a good scientific explanation why. So finally, there was a landmark paper published on the VACs that showed that the machine was taking ten cycles to do every intstruction, which was a factor of two higher than even the VACs designers believed it to be. And, we could show these RISC machines were doing one and a half to two instructions per cycle. So now we knew where the performance advantage was. And that really clarified it. But everybody said, you know well maybe everything’s not equal. Finally, though, the … guys wrote a very famous, carefully documented paper, by Philipe Bandacar and Doug Clarke that’s cited in our book that’s something like a comparison of a RISC machine against a CISC approach. It really documents exactly this and demonstrates the fact that there really is something fundamental here, that it is a more effective way to build machines. So that’s a great paper. That was sort of a capstone. It said, "All right guys, the argument’s over."

But, it was very threatening. Here we said you can build these microprocessors and because it’s microprocessor technology, maybe it costs half as much to begin with, and by the way it’s two or three or four times faster. And this is why IBM cancelled their project. IBM cancelled the project because the technology was too disruptive, and it just threw all of this into, the entire market into upheavel. So that’s a good paper to look at. Interesting perspective.