Conclusion

Conclusion

We have taken one specific tour of computer systems. But why?

Conclusion

A lot has been said about making code efficient: fewer instructions, less time, etc.

Sometimes efficiency of your code just doesn't matter: if your code doesn't run very often/​long, or spends almost all of its time waiting for network, or only makes a few calls to some library functions (Pandas, Node.js, etc) you have no control over, or …, then it's not worth the effort.

Conclusion

But sometimes efficiency does matter.

Every instruction has a cost: in time, electricity, battery life, CO₂ emissions, cloud provider bills, etc.

Write good code. Sometimes understanding the system is what lets you do that. (Sometimes it's a better algorithm, different library choice, different programming language choice, … .)

Conclusion

Can we count on Moore's Law or something similar to make our code magically faster next year? Probably not. The number of transistors is still increasing, but the gains are going to number of cores, vector instructions, etc.

To be good programmers, we need to be able to take advantage of the features the processor gives us.

Conclusion

Is the an absolute speed limit? I know of one. Let's look at an i9 14900K.

• Die size: 23.8 × 10.8 mm. Long dimension: \(2.38\times10^{-2}\,\mathrm{m}\).
• Highest processor clock: 5.8 GHz or \(1.72\times10^{-10}\,\mathrm{s/cycle}\).
• Speed of light: \(3.00\times10^{8}\,\mathrm{m/s}\)

So the absolute minimum time needed to propagate a signal across the die:

Conclusion

The die size includes all of the cores: an individual core is a fraction of that so maybe we don't expect to get all the way across in one cycle. The speed of electricity is a high fraction the speed of light, but slightly less.

Maybe the transistors can be shrunk to make the distances smaller but quantum physics interrupts that.

One way or another: modern processors are not very far from an absolute limit on raw speed. We need to get speed from good code, not magic improvements.

Not Covered

Some topics I wish we had time for but are going to fall off the end of the semester, with a one-sentence summary if you're interested:

• CISC vs RISC: it's in textbooks, but the distinction isn't super meaningful for modern processors.
• Meltdown & Spectre: security vulnerabilities that take advantage of speculative and out-of-order execution to leak information.

Not Covered

• Other places to do computation:
  • A coprocessor specialized for particular computations (e.g. Xeon Phi, an AI accelerator, a TLS accelerator, an FPGA).
  • Specifically, GPUs: think of a processor with hundreds or thousands of simple cores with great SIMD functionality.

Not Covered

On CPUs vs GPUs, a comparison of top-end consumer options:

Not Covered

I don't feel too bad about missing any of those. Semesters have finite length.

Other Processors

We have been focussing on x86-64 processors.

They have a lot of historical baggage, but are very common in real computers you have access to. There is also a massive collection of tools and documentation available, which is nice (and why I chose them for this course).

All of the concepts of this course apply to other modern processors. Details differ, but the ideas are the same.

Other Processors

The other processor architecture that you're likely to encounter at the moment is ARM. There are a wide variety of ARM processors: the design is licensed to many companies.

Your phone's or tablet's processor is likely ARM. So are Apple Silicon processors. Also, NVIDIA Grace, AWS Graviton, Raspberry Pis.

Other Processors

Like x86, ARM isn't new and has gone through many changes. The 64-bit ARM architecture is often referred to as AArch64.

Generally modern ARM processors have: 31 general purpose registers + a stack pointer; pipelined and superscalar; L1, L2, L3 cache; 128-bit wide SIMD instructions (Neon).

Other Processors

Even if you don't know ARM assembly, you can probably read it , at least a little. Maybe with some practice.

Conclusion

So, what did we learn?

Conclusion

Probably: don't write assembly to solve problems.

But: you have to know some assembly and how the system works to use it to its fullest.

Unless: you have to write assembly to solve that particular problem in a very specific way that can't be expressed any other way.

Conclusion

Computers are more complex than they seem.

• Processors make it look like they execute one instruction at a time.
• The way you arrange your data in memory doesn't seem like it matters: it's random access memory.
• Floating point numbers look like real numbers.
• Your data often doesn't look like it's made of bits.

… but none of those are true if you look closely. Abstractions leak.

Conclusion

I propose the most important things from this course to remember a year from now:

• Bit manipulation is very fast.
• Compilers are doing their best to write assembly for us. Their best is almost always good enough.
• Processor cache and memory locality matter.
• Predictable/​unpredictable branches affect performance by a lot.
• With the right operations and data layout, SIMD instructions are a huge speed gain.
• godbolt.org.