Its D-day, the project is ready functionality wise, everyone’s beaming, code is checked in, bug count is in single digits and the project is ready to release. Then out of the blue, the project gets swamped by swarms of last minute glitches relating to performance. From memory leaks, to database bottlenecks, to pure response time, series of firefightings & heated meetings ensues, causing embarrassments and slipped schedules all around. Sounds familiar?
However as in most such cases, true improvements or fixes are rarely possible due to the effort it represents. Rarely is the architecture found flexible enough to accommodate improvements to the tune of an order of magnitude, without major rewrites. Instead, cover ups and shove it under the seat approaches or spec reductions and upscaling of system requirements is what is attempted at. Depending on the soft skills and magnanimity of the team involved, the product is sometimes pushed out to face endless cycles of customer support or better still it is killed off before being released.
Is not premature optimization evil (advocated first by Tony Hoare and restated by Donald Knuth)? So what gives?
Turns out that for most people, ‘functionally complete’ does not implicitly translate into works-fast. I wonder if the same folks would consider a car functionally complete if 10 miles is the top speed at which it can go.
In this particular case, the primary culprit was that none of the engineers who had worked on the project had much experience with the particular database that was being used. But the architecture was designed around a centrally important database to hold and serve all the data that flowed through the system . Consequently bad database coding practices threw the worst punch when it came to performance. There was no way the product could be fast, without major rewrites. The net output was pretty, but somewhat un-usable & could not be justified to be pushed out.
The keyword here is the term “premature”. Knuth or Hoare never meant the advise as an abandonment of basic sanitary performance levels. Given the nature of the advise, one would expect people to not worry about the teeny tiny levels of optimizations like moving that extra if-check out of the loop or manually unrolling instructions etc unless it was measured and found to be warranted. But not considering performance during the initial and intermediary stages is nothing but negligent design.
However it is hard to think in terms of basic engineering once you start implementing code, in terms of alien frameworks, tools & libraries. Implementation is hidden and the net effect comes out only towards the end. (case for open source?) Unless someone with expertise in each of these closed areas is present, its difficult to get things right, the first time a new team attempts using these technologies.
It is unpractical though, to expect to have an expert in each of the scores of tools / frameworks (count your acronyms) that are part of any modern projects. Normal IT shops should therefore seriously consider premature planning / architect-ing for optimization. Getting extremely serious about performance from the very beginning, would be a good idea. You can wait to get paranoid, towards the end. Adopting best practices & making it a point to adhere to these across the entire code base, before you start churning out code is sensible engineering and prudent planning. Anything short of this is plain and simple negligence.
Googling premature optimization, surprisingly presented similar thoughts echoed around the web. Wise – advises abound in the collective wisdom of the web.
Another take on same idea – cpu cycles might be cheap, but customer time waiting on that slow product of yours a’int.
Advice from Joel – don’t start a new project without at least one architect with several years of solid experience in the language, classes, APIs, and platforms you’re building on.
July 12, 2007 at 10:21 am
Ok, you say premature optimization is evil. What i say from database point is that, optimization starts from day 1. May be its the other way in software, but when you start design the database, start design the schema, there you start performance optimizations. Planning is the key word i would like to use. If plan has leaks, so do software, no matter what it does!
July 12, 2007 at 10:27 am
In fact i was advocating premature optimization
October 10, 2007 at 11:53 am
No one has ever written anywhere that a car will b functionally complete ONLY if it goes faster than 10 miles per sec. How can u say a software is functionally complete ONLY if it works fast? If it does what it’s supposed to do it is functionally complete isn’t it? Working fast is an extra requirement rt? It wud b good if the car goes a little more faster. Similarly it wud b better if the software performs the task a bit more faster.
October 10, 2007 at 11:59 am
Would you shell out 10,000$ for a car that goes only at 10 miles per hour?
(A cycle – 150$ can go faster than that)
October 25, 2007 at 2:20 pm
The keyword in “premature optimization” is “premature” – and the way I understand “premature” in this case is either “ahead of it’s time” or “in the wrong place”. This does mean that optimizing before you release is wrong – it means that spending time on optimizing pieces of code which might or might not be causing bottlenecks before you know where your bottlenecks are, is wrong.
If you’re building a piece of software that, as part of it’s initial requirements, is designed to handle heavy load – what you need to do is first of all *define exactly* the amount of load you need your software to handle as parts of your requirements. Then, build your software in a way that will be best performing – but don’t spend too much time optimizing the performance of parts without actually having some intelligent clue of how cost effective what your doing is. Instead, spend your time designing your software in a way that it can be easily optimized (a simple example: so that you’ll be able to move from one RDMMS to another easily) when you do have a clue about where the bottlenecks are.
Oh, and also if your goals define the stress you’re supposed to handle, stress testing should be an early part of your testing phase – this way you can decide whether your code handles the load it’s supposed to, and if not, find your bottlenecks (a good profiler is all you need in some cases) and then – optimize (when that time is right that is!)
October 25, 2007 at 2:31 pm
Shahar,
My experience has been that
Defining the load & performance estimates is one thing
Obtaining the said performance is usually another.
There is no way the performance levels one require can be obtained without actually planning & designing around it. (which is what you pointed out too). The fact that folks do not consider this in their designs is my peeve.