David Snowden has an item on Reality avoidance that makes a connection to some thoughts on project execution in the Critical Chain Project Management world.
Maybe its time we stopped ascribing cause without reason, withdrawing into a fatalistic shell or praying for salvation and just got stuck in to try and make things better?
In project environments, there are higher level processes that could occur around how to get better over time. Frequently this comes in at the end of projects with lessons learned discussions or debriefs or the like. The general question is, "What has caused this delay?" and "How can we prevent it in the future?" In happy circumstances you might even get to ask, "Why was it early?" and "Can we replicate it?"
Sometimes a project team can identify some causes of big delays or delays that were particularly annoying. But it is not always clear whether a given cause is the source of delays. The more common answer is, "We have no idea of the cause of delays / early delivery." Generally, this is because projects are complex and delays throughout the project interact and create a mess of spaghetti from which no clear answer emerges.
So, what is to be done? To find out the real reasons behind delays, it's better to discover them as they happen. This suggests that it should be part of the ongoing project management activities at the task level, rather than at the project level. Over time, the project and the whole collection of projects should collect reasons for task delays over the entire project network as it executes.
But, you want a real process of ongoing improvement. You want to know how to be better the next time. It is not enough simply to catalog a list of delays. They need to be tied to the impact they have had. Not only that, you must understand the collective impact of the delay to the project. If a raw material delay comes up a lot, but doesn't have a great impact overall, then it is not something to worry about. However if another raw material is delayed infrequently but kills the project, then there is something to address.
Another piece to this puzzle is in how you ask for the cause of these delays. Asking, "Why are you late?" doesn't always give the correct response, as people like to point at anything but themselves. And pointing out delays already puts the wrong pressure on the responsible parties. Rather, ask everyone (all the active tasks), whether they are making progress. If not, ask, "What are you waiting for?" This question should give real causes of delays.
And remember, you aren't interested in task-specific dates and holding people's feet to the fire for them, you are interested in the overall success of the project. How are the collective actions of everyone on the project impacting the project?