"depends a great deal on whether or not people using these tools are willing to make real predictions and then evaluate them against the actual outcomes, as opposed to simply looking for interesting patterns after the fact and reporting on the ones that look good."
Sounds a lot like a good Plan-Do-Check-Act cycle, to me.
The PDCA cycle is always worthwhile. But to work really well, it needs to be honest, as quoted above. It requires a stated, written expectation ahead of time and an honest, reflective review after the fact. It can't "pick and choose" the facts it wants to support.
In my experience, it is the reluctance to reflect on results and how they compare with the plan that is the weak link for many PDCA efforts. Once done with a project or a fiscal year, the weary warriors just want to get on with something new. Unfortunately, when we do this, we are often doomed to repeat the same errors that made us miss the mark in the first place. Reflection and honesty are the best antidote. But we often don't want to take that medicine.
Steve Spear's excellent book, Chasing the Rabbit is the best description of PDCA out there, with a myriad of examples from multiple industries. I commend it highly to you. It's in the top 5 books about Lean out there.
Click here to subscribe to Learning about Lean by email.