SIPS and SLURPS

In his book, The Flaw of Averages, Stanford professor Sam Savage provides an entertaining account of how making decisions based only on the average case regularly leads to biases that are often extremely costly. Through a series of witty examples, he convincingly argues that effective risk management and decision making should consider explicitly the ramifications of uncertainty. He advocates the use of probability as a methodology for expressing and communicating the uncertainties present in estimates.

In a two-part series of articles in OR/MS Today,  Sam Savage, Stefan Scholtes, and Daniel Zweidler describe what they call Probability Management. They propose practical pragmatic methods to enable any organization to make effective use of information about uncertainty. They present a framework for estimating, maintaining, communicating and utilizing probability distributions within an organization. By embracing the framework, decision makers can potentially eliminate the flaw of averages. Analytica is an ideal tool for implementing and leveraging this framework.

The key elements of this new framework are called SIPs and SLURPs. When a subject matter expert provides an estimate for a quantity as a probability distribution – such as, when a program manager provides an estimated time to completion – they propose the distribution in the form of a Stochastic Information Packet (SIP). A SIP is essentially a Monte Carlo sample of possible values, using a standard sample size, with a distribution representative of the possible outcomes. Importantly, the SIP is treated as the representation of the value and uncertainty of the quantity. To capture relationships between quantities, multiple SIPs are bundled into a SLURP (Stochastic Library Unit with Relationship Preserved). SIPs and SLURPs may be exchanged between people within the organization, and used directly in decision models. By managing a standardized set of SIPs and SLURPs within an organization, probabilistic estimates from different groups within an organization can be combined within models in a coherent fashion.

The sample values within SIPs and SLURPs appear in a random order, as would be the case in a Monte Carlo sample, but the specific ordering of the samples is critical as it captures the relationships between quantities. Suppose one SIP represents the remaining cost to complete a construction project and another SIP is the remaining time to completion. In scenarios, or samples, with an exceptionally low cost, the remaining time will also usually be small. Likewise, cost overruns usually coincide with delays. These two SIPs are coherent when the ordering of samples captures this relationship, meaning that the nth point of remaining_cost should correspond to the same scenario as the nth point of remaining_time. Coherence in this fashion captures the correlation between the quantities, as well as other more subtle dependencies that may not be apparent in correlations. Remaining cost and time are SIPs that should be bundled within the same SLURP.

According to Sam Savage, The Flaw of Averages, “without using the terminology [of SIPs and SLURPs], Analytica has been providing, what I would call, a full Probability Management solution from the beginning.” Each uncertain variable in an Analytica model has a sample value indexed by the system Run index, and this sample value serves as Analytica’s representation of uncertainty. In Savage’s terminology, the sample value for a scalar Analytica variable is a SIP. As downstream results are computed from existing quantities, Analytica’s intelligent array-abstraction mechanism applies the computations to each point in the underlying sample values, with the resulting sample values also indexed by Run. In so doing, all variables within the model retain coherence. A multidimensional variable (containing other dimensions such as Time) is an example of a SLURP, but even more comprehensively, the full Analytica model constitutes a SLURP, with all variables forming a coherent package of uncertainties.

As an ordered vector of numbers, a SIP may seem a bit awkward to share with your colleagues. It is easier to pass your colleague a single self-contained entity, such as a single number, than a collection of numbers. To address this, ProbabilityManagement.org has introduced the DIST™ standard, which provides a representation DST Graph of an entire SIP as a single text value, packaged as XML. A SLURP is shared as a collection of DISTs. Our own Lonnie Chrisman (CTO of Lumina) has contributed substantially to improving the DIST representation, resulting in a revised standard, DIST 1.1. The Analytica function, SipEncode and SipDecode, rapidly transforms a SIP to and from a DIST representation. The DIST is more compact than the standard numerical representation by a factor of 6 (at single resolution) or a factor of 3 (at double resolution), resulting in a smaller database – and what is usually more important, dramatically reducing data read and write times for large SIPs or SLURPs. DISTs may also be exchanged with other software that supports the standard, including Frontline Systems’ Risk Solver and XLSim, for use with Crystal Ball or @Risk. Analytica can handle much larger DIST sample sizes than spreadsheets, given its blazing encoding and decoding speeds, e.g., about 1/5th of a second to encode or decode a DIST with a one million sample.

With the high availability of computational resources today, the availability of user-friendly software such as Analytica, and the increasing awareness of how important it is to account for uncertain outcomes, there is little uncertainty that the current trend of rapidly increasing adoption of Probability Management practices will continue in years to come.