Archive for the 'Forecasting' Category

Strategic Progamming

Over the last 40-odd years, a branch of Artificial Intelligence called AI Planning has developed.  One way to view Planning is as automated computer programming: 

  • Write a program that takes as input an initial state, a final state (“a goal”), and a collection of possible atomic actions, and  produces as output another computer programme comprising a combination of the actions (“a plan”) guaranteed to take us from the initial state to the final state. 

A prototypical example is robot motion:  Given an initial position (e.g., here), a means of locomotion (e.g., the robot can walk), and a desired end-position (e.g., over there), AI Planning seeks to empower the robot to develop a plan to walk from here to over there.   If some or all the actions are non-deterministic, or if there are other possibly intervening effects in the world, then the “guaranteed” modality may be replaced by a “likely” modality. 

Another way to view Planning is in contrast to Scheduling:

  • Scheduling is the orderly arrangement of a collection of tasks guranteed to achieve some goal from some initial state, when we know in advance the initial state, the goal state, and the tasks.
  • Planning is the identification and orderly arrangement of tasks guranteed to achieve some goal from some initial state, when we know in advance the initial state, the goal state, but we don’t yet know the tasks;  we only know in advance the atomic actions from which tasks may be constructed.

Relating these ideas to my business experience, I realized that a large swathe of complex planning activities in large companies involves something at a higher level of abstraction.  Henry Mintzberg called these activities “Strategic Programming”

  • Strategic Programming is the identification and priorization of a finite collection of programs or plans, given an initial state, a set of desirable end-states or objectives (possibly conflicting).  A program comprises an ordered collection of tasks, and these tasks and their ordering we may or may not know in advance.

Examples abound in complex business domains.   You wake up one morning to find yourself the owner of a national mobile telecommunications licence, and with funds to launch a network.  You have to buy the necessary equipment and deploy and connect it, in order to provide your new mobile network.   Your first decision is where to provide coverage:  you could aim to provide nationwide coverage, and not open your service to the public until the network has been installed and connected nationwide.  This is the strategy Orange adopted when launching PCS services in mainland Britain in 1994.   One downside of waiting till you’ve covered the nation before selling any service to customers is that revenues are delayed. 

Another downside is that a competitor may launch service before you, and that happened to Orange:  Mercury One2One (as it then was) offered service to the public in 1993, when they had only covered the area around London.   The upside of that strategy for One2One was early revenues.  The downside was that customers could not use their phones outside the island of coverage, essentially inside the M25 ring-road.   For some customer segments, wide-area or nationwide coverage may not be very important, so an early launch may be appropriate if those customer segments are being targeted.  But an early launch won’t help customers who need wider-area coverage, and – unless marketing communications are handled carefully – the early launch may position the network operator in the minds of such customers as permanently providing inadequate service.   The expectations of both current target customers and customers who are currently targets need to be explicitly managed to avoid such mis-perceptions.

In this example, the different coverage rollout strategies ended up at the same place eventually, with both networks providing nationwide coverage.  But the two operators took different paths to that same end-state.   How to identify, compare, prioritize, and select-between these different paths is the very stuff of marketing and business strategy, ie, of strategic programming.  It is why business decision-making is often very complex and often intellectually very demanding.   Let no one say (as academics are wont to do) that decision-making in business is a doddle.   Everything is always more complicated than it looks from outside, and identifying and choosing-between alternative programs is among the most complex of decision-making activities.




Taking a view vs. maximizing expected utility

The standard or classical model in decision theory is called Maximum Expected Utility (MEU) theory, which I have excoriated here and here (and which Cosma Shalizi satirized here).   Its flaws and weaknesses for real decision-making have been pointed out by critics since its inception, six decades ago.  Despite this, the theory is still taught in economics classes and MBA programs as a normative model of decision-making.  

A key feature of MEU is the the decision-maker is required to identify ALL possible action options, and ALL consequential states of these options.   He or she then reasons ACROSS these consequences by adding together the utilites of the consquential states, weighted by the likelihood that each state will occur.  

However, financial and business planners do something completely contrary to this in everyday financial and business modeling.   In developing a financial model for a major business decision or for a new venture, the collection of possible actions is usually infinite and the space of possible consequential states even more so.  Making human sense of the possible actions and the resulting consequential states is usually a key reason for undertaking the financial modeling activity, and so cannot be an input to the modeling.  Because of the explosion in the number states and in their internal complexity, business planners cannot articulate all the actions and all the states, nor even usually a subset of these beyond a mere handful.  

Therefore, planners typically choose to model just 3 or 4 states - usually called cases or scenarios – with each of these combining a complex mix of (a) assumed actions, (b) assumed stakeholder responses and (c) environmental events and parameters.  The assumptions and parameter values are instantiated for each case, the model run, and  the outputs of the 3 or 4 cases compared with one another.  The process is usually repeated with different (but close) assumptions and parameter values, to gain a sense of the sensitivity of the model outputs to those assumptions.     

Often the scenarios will be labeled “Best Case”, “Worst Case”, “Base Case”, etc to identify the broad underlying  principles that are used to make the relevant assumptions in each case.   Actually adopting a financial model for (say) a new venture means assuming that one of these cases is close enough to current reality and its likely future development in the domain under study- ie, that one case is realistic.   People in the finance world call this adoption of one case “taking a view” on the future. 

Taking a view involves assuming (at least pro tem) that one trajectory (or one class of trajectories) describes the evolution of the states of some system.  Such betting on the future is the complete opposite cognitive behaviour to reasoning over all the possible states before choosing an action, which the protagonists of the MEU model insist we all do.   Yet the MEU model continues to be taught as a normative model for decision-making to MBA students who will spend their post-graduation life doing business planning by taking a view.

 




Time, gentlemen, please

Harper Charley SerengetiSpaghetti

Much discussion again over at Language Log over a claim of the form “Language L has no word for concept C”.  This time, it was the claim by Wade Davis (whose strange use of past tense indicates he has forgotten or is unaware that many Australian Aboriginal languages are still in use) that:

In not one of the hundreds of Aboriginal dialects and languages was there a word for time.”

The rebuttal of this claim by Mark Liberman was incisive and decisive.   Davis was using this claim to support a more general argument:  that traditional Australian Aboriginal cultures had different notions of and metaphors for time to those we mostly have in the modern Western world.

We in the contemporary educated West typically use a spatial metaphor for time, where the past is in one abstract place, the present in another non-overlapping abstract place, and the future in yet a third non-overlapping abstract place.    In this construal of time, causal influence travels in one direction only:  from the past to the present, and from the present to the future.   Nothing in either the present  or the future may influence the past, which is fixed and unchangeable.   Events in the future may perhaps be considered to influence the present, depending on how much fluidity we allow the present to have.  However, most of us would argue that it is not events in the future that influence events in the present, but our present perceptions of possible future events that influence events and actions in the present.

Modern Western Europeans typically think of the place that represents the past as being behind them, and the future ahead.   People raised in Asian cultures often think of the abstract place that is the past as being below them (or above them), and the future above (or below).   But all consider these abstract places to be non-overlapping, and even non-contiguous.

Traditional Australian Aboriginal cultures, as Davis argues, construe time very differently, and influences may flow in all directions.   A better spatial metaphor for Aboriginal notions of time would be to consider a modern city, where there are many different types of transport and communications, each viewable as a network:  rivers, canals, roads, bus-only road corridors, railways, underground rail tunnels, underground sewage or water drains, cycleways, footpaths, air-transport corridors, electricity networks, fixed-link telecommunications networks, wireless telecommunications networks, etc.    A map of each of these networks could be created (and usually are) for specific audiences.  A map of the city itself could then be formed from combining these separate maps, overlaid upon one another as layers in a stack.   Each layer describes a separate aspect of reality, but the reality of the actual entire city is complex and more than merely the sum of these parts.  Events or perceptions in one layer may influence events or perceptions in other layers, without any limitations on the directions of causality between layers.

Traditional Aboriginal notions of time are similar, with pasts, the present and futures all being construed as separate layers stacked over the same geographic space – in this case actual geographic country, not an abstract spatial representation of time.  Each generation of people who have lived, or who will live, in the specific region (“country” in modern Aboriginal English) will have created a layer in the stack.   Influence travels between the different layers in any and all directions, so events in the distant past or the distant future may influence events in the present, and events in the present may influence events in the past and the future.

Many religions – for example, Roman Catholicism, Hinduism, and African cosmologies – allow for such multi-directional causal influences via a non-material realm of saints or spirits, usually the souls of the dead, who may have power to guide the actions of the living in the light of the spirits’ better knowledge of the future.   Causal influence can thus travel, via such spirit influences, from future to present.  Similarly, the view of Quantum Mechanics of space-time as a single 4-dimensional manifold allows for influences across the dimension of time as well as those of space.

I am reminded of an experience I once witnessed where the only sensible explanation of a colleague’s passionate enthusiasm for a particular future course of action was his foreknowledge of the specific details of the outcome of that course of action.  But these details he did not know and could not have known at the time of his enthusiasm,  prior to the course of action being executed.  In other words, only a causal influence from future to present provided a sensible explanation for this enthusiasm, and this explanation only became evident as the future turned into the present, and the details of the outcome emerged.  Until that point, he could not justify or explain his passionate enthusiasm, which seemed to be a form of madness, even to him.    Contemporary Western cosmology does not provide such time-reversing explanations, but many other cultures do; and current theories of quantum entanglement also seem to.

Contemporary westerners, particularly those trained in western science, have a hard time understanding such alternative cosmologies, in my experience.  I have posted before about the difficulties most westerners have, for instance,  in understanding Taoist/Zen notions of synchronicity of events, which westerners typically mis-construe as random chance.




Glasperlenspielen

Lars Pålsson Syll on “orthodox, mainstream, neoclassical economics”:

Economic theory today consists mainly in investigating economic models.

Neoclassical economics has since long given up on the real world and contents itself with proving things about thought up worlds. Empirical evidence only plays a minor role in economic theory (cf. Hausman [1997]), where models largely functions as a substitute for empirical evidence.  But “facts kick”, as Gunnar Myrdal used to say. Hopefully humbled by the manifest failure of its theoretical pretences, the one-sided, almost religious, insistence on mathematical deductivist modeling as the only scientific activity worthy of pursuing in economics will give way to methodological pluralism based on ontological considerations rather than formalistic tractability.

If not, we will have to keep on wondering – with Robert Solow and other thoughtful persons – what planet the economic theoretician is on.”  [page 54]

I agree with the general thrust of this essay, which resonates with some of my own thoughts on the Glass Bead Game of Economics, for example,  here and here.

Mind you, I don’t agree with everything that Syll says in this essay.  For example, he argues that good predictive capabilities require models to bear resemblance to their target domains.    But we know many counter-examples to this claim, from Newton’s model of planetary motion to Friedman’s billiard players.    Prediction and explanation are two orthogonal dimensions of a model, which may or may not be related in any particular case.

His essay also overlooks the fact the the so-called “real world” which is the target domain of economic models contains, at least in the case of macro-economics, mostly humanly-constructed artefacts, such as the “variables” known as inflation and unemployment rates.   Having sat in working parties defining and redefining such artefacts, I am always surprised that any economist could possibly imagine they are modeling an independent reality.

Reference:

Lars Pålsson Syll [2010]:  What is (wrong with) economic theory?  Real-world Economics Review, 55: 23-57.




Shackle on Rational Expectations

The Rational Expectations model in economics assumes that each economic agent (whether an individual or a company) can predict the future as perfectly as the modelers themselves.   To anyone living outside the rarified bubble of mathematical economics, this is simply ridiculous.   It is clear that no one associated with that theory has ever made any real business decisions, or suffered their consequences.

Here is non-mainstream economist George Shackle, writing to Bryan Hopkins on 1980-08-20:

‘Rational expectations’ remains for me a sort of monster living in a cave. I have never ventured into the cave to see what he is like, but I am always uneasily aware that he may come out and eat me. If you will allow me to stir the cauldron of mixed metaphors with a real flourish, I shall suggest that ‘rational expectations’ is neo-classical theory clutching at the last straw.

Observable circumstances offer us suggestions as to what may be the sequel of this act or that one. How can we know what invisible circumstances may take effect in time-to come, of which no hint can now be gained? I take it that ‘rational expectations’ assumes that we can work out what will happen as a consequence of this or that course of action. I should rather say that at most we can hope to set bounds to what can happen, at best and at worst, within a stated length of time from ‘the present’, and can invent an endless diversity of possibilities lying between them. [Italics in original]

Of course, Shackle had actual real-world experience of investment decision-making from his experience during WW II on national infrastructure planning.

Reference:

George L. S. Shackle [1980]:  Letter to Bryan Hopkins.  Quoted in:  Stephen L. Littlechild [2003]: Reflections on George Shackle:  Three Excerpts from the Shackle Collection.  The Review of Austrian Economics, 16 (1): 113-117.




The Great British Rail Network Franchise Disaster of 2012

The British papers are full of stories about The Great British Rail Network Franchise Disaster of 2012.  Like Bristow’s Great Tea Trolley Disaster of 1967, we may never learn the real reasons behind the disaster – errors are alleged in calculations (arithmetic errors? using multi-line spreadsheets?) undertaken by senior civil servants, now suspended.  But one item leapt out to me:

Government sources said “heads will definitely roll in the department” over the affair, adding that “the minister cannot be expected to be responsible for a very technical models with hundreds of lines in a spreadsheet”.

The key error seems to have been to underestimate the potential value of the franchise – where the company pays a premium to the Government, rather than receiving a subsidy.

The department said mistakes had been made over estimates of the number of passengers who would use the route and the way inflation was calculated. Three civil servants have been suspended.

Why on earth are government civil servants estimating future passenger numbers and rates of inflation?  Surely, that is the business of the bidders.   Only the bidders, after all, have the expertise, the experience, and the motivated self-interest to make these forecasts as accurately and realistically as possible.  The Government should be making its franchise decision on whatever criteria it thinks appropriate (eg, the numbers of jobs created, the novelty of services provided, the public fares charged, the money payments offered for the franchise, etc), but not trying to second-guess the business plans of the train operators.   Any demand forecast will depend on assumptions about the actual services offered, the actual prices charged for these services, and the actions undertaken to market, promote, distribute, and sell them, and none of these assumptions are within the purview of the Government.

Indeed, not only do civil servants not know these marketing plans, civil servants – in my extensive experience of submitting telecommunications licence applications – do not even have the expertise needed to assess such plans.     How can they tell whether a marketing plan is effective or not?  Feasible or not?  Sensible or not?  Even experienced marketers can get market planning wrong, so how much more so civil servants with no commercial experience at all, no direct stake in the outcome, and no ear to the market ground?  A famous British example of marketing ignorance by civil servants was the refusal by British Treasury officials during the 1960s to approve (what is now) British Telecom’s proposed telecoms switch upgrades, since the proposed switches allowed for itemized billing of calls:  What user would need that? asked the refusenik officials.

A decade of telecommunications licences awarded by beauty contests finally convinced Governments around the world to put aside any attempt to plan the businesses involved, and just ask potential operators to pay what they think each licence is worth, via auctions.   Of course, British regional rail network franchises are monopolies, so it is appropriate for franchise allocation decisions to be based on criteria additional to the amount of money offered for the franchise.   It is even appropriate for these criteria to include subjective and qualitative factors, such as the degree of risk of the bidder going bankrupt during the franchise period.    Even so, I cannot see a need for a Government to be predicting customer demand,  or even assessing the predictions of customer demand made by the bidders.  They should leave that job to the people with the most to lose for getting the forecasts wrong.

If, for some reason, the Government does need its own independent forecast of demand, it should outsource the creation of the forecast (strictly, the forecast model) to some outside entity with the expertise, the experience, and the motivated self-interest to make these forecasts (or model) as accurately and realistically as possible.   Outsourcing would also more likely ensure that the generation of such demand forecasts is independent from their use in any evaluation of franchise bids, so that neither decision  – deciding the forecasts nor choosing the franchise winners – could corruptly influence the other.




Embedded network data

In June, I saw a neat presentation by mathematician Dr Tiziana Di Matteo on her work summarizing high-dimensional network data.  Essentially, she and her colleagues embed their data as a graph on a 2-dimensional surface.   This process, of course, loses information from the original data, but what remains is (argued to be) the most important features of the original data.

Seeing this, I immediately thought of the statistical moments of a probability distribution – the mean, the variance, the skewness, the kurtosis, etc.   Each of these summarizes an aspect of the distribution – respectively, its location, its variability, its symmetry, its peakedness, etc.  The moments may be derived from the coefficients of the Taylor series expansion (the sum of derivatives of increasing order) of the distribution, assuming that such an expansion exists.

So, as I said to Dr Di Matteo, the obvious thing to do next (at least obvious to me) would be to embed their original network data in a sequence of surfaces of increasing dimension:  a 3-dimensional surface, a 4-dimensional surface, and so on, akin to the Taylor series expansion of a distribution.     Each such embedding would retain some features of the data and not others.  Each embedding would thus summarize the data in a certain way.   The trick will be in the choice of surfaces, and the appropriate surfaces may well depend on features of the original network data.

One may think of these various sequences of embeddings or Taylor series expansions as akin to the chain complexes in algebraic topology, which are means of summarizing the increasing-dimensional connectedness properties of a topological space.  So there would also be a more abstract treatment in which the topological embeddings would be a special case.

References:

M. Tumminello, T. Aste, T. Di Matteo, and R. N. Mantegna [2005]:  A tool for filtering information in complex systems.  Proceedings of the National Academy of Sciences of the United States of America (PNAS), 102 (30) 10421-10426.

W. M. Song, T. Di Matteo and T. Aste [2012]:  Hierarchical information clustering by means of topologically embedded graphs. PLoS ONE, 7:  e31929.




Forecasting using social media

is all the rage among marketers.   A nice application is here, courtesy of Ohal – forecasting availability of Boris Bikes in London by means of Twitter and Facebook posts.  In this application, the software learns the key words used as forecast inputs.




Imaginary beliefs

In a discussion of the utility of religious beliefs, Norm makes this claim:

A person can’t intelligibly say, ‘I know that p is false, but it’s useful for me to think it’s true, so I will.’ “

(Here, p is some proposition – that is, some statement about the world which may be either true or false, but not both and not neither.)

In fact, a person can indeed intelligibly say this, and pure mathematicians do it all the time.   Perhaps the example in mathematics which is easiest to grasp is the use of the square root of minus one, the number usually denoted by the symbol i.   Negative numbers cannot have square roots, since there are no numbers which when squared (multiplied by themselves) lead to a negative number.  However, it turns out that believing that these imaginary numbers do exist leads to a beautiful and subtle mathematical theory, called the theory of complex numbers. This theory has multiple practical applications, from mathematics to physics to engineering.  One area of application we have known for about a  century is the theory of alternating current in electricity;  blogging – among much else of modern life – would perhaps be impossible, or at least very different, without this belief in imaginary entities underpinning the theory of electricity.

And, as I have argued before (eg, here and here), effective business strategy development and planning under uncertainty requires holding multiple incoherent beliefs about the world simultaneously.   The scenarios created by scenario planners are examples of such mutually inconsistent beliefs about the world.   Most people – and most companies – find it difficult to maintain and act upon mutually-inconsistent beliefs.   For that reason the company that pioneered the use of scenario planning, Shell, has always tried to ensure that probabilities are never assigned to scenarios, because managers tend to give greater credence and hence attention to scenarios having higher-probabilities.  The utilitarian value of scenario planning is greatest when planners consider seriously the consequences of low-likelihood, high-impact scenarios (as Shell found after the OPEC oil price in 1973), not the scenarios they think are most probable.  To do this well, planners need to believe statements that they judge to be false, or at least act as if they believe these statements.

Here and here I discuss another example, taken from espionage history.




Alan Greenspan in 2004

Alan Greenspan, then Chairman of the US Federal Reserve Bank System, speaking in January 2004, discussed the failure of traditional methods in econometrics to provide adequate guidance to monetary policy decision-makers.   His words included:

Given our inevitably incomplete knowledge about key structural aspects of an ever-changing economy and the sometimes asymmetric costs or benefits of particular outcomes, a central bank needs to consider not only the most likely future path for the economy but also the distribution of possible outcomes about that path. The decisionmakers then need to reach a judgment about the probabilities, costs, and benefits of the various possible outcomes under alternative choices for policy.”

The product of a low-probability event and a potentially severe outcome was judged a more serious threat to economic performance than the higher inflation that might ensue in the more probable scenario.”