Living in the Moment

Short blog this morning .... something a little different

I have had a life long love affair with the game of tennis, having played competitively and recreationally. After many years of practice I have finally begun to understand the game. Over the last 5 years my game has improved significantly, and even out of shape as I am now, I am playing better than I ever have at my competitive peak. The key learning is to live in the moment, and forget about outcomes. When I was younger, I focused so much on the score, worrying about what people would think if I won or lost. What I learned was that you cannot achieve an outcome if you don't love executing the moments and actions you need to get there.

I have learned to not worry about scores and outcomes. I focus on executing each shot, enjoying a well placed forehand, a good volley, a backhand slice. I can now reach a level of calmness, not having external thoughts and worries, when I play that allows me to perform to my potential which then achieves the outcomes I have so desired.

The learning also applies to the work world. Focusing on your day to day activities, learning to love those and doing the best you can in the moment, is the sure fire way to success. I have been working hard to bring lessons learned from recreation to my day to day business life. I'll keep you posted.

Appliance as a Business Solution

We've been working on developing applianced based business solutions.... (keyword business)

what is an appliance according to me (the first list is technology focused, if you're a business person read-on beyond this list...)

1. Hardware
2. Database Software
3. ETL Software
4. Reporting Software
5. Advanced Statistical Analysis Software
6. Systems Integration Assets (Data Models, ETL Code, Standard Reports, Automated Statistical Modeling)
7. Business Process Workflow
8. Financial Model
9. Remote Operational Management

What does the business appliance do ... according to me

1. It improves a specific business process (i.e. increase revenue or reduces cost) by using advanced analytics.
2. the output of the appliance is integrated into a business process with defined user workflows that generate operational business activity
3. The output of the appliance also generates a P&L and Balance Sheet actuals & forecast which resulted or will result from the operational business activity executed by using the appliance

What type of business processes are amenable to an appliance model ... according to me again

1. Processes where a small set of decision metrics are obvious to act upon
2. Processes that generate operational data

Some Examples

1. fraud detection... output metric is a scored claim where the score value indicates whether or not the claim should be investigated. The result of claims investigation is less claims paid out. 10 to 15% of claims are typically fraudulent, eliminating 50% of fraud will more than double bottom-line results
2. Price optimization for retail ... output is a recommended price on a product ... when executed price optimization can deliver up to 5% sales and margin increase which for a typical retailer will double bottom line profits...
3. Forecasting for retail and manufacturing ... output is recommended inventory levels ... when executed for rertail will reduce stockouts and markdowns ... when executed for manufacturing will increase inventory turns or capital tied up in inventory... both can double bottom-line results.
4. Customer Loyalty ... output is recommended marketing campaigns to specific customer sets ... when executed will increase frequency, transaction revenue, transaction gross margin, and customer lifetime which can add 5% to top line or gross margin which again would double a typical retailers bottom line results...
5. A huge list of other business opportunities which achieve dramatic results

from a technology perspective what distinguishes a business appliance from software which could deliver the above 4 examples ... according to me

1. The appliance is pre-built and avoids a 12-18 months systems integration project. Most large integration projects fail
2. The appliance has business workflow which integrates output of the appliance into existing operational business process which ensures that output is acted upon.
3. The appliance has financial model output which integrates into P&L and Balance Sheet tracking and forecasting to ensure the top-line and bottom-line results are achieved.
4. The appliance is a black-box to customers ... it has a data interface to receive data and output windows (workflow, reporting and financial output) requiring no administration except a power and network cable.
5. the appliance's day to day function is remotely administered to ensure data is loaded properly, data is backed up, statistic models are tuned, financial results are achieved...
6. The appliance can be delivered as a hosted or SaaS model...
7. The appliance can be virtualized...
8. The appliance can be deployed in 20-30 days from order
9. The client will achieve business results in 21-31 days from order

Why would a company consider an appliance .... according to me

1. Avoids risky system integration projects
2. Delivers results quickly
3. Is business focused with clear integration point to business process and shareholder/public financial reporting
4. The appliance deliverable is a business result ... not a technology system ...the appliance is a means to an end.
5. Corporate Information Technology departments are not good at delivering strategic results... The appliance requires minimal IT involvement
6. Corporate IT departments can learn to "appliance-ize" solutions
7. Technology vendors (hardware and software) do not deliver on the sales proposition... the appliance does.

My company plans to roll-out many appliance solutions in the coming months ....

What do you think? Sound interesting? Feedback welcome ....

Five thoughts ... one blog

Long time no blog ... been very busy, which is great!

Just wanted to put down some short statements that I always use with clients, colleagues, friends related to the power of information.

1) You can't sell pink basketballs... The fundamental driver of business growth is the quality of your company's value proposition and how authentically (see http://www.ted.com/index.php/talks/joseph_pine_on_what_consumers_want.html) you execute it. You can build great management methods, execute your operational processes to perfection, use advanced analytics in all parts of your business and do it all for nothing if your customers don't like your value proposition. Great companies deliver great value to their customers. Facing the facts on this one is a hard thing to do.

2) If you want to increase revenue/profit ... start right now... in the words if Nike Corporation "Just Do It". Integrate advanced analytics and your operational processes... you could be a matter of weeks away from top-line/bottom-line growth. Change is scary but the results are worth it. You have a 100% chance of success applying analytic insight. Read the books "Freakonomics, Supercrunchers, Competing on Analytics, Smart Enough systems,....".

3) Companies don't have relationships with their customers. I don't know anyone at the bank, telco, insurance company, retailers I do business with. We don't trade family pictures, vacation... Getting hung up on the terminology of relationship I believe is dangerous. Again, customers seek value propositions. Offering an authentic value proposition is the secret sauce, understanding what customers value and informing them of the value propositions you offer is key. Doing this in a profitable manner is critical. Customer insight ... yes! Creating a great value proposition(s)... yes! Executing brilliantly ... yes! Marketing, promoting and operationally executing your value propositions ... yes! Track the results of your operational processes ... yes! Analyzing your processes to create insight and execute better ... yes! If all these are the definition of a "relationship" then yes! .... but not the dictionary definition of relationship.

4) Stop seeking the silver bullet. Execute your business model. It's boring... you have to work harder... if you're a bank, be a bank (not some high risk investment house), I find it hard to imagine that you could ever lose money being a bank or insurance company. I think executives seek the silver bullet and end up straying from the core business and lose the ability to execute the core business model well. The hamster on the treadmill wins. Always has, always will ...

5) Test and learn ... conduct experiments every day ... measure the results of those experiments and improve your operational processes. The current business paradigm makes operational changes once every decade and rides the process until it fails. Continuous tweaking is the right way... approve 1000 loans randomly, underwrite 1000 policies randomly and see what happens. Build a new loan approval or underwriting model from the results and see if it significantly different than your current model.

That's it for now. Would love your comments...

A rant

Something completely different...

Does anyone else think it is a huge conflict of interest to allow police to use the proceeds of applying the traffic act, that is they get to keep the monies collected from handing out traffic tickets? Isn't this the violation of some basic legal prinicple?

Police are incented to hand out more tickets so they can stay employed and hire more policemen to hand out tickets. It is human nature to want to stay employed and delude yourself into thinking you are serving the greater good when you're not necessarily doing so ...

For example, in my neighbourhood their is a school where the speed limit is 50km/h. The police hand out tickets like crazy all weekend long. They are not there when the kids are in school or just before or after school and at lunch. How is this a benefit to society?

Police should hand out traffice tickets. The money should be collected by the government and put into the big pot. The government should propose levels of policing and funding for police as their platform based on the consensus of policy makers and citizens and we should elect the government which represents our beliefs...

Police using the proceeds of the Traffic Act is undemocratic, a conflict of interest and does not necessarily lead to safer roads (if that is the objective of traffic laws).

I'll probably get ticketed like crazy from here on in ....

Multi-Dimensional Currency

Another idea that I have been thinking about for a long time... this current recession motivates this blog.

Currency is currently one dimensional, that is, it has one value and all currencies are just a variation of the same thing apart by only a factor. My idea, which I haven't completely worked out, is that currency should be multi-dimensional, that is, it has different value dimension depending on the usage domain.

In Canada, we might consider healthcare a form of alternate currency for citizens. Healthcare is universally free for the most part, we contribute via employee benefits to additional health coverages like dentists, drug and eyecare etc. The value of the services received for some individuals who use the system is much more than what it would cost them individually and for some it is less. We could have a bank account that we tap into which has monies in it placed by the government which is the equal balance each citizen receives annually for our tax contributions. This account might have atleast $160 billion (total cost of health care)/30 million (approx population of Canada) or just over $5,000. We could see the money come and go as we use the healthcare system but we cannot use it for anything else. Availability of healthcare is independent of whether there is a recession or not.

Another example of alternate currencies are loyalty rewards that we accumulate. Again they have value but are only usable with certain rules attached to their usage.

My first thought is to attach all your loyalty points from all the programs you participate in, your healthcare dollars and the actual cash you have to one account or institution, maybe the bank of your choice or a central bank. If this happened you could see the multiple dimensions of the currency that you have and you could have one card that you could use to access this value in the appropriate situations.

Another thought. Like healthcare can we decouple certain rights from the fluctuations of the economy. Shouldn't we all have the right to food, shelther, clothing in addition to healthcare if we are productive members of society. I don't mean this in a communist sense, that everyone has exactly the same but we all get a minimum based on the fact that we go to work everyday or contribute in some other important way like raising children etc. Some of has who have the fortune to earn more can use the additional earnings to purchase additional food, shelther, clothing, leisure etc. with the additional income earned. I don't advocate innefficent government running with these additional programs but I don't have an alternative at this moment. These additional accounts would have to be funded from the productivity of all participants in the economy.

We could add these currency dimensions to our one currency account, which now contains, healthcare dollars, loyalty points, food account, shelter account, clothing account, your cash and credit. Only your cash and credit are exposed to market fluctuations.

We could also add tax credits or government loyalty points as another dimension to our one currency account (see my previous post on zero sum currencies). Now we would have all these different dimensions of value available in one place and maybe the actual real money value of all these dimensions accessible in that one place also accessible by one smart card in our wallet.

What would happen if we allowed individuals to move value from one dimension to another, if their behaviour led to excess in some dimensions and shortfalls in another. The rules for moving from one dimension to another to be defined maybe by government programs and their incentives to promote behaviour that leads to government budget efficiencies or by loyalty programs or corporations. This would be like turning some portion of government and corporate spending to individuals and allowing individuals to arbitrage government programs, loyalty points, cash etc. in unique and unimaginable ways that would drive orders of efficiency not otherwise predictable at an aggregate level. The government portion is only be possible first by aggregating the value of all of the productivity of the economies participants and then distributing it back to the individuals with programs and incentives for behaviour which allow all the value to be utilized. Loyalty programs do essentially the same thing, aggregate consumer spending and use the aggregate buying power to create rewards not individually achievable and then distributing the buying power back to individuals. The loyalty program seems to create incredible value out of nothing. (It creates value out of the inefficiency of retail). We can similarly get much more value out of government programs because guess what ... they're inefficient too.

The benefits,

1) Basic human needs decoupled from business fluctuations. Recessions hurt dumb businesses not the individuals who show up to work everyday and have very little say in what businesses do.

2) Opportunities to create significant economic efficiencies by creating one multi-dimensional currency accessible (or at least viewable) in one place attached to one smart card which allow individuals to move value across dimensions and allow individuals to seek the personalized efficiencies. Let's call this economic micro optimization ...

3) a minor benefit would be to eliminate all the cards (loyalty, credit, debit, ...) and replace them with one.

I haven't worked out all the details but hopefully presented enough of an idea or ideas that if some smart individual or economist hopped on this, they could either rip it apart or run with it.

I'll continue to think about it and work out the details to the best of my ability ... and maybe launch another information based business.

The power of information....

Zero Sum Currencies

An idea I have thought about for a long time...

First, what is a zero sum currency. One example I know of, but there many, is the futures market. The total return on commodities futures is zero but the market performs an important function, it lowers the risk for individuals or business who make a living digging commodities out of the ground or raising livestock or growing food. For example, a farmer needs to achieve a minimum sale price for his/her goods to make a profit. They would sell a futures contract for an amount of their goods (quality, lot size, delivery, etc.) all specified by the standardized futures contracts sold on the various markets that would guarantee them a price on a specified date.

Buyers of the contract would either be a business who uses the farmer's goods to make their product and would like to guarantee a specific purchase price that fits their business model or a speculator who makes money by predicting a price that will be higher than the farmers guarantee and can sell the contract for a profit before taking delivery. The commodity buyers and sellers use the instrument to reduce the impact of the commodity price on their business and view it as a cost of doing business, it is like insurance. Total wins and losses across all contracts sum to zero but importantly the risk for businesses depending on the commodity price is reduced.

Other examples are the Kyoto protocol or Carbon taxes which are types of zero sum currencies where the polluters pay the non-polluters with the overall objective of reducing greenhouse gases or carbon emissions.

The government should do this for its citizens for all the programs it plans to execute. For example, if the government's healthcare program is designed to improve healthcare quality and reduce costs it should publish metrics about the current state and what it wants to accomplish. Maybe the metrics would be number of visits, cost per visit and number of deaths by age band. The objective being to reduce number of visits, reduce cost per visit and reduce number of deaths by age band. (I am sure you could come up with better metrics but for arguments sake lets consider these)

The government knows how many visits to the healthcare system there are because here in Canada we show our healthcard everytime we get medical attention, and they know how much each visit costs because the healthcare provider bills the government for that visit. The government would also know deaths and ages of the deceased.

Now the goverment healthcare experts come up with their brilliant ideas to incent citizens to behave in a way to achieve the metrics. For example, the government might pay for gym memberships because being healthy is proactive and you will go to doctor less. How will they pay? They should create a zero sum currency, in the form of tax credits. Everytime you go to the gym you present your healthcard which the government can track. Perhaps the government also thinks that everyone should see the doctor once per year for a physical, something they can already track, because that would allow early detection or doctor recommended preventative activities to reduce your dependence on healthcare. The overall result being improved health and longer lifespans over time.

The citizens who believe in the governments plan will take advantage of the opportunity and earn tax credits, the citizens who don't believe or care will be taxed more to offset the cost of the tax credits. Lets say everyone climbs on the bandwagon and the tax increases don't offset the tax credits. In this case, the monies will then be taken from the existing healthcare budgets we are already taxed for reducing the overall cost of healthcare and putting money back in our pockets. If no one participates , i believe a less likely scenario, then we will all pay more tax which the government can use to put towards the current healthcare model. If the theory of gym visits, GP visits works we should see the death rate and cost of healthcare go down. The tax credit is a zero sum currency (tax credits offset by tax increases and reduction in healthcare funding equals zero) and healthcare gets cheaper and better. Obviously an overly simplified example but you get the idea.

Side benefits are transparency, we all know what the governments programs are and how they translate into results and what is expected of us. Transparency brings accountability and allows us to hold our politicians accountable (see previous posts on transparency). You can't improve what you don't measure (see previous posts on this topic)

This is kind of government loyalty program. Instead of using hard dollars you could use a different currency or points which the government translates into dollars at the end of the tax year based on how much tax revenue they have collected. Repeat this concept for all government programs.

Business should also follow. Some businesses already have loyalty programs but haven't set them up as zero sum currencies. Most companies cannot determine what loyalty is doing for them partly because there are no concrete objectives or they have difficulty measuring the results or they don't have clearly marketed customer incentives except to buy more spend more. Setting up a zero sum currency for loyalty is critical. I think the next generation of loyalty programs should align social responsiblity, corporate values and consumer behaviour to create more interesting and authentic experiences for consumers.

Check this ted.com talk out http://www.ted.com/index.php/talks/joseph_pine_on_what_consumers_want.html

I'll post more on loyalty programs later ...

My bottom line, zero sum currencies everywhere set up to improve the world in measurable, transparent and accountable ways. Analytics used to discover the levers that create the incentives and penalties.

Disruptive Innovation versus Continuous Improvement

I was at a seminar last week and one of the speakers shared a few great quotes which stimulated some thought and led to this. The two quotes were "Edison’s electric light did not come about from the continuous improvement of the candle…"-Oren Harari and "If I had asked people what they wanted, they’d have told me ‘a faster horse’!"-Henry Ford.

These quotes led me to consider what today's business leader's are looking for. Are they seeking the next great innovation that will change their business or are they trying to improve their existing core operational processes and get more efficient. I think the answer is both but I believe that continuous improvement is a lower strategic priority. I'll tell you what leads me to this belief and what its implications are for analytics professionals.

I have been working as a data mining/management consultant for 20 years helping large corporations get efficient by improving marketing efficiency, improve location selection, make less risky loans, eliminate fraud, improve manufacturing throughput - in a nutshell, trying to make analytics a strategic priority by marrying analytics and core operational process. Everytime a client has acted on the analytic recommendations (perhaps 1 out of 10 times) the results have been staggering in terms of return on investment. The other 9 out of 10 times, a great business case, clear findings and definitive recommendations have led to inaction. I have struggled to understand why many clients are not willing to act. In previous posts, I talked about transparency and willingness to act and these are some of the reasons. My current realization is that companies seek disruptive innovations because

1) A new innovation comes in an form easily recognizable to the experiences of a business executive, it's a new business model (internet channels) , a new product ( a light bulb, a car ), a new value proposition (combo meals at quick service) ... you get the idea

2) Innovations are exciting and understood by an organization's customers.

3) Continuous improvement requires analytic capabilities (not in abundance), sophisticated processes (measurement, tracking, forecasting), is not exciting (most retailers would rather have new iPod that sells like crazy than predictive forecasting models to improve merhcandise buying), not easy to understand, and invisible (Aside definition of invisible: bad direct marketing starts with a list of names randomly chosen, after building a data warehouse, hiring statisticians, building mathematical models, spending significant time and effort improving the process you end up with a list of names. The new and improved list looks no different than the original list. To prove it is better you have to set up and execute design of experiments and track business execution and generate sophisticated reporting to show on paper which list was better. Not obvious to say the least. The output of most analytic based continuous improvement activities leads to output which on the surface looks no different than what you started with).

4) In my experience people look for a silver bullet (disruptive innovation). Continuous improvement is not easy ... it is continuing to execute and be the "hamster on a treadmill"


My conclusion here is that analytics will always be a second class citizen to disruptive innovation unless we educate a new generation of business leaders and create new business management paradigms based on analytics. The implication for analytics professionals is that we will not get the strategic executive support required for significant corporate initiatives, we will continue to have to seek those few management individuals who understand the power of information and that the business intelligence/analytics industry will not achieve that status and promise that it should.

My belief is that continuous improvement through analytics will generate double digit bottom line gains (see my original Power of Information post) and that year over year continuous improvement will generate more return than disruptive innovation which happens on the order of a decade in most industries.

Companies who have focus on continuous improvement and use analytics to understand every operational facet of their business are those companies that will not only survive this recession but grow and gain the marketshare lost by their peers who have been looking for that silver bullet and hence have less insight into operations and less ability to make critical decisions about survival.

My recommendation is that companies should focus more on continuous improvement while keeping an eye out for disruptive innovation. If your organization is not analyzing your operations, start now. The business return from analytics can happen in weeks. It could be the difference between survival or failure.

The Recession

I just experienced a very enlightening lunch event that I was invited to. Refreshing first because I am usually the one taking clients and prospects to lunch, sporting events and the like but this time I was the prospect and was invited to hear the Honorable Minister of Finance, Jim Flaherty address the Toronto Board of Trade... a pleasant change.

I really had no expectations for the speech I was about to hear but was keenly interested in what one of our leaders at the centre of Canada's efforts to minimize the recession had to say.

I learned several things which I thought I would share. First some pertinent facts, according to Mr. Flaherty

1) Canada entered the recession with no deficit, contrary to many other countries in the G20
2) Canada's current national debt to GDP ratio is 29.1%, contrary to the United States in the mid 40's and Japan much higher (somewhere near or above 100% according to wikipedia)
3) Mr. Flaherty by running deficits for the next 3 to 4 years will increase the debt to GDP ratio to 32.2% and then immediately bring it back down in the years coming out of the recession
4) Canada is running a higher deficit than most G20 countries who all agreed to run a minimum of 2.0% to stabilize global economies. Canada has decided to run a 3.5% deficit. The United States will run a much higher deficit due to the dramatically worse situation there.
5) The planned deficits are not long term programs but short term funds that have a "use it or lose it clause" which will ensure that we do not end of up with long term debt from entrenched or structured programs.

All facts which I find very heartening and postive.

The Minister also shared some personal anecdotes and did some "cool" name dropping. Saying how in his conversations with fellow finance ministers, the US Secretary of the Treasury, the UK's Chancellor of the Exchequer that all of these folks expressed praise for how Canada's financial system and government were well run, stable and able to deal with the current situation. Minister Flaherty is also co-chairing a committee to deliver a report to the leaders of the G8 countries in early April. Again very positive to know that Canada is playing a significant role in sharing our views and progress with world leaders about this current situation.

Finally on a personal note he said [I'll quote him as best as I can] "I plan to take the Island Ferry to the airport to catch a flight on Porter air later. It reminds me of many of our parents and grandparents who came to this country by ship from Europe and other parts of the world. Their experiences and challenges in building a life in this country were much more difficult and trying than what we are currently experiencing. We are Canadians and we will persevere and come out of this recession strong and sooner than most" or something to that effect. I certainly echo that perspective. Although the current times are difficult most of us are still very fortunate to lead a great life here in Canada.

I found Mr. Flaherty to be very passionate and sincere about his desire to help Canadians and he conveyed a sense of his diligince and urgency to work with our local, provincial and federal leaders as well as his international counterparts to "cushion" Canadian families and businesses from the effects of the recession. This gave me great confidence.

I continue as ever to work hard to build my business and provide employment opportunities to our staff and contribute in a small way to our economy and country.

Data Preparation

My view is that feature extraction or variable creation or data preparation is the largest single activity which determines the quality of data mining results. I believe that feature extraction has an order of magnitude larger impact on the final result of a data mining exercise than does the algorithm utilized.

I have several motivations for data preparation

1) Current data mining tools cannot directly ingest the normalized data found in OLTP and data warehouse systems. They require the data to be "flattened" or "de-normalized", creating attributes related to the entities of interest (i.e. customers, locations, transactions, suppliers etc.) Aside - Algorithms which can mine the data in normalized format would be fabulous. Something I've thought about for years but haven't figured out yet.

2) The creation of variables creates context or relates an atomic transaction to the complete set of information about a particular entity. I.e. the fact that I had a credit card transaction at 2:47pm on February 16, 2009 for $10.11 at Starbucks is of little relevance or use in this form. relating this transaction to my entire transaction behaviour is the critical leap i.e. This transaction represents 4% of my week's spend, is 10% more than I spent at starbuck's last week, is 25% of my quarterly coffee purchases etc.

3) The selection of which features or variables to be created is guided my engineering background. Let us develop features which describe all the properties of a system. In engineering systems like air flow around and aircraft we measure temperature, pressure, density, 3 dimensional velocity, viscosity, etc. In a retail system, for example, if we are modeling customer behaviour we should ask what are the properties which describe customer shopping behaviour over a period of time. Recency, frequency, montary value, product mix, channel mix, consistency, trends, counters of/from major events, time series metrics etc. are many of the feature categories we can derive from a customers transaction series. We can create millions of features from a customers transaction series using this approach. ie. days since recent transaction, number of transactions by month, montary value of transactions by month. If we calculate this for the last 3 years by month, for 20 product categories we have 3x12X20x3 variables. If we further calculate month over month changes, year over year changeswe can further double the variables. Depending on the granularuity of time and product you can easily see how we can derive millions of variables. The largest number of variables I created for a data mining project was on the order of 25,000. I typically create ~1,000 attributes.

The exercise in step 3 results in a flattened data set where we have thousands of variables for each entity that we are modelling. This extensive set of variables provides a very rich description of an entities behaviour over a period of time. It provides the context about an entity which allow us to make intelligent conclusions, predictions or assumptions about that entity which are useful to improved decision making.

I apply the same variable creation method to all problems in all industries. The categories recency, frequency, monetary value, product mix, channel mix, trends, stochastics, counters etc. all have analogies in every problem I have countered. I prefer to describe a system and let the algorithms and feature selection approaches determine which features are important to the problem at hand, as opposed to hypothesizing a small set of features based on my knowledge.

In my experience, I can generate better data mining results through data preparation or feature extraction by building a rich description of a system's entities over time relating atomic transactions or events to the entity behaviour as a whole than by focusing on improving an algorithm's performance.

Comments welcome.