Something completely different...
Does anyone else think it is a huge conflict of interest to allow police to use the proceeds of applying the traffic act, that is they get to keep the monies collected from handing out traffic tickets? Isn't this the violation of some basic legal prinicple?
Police are incented to hand out more tickets so they can stay employed and hire more policemen to hand out tickets. It is human nature to want to stay employed and delude yourself into thinking you are serving the greater good when you're not necessarily doing so ...
For example, in my neighbourhood their is a school where the speed limit is 50km/h. The police hand out tickets like crazy all weekend long. They are not there when the kids are in school or just before or after school and at lunch. How is this a benefit to society?
Police should hand out traffice tickets. The money should be collected by the government and put into the big pot. The government should propose levels of policing and funding for police as their platform based on the consensus of policy makers and citizens and we should elect the government which represents our beliefs...
Police using the proceeds of the Traffic Act is undemocratic, a conflict of interest and does not necessarily lead to safer roads (if that is the objective of traffic laws).
I'll probably get ticketed like crazy from here on in ....
Tuesday, March 24, 2009
Monday, March 23, 2009
Multi-Dimensional Currency
Another idea that I have been thinking about for a long time... this current recession motivates this blog.
Currency is currently one dimensional, that is, it has one value and all currencies are just a variation of the same thing apart by only a factor. My idea, which I haven't completely worked out, is that currency should be multi-dimensional, that is, it has different value dimension depending on the usage domain.
In Canada, we might consider healthcare a form of alternate currency for citizens. Healthcare is universally free for the most part, we contribute via employee benefits to additional health coverages like dentists, drug and eyecare etc. The value of the services received for some individuals who use the system is much more than what it would cost them individually and for some it is less. We could have a bank account that we tap into which has monies in it placed by the government which is the equal balance each citizen receives annually for our tax contributions. This account might have atleast $160 billion (total cost of health care)/30 million (approx population of Canada) or just over $5,000. We could see the money come and go as we use the healthcare system but we cannot use it for anything else. Availability of healthcare is independent of whether there is a recession or not.
Another example of alternate currencies are loyalty rewards that we accumulate. Again they have value but are only usable with certain rules attached to their usage.
My first thought is to attach all your loyalty points from all the programs you participate in, your healthcare dollars and the actual cash you have to one account or institution, maybe the bank of your choice or a central bank. If this happened you could see the multiple dimensions of the currency that you have and you could have one card that you could use to access this value in the appropriate situations.
Another thought. Like healthcare can we decouple certain rights from the fluctuations of the economy. Shouldn't we all have the right to food, shelther, clothing in addition to healthcare if we are productive members of society. I don't mean this in a communist sense, that everyone has exactly the same but we all get a minimum based on the fact that we go to work everyday or contribute in some other important way like raising children etc. Some of has who have the fortune to earn more can use the additional earnings to purchase additional food, shelther, clothing, leisure etc. with the additional income earned. I don't advocate innefficent government running with these additional programs but I don't have an alternative at this moment. These additional accounts would have to be funded from the productivity of all participants in the economy.
We could add these currency dimensions to our one currency account, which now contains, healthcare dollars, loyalty points, food account, shelter account, clothing account, your cash and credit. Only your cash and credit are exposed to market fluctuations.
We could also add tax credits or government loyalty points as another dimension to our one currency account (see my previous post on zero sum currencies). Now we would have all these different dimensions of value available in one place and maybe the actual real money value of all these dimensions accessible in that one place also accessible by one smart card in our wallet.
What would happen if we allowed individuals to move value from one dimension to another, if their behaviour led to excess in some dimensions and shortfalls in another. The rules for moving from one dimension to another to be defined maybe by government programs and their incentives to promote behaviour that leads to government budget efficiencies or by loyalty programs or corporations. This would be like turning some portion of government and corporate spending to individuals and allowing individuals to arbitrage government programs, loyalty points, cash etc. in unique and unimaginable ways that would drive orders of efficiency not otherwise predictable at an aggregate level. The government portion is only be possible first by aggregating the value of all of the productivity of the economies participants and then distributing it back to the individuals with programs and incentives for behaviour which allow all the value to be utilized. Loyalty programs do essentially the same thing, aggregate consumer spending and use the aggregate buying power to create rewards not individually achievable and then distributing the buying power back to individuals. The loyalty program seems to create incredible value out of nothing. (It creates value out of the inefficiency of retail). We can similarly get much more value out of government programs because guess what ... they're inefficient too.
The benefits,
1) Basic human needs decoupled from business fluctuations. Recessions hurt dumb businesses not the individuals who show up to work everyday and have very little say in what businesses do.
2) Opportunities to create significant economic efficiencies by creating one multi-dimensional currency accessible (or at least viewable) in one place attached to one smart card which allow individuals to move value across dimensions and allow individuals to seek the personalized efficiencies. Let's call this economic micro optimization ...
3) a minor benefit would be to eliminate all the cards (loyalty, credit, debit, ...) and replace them with one.
I haven't worked out all the details but hopefully presented enough of an idea or ideas that if some smart individual or economist hopped on this, they could either rip it apart or run with it.
I'll continue to think about it and work out the details to the best of my ability ... and maybe launch another information based business.
The power of information....
Currency is currently one dimensional, that is, it has one value and all currencies are just a variation of the same thing apart by only a factor. My idea, which I haven't completely worked out, is that currency should be multi-dimensional, that is, it has different value dimension depending on the usage domain.
In Canada, we might consider healthcare a form of alternate currency for citizens. Healthcare is universally free for the most part, we contribute via employee benefits to additional health coverages like dentists, drug and eyecare etc. The value of the services received for some individuals who use the system is much more than what it would cost them individually and for some it is less. We could have a bank account that we tap into which has monies in it placed by the government which is the equal balance each citizen receives annually for our tax contributions. This account might have atleast $160 billion (total cost of health care)/30 million (approx population of Canada) or just over $5,000. We could see the money come and go as we use the healthcare system but we cannot use it for anything else. Availability of healthcare is independent of whether there is a recession or not.
Another example of alternate currencies are loyalty rewards that we accumulate. Again they have value but are only usable with certain rules attached to their usage.
My first thought is to attach all your loyalty points from all the programs you participate in, your healthcare dollars and the actual cash you have to one account or institution, maybe the bank of your choice or a central bank. If this happened you could see the multiple dimensions of the currency that you have and you could have one card that you could use to access this value in the appropriate situations.
Another thought. Like healthcare can we decouple certain rights from the fluctuations of the economy. Shouldn't we all have the right to food, shelther, clothing in addition to healthcare if we are productive members of society. I don't mean this in a communist sense, that everyone has exactly the same but we all get a minimum based on the fact that we go to work everyday or contribute in some other important way like raising children etc. Some of has who have the fortune to earn more can use the additional earnings to purchase additional food, shelther, clothing, leisure etc. with the additional income earned. I don't advocate innefficent government running with these additional programs but I don't have an alternative at this moment. These additional accounts would have to be funded from the productivity of all participants in the economy.
We could add these currency dimensions to our one currency account, which now contains, healthcare dollars, loyalty points, food account, shelter account, clothing account, your cash and credit. Only your cash and credit are exposed to market fluctuations.
We could also add tax credits or government loyalty points as another dimension to our one currency account (see my previous post on zero sum currencies). Now we would have all these different dimensions of value available in one place and maybe the actual real money value of all these dimensions accessible in that one place also accessible by one smart card in our wallet.
What would happen if we allowed individuals to move value from one dimension to another, if their behaviour led to excess in some dimensions and shortfalls in another. The rules for moving from one dimension to another to be defined maybe by government programs and their incentives to promote behaviour that leads to government budget efficiencies or by loyalty programs or corporations. This would be like turning some portion of government and corporate spending to individuals and allowing individuals to arbitrage government programs, loyalty points, cash etc. in unique and unimaginable ways that would drive orders of efficiency not otherwise predictable at an aggregate level. The government portion is only be possible first by aggregating the value of all of the productivity of the economies participants and then distributing it back to the individuals with programs and incentives for behaviour which allow all the value to be utilized. Loyalty programs do essentially the same thing, aggregate consumer spending and use the aggregate buying power to create rewards not individually achievable and then distributing the buying power back to individuals. The loyalty program seems to create incredible value out of nothing. (It creates value out of the inefficiency of retail). We can similarly get much more value out of government programs because guess what ... they're inefficient too.
The benefits,
1) Basic human needs decoupled from business fluctuations. Recessions hurt dumb businesses not the individuals who show up to work everyday and have very little say in what businesses do.
2) Opportunities to create significant economic efficiencies by creating one multi-dimensional currency accessible (or at least viewable) in one place attached to one smart card which allow individuals to move value across dimensions and allow individuals to seek the personalized efficiencies. Let's call this economic micro optimization ...
3) a minor benefit would be to eliminate all the cards (loyalty, credit, debit, ...) and replace them with one.
I haven't worked out all the details but hopefully presented enough of an idea or ideas that if some smart individual or economist hopped on this, they could either rip it apart or run with it.
I'll continue to think about it and work out the details to the best of my ability ... and maybe launch another information based business.
The power of information....
Friday, March 20, 2009
Zero Sum Currencies
An idea I have thought about for a long time...
First, what is a zero sum currency. One example I know of, but there many, is the futures market. The total return on commodities futures is zero but the market performs an important function, it lowers the risk for individuals or business who make a living digging commodities out of the ground or raising livestock or growing food. For example, a farmer needs to achieve a minimum sale price for his/her goods to make a profit. They would sell a futures contract for an amount of their goods (quality, lot size, delivery, etc.) all specified by the standardized futures contracts sold on the various markets that would guarantee them a price on a specified date.
Buyers of the contract would either be a business who uses the farmer's goods to make their product and would like to guarantee a specific purchase price that fits their business model or a speculator who makes money by predicting a price that will be higher than the farmers guarantee and can sell the contract for a profit before taking delivery. The commodity buyers and sellers use the instrument to reduce the impact of the commodity price on their business and view it as a cost of doing business, it is like insurance. Total wins and losses across all contracts sum to zero but importantly the risk for businesses depending on the commodity price is reduced.
Other examples are the Kyoto protocol or Carbon taxes which are types of zero sum currencies where the polluters pay the non-polluters with the overall objective of reducing greenhouse gases or carbon emissions.
The government should do this for its citizens for all the programs it plans to execute. For example, if the government's healthcare program is designed to improve healthcare quality and reduce costs it should publish metrics about the current state and what it wants to accomplish. Maybe the metrics would be number of visits, cost per visit and number of deaths by age band. The objective being to reduce number of visits, reduce cost per visit and reduce number of deaths by age band. (I am sure you could come up with better metrics but for arguments sake lets consider these)
The government knows how many visits to the healthcare system there are because here in Canada we show our healthcard everytime we get medical attention, and they know how much each visit costs because the healthcare provider bills the government for that visit. The government would also know deaths and ages of the deceased.
Now the goverment healthcare experts come up with their brilliant ideas to incent citizens to behave in a way to achieve the metrics. For example, the government might pay for gym memberships because being healthy is proactive and you will go to doctor less. How will they pay? They should create a zero sum currency, in the form of tax credits. Everytime you go to the gym you present your healthcard which the government can track. Perhaps the government also thinks that everyone should see the doctor once per year for a physical, something they can already track, because that would allow early detection or doctor recommended preventative activities to reduce your dependence on healthcare. The overall result being improved health and longer lifespans over time.
The citizens who believe in the governments plan will take advantage of the opportunity and earn tax credits, the citizens who don't believe or care will be taxed more to offset the cost of the tax credits. Lets say everyone climbs on the bandwagon and the tax increases don't offset the tax credits. In this case, the monies will then be taken from the existing healthcare budgets we are already taxed for reducing the overall cost of healthcare and putting money back in our pockets. If no one participates , i believe a less likely scenario, then we will all pay more tax which the government can use to put towards the current healthcare model. If the theory of gym visits, GP visits works we should see the death rate and cost of healthcare go down. The tax credit is a zero sum currency (tax credits offset by tax increases and reduction in healthcare funding equals zero) and healthcare gets cheaper and better. Obviously an overly simplified example but you get the idea.
Side benefits are transparency, we all know what the governments programs are and how they translate into results and what is expected of us. Transparency brings accountability and allows us to hold our politicians accountable (see previous posts on transparency). You can't improve what you don't measure (see previous posts on this topic)
This is kind of government loyalty program. Instead of using hard dollars you could use a different currency or points which the government translates into dollars at the end of the tax year based on how much tax revenue they have collected. Repeat this concept for all government programs.
Business should also follow. Some businesses already have loyalty programs but haven't set them up as zero sum currencies. Most companies cannot determine what loyalty is doing for them partly because there are no concrete objectives or they have difficulty measuring the results or they don't have clearly marketed customer incentives except to buy more spend more. Setting up a zero sum currency for loyalty is critical. I think the next generation of loyalty programs should align social responsiblity, corporate values and consumer behaviour to create more interesting and authentic experiences for consumers.
Check this ted.com talk out http://www.ted.com/index.php/talks/joseph_pine_on_what_consumers_want.html
I'll post more on loyalty programs later ...
My bottom line, zero sum currencies everywhere set up to improve the world in measurable, transparent and accountable ways. Analytics used to discover the levers that create the incentives and penalties.
First, what is a zero sum currency. One example I know of, but there many, is the futures market. The total return on commodities futures is zero but the market performs an important function, it lowers the risk for individuals or business who make a living digging commodities out of the ground or raising livestock or growing food. For example, a farmer needs to achieve a minimum sale price for his/her goods to make a profit. They would sell a futures contract for an amount of their goods (quality, lot size, delivery, etc.) all specified by the standardized futures contracts sold on the various markets that would guarantee them a price on a specified date.
Buyers of the contract would either be a business who uses the farmer's goods to make their product and would like to guarantee a specific purchase price that fits their business model or a speculator who makes money by predicting a price that will be higher than the farmers guarantee and can sell the contract for a profit before taking delivery. The commodity buyers and sellers use the instrument to reduce the impact of the commodity price on their business and view it as a cost of doing business, it is like insurance. Total wins and losses across all contracts sum to zero but importantly the risk for businesses depending on the commodity price is reduced.
Other examples are the Kyoto protocol or Carbon taxes which are types of zero sum currencies where the polluters pay the non-polluters with the overall objective of reducing greenhouse gases or carbon emissions.
The government should do this for its citizens for all the programs it plans to execute. For example, if the government's healthcare program is designed to improve healthcare quality and reduce costs it should publish metrics about the current state and what it wants to accomplish. Maybe the metrics would be number of visits, cost per visit and number of deaths by age band. The objective being to reduce number of visits, reduce cost per visit and reduce number of deaths by age band. (I am sure you could come up with better metrics but for arguments sake lets consider these)
The government knows how many visits to the healthcare system there are because here in Canada we show our healthcard everytime we get medical attention, and they know how much each visit costs because the healthcare provider bills the government for that visit. The government would also know deaths and ages of the deceased.
Now the goverment healthcare experts come up with their brilliant ideas to incent citizens to behave in a way to achieve the metrics. For example, the government might pay for gym memberships because being healthy is proactive and you will go to doctor less. How will they pay? They should create a zero sum currency, in the form of tax credits. Everytime you go to the gym you present your healthcard which the government can track. Perhaps the government also thinks that everyone should see the doctor once per year for a physical, something they can already track, because that would allow early detection or doctor recommended preventative activities to reduce your dependence on healthcare. The overall result being improved health and longer lifespans over time.
The citizens who believe in the governments plan will take advantage of the opportunity and earn tax credits, the citizens who don't believe or care will be taxed more to offset the cost of the tax credits. Lets say everyone climbs on the bandwagon and the tax increases don't offset the tax credits. In this case, the monies will then be taken from the existing healthcare budgets we are already taxed for reducing the overall cost of healthcare and putting money back in our pockets. If no one participates , i believe a less likely scenario, then we will all pay more tax which the government can use to put towards the current healthcare model. If the theory of gym visits, GP visits works we should see the death rate and cost of healthcare go down. The tax credit is a zero sum currency (tax credits offset by tax increases and reduction in healthcare funding equals zero) and healthcare gets cheaper and better. Obviously an overly simplified example but you get the idea.
Side benefits are transparency, we all know what the governments programs are and how they translate into results and what is expected of us. Transparency brings accountability and allows us to hold our politicians accountable (see previous posts on transparency). You can't improve what you don't measure (see previous posts on this topic)
This is kind of government loyalty program. Instead of using hard dollars you could use a different currency or points which the government translates into dollars at the end of the tax year based on how much tax revenue they have collected. Repeat this concept for all government programs.
Business should also follow. Some businesses already have loyalty programs but haven't set them up as zero sum currencies. Most companies cannot determine what loyalty is doing for them partly because there are no concrete objectives or they have difficulty measuring the results or they don't have clearly marketed customer incentives except to buy more spend more. Setting up a zero sum currency for loyalty is critical. I think the next generation of loyalty programs should align social responsiblity, corporate values and consumer behaviour to create more interesting and authentic experiences for consumers.
Check this ted.com talk out http://www.ted.com/index.php/talks/joseph_pine_on_what_consumers_want.html
I'll post more on loyalty programs later ...
My bottom line, zero sum currencies everywhere set up to improve the world in measurable, transparent and accountable ways. Analytics used to discover the levers that create the incentives and penalties.
Wednesday, March 18, 2009
Disruptive Innovation versus Continuous Improvement
I was at a seminar last week and one of the speakers shared a few great quotes which stimulated some thought and led to this. The two quotes were "Edison’s electric light did not come about from the continuous improvement of the candle…"-Oren Harari and "If I had asked people what they wanted, they’d have told me ‘a faster horse’!"-Henry Ford.
These quotes led me to consider what today's business leader's are looking for. Are they seeking the next great innovation that will change their business or are they trying to improve their existing core operational processes and get more efficient. I think the answer is both but I believe that continuous improvement is a lower strategic priority. I'll tell you what leads me to this belief and what its implications are for analytics professionals.
I have been working as a data mining/management consultant for 20 years helping large corporations get efficient by improving marketing efficiency, improve location selection, make less risky loans, eliminate fraud, improve manufacturing throughput - in a nutshell, trying to make analytics a strategic priority by marrying analytics and core operational process. Everytime a client has acted on the analytic recommendations (perhaps 1 out of 10 times) the results have been staggering in terms of return on investment. The other 9 out of 10 times, a great business case, clear findings and definitive recommendations have led to inaction. I have struggled to understand why many clients are not willing to act. In previous posts, I talked about transparency and willingness to act and these are some of the reasons. My current realization is that companies seek disruptive innovations because
1) A new innovation comes in an form easily recognizable to the experiences of a business executive, it's a new business model (internet channels) , a new product ( a light bulb, a car ), a new value proposition (combo meals at quick service) ... you get the idea
2) Innovations are exciting and understood by an organization's customers.
3) Continuous improvement requires analytic capabilities (not in abundance), sophisticated processes (measurement, tracking, forecasting), is not exciting (most retailers would rather have new iPod that sells like crazy than predictive forecasting models to improve merhcandise buying), not easy to understand, and invisible (Aside definition of invisible: bad direct marketing starts with a list of names randomly chosen, after building a data warehouse, hiring statisticians, building mathematical models, spending significant time and effort improving the process you end up with a list of names. The new and improved list looks no different than the original list. To prove it is better you have to set up and execute design of experiments and track business execution and generate sophisticated reporting to show on paper which list was better. Not obvious to say the least. The output of most analytic based continuous improvement activities leads to output which on the surface looks no different than what you started with).
4) In my experience people look for a silver bullet (disruptive innovation). Continuous improvement is not easy ... it is continuing to execute and be the "hamster on a treadmill"
My conclusion here is that analytics will always be a second class citizen to disruptive innovation unless we educate a new generation of business leaders and create new business management paradigms based on analytics. The implication for analytics professionals is that we will not get the strategic executive support required for significant corporate initiatives, we will continue to have to seek those few management individuals who understand the power of information and that the business intelligence/analytics industry will not achieve that status and promise that it should.
My belief is that continuous improvement through analytics will generate double digit bottom line gains (see my original Power of Information post) and that year over year continuous improvement will generate more return than disruptive innovation which happens on the order of a decade in most industries.
Companies who have focus on continuous improvement and use analytics to understand every operational facet of their business are those companies that will not only survive this recession but grow and gain the marketshare lost by their peers who have been looking for that silver bullet and hence have less insight into operations and less ability to make critical decisions about survival.
My recommendation is that companies should focus more on continuous improvement while keeping an eye out for disruptive innovation. If your organization is not analyzing your operations, start now. The business return from analytics can happen in weeks. It could be the difference between survival or failure.
These quotes led me to consider what today's business leader's are looking for. Are they seeking the next great innovation that will change their business or are they trying to improve their existing core operational processes and get more efficient. I think the answer is both but I believe that continuous improvement is a lower strategic priority. I'll tell you what leads me to this belief and what its implications are for analytics professionals.
I have been working as a data mining/management consultant for 20 years helping large corporations get efficient by improving marketing efficiency, improve location selection, make less risky loans, eliminate fraud, improve manufacturing throughput - in a nutshell, trying to make analytics a strategic priority by marrying analytics and core operational process. Everytime a client has acted on the analytic recommendations (perhaps 1 out of 10 times) the results have been staggering in terms of return on investment. The other 9 out of 10 times, a great business case, clear findings and definitive recommendations have led to inaction. I have struggled to understand why many clients are not willing to act. In previous posts, I talked about transparency and willingness to act and these are some of the reasons. My current realization is that companies seek disruptive innovations because
1) A new innovation comes in an form easily recognizable to the experiences of a business executive, it's a new business model (internet channels) , a new product ( a light bulb, a car ), a new value proposition (combo meals at quick service) ... you get the idea
2) Innovations are exciting and understood by an organization's customers.
3) Continuous improvement requires analytic capabilities (not in abundance), sophisticated processes (measurement, tracking, forecasting), is not exciting (most retailers would rather have new iPod that sells like crazy than predictive forecasting models to improve merhcandise buying), not easy to understand, and invisible (Aside definition of invisible: bad direct marketing starts with a list of names randomly chosen, after building a data warehouse, hiring statisticians, building mathematical models, spending significant time and effort improving the process you end up with a list of names. The new and improved list looks no different than the original list. To prove it is better you have to set up and execute design of experiments and track business execution and generate sophisticated reporting to show on paper which list was better. Not obvious to say the least. The output of most analytic based continuous improvement activities leads to output which on the surface looks no different than what you started with).
4) In my experience people look for a silver bullet (disruptive innovation). Continuous improvement is not easy ... it is continuing to execute and be the "hamster on a treadmill"
My conclusion here is that analytics will always be a second class citizen to disruptive innovation unless we educate a new generation of business leaders and create new business management paradigms based on analytics. The implication for analytics professionals is that we will not get the strategic executive support required for significant corporate initiatives, we will continue to have to seek those few management individuals who understand the power of information and that the business intelligence/analytics industry will not achieve that status and promise that it should.
My belief is that continuous improvement through analytics will generate double digit bottom line gains (see my original Power of Information post) and that year over year continuous improvement will generate more return than disruptive innovation which happens on the order of a decade in most industries.
Companies who have focus on continuous improvement and use analytics to understand every operational facet of their business are those companies that will not only survive this recession but grow and gain the marketshare lost by their peers who have been looking for that silver bullet and hence have less insight into operations and less ability to make critical decisions about survival.
My recommendation is that companies should focus more on continuous improvement while keeping an eye out for disruptive innovation. If your organization is not analyzing your operations, start now. The business return from analytics can happen in weeks. It could be the difference between survival or failure.
Friday, March 6, 2009
The Recession
I just experienced a very enlightening lunch event that I was invited to. Refreshing first because I am usually the one taking clients and prospects to lunch, sporting events and the like but this time I was the prospect and was invited to hear the Honorable Minister of Finance, Jim Flaherty address the Toronto Board of Trade... a pleasant change.
I really had no expectations for the speech I was about to hear but was keenly interested in what one of our leaders at the centre of Canada's efforts to minimize the recession had to say.
I learned several things which I thought I would share. First some pertinent facts, according to Mr. Flaherty
1) Canada entered the recession with no deficit, contrary to many other countries in the G20
2) Canada's current national debt to GDP ratio is 29.1%, contrary to the United States in the mid 40's and Japan much higher (somewhere near or above 100% according to wikipedia)
3) Mr. Flaherty by running deficits for the next 3 to 4 years will increase the debt to GDP ratio to 32.2% and then immediately bring it back down in the years coming out of the recession
4) Canada is running a higher deficit than most G20 countries who all agreed to run a minimum of 2.0% to stabilize global economies. Canada has decided to run a 3.5% deficit. The United States will run a much higher deficit due to the dramatically worse situation there.
5) The planned deficits are not long term programs but short term funds that have a "use it or lose it clause" which will ensure that we do not end of up with long term debt from entrenched or structured programs.
All facts which I find very heartening and postive.
The Minister also shared some personal anecdotes and did some "cool" name dropping. Saying how in his conversations with fellow finance ministers, the US Secretary of the Treasury, the UK's Chancellor of the Exchequer that all of these folks expressed praise for how Canada's financial system and government were well run, stable and able to deal with the current situation. Minister Flaherty is also co-chairing a committee to deliver a report to the leaders of the G8 countries in early April. Again very positive to know that Canada is playing a significant role in sharing our views and progress with world leaders about this current situation.
Finally on a personal note he said [I'll quote him as best as I can] "I plan to take the Island Ferry to the airport to catch a flight on Porter air later. It reminds me of many of our parents and grandparents who came to this country by ship from Europe and other parts of the world. Their experiences and challenges in building a life in this country were much more difficult and trying than what we are currently experiencing. We are Canadians and we will persevere and come out of this recession strong and sooner than most" or something to that effect. I certainly echo that perspective. Although the current times are difficult most of us are still very fortunate to lead a great life here in Canada.
I found Mr. Flaherty to be very passionate and sincere about his desire to help Canadians and he conveyed a sense of his diligince and urgency to work with our local, provincial and federal leaders as well as his international counterparts to "cushion" Canadian families and businesses from the effects of the recession. This gave me great confidence.
I continue as ever to work hard to build my business and provide employment opportunities to our staff and contribute in a small way to our economy and country.
I really had no expectations for the speech I was about to hear but was keenly interested in what one of our leaders at the centre of Canada's efforts to minimize the recession had to say.
I learned several things which I thought I would share. First some pertinent facts, according to Mr. Flaherty
1) Canada entered the recession with no deficit, contrary to many other countries in the G20
2) Canada's current national debt to GDP ratio is 29.1%, contrary to the United States in the mid 40's and Japan much higher (somewhere near or above 100% according to wikipedia)
3) Mr. Flaherty by running deficits for the next 3 to 4 years will increase the debt to GDP ratio to 32.2% and then immediately bring it back down in the years coming out of the recession
4) Canada is running a higher deficit than most G20 countries who all agreed to run a minimum of 2.0% to stabilize global economies. Canada has decided to run a 3.5% deficit. The United States will run a much higher deficit due to the dramatically worse situation there.
5) The planned deficits are not long term programs but short term funds that have a "use it or lose it clause" which will ensure that we do not end of up with long term debt from entrenched or structured programs.
All facts which I find very heartening and postive.
The Minister also shared some personal anecdotes and did some "cool" name dropping. Saying how in his conversations with fellow finance ministers, the US Secretary of the Treasury, the UK's Chancellor of the Exchequer that all of these folks expressed praise for how Canada's financial system and government were well run, stable and able to deal with the current situation. Minister Flaherty is also co-chairing a committee to deliver a report to the leaders of the G8 countries in early April. Again very positive to know that Canada is playing a significant role in sharing our views and progress with world leaders about this current situation.
Finally on a personal note he said [I'll quote him as best as I can] "I plan to take the Island Ferry to the airport to catch a flight on Porter air later. It reminds me of many of our parents and grandparents who came to this country by ship from Europe and other parts of the world. Their experiences and challenges in building a life in this country were much more difficult and trying than what we are currently experiencing. We are Canadians and we will persevere and come out of this recession strong and sooner than most" or something to that effect. I certainly echo that perspective. Although the current times are difficult most of us are still very fortunate to lead a great life here in Canada.
I found Mr. Flaherty to be very passionate and sincere about his desire to help Canadians and he conveyed a sense of his diligince and urgency to work with our local, provincial and federal leaders as well as his international counterparts to "cushion" Canadian families and businesses from the effects of the recession. This gave me great confidence.
I continue as ever to work hard to build my business and provide employment opportunities to our staff and contribute in a small way to our economy and country.
Tuesday, February 17, 2009
Data Preparation
My view is that feature extraction or variable creation or data preparation is the largest single activity which determines the quality of data mining results. I believe that feature extraction has an order of magnitude larger impact on the final result of a data mining exercise than does the algorithm utilized.
I have several motivations for data preparation
1) Current data mining tools cannot directly ingest the normalized data found in OLTP and data warehouse systems. They require the data to be "flattened" or "de-normalized", creating attributes related to the entities of interest (i.e. customers, locations, transactions, suppliers etc.) Aside - Algorithms which can mine the data in normalized format would be fabulous. Something I've thought about for years but haven't figured out yet.
2) The creation of variables creates context or relates an atomic transaction to the complete set of information about a particular entity. I.e. the fact that I had a credit card transaction at 2:47pm on February 16, 2009 for $10.11 at Starbucks is of little relevance or use in this form. relating this transaction to my entire transaction behaviour is the critical leap i.e. This transaction represents 4% of my week's spend, is 10% more than I spent at starbuck's last week, is 25% of my quarterly coffee purchases etc.
3) The selection of which features or variables to be created is guided my engineering background. Let us develop features which describe all the properties of a system. In engineering systems like air flow around and aircraft we measure temperature, pressure, density, 3 dimensional velocity, viscosity, etc. In a retail system, for example, if we are modeling customer behaviour we should ask what are the properties which describe customer shopping behaviour over a period of time. Recency, frequency, montary value, product mix, channel mix, consistency, trends, counters of/from major events, time series metrics etc. are many of the feature categories we can derive from a customers transaction series. We can create millions of features from a customers transaction series using this approach. ie. days since recent transaction, number of transactions by month, montary value of transactions by month. If we calculate this for the last 3 years by month, for 20 product categories we have 3x12X20x3 variables. If we further calculate month over month changes, year over year changeswe can further double the variables. Depending on the granularuity of time and product you can easily see how we can derive millions of variables. The largest number of variables I created for a data mining project was on the order of 25,000. I typically create ~1,000 attributes.
The exercise in step 3 results in a flattened data set where we have thousands of variables for each entity that we are modelling. This extensive set of variables provides a very rich description of an entities behaviour over a period of time. It provides the context about an entity which allow us to make intelligent conclusions, predictions or assumptions about that entity which are useful to improved decision making.
I apply the same variable creation method to all problems in all industries. The categories recency, frequency, monetary value, product mix, channel mix, trends, stochastics, counters etc. all have analogies in every problem I have countered. I prefer to describe a system and let the algorithms and feature selection approaches determine which features are important to the problem at hand, as opposed to hypothesizing a small set of features based on my knowledge.
In my experience, I can generate better data mining results through data preparation or feature extraction by building a rich description of a system's entities over time relating atomic transactions or events to the entity behaviour as a whole than by focusing on improving an algorithm's performance.
Comments welcome.
I have several motivations for data preparation
1) Current data mining tools cannot directly ingest the normalized data found in OLTP and data warehouse systems. They require the data to be "flattened" or "de-normalized", creating attributes related to the entities of interest (i.e. customers, locations, transactions, suppliers etc.) Aside - Algorithms which can mine the data in normalized format would be fabulous. Something I've thought about for years but haven't figured out yet.
2) The creation of variables creates context or relates an atomic transaction to the complete set of information about a particular entity. I.e. the fact that I had a credit card transaction at 2:47pm on February 16, 2009 for $10.11 at Starbucks is of little relevance or use in this form. relating this transaction to my entire transaction behaviour is the critical leap i.e. This transaction represents 4% of my week's spend, is 10% more than I spent at starbuck's last week, is 25% of my quarterly coffee purchases etc.
3) The selection of which features or variables to be created is guided my engineering background. Let us develop features which describe all the properties of a system. In engineering systems like air flow around and aircraft we measure temperature, pressure, density, 3 dimensional velocity, viscosity, etc. In a retail system, for example, if we are modeling customer behaviour we should ask what are the properties which describe customer shopping behaviour over a period of time. Recency, frequency, montary value, product mix, channel mix, consistency, trends, counters of/from major events, time series metrics etc. are many of the feature categories we can derive from a customers transaction series. We can create millions of features from a customers transaction series using this approach. ie. days since recent transaction, number of transactions by month, montary value of transactions by month. If we calculate this for the last 3 years by month, for 20 product categories we have 3x12X20x3 variables. If we further calculate month over month changes, year over year changeswe can further double the variables. Depending on the granularuity of time and product you can easily see how we can derive millions of variables. The largest number of variables I created for a data mining project was on the order of 25,000. I typically create ~1,000 attributes.
The exercise in step 3 results in a flattened data set where we have thousands of variables for each entity that we are modelling. This extensive set of variables provides a very rich description of an entities behaviour over a period of time. It provides the context about an entity which allow us to make intelligent conclusions, predictions or assumptions about that entity which are useful to improved decision making.
I apply the same variable creation method to all problems in all industries. The categories recency, frequency, monetary value, product mix, channel mix, trends, stochastics, counters etc. all have analogies in every problem I have countered. I prefer to describe a system and let the algorithms and feature selection approaches determine which features are important to the problem at hand, as opposed to hypothesizing a small set of features based on my knowledge.
In my experience, I can generate better data mining results through data preparation or feature extraction by building a rich description of a system's entities over time relating atomic transactions or events to the entity behaviour as a whole than by focusing on improving an algorithm's performance.
Comments welcome.
Monday, May 19, 2008
Transparency, Change and Equality
I spent the last month or so meticulously planning my next bajillion blogs in a very structured manner and along the way realized that blogging should be a little more spontaneous otherwise I should just write a book. So I will be a little more random in the order of topics that I write about but they will still relate to my original premise. The power of information.
Many years ago a large fraction of people were paid or evaluated by piece work. You may have spent the day picking apples in an orchard and at the end of the day you received your pay based on the number of apples you picked. The harder you worked, the more you achieved. Our corporate world lost the ability to see into the labour of the individuals executing its business processes. In recent years the introducion of technology and workflow systems creates visbility.
If your objective is to improve your business processes then you must be able to measure the process at a granular level. You cannot improve what you cannot measure. If people are involved in the execution of a process then you need to be able to measure their productivity. Creating information systems that promote the execution of a particular workflow, track it at all levels and measure the result are key. Many such workflow systems exist, ERP systems, Sales Forces automation, etc. Management should know which sales person is making more cold calls than the next, they should know which programmer generates less lines of code than the average ... you get the idea. This is transparency, the ability to see a business process at work at all levels.
Whenever I have worked with clients to try to improve their business processes by providing intelligence based on careful analysis of the data created by the processes I have run into significant change management issues. We know that change is not easy, there is a whole branch 0f consulting dedicated to change management. Transparency, however, creates its own change issues. Creating information about a process, creates transparency and therefore creates accountability (i.e. if every one knew you did half the work of the person next to you, you might be held accountable in some fashion. Your peers might ostracize you, management might dock your pay or at least pass you over for raises etc. )People do not like to be measured in a transparent fashion because it forces them to be accountable on a daily basis for results or contribution delivered. In my experience, the biggest barrier to leveraging the power of information is the willingness to act. If a company wants to improve a business process it needs to "Just do it". There is no barrier to change, never has been, never will be. The fear of transparency is, I believe, a major barrier to information based business process improvement.
One way to reduce the fear associated with transparency and hence reduce the barriers to change is to level the playing field or create equality around certain information. We know that in a transaction or negotiation the party who has better information has a distinct advantage. Corporate systems are typically set up so that management has access to information that employees do not. When it comes to measuring the effectiveness of an employee's contribution to a business process we should include the contribution of all levels of management to that process. Perhaps if all levels of the corporation have equal visibility into a process and are therefore held equally accountable to the execution of a process then we may lower some of the barriers related to information based change.
So in summary, you can't change what you can't measure, creating transparency into a process allows you to measures its results, transparency creates acountability, equality in transparency for all stakeholders of a process reduces the fear associated with accountability and can remove one of the key barriers to change ... willingness to act.
These same concepts can be extended to political systems, social systems etc. not just the corporate world.
Look forward to your comments...
Many years ago a large fraction of people were paid or evaluated by piece work. You may have spent the day picking apples in an orchard and at the end of the day you received your pay based on the number of apples you picked. The harder you worked, the more you achieved. Our corporate world lost the ability to see into the labour of the individuals executing its business processes. In recent years the introducion of technology and workflow systems creates visbility.
If your objective is to improve your business processes then you must be able to measure the process at a granular level. You cannot improve what you cannot measure. If people are involved in the execution of a process then you need to be able to measure their productivity. Creating information systems that promote the execution of a particular workflow, track it at all levels and measure the result are key. Many such workflow systems exist, ERP systems, Sales Forces automation, etc. Management should know which sales person is making more cold calls than the next, they should know which programmer generates less lines of code than the average ... you get the idea. This is transparency, the ability to see a business process at work at all levels.
Whenever I have worked with clients to try to improve their business processes by providing intelligence based on careful analysis of the data created by the processes I have run into significant change management issues. We know that change is not easy, there is a whole branch 0f consulting dedicated to change management. Transparency, however, creates its own change issues. Creating information about a process, creates transparency and therefore creates accountability (i.e. if every one knew you did half the work of the person next to you, you might be held accountable in some fashion. Your peers might ostracize you, management might dock your pay or at least pass you over for raises etc. )People do not like to be measured in a transparent fashion because it forces them to be accountable on a daily basis for results or contribution delivered. In my experience, the biggest barrier to leveraging the power of information is the willingness to act. If a company wants to improve a business process it needs to "Just do it". There is no barrier to change, never has been, never will be. The fear of transparency is, I believe, a major barrier to information based business process improvement.
One way to reduce the fear associated with transparency and hence reduce the barriers to change is to level the playing field or create equality around certain information. We know that in a transaction or negotiation the party who has better information has a distinct advantage. Corporate systems are typically set up so that management has access to information that employees do not. When it comes to measuring the effectiveness of an employee's contribution to a business process we should include the contribution of all levels of management to that process. Perhaps if all levels of the corporation have equal visibility into a process and are therefore held equally accountable to the execution of a process then we may lower some of the barriers related to information based change.
So in summary, you can't change what you can't measure, creating transparency into a process allows you to measures its results, transparency creates acountability, equality in transparency for all stakeholders of a process reduces the fear associated with accountability and can remove one of the key barriers to change ... willingness to act.
These same concepts can be extended to political systems, social systems etc. not just the corporate world.
Look forward to your comments...
Subscribe to:
Posts (Atom)
