Data Scientists ... where are they?

The following blog was inspired by an article


http://www.kdnuggets.com/2013/12/unicorn-data-scientists-vs-data-science-teams-discussion.html

and was posted as a comment in response to it ....




I was intrigued by the article as I have been struggling to find practitioners to hire who have the required skills. I agree wholeheartedly with the venn diagrams in the article.
At makeplain we deliver improved outcomes to our clients derived using advanced analytic insights from operational data. I personally have been using the three skill sets plus business domain knowledge for 20 years to deliver operational recommendations to our customers.

I believe that all three skills are required in one individual to most effectively accomplish the task of creating actionable insights from large volumes of operational data. Individuals with these skills are few and far between, most having learned the skills from practical experience. A team of specialists cannot effectively deliver the required result in a timely fashion because the process is iterative requiring some back and forth between domains. The lack of cross-disciplinary knowledge on a team results in deliverables between team members having significant gaps. i.e. a database specialist cannot build a high quality analytic file to hand off to the mathematician and the mathematician cannot adequately describe the requirement to the database specialist as they do not understand the nuances of manipulating potentially terabytes of data. The mathematician who designs an algorithm does not know enough about the computer science implications of a large computation requiring parallelization and this results in impractical analytic processes. you get the idea.. Certainly there are individuals who have multi-disciplinary knowledge in more than one domain and teams containing these individuals get the best results. Nothing beats teams of individuals who have all three capabilities and that is what we try to train at makeplain.

Business domain knowledge is a secondary requirement if you have a team leader who brings this to the team but lone data scientists require business domain knowledge as well.
We have had most success hiring engineers as their background tends to be more general and they get moderate exposure to all three domains. Mathematicians and computer scientists tend to be more specialized (mathematicians being most specialized). We find it easier to teach more math to engineers than database skills to a mathematician. Certainly some individuals take out-of-program electives that give some broader exposure and we typically seek out these individuals as prospective hires.

I believe the current lack of individuals with the requisite skills is slowing the adoption of advanced analytics by corporations and has also been the barrier in the past. We certainly see a lot of current discussion (Big Data, Cloud Analytics, Data Science etc.) about the topic and I have seen more recent adoption by corporations than in the past 20 years. My personal belief is that there will never be enough data scientists to fill the business need in its current form; only a stable percentage of the population has the interest and requisite skills to pursue the necessary education. There is a minimum educational requirement to pursue this career. Only exceptional individuals could learn this on the job without quantitative post-secondary education. I think that in the near future we will automate the end to end process (across all three domains) of insight creation and business recommendation/execution and embed these processes into operational systems creating expert operational systems. In this possible future we won't require armies of non-existing data scientists and we can direct the realistically smaller number of practitioners to address analytic problems that are difficult or impossible to completely automate. Automated analytics delivering a 7 out of 10 grade, can be applied to the 10's of millions of "near-random" decisions that corporations operationally execute each and every day.

At makeplain we are trying to accomplish this level of automation and have made significant process towards this vision over the last 10 years made easier every day by Moore's law and other machine learning/database advancements brought to market by many great companies.

Significant adoption of advanced analytics by corporations and government used strategically and tactically to make daily operational decisions can create significant economic efficiency taking the pressure off our current debt-laden and slowly crumbling economies.

Comments welcome....

Act on Analytics Now! Grow your business in the next 20 days

Its been quite some time since I posted anything. As always I've been very busy with clients, which I am thankful for!

I wanted to share with you three very simple steps that you can use to grow your business profits by more than 10% starting right now and get results within 20 days ... There is no excuse to not implement advanced analytics (other than you don't know how to) because it works. It is smart business. I have been analyzing my clients businesses for close to 20 years and I have been able to deliver incredible returns when my clients have been willing to act.

Be willing to act and do something new... if you always do what you've always done you'll always get what you always got (I think I've said that before)

Your organization's willingness to do something literally right now is the most important step... It is possible in most industries to see the results of intelligent decision making and intelligent operations within 20 business days.

Step 1 ... Decide to improve your business within 20 days.

2. Analyze your business by segmenting it into a lower level of detail than you do today ... If one of your business segments at your lowest level of detail today has an annual performance metric of 10 it will always be true that at levels of detail lower than you measure some sub-segments will perform below 10 and some will perform above 10.

This is where advanced analytics comes in ... segment your business at a lower level of detail and find the under and over performing sub-segments. Look at how you spend operational dollars against the sub-segments. Do asset allocation!!! Spend more on the over-perfoming sub-segments and spend less on the under-performing segments. Re-Allocate your operational dollars without increasing overall spend. If you do this your segment performance will increase.  

Where you are currently over-allocating resources to an underperforming part of your business you are making bad decisions ...

Step 2 ... Look at your business in a lower level of detail and re-allocate your operational dollars to over-performing parts of your business and away from under-performing parts of your business.


3. Set a growth plan ... Build a resource re-allocation theory to achieve it... create a new operating plan which is the implementation of your theory... track the results of your new plan ... adjust your theory accordingly.

Choose your first cycle to be 20 business days long ... Several example theories could be ... the more direct marketing we do to our best customers and best prospects the more revenue we will generate ... or the more we increase our prices for products and services with low price elasticity and decrease our prices for related products with higher price elasticity the more revenue we will generate ... or the more fraud we detect and eliminate from claims that are more likely fraudulent the more profit we will generate .. or the better retail locations we open the more revenue we will generate ... or the better assortment of core products we stock the more revenue we will generate and the less overstock we will have ...

A theory identifies a sub-segment(s) and a re-allocation of resources and sets a quantitative target... The examples above were missing the quantitative elements. I'll repeat ... If we identify and investigate 100,000 claims that are suspected fraud (by using advanced analytics to identify) we will reduce the claims paid out against the 100,000 claims by 10%... If we identify our best customers using advanced analytics and we increase our marketing spend to promote offers to these customers by 500% (and accordingly reduce spend to our "not-best" customers) then we will increase best customer average transaction size by 10% and their transaction frequency by 10% resulting in the business growing by 20% ... etc. etc.

It is possible to identify 100,000 potentially fraudulent claims in 10 days ... it is possible to identify and understand your best customers  in 10 days ... it is possible to identify high and low price elastic products in 10 days ... etc. etc.

You can investigate 100 claims in the next 10 days ... send out emails to your best customers with attractive offers in the next 10 days ... it is possible to increase and decrease prices in the next 10 days ... notice I chose quick cycle time activities ... some activities like assortment plans or site location are longer cycle plans which take longer to execute and track although you can create the plan in 10 days ....


After 20 days maybe you reduced fraud by only 7% not 10% in the 100 claims you investigated ... great! ... adjust your operating plan to look at 10/7*100,000 claims instead if you want 10% reduction in fraud. After 10 days of price changes maybe revenue went up 3% instead of the 2% that you thought... great you don't have to make as many price changes to get your 2% growth. you get the idea. Adjust your plan.

Step 3 ... Set a target ... create a plan based on lower levels of  detail than you measure today in 10 days ... execute the new plan in the next 10 days... see what happened ... adjust your plan ... get your employees to do this all over your business again and again forever ....

Any naysayer is not thinking out of the box ... if you think you are leveraging advanced analytics today and your business is not dramatically improving then you're most likely not doing it or you are doing it wrong ...

COMPLETELY DO-ABLE IN ALL INDUSTRIES ... in the words of Nike ... "Just Do it"


I challenge you all ...

correct application of analytics = more understanding about your business = better business results

Let me know how it works out ..

Evaluating Change

When you are considering changing the way you operate your business apply the same critical assessment of the change you are considering to the status quo and you should quickly conclude that change is the correct course.

Incomplete improvements to the current process are always better than no improvement at all. Achieving all possible improvements is perfection and this is not possible.

Overcome the fear of change by lowering your expectation of perfection. I believe expectation of perfection is a natural human psychological barrier we put up to avoid change and stepping into what we perceive to be the unknown.

I experience this resistance from most of the clients and companies I work with when they are considering adding analytics to core operational processes. It is quite simple and true that better information leads to better decisions ... period.

Perfection does not exist so move on and step into change ... you can experience incredible business results in as little as a few weeks. Your willingness to change is the only barrier.

Money, Money, Money

Is $1 worth $1 dollar in all situations... is money linear?

I believe that money is highly non-linear and our 1000's of years old concept of money needs an overhaul.

Some questions ....

If I am a millionaire how do I value an incremental dollar offered to me versus a homeless person?

If I am a millionaire and I need to get out of a parking garage and my bill is $10 but I only have $9 in cash, what is $1 worth?

If I want to spend $100 million on televisions will the cost per TV be the same as if I spent $3,000 on TV's?

If I love books and hate music would spending $1000 on books be the same as spending $1000 on CDs to paying a $1000 tax bill?

Is earning $1000 at work equivalent to winning a $1000 lottery?

Are 100,000 loyalty program points which I can buy a $1000 TV with worth $1000 in cash? Are 100,000 loyalty program points which I can buy a $1000 plane ticket with worth a different amount of cash than the 100,000 points I can buy the TV with?

Is a $10 million offer to buy TVs from a $10 billion dollar retailer worth the same as a $10 million offer to buy the same TVs from the $100 million dollar manufacturer of the TVs?

Why do rich celebrities get a $40,000 gift bag at the academy awards for free? Are those gifts actually worth $40,000 to the giver(s) or the receiver?

Why is there an adage which says "It is much hard to make your first million than your second"?

Why is it that the more money you have the more you get for free?

Is paying $100,000 for products/services to a large corporation with cultural beliefs about salary, compensation, worth the same as paying a different consulting firm with different beliefs $100,000 for the same products/services?

The value of money is a function many things... incremental utility of the buyer, incremental utility of the seller, the marketplace, time, the situational utility, quality, belief systems, what the economic value of your currency is plus many other variables...

I don't believe that supply, demand, price or traditional macro/micro economic concepts can accurately account for the situations described above.

Big questions ... how much inefficiency exists in our economic, political and social systems because we have such a simple concept of money? Can we create new value out of this inefficiency that would increase GDP, reduce unemployment etc.? Can we change macroeconomic properties by optimizing the micro-utility of trillions of transactions.

Can we create a new concept of money that allows us to maximize our utility better than we currently do? or can we broaden the definition of a transaction that recognizes more than just our current system of money that will impact our utility and on aggregate will impact macroeconomic properties like GDP.


Yes, Yes, Yes ... we can use information technology and analytics and the willingness to change to do so.

Better use of information can create value out of seemingly nothing. It is there for the taking by any individual, corporation or organization.

Change is Everything

Technology implementations, statistical analysis, discovering actionable insights, changing business processes, developing information use cases, measuring financial results, modeling business performance... All easy things to accomplish. I have been involved in dozens of projects where we have delivered some or all of these things. Clients have been on the precipice of dramatic business gains... yet many analytic projects do not deliver as promised.

Getting organizations and people to change is very difficult and is the real challenge when delivering strategic analytic solutions. By strategic analytic solutions, I mean solutions which could double a company's bottom-line performance.

Overlooking change management issues will doom projects to the dustbin.

The types of analytic solutions in the marketplace, fraud detection for insurance and banking, pricing optimization and assortment planning for retail, forecasting for retail and manufacturing, retail customer loyalty require organizations to be able to create, assess and act upon the output of sophisticated analytics and reporting and to be able to measure the impact of your actions, learn from those actions and adjust your operating plan.

There are two types of change required: surgical change to business processes that are impacted by information and strategic change which requires an organization to be change its management style and mindset to become analytically driven and willing to experiment.

Surgical change while narrow in scope can be painful. Employees who execute business processes may not have the skills to assess, review and take action based on analytic insight and business intelligence reporting. Employees who were otherwise well regarded and successful in their positions may find themselves floundering to adapt to the changes. With information based business execution comes transparency and accountability, further compounding employees willingness to accept change.

Changing the management mindset in an organization is even more difficult. When decision making has been largely based on intuition, or "how we've always done it" and the impact of decisions has not been measured, it becomes a monumental task to get employees, middle managers and executives to change the way they make decisions and learn how to make better decisions. I don't know many companies who track their decision performance!

Recognizing these issues before embarking on strategic analytic initiatives will improve the likelihood for success. Setting appropriate expectations for time to results should be set. Implementing a change management program and plan is mandatory. Crawl, walk and run is highly recommended. Expect the change management issues to long outlive the technology implementations.

Seek outside help as change from within is very difficult. Select a change management method that accomodates both surgical change and organization paradigm shift.

Analytic technology implementations take 2 to 4 months. If you can overcome the change issues, I think every organization is 12 to 18 months away from doubling and more their bottom-line profits. Start now...


Comments welcome

Living in the Moment

Short blog this morning .... something a little different

I have had a life long love affair with the game of tennis, having played competitively and recreationally. After many years of practice I have finally begun to understand the game. Over the last 5 years my game has improved significantly, and even out of shape as I am now, I am playing better than I ever have at my competitive peak. The key learning is to live in the moment, and forget about outcomes. When I was younger, I focused so much on the score, worrying about what people would think if I won or lost. What I learned was that you cannot achieve an outcome if you don't love executing the moments and actions you need to get there.

I have learned to not worry about scores and outcomes. I focus on executing each shot, enjoying a well placed forehand, a good volley, a backhand slice. I can now reach a level of calmness, not having external thoughts and worries, when I play that allows me to perform to my potential which then achieves the outcomes I have so desired.

The learning also applies to the work world. Focusing on your day to day activities, learning to love those and doing the best you can in the moment, is the sure fire way to success. I have been working hard to bring lessons learned from recreation to my day to day business life. I'll keep you posted.

Appliance as a Business Solution

We've been working on developing applianced based business solutions.... (keyword business)

what is an appliance according to me (the first list is technology focused, if you're a business person read-on beyond this list...)

1. Hardware
2. Database Software
3. ETL Software
4. Reporting Software
5. Advanced Statistical Analysis Software
6. Systems Integration Assets (Data Models, ETL Code, Standard Reports, Automated Statistical Modeling)
7. Business Process Workflow
8. Financial Model
9. Remote Operational Management

What does the business appliance do ... according to me

1. It improves a specific business process (i.e. increase revenue or reduces cost) by using advanced analytics.
2. the output of the appliance is integrated into a business process with defined user workflows that generate operational business activity
3. The output of the appliance also generates a P&L and Balance Sheet actuals & forecast which resulted or will result from the operational business activity executed by using the appliance

What type of business processes are amenable to an appliance model ... according to me again

1. Processes where a small set of decision metrics are obvious to act upon
2. Processes that generate operational data

Some Examples

1. fraud detection... output metric is a scored claim where the score value indicates whether or not the claim should be investigated. The result of claims investigation is less claims paid out. 10 to 15% of claims are typically fraudulent, eliminating 50% of fraud will more than double bottom-line results
2. Price optimization for retail ... output is a recommended price on a product ... when executed price optimization can deliver up to 5% sales and margin increase which for a typical retailer will double bottom line profits...
3. Forecasting for retail and manufacturing ... output is recommended inventory levels ... when executed for rertail will reduce stockouts and markdowns ... when executed for manufacturing will increase inventory turns or capital tied up in inventory... both can double bottom-line results.
4. Customer Loyalty ... output is recommended marketing campaigns to specific customer sets ... when executed will increase frequency, transaction revenue, transaction gross margin, and customer lifetime which can add 5% to top line or gross margin which again would double a typical retailers bottom line results...
5. A huge list of other business opportunities which achieve dramatic results

from a technology perspective what distinguishes a business appliance from software which could deliver the above 4 examples ... according to me

1. The appliance is pre-built and avoids a 12-18 months systems integration project. Most large integration projects fail
2. The appliance has business workflow which integrates output of the appliance into existing operational business process which ensures that output is acted upon.
3. The appliance has financial model output which integrates into P&L and Balance Sheet tracking and forecasting to ensure the top-line and bottom-line results are achieved.
4. The appliance is a black-box to customers ... it has a data interface to receive data and output windows (workflow, reporting and financial output) requiring no administration except a power and network cable.
5. the appliance's day to day function is remotely administered to ensure data is loaded properly, data is backed up, statistic models are tuned, financial results are achieved...
6. The appliance can be delivered as a hosted or SaaS model...
7. The appliance can be virtualized...
8. The appliance can be deployed in 20-30 days from order
9. The client will achieve business results in 21-31 days from order

Why would a company consider an appliance .... according to me

1. Avoids risky system integration projects
2. Delivers results quickly
3. Is business focused with clear integration point to business process and shareholder/public financial reporting
4. The appliance deliverable is a business result ... not a technology system ...the appliance is a means to an end.
5. Corporate Information Technology departments are not good at delivering strategic results... The appliance requires minimal IT involvement
6. Corporate IT departments can learn to "appliance-ize" solutions
7. Technology vendors (hardware and software) do not deliver on the sales proposition... the appliance does.

My company plans to roll-out many appliance solutions in the coming months ....

What do you think? Sound interesting? Feedback welcome ....