Power BI For Data Science

Please login or click SIGN UP FOR FREE to create your PowerBIUG account to join this user group.
  • 1.  Why business fail at machine learning

    Top Contributor
    Posted Feb 13, 2019 02:26 PM
    Hey community,

    Allow me to share with you all this interesting blog written by 'Cassie Kozykov' that I find very interesting. Excuse me that this is not related to any of the Microsoft product, but I think it worth to take a read and get some context about what is happening in the industry.

    I must say that I have seen and experienced all this, as my background comes from finance and business, not from IT or Software.

    Here is the blog


    Why businesses fail at machine learning


    I'd like to let you in on a secret: when people say 'machine learning' it sounds like there's only one discipline here. There are two, and if businesses don't understand the difference, they can experience a world of trouble.

    A tale of two machine learnings

    Imagine hiring a chef to build you an oven or an electrical engineer to bake bread for you. When it comes to machine learning, that's the kind of mistake I see businesses making over and over.

    If you're opening a bakery, it's a great idea to hire an experienced baker well-versed in the nuances of making delicious bread and pastry. You'd also want an oven. While it's a critical tool, I bet you wouldn't charge your top pastry chef with the task of knowing how to build that oven; so why is your company focused on the equivalent for machine learning?

    Are you in the business of making bread? Or making ovens?

    These are different businesses! Unfortunately, too many machine learning projects fail because the team doesn't know whether they're supposed to build the oven, the recipe, or the bread.

    Machine learning research

    What they don't tell you is that all those machine learning courses and textbooks are about how to build ovens (and microwaves, blenders, toasters, kettles… the kitchen sink!) from scratch, not how to cook things and innovate with recipes.

    If you build machine learning algorithms, your focus is general purpose tools for others to use. (Kitchen appliances, if you prefer the analogy.) This business is called machine learning research and is typically done by places like academia or Google.

    When it comes to machine learning, many organizations are in the wrong business.

    You need quite a lot of education to be in this line of <g class="gr_ gr_544 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="544" data-gr-id="544">work,</g> because there's a long history here. Some popular algorithms have been around for centuries. For example, the method of least squares for <g class="gr_ gr_542 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-del replaceWithoutSep" id="542" data-gr-id="542">regression,</g> was published in 1805. Trust me, humanity has come a long way in 200 years.

    Today, there are some pretty sophisticated appliances out there… how are you going to build a better microwave if you don't know how this one works? Of <g class="gr_ gr_537 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation only-ins replaceWithoutSep" id="537" data-gr-id="537">course</g> you need all that immersive study! Becoming a researcher takes years and there's a good reason that the 101 course starts with the basics of calculus.

    Applied machine learning

    Most businesses just want to get cooking - to solve their business problems. They have no interest in selling microwaves, and yet often make the mistake of trying to build those appliances from scratch. It's hard to blame them - the current hype and education cycle dominantly focuses on research, instead of <g class="gr_ gr_545 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar only-ins doubleReplace replaceWithoutSep" id="545" data-gr-id="545">application</g>.

    If you're innovating with recipes, don't reinvent the wheel. Those microwaves exist already. You can get them for free from many places. And if setting up your own machine learning kitchen sounds like a chore, providers like Google Cloud Platform let you use theirs, complete with appliances, ingredients, and recipe books.

    If you're innovating in the kitchen, don't reinvent the wheel.

    For most applications, your team doesn't need to understand the mathematics of backpropagation in neural networks any more than a chef needs to know the wiring diagram for a microwave. But there's a lot that you do need to know if you're planning on running an industrial-scale kitchen, everything from curating your ingredients to checking that your dishes are good before you serve them.

    Which of these are you selling? The right team to hire depends on your answer.

    Crashing and burning with machine learning

    Unfortunately, I see a lot of businesses failing to get value from machine learning because they don't realize that the applied side is a very different discipline from the algorithms research side. Instead, leaders try to start their kitchens by hiring those folks who've been building microwave parts their whole lives but have never cooked a thing. What could possibly go wrong? If that works out, it's because you got lucky and accidentally hired an engineer who is a great chef.

    But usually, you're not lucky. There are only so many hours in one lifetime, and if you spend them learning how <g class="gr_ gr_553 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar only-del replaceWithoutSep" id="553" data-gr-id="553">a microwave</g> is wired, you've got fewer to devote to mastering the art of pastry or business. Where - and when! - would your PhD-trained artificial intelligence researcher have gained the skills required for applied machine learning? If you set your heart on the hybrid who's an expert in both, no wonder you're complaining about the talent shortage!


    If you try to start a restaurant by hiring folks who've been building microwave parts their whole lives but have never cooked a thing… what could possibly go wrong?


    Whom should you hire instead? Just like in an industrial kitchen, you need an interdisciplinary team with leadership that understands this space. Otherwise, projects fizzle and go nowhere.

    Hiring the right team for the job

    If you're selling cutting-edge appliances, hire researchers. If you're innovating in recipes to sell food at scale, you need people who figure out what's worth cooking / what the objectives are (decision-makers and product managers), people who understand the suppliers and the customers (domain experts and social scientists), people who can process ingredients at scale (data engineers and analysts), people who can try many different ingredient-appliance combinations quickly to generate potential recipes (applied ML engineers), people who can check that the quality of the recipe is good enough to serve (statisticians), people who turn a potential recipe into millions of dishes served efficiently (software engineers), people who keep the interdisciplinary team on track (project/program managers), and people who ensure that your dishes stay top notch even if the delivery truck brings you a ton of potatoes instead of the rice you ordered (reliability engineers).

    While these needn't be separate individuals, be sure you've got each role covered. And before you fling your rotten tomato at me for providing such an incomplete caricature, I'll freely admit that there's much more to say about hiring for applied machine learning. I've outsourced that to other posts, including this one.

    Speaking of outsourcing, if your team has tried all existing tools and can't make a recipe that meets your business objectives, it makes sense to think about adding skills in building appliances (researcher). Whether or not you hire that person to your permanent staff or outsource the job to an experienced algorithms research firm depends on the scale and maturity of your operation.

    Another reason to connect with researchers is that your prototype is so successful that using custom-built appliances makes sense at the massive scale you're lucky enough to operate at. (What a great problem to have!)

    Decision intelligence

    Experts should be talking about this, but they aren't. They're not owning up to the fact that <g class="gr_ gr_549 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar multiReplace" id="549" data-gr-id="549">there's</g> really two machine learnings here, and so the world is training people in building all these algorithms but not in using them.

    My team is working to fix that. We've created a new discipline to cover the applied side and we've already trained over 15,000 staff members in it. We're calling it decision intelligence engineering, and it spans all the applied aspects of machine learning and data science.

    To put it another way, if research machine learning is building microwaves and applied machine learning is using microwaves, decision intelligence engineering is using microwaves safely to meet your goals and using something else when you don't need a microwave.

    Good luck and have fun!

    When it comes to applied machine learning, the hardest part is knowing what you want to cook and how you plan to check it before you serve it to your customers. That part is actually not that hard - just don't forget to do it.

    As for the rest, solving business problems with machine learning is far easier than most people think. Those gleaming kitchens are waiting for you to <g class="gr_ gr_564 gr-alert gr_gramm gr_inline_cards gr_run_anim Grammar only-ins replaceWithoutSep" id="564" data-gr-id="564">come</g> play in them. Dive in as you would in a real kitchen. Start tinkering! Every time I meet someone who believes they need to take a traditional machine learning algorithms course - or, goodness! a whole degree - in order to get started, I can't help but imagine them refusing to use microwaves until they built one themselves. Don't fall for the lie that says you need a <g class="gr_ gr_565 gr-alert gr_gramm gr_inline_cards gr_run_anim Punctuation multiReplace" id="565" data-gr-id="565">PhD</g> to do amazing things with machine learning. Instead, what you really need is a bit of human creativity. Good luck and have fun!



    ------------------------------
    Pablo Moreno
    Data Intelligence Engineer
    Panama

    ------------------------------


  • 2.  RE: Why business fail at machine learning

    Top Contributor
    Posted Feb 13, 2019 03:38 PM
    Excellent share @Pablo Moreno, please also share the link of the article. This is something i have been trying to advocate but you know with all the hype of ML / AI people don't want to listen the other side of story. ​

    Wonderful write up @Cassie Kozyrkov

    Regards,


    ------------------------------
    Hasham Bin Niaz
    Sr. BI Consultant
    Karachi, Pakistan
    ------------------------------



  • 3.  RE: Why business fail at machine learning

    Top Contributor
    Posted Feb 13, 2019 03:40 PM
    Hmm got the link by searching through her profile; here is the original link for others to refer;

    https://hackernoon.com/why-businesses-fail-at-machine-learning-fbff41c4d5db

    ------------------------------
    Hasham Bin Niaz
    Sr. BI Consultant
    Karachi, Pakistan
    ------------------------------



  • 4.  RE: Why business fail at machine learning

    Top Contributor
    Posted Feb 13, 2019 04:48 PM
    Hello @Hasham Niaz

    Yes, sorry that I missed the link, but I see that you have it now.

    https://hackernoon.com/why-businesses-fail-at-machine-learning-fbff41c4d5db

    ------------------------------
    Pablo Moreno
    Data Intelligence Engineer
    Panama
    pmorenogonzalez@gmail.com
    ------------------------------