Why every domain expert MUST use data science
Amazing technology is surrounding us, sometimes we don’t even realize it. Whether it’s a smart-watch, a blood pressure control pill or a radar enabling self-driving capabilities, it is backed by years of orchestrated effort, led by multiple domain experts juggling between endless details and considerations, which are very hard to capture and analyze simultaneously.
This is getting highly challenging as systems are becoming more and more complex and the pressure to meet timelines and goals is intense. Companies are constantly looking to find the next groundbreaking medication or treatment, push the performance of sensors or just get that extra percentage in yield. Missing on performance or being late to market usually means losing it completely, sometimes amounting to billions of dollars.
In an effort to better handle these challenges, companies are looking to leverage the maturing ML and AI technologies across the life-cycle of products — from research and development through manufacturing. It is still used sparsely though as companies are struggling to understand how to best deploy these powerful techniques.
One approach is “let’s give it to a data expert”. In this case, designated data science teams are formed or an external AI-projects company is hired (in both scenarios hundreds of thousands of dollars and several months are invested). And although this offers real access to AI technology, the downsides of it are:
1. Usually, considerable time and effort are required from in-house experts for bringing the data science experts up to speed with the relevant background. Just think about the complexities and knowledge required for developing your product — there’s a good reason why companies invest so much in R&D (15–25% of their revenue).
2. R&D takes iterations, and by extension analyzing data. Data Science teams aren’t generally available on-demand, and response time is crucial for these fast-paced hi-tech companies.
The result is the common situation of data science teams working in an isolated environment, which is never a good idea (as nicely described in this Medium article ).
A more common approach is for the domain experts to do the data analysis by themselves, providing them with the agility and intimate interaction with the data they need.
But how can they leverage AI and ML without being data scientists?
The tools available today for domain experts require relatively high levels of proficiency in programming and ML methods (such as TensorFlow, PyTorch, and Keras), which most of them do not hold, and for a good reason. They are experts in their field and this is not their focus.
There are however a couple of simple steps that are key if you’d like to start using these methods to analyze your product’s data -
1. Try to formalize the question first. Sounds trivial, but you won’t believe how often people don’t start with this simple yet crucial step.
2. Identify the parameter or KPI. A KPI (key performance indicator) is what you’d like to predict, investigate or explain. It is usually the single most important thing that your product or process is optimized for. It may be the yield on your assembly line or the accuracy of your leakage detection sensor.
3. Identify all parameters that may affect it and gather the relevant data whether from experiments, testers or simulations. Some insights as to your data quality can even be detected at this point. Maybe you’ll realize you need to add some tests or clear up many data points that are not relevant for what you want to explore. Once you have this, you can start thinking about the best approach to analyze it.
Anyway, we can all agree that it would be great to have access to these technologies without being a data scientist. By definition, it means that a self-service platform, preferably code-free, is required.
Obviously, not an easy task — several challenges need to be overcome:
1. Robustness of such a platform to support diverse use cases
2. Explain-ability of the AI model and insights
3. Seamless integration with current tools and working methods of domain experts
However, once we achieve that, companies will be able to apply the best possible data analysis for every single decision, a long-sought-after goal.
One last thing
Any following thoughts or just curious about how we’re tackling this challenge @VantiAnalytics?
Drop us a line here