Is Machine Learning always a good choice?

When Is Machine Learning Appropriate?

‍

Problems Too Complex to Program

When the problem is complex, codifying all the solution rules is very difficult.

Example: Spam Detector

If you opt for a traditional programming solution for such a complex problem, you'll eventually end up with so many conditions and exceptions in your code that it will become very difficult to maintain.

Example: Credit Scoring

Hundreds of numbers represent each borrower: age, salary, account balance, frequency of previous payments, marital status, number of children, car make and year, mortgage balance, etc. Some of these numbers may be important in making decisions, some may be less important individually but become more significant when considered in combination with other numbers.

When the Problem Changes Over Time

Some problems evolve, so software must be regularly updated, leading to frustration, increased error chances, and significant testing and deployment burdens. A properly designed system powered by machine learning algorithms can adapt to new data patterns.

Problems Related to Perception

Example: Object detection on Video - Traditional approaches involve creating special filters to apply to images and using heuristics to identify objects. Convolutional neural networks can effectively solve such problems by identifying attributes from hundreds of thousands of images.

When the Problem Involves Phenomena That Are Not Well Analyzed but Observable

Example: Social Media Interactions or recommendation engines - Predicting individual preferences based on observed actions is challenging. However, machine learning models implemented in social networks can recommend content or connections based on actions like comments and likes.

Problems with a Small Number of Potential Solutions

Example: yes/no in spam classifiers or a single number in property valuations.

What are the red flags that may indicate machine learning is not suitable?

You can also look at it from the opposite angle.

Any change in system behavior compared to its past must be explainable.

Machine learning models adapt to new data, potentially changing their decision-making patterns over time. If these changes cannot be explained or anticipated, it might lead to trust issues or difficulties in managing the system.

Example: A fraud detection system that suddenly starts flagging previously unproblematic transactions could cause customer service issues if the reasons for these changes are not understandable.

The cost of a mistake made by the system is too high.

In critical applications where errors can have severe consequences, such as medical diagnosis or autonomous vehicle control, the risk associated with machine learning errors might be unacceptable.

Example: An incorrect machine learning-based diagnosis in a medical application could lead to improper treatment, endangering a patient's life.

Every action or decision made by the system must be explainable.

Modern machine learning models, especially those based on deep learning, often operate as "black boxes," making it difficult to understand how they arrived at a specific decision. This can be problematic in industries where regulatory compliance requires clear explanations for decisions, such as finance and healthcare.

Example: In a banking scenario, if a loan application is rejected by a machine learning model, regulators and customers may demand a detailed explanation of the factors that influenced this decision.

Obtaining the right data is too difficult or impossible.

Machine learning models require large amounts of high-quality, relevant data. If such data are scarce, expensive to acquire, or contain too much noise, building an effective model might not be feasible.

Example: Predicting rare events, such as specific types of machinery failures that have only occurred a handful of times across an industry, may not be possible due to a lack of sufficient training data.

The problem can be solved with traditional software development at lower costs.

If a problem is well-defined and does not require the pattern recognition capabilities of ML, traditional programming might be more efficient and cost-effective.

Example: A simple web application for booking appointments might not need machine learning to function effectively.

Simple heuristics would work quite well.

Sometimes, a simple rule-based system can perform just as well as, or even outperform, a complex machine learning model, especially in cases where the problem space is not very complex.

Example: A rule that flags any transaction over a certain amount for review could be just as effective for certain types of financial fraud detection as a complex machine learning system.

You can manually create logic that will produce the desired outcome for all possible input data combinations (if not too many).

If the number of input data combinations is manageable and the logic for handling them can be explicitly defined, a traditional algorithm might be more transparent and easier to manage.

Example: A vending machine's software, which only needs to manage a finite set of product selections and payments, can be effectively programmed without ML.

Conclusion  

Deciding whether machine learning is the right solution involves understanding both the nature of the problem and the capabilities of ML technologies. By considering the appropriateness and potential limitations of machine learning, businesses can make informed decisions about integrating these technologies into their digital transformation strategies.

‍