Why organisations are poised to embrace machine learning

Article by Snowflake senior sales engineer Rishu Saxena.

Once a technical novelty seen only in software development labs or enormous organisations, machine learning (ML) is poised to become an important tool for large numbers of Australian and New Zealand businesses.

Lured by promises of improved productivity and faster workflows, companies are investing in the technology in rising numbers. According to research firm Fortune Business Insights, the ML market will be worth US$117.19 billion by 2027.

 

Removal of roadblocks

Historically, ML was perceived to be an expensive undertaking that required massive upfront investment in people, as well as both storage and compute systems. Recently, many of the roadblocks that had been hindering adoption have now been removed.

One such roadblock was not having the right mindset or strategy when undertaking ML-related projects. Unlike more traditional software development, ML requires a flexible and open-ended approach. Sometimes it won’t be possible to assess the result accurately, and this could well change during deployment and preliminary use.

A second roadblock was the lack of ML automation tools available on the market. Thanks to large investments and hard work by computer scientists, the latest generation of auto ML tools are feature-rich, intuitive and affordable.

Those wanting to put them to work no longer have to undertake extensive data science training or have a software development background. Dubbed citizen data scientists, these people can readily experiment with the tools and put their ideas into action. 

The way data is stored and accessed by ML tools has also changed. Advances in areas such as cloud-based data warehouses and data lakes means an organisation can now have all its data in a single location. This means the ML tools can scan vast amounts of data relatively easily, potentially leading to insights that previously would have gone unnoticed.

The lowering of storage costs has further assisted this trend. Where an organisation may have opted to delete or archive data onto tape, that data can now continue to be stored in a production environment, making it accessible to the ML tools.

 

Steps to ML success

For those organisations looking to embrace ML and experience the business benefits it can deliver, there are a series of steps that should be followed:

Keep things simple

When starting with ML, don’t try to run before you walk. Begin with small, stand-alone projects that give citizen data scientists a chance to become familiar with the machine learning process, the tools, how they operate, and what can be achieved. Once this has been bedded down, it’s then easier to gradually increase the size and scope of activities.

Select appropriate tools

To start your ML journey, lean on the vast number of auto ML tools available on the market instead of using open source notebook based IDEs that require high levels of skills and familiarity with ML. 

There is an increasing number of ML tools on the market, so take time to evaluate options and select the ones best suited to your business goals. This will also give citizen data scientists required experience before any in-house development is undertaken.

Encourage staff involvement

ML is not something that has to be the exclusive domain of the IT department. Encourage the growth of a pool of citizen data scientists within the organisation who can undertake projects and share their growing knowledge.

Centralise the organisation’s data

To enable ML tools to do as much as possible, centralise the storage of all data in your organisation. One option is to make use of a cloud-based data platform that can be readily scaled as data volumes increase.

Monitor results

Once projects have been underway for some time, closely monitor the results being achieved. This will help to guide further investments and shape the types of projects that will be completed in the future.

Grow the organisation’s capabilities

Once knowledge and experience levels within the organisation have increased, consider tackling more complex projects. These will have the potential to add further value to the organisation and ensure that stored data is generating maximum value.

The potential for ML to support organisations, help them to achieve fresh insights, and streamline their operations is vast. By starting small and growing over time, it’s possible to keep costs under control while achieving benefits in a relatively short space of time.