After having spent over two decades developing and applying AI and cognitive technologies — from early-stage startups to Fortune 50 companies— I have learned that successful initiatives balance technology and business principles to build superior products that solve their customer problems and react quickly as the customer needs evolve.
AI/ML techniques have since been democratized by the ready availability of frameworks that enable anyone to extract insights from their datasets using state-of-the-art algorithms on infinitely scalable cloud infrastructure.
Over the years, I’ve also seen plenty of failed AI projects for all kinds of reasons: from insufficient or low-value data; to team dynamics and AI-readiness; to not having a clear direction for the product or company. While the AI strategy is key — so is the business strategy and the alignment with a clear vision.
Using my years of expertise, I focus on setting the right AI strategy.
Adding in traditional AI/ML support via a systems design approach.
During my PhD research, which was funded by Intel at the University of Illinois at Urbana-Champaign, I explored how system design principles could be applied to enable desktop computers to solve supercomputer class problems.
The outcome was a set of CPU design elements to optimize computing over large datasets.
I was exposed early on to the design principles espoused by Uhlrich and Eppinger in the book Product Design and Development.
I learned that the notion of a system includes the customer needs, design for manufacturing, prototyping and industrial design.
At HP, I was given an exciting high-visibility AI project to manage. I collaborated with The New York Times on pioneering their content personalization and targeting system. Other AI initiatives I led included an approach for a digital printing press, together with audience engagement systems to improve engagement of customers with digital media. These were system-design projects where insights extracted from large datasets were applied to solve the apt customer problems.
After a couple of decades at HP I was itching to leave and head into startupland.
I worked at emerging startups in CTO roles to build AI-based SaaS products. I learned the value of Agile methods and of Lean Design principles as expressed in the book The Lean Startup by Eric Ries.
A key attribute that AI startups have is a relentless focus on continual experimentation to extract viable signal from large datasets by applying the appropriate modeling techniques, which are improving at a rapid pace.
I became super-focused on extracting signal from data!
Determining the signal in your data via experimentation.
AI-infused projects differ from other software development projects in that extracting actionable signal from datasets is a key success factor.
Some differences include:
Traditional software is code-centric
- Traditional software is prone to logical bugs in the code
- Traditional software updates are handled by Devops. Testing processes are well understood and organizations are able to react quickly to issue patch releases.
Traditional AI/ML projects are data-centric.
- Traditional AI projects rely on algorithms to extract the training patterns from data. As new data is used to train the models, there can be issues with model drift, or even correctness.
- The emerging discipline of MLOps is responsible for deploying model updates in collaboration with the data scientists.
In Traditional AI projects, model variations can significantly affect the behavior of the product. Also, a change in one part of the system can propagate elsewhere. In AI projects modeling is hard, but scaling up is even harder.
After years of living first-hand of these differences, we have developed an agile approach to data-centric AI-first projects that includes the signal data experimentation phase of an AI/ML project.
The process looks like this:
The main steps for Traditional AI/ML data signal experimentation are as follows:
- Gather the dataset: Ideally these are proprietary to you, i.e., “first party” data. However, it might make sense to also consider public or 3rd party datasets that can be used to add value to your product.
- Research the latest literature: The field of AI is evolving rapidly. Research labs, including those at Google, Facebook, and Microsoft are constantly publishing new algorithms and approaches. It is worthwhile to do a literature search and summarize the approaches others are using in solving similar problems.
- Choose an appropriate ML algorithm: Again there are a number of frameworks and algorithms available to solve a particular problem. It is important to narrow these down to a few candidates.
- Frame experiments: At this stage you make assertions about how many experiments to run and for how long, which hyperparameters to modify, how to split the datasets between training sets and evaluation sets, what the success criteria are etc.
- Run Experiments: You will run a large number of experiments in order to assess the viability of signal extraction.
- Evaluate results and present to stakeholders: Based on the results conduct a review with stakeholders. If the results are promising you may need to iterate the above steps to improve the approach and make recommendations.
I’ve built out this process in my experience both at large companies and at startups multiple times.
For example, I led a startup team in building an audience targeting system for app downloads. The initial design of the algorithms was iterative as shown above. Also, as we scaled up the product, we chose to grow the training sets by about three orders of magnitude to improve our predictions.
Several times in our product journey we had to take a step back and rerun the above process from the ground up. This led us to redesign significant aspects of our data pipeline to apply different frameworks and cloud mechanisms as were appropriate to the overall system. The above approach was critical to maintaining our sanity as the system dimensions changed with increasing scale.
Bringing it together: Enhancing product strategy with traditional AI/ML
System design principles continue to be critical to successful delivery of game-changing products to serve your customer needs.
The following questions of the system must be considered in an AI-first project where your company can lead:
- Can your data be leveraged to build predictions that are useful to our customer?
- An experimental mindset helps you be agile in identifying available insight, aka “signal”, in your data and deliver it quickly within the product
- Will your product be sufficiently differentiated from the competition?
- In fact can your product be the dominant player in its own category?
- How can your company react quickly to rapidly-changing customer needs, and to emerging competitive pressures, to manage your business actively towards successful outcomes?
Businesses have tried to enhance their products with AI and there are lots of reasons for why they failed. Your AI-led technology efforts need to combine with the business strategy.
Great companies combine the two. So should you.
