We worked with a leading provider of digital media player solutions in the digital signage industry to define an AI-powered shopper measurement system using computer-vision for in-store retail kiosks.
The goal was to create a computer-vision AI-powered shopper measurement system that could measure the real-time viewer demographics of digital content. This would enable targeting of product information appropriately to potential buyers.
Assessment
Our asssessment revealed the following:
- Mature offering in the media player business. The analytics platform is being maintained by a relatively small technical team.
- Leadership team is well versed with current trends in Digital Signage and recognizes the competitive pressure from Internet advertising, which provides detailed targeting analytics to guide ad campaigns.
- Digital media player market was fragmented and is getting commoditized — it is hard to differentiate. The hardware is cost sensitive and is built with off-the-shelf parts with low-cost offshore manufacturing.
- No in-house AI expertise but the team is aware of trends such as autonomous vehicles, which use digital cameras to understand the objects in the environment while preserving privacy.
- Opportunity to add high-margin features that would differentiate the product and enable the client to increase market share.
The assessment results are summarized below.
AI Business Problem and Value
The digital signage market is fragmented and is getting commoditized. By adding demographic capabilities, the software solution could be a lot more valuable to digital media buyers by enabling the targeting of content to the actual viewer demographic.
Customer interviews have indicated that existing customers are likely to upgrade to the new demographic features and there is potential to increase market share significantly by recruiting new customers. Also, competitors are beginning to tout audience measurement capabilities.
Management waa sensitive to protecting PII (Personally Identifiable Information). While the sensor is a digital camera, and the solution requires capturing images/video for information extraction, all images/videos must be protected with encryption and deleted within 24 hours. All analysis is to be done locally and no transfer of images or video will be made to the Internet.
AI Use Cases
We worked with the client to define the requirements on the proposed AI-based solution and identified the following potential AI use cases for a potential representative customer.
Client dashboards and reporting was enhanced to include:
- Metrics on which content was viewed by who, when, and for how long
- Demographic summaries to include gender and age range —with > 80% precision.
- We added mood metrics (happy, sad, unaffected, etc.) to measure engagement effectiveness.
Prediction Variables
After identifying the key use cases, we identified the key prediction variables that would be generated by a data-driven ML system:
- Predict face gender and age classification.
- Also determine customer interest through sentiment analysis of facial expressions.
Modeling Approach
The applicable AI technique for the desired use cases is the extraction of information from camera images, specifically the identification of images of faces from images and classification of the faces by demographics.
In collaboration with the client we designed a set of experiments with an off-the-shelf image classifier package trained with the annotated images.
The goal was to exceed 80% accuracy. In practice we were able to achieve 95% accuracy in detecting faces, 90% accuracy in gender identification and 85% accuracy in age-range identification.
We recommended the following architecture choices:
- Periodic refresh of models done in cloud with the model pushed to the media players.
- Actual prediction was done entirely within the media player.
- No change was needed to the media player except for the addition of an inexpensive USB webcam.
- A standardized field setup/calibration process was designed to enable the installers to correctly deploy the solution to the field and to do periodic validations of the predictions.
Data Strategy
The client did not have a training set to train and validate the apt classifiers.
We did a survey of data annotation vendors, conducted RFPs with potential vendors and contracted with an overseas vendor to receive over 100K images for evaluation.
A random set of images was examined by in house resources for validation.
Results
The client accepted the findings and rolled out the ML server in a product that was released in a private beta and is now available to its select clients for a full release testing.
Leadership team and AI readiness
Identified team gap expertise and presented internal training and external consultancy plan.
Data pipeline
Identified data providers to source annotated training sets for the ML pipeline. An apt data architecture was prototyped and presented to the client.
AI algorithms, models, architecture and deployment
Surveyed approaches for extracting the apt predictions. Created an experimental framework and set the goal of 80% accuracy. Ran a number of experiments to establish that this was achievable in practice.
Architecture and Deployment
AI MVP produced a working prototype of the product to validate with beta customers and to estimate the cloud resources needed to deliver the desired experience. Security issues were addressed, and we integrated media player metrics in the dashboard.
Product, Roadmap, Value Prop and Competitive
Further work was needed to help the client map the product roadmap to the new category.
