Client

PMY Group is an independent technology company. They specialize in the provision of technology solutions to the infrastructure, public venues, and major events globally. PMY supports many other Australian stadia, including the Sydney Cricket Ground (Sydney), Etihad (now Marvel) Stadium (Melbourne), and Optus Stadium (Perth), to embrace technology and enhance their venues. They have also been engaged by more than 100 clients globally, involving more than $400m of smart technology transactions.

The Challenge

PMY approached us with a challenge of real-time visual inference of counting the number of people present in each court at the US Open tournament.

The Process

Lotus Labs developed a custom object detection model that detects crowds and inferences the number of people in the crowd. We created a model that generated density estimation maps (presence of the trained object per pixel) and efficiently annotated the training data provided by PMY. We developed the model using PyTorch.

The model structure combines two modules. The first module initially extracts the input image features (lines, angles, etc.) and later captures as many combinations as possible to learn the image’s abstractions. The second module extracts and learns the textures/patterns of the image based on the information extracted previously.

The output of the model is an estimated density matrix with a size proportional to the input image. The higher values in the matrix represent the areas where the model recognized the objects it was trained to estimate. Typically, if we overlay the density matrix as a heatmap on the original image, we should see a halo around each object decreasing in intensity from the

center to the edges.

The Results

We successfully deployed the padme.ai platform at US Open 2021. The model accurately counted the number of people present at the stadium with a mean absolute error of 9.07.

‍