Published: Jul 23, 2022

giving video analytics applications the power to tackle unknowns

Put simply, video segmentation is a task where objects are identified in an image frame – essential for video analysis and comprehensive understanding. Its applications are numerous in today’s modern world, its technology ubiquitous - occupying areas of technology we are oblivious to – yet indispensable to our daily lives. Spread across industries such as transport, healthcare, agriculture, or retail - trickling down to specific areas such as workplace safety, cancer detection, security, traffic monitoring, or even predictive maintenance in construction.

As always, efforts to improve technology never cease, and while existing models work well for certain domains, such as the outdoors, where large amounts of manually annotated labelled data are available, problems do arise when these models encounter new domains such as poorly lit indoor spaces or industrial locations, with noticeable drops in performance.

Domain Adaptive Video Segmentation (DA – video segmentation) addresses this gap by adapting existing models to work in new and previously unseen domains.

In general, video segmentation algorithms are trained using supervised learning, meaning it requires annotation of large amounts of data. For certain use cases, it may be practically impossible to collect such amounts of labelled data - being prohibitively expensive - for example, in an industrial or office setting. What DA – video segmentation does is adopt models trained in one domain to different domains – allowing the use of the trained model on several types of new data, without time-consuming and expensive annotation processes. For example, by this method, a model trained for autonomous driving scene understanding on industrial setting can be adopted to be used for office scene understanding.

With immense potential in innumerable industries, video analytics has not penetrated well due to the lack of data – possibly sensitive data, or the expense associated with recording and labelling. In these cases, the DA – video segmentation technique offers an approach to train these models in new domains, providing a bridge to penetration in these industries. This economic factor cannot be ignored, and in many cases has proven to be a stumbling block in technology adoption.

Beginning way back in the 1990s, video analytics has gradually found a niche toward the betterment of society. With upcoming technology that has yet to become commonplace – driverless cars, self-driving aircraft, or delivery robots, this has become all but essential to future quality of life. Consider a visually impaired individual who may benefit from more accurate details of his or her surroundings, more accurate CT imaging with a patient, or laser precision in blood loss measurement during childbirth.

DA-video segmentation can be used for applications like workplace safety where domain-specific data is not easily available. For example, in office spaces, the model can be used to detect unsafe actions, near misses, overstacking of boxes, blockage of exit paths, overloading of electrical sockets, etc. Plans are in place to set up a platform based on DA-video segmentation for training as well as generating data where data is not easily available.

“NCS has developed and deployed high-performance Video Analytics solutions.  This innovation will enable these products and platforms to solve customers’ pain points and problem statements quickly with a low total cost of ownership”, says Dr. Sunil Sivadas, Practice Lead, NEXT Gen Tech.

*This article is derived from a joint innovation with researchers from Singtel Cognitive and AI Lab for Enterprises (SCALE) in Nanyang Technological University, Singapore.  

References

(1) Dayan Guan, Jiaxing Huang, Aoran Xiao, Shijian Lu

"Domain adaptive video segmentation via temporal consistency regularization"

IEEE/CVF International Conference on Computer Vision. 2021. (Link)