Opinion: How todays intelligent video applications can improve public safety

By Claude Kee, a director of ANV, supplier of the Aimetis intelligent IP video management software

Whether attempting to reduce crime in communities, defend the homeland, or deal with natural or manmade threats, public safety agencies at all levels are faced with significant challenges. To overcome them, many companies are implementing surveillance systems with intelligent video applications that automaticallyanalyze video data to collect useful information.

Intelligent video systems accomplish their tasks by using complex mathematical algorithms to extract moving objects or other recognisable forms from the recorded video, while filtering out irrelevant images or movement. Decision-making rules govern the data search to determine if the events recorded in the video are normal, or if they should be flagged as alerts tosecurity staff or police.

Historically, organisations viewed live video feeds or used simple motion detection applications to be alerted to potentially dangerous activities. But studies have shown that after 12 minutes of continuous video monitoring, anoperator will often miss up to 45 percent of screen activity.

After 22 minutes of viewing, up to 95 percent is overlooked. Meanwhile, manually looking through recorded video for incidents is so time consuming that many agencies actually view as little as 1 percent or less of the footage they record. So despite increasing their surveillance coverage, public safety agencies may actually experience greater security risks without additional technology support. And while motion detection applications are useful, they often produce so many false positives that real incidents may be ignored.

Fortunately, surveillance technology has evolved tremendously in recent years. Analytic technology today can filter out naturally caused pixel changes, such as lighting, cloud movements and blowing leaves, making applications more accurate. Sophisticated technology is used to compare pixels in a referenceframe to each subsequent frame in the video to determine which objects are moving.

After detecting motion, it categorises what it sees (a car, luggage, a human, etc.), then segments it by color, size and direction of motion. Once this metadata is gathered, the software can then perform a number of applications, such as people counting, crowd detection, object tracking and perimeter protection.

Perimeter Protection
Used primarily to protect buildings such as government or utility buildings during off hours, law enforcement agencies have also found success with perimeter protection in keeping individuals out of parks and other areas prone to criminal activity when closed. Perimeter breach detects instances of peoplecrossing a virtual, user-defined line at defined times in a more accurate manner than basic motion detection applications. This application can be used for areas directly outside of buildings, for example, such as doorways and loading docks, so that on-site security guards can be alerted immediately from central command if someone is detected after hours or on weekends.

Loitering
While it is normal for individuals to stop at times while walking, loitering applications can be used at building entry points, parking garages and other areas to alert public safety officials when someone seems to be loitering too long or in a vulnerable area. For example, one organisation uses cameras and intelligent video in a parking garage to determine when individuals stop outside of cars for more than 45 seconds.

When they do, an alert is sent to a roving guard so he can check out the circumstances. Loitering applications can also be combined with pan/tilt/zoom (PTZ) cameras that automatically turn to and focus on suspects to determine if there is an innocent situation or a break-in.

Left-Item Protection
In non-busy environments, “left-item” applications have proven useful in identifying when individuals leave objects on the floor and then walk away.Like loitering applications, alerts can notify roving guards, or PTZ cameras can first examine if the situation is potentially dangerous. These applications are often used in low- to medium-traffic public buildings or on quiet sidewalks outside of buildings of concern.

Integration with Traffic Systems
Several cities have implemented intelligent video systems that are integrated with their traffic systems to determine when individuals drive through red lights at dangerous intersections. Real-time alerts notify officers of the event, and video backup can be used to corroborate events and identify vehicles.

Forensics
While today’s intelligent video technology is very good, it isn’t perfect. As a result, you may be able to determine that an object was left behind, but not know whether it was simply an individual who put down a suitcase to purchase a soft drink or a terrorist who left behind an explosive device.

Likewise, it’s hard to know whether an individual loitering next to a car in a parking garage simply couldn’t locate his keys or was attempting to break in. Particularly when the event in question didn’t trigger an alert, you need human operators to review video to filter out innocent events, especially in very crowded areas where the accuracy of theanalytics is further reduced.

As law enforcement officers know, the first hours following a crime are themost important. Intelligent video’s forensic capabilities can be crucial in getting to the actual event quickly. Faced with 48 hours of video, intelligent video analytics may be able to parse out 20 qualifying events in a matter of seconds and then enable law enforcement officers to pin down the actual incident of interest in a matter of minutes.

Environmental Factors
While intelligent video applications are extremely advanced, six environmental factors will impact their success. As a result, working with trained and experienced integrators and suppliers to account for these issues is critical.

Camera Angle: The angle of the camera can influence several factors used in video analytics, including perspective, occlusion and segmentation of objects. As a result, the type of camera and its placement should be carefully selected.

Distance to Object: The object’s pixel size is an important element to video analytics. Most video analytics require a minimum pixel size (e.g., 15 x 15). Conversely, if the pixel sizes of the objects are too large, that too can distort the performance of the analytics (e.g. reflecting light into the camera). Thermal cameras perform over longer distances with higher accuracy.

Lighting Level: Lighting can influence video analytics in a few ways. First, for video analytics to detect objects, there must be some minimum light available (unless infrared or thermal cameras are being used). Second, abrupt changes in lighting (e.g., opening of doors) can cause false conclusions. Once again, consider the use of outside lighting, day/night cameras and/or infrared or thermal cameras, depending upon the application and environment.

Degree of Activity: The degree of activity or “busyness” of an environment has influence over the performance of video analytics. Generally, the higher the level of activity, the more false conclusions will be drawn by the video analytics algorithm. That’s why loitering and left-item applications are less suited for heavy traffic areas such as airports and train stations.

Weather: The volatility and variance of weather (sun, rain, snow, wind, trees, clouds, shadows, etc.) can cause false conclusions for video analytics, especially in outdoor environments. Weather also has an impact on video analytics in indoor environments where there exist large glass windows and doors and the mentioned conditions create changes to the scene viewed by the indoor camera. Some sophisticated cameras can compensate for these variableconditions.

Backgrounds: The degree of change to the background of a camera view can impact the performance of video analytics. For instance, if the view of thecamera includes a constantly moving escalator, this could result in false conclusions and needs to be taken into account when developing or installing a solution.

Computational Factors
Beyond the environment, public safety organisations should consult with integrators and vendors regarding the features of their video surveillance systems.

Processing Power: There are many different analytics engines. One can require 10 times as much CPU as another. More CPU is usually required if you want to detect small objects moving quickly. This is because to accomplish this, engines need to run more quickly to run at both a high resolution (todetect small objects) and a high frame rate (to track fast objects).

Resolution: Normally you can record video at 4CIF (704 x 480 resolution) and do analysis at CIF (352 × 288 resolution) to save CPU. But to detect very small objects during analysis, you may have to run video at 4CIF. The higher the resolution, the more network traffic and storage is required.

Frame Rate: Most analytics engines need between five and eight frames per second (FPS), with a record rate even higher. Faster moving objects require higher FPS for tracking. Even left item detection analytics often use motion tracking to cut down on false positives. Higher frame rates also result in more network traffic and storage requirements.

Hard Disk: If you want to be able to search through analyzed footage (e.g., objects moving near a car), you will need to store the XML metadata produced by the analytics engine. This is normally a negligible amount of HD—a few percent of video storage requirements.

Memory: An analytics engine usually requires an additional 10MB to 100MB when run on a PC. Higher resolutions need more memory. Video analytics algorithms can vary greatly on the amount of computational power needed to perform adequately.

Before implementing any analytic applications, make sure you fully understand your goals and accuracy needs. Then consider the appropriate applications along with environmental and computational factors affecting performance. A school might be happy with 90 percent accuracy to detect vandalism, but that same percentage would be dangerously low for critical applications that protect the safety of dignitaries. So take all of these factors into account to reduce risks and enhance the safety and security of citizens and public buildings.