It has been six months since the start of our first pilot in Stratumseind, Eindhoven. Hence it’s time to write about our expectations, experiences, results, and outlooks. We’ve performed this pilot in collaboration with Axis Communications and the municipality of Eindhoven. This blogpost briefly discusses the results from this pilot and lays out our plan for the future.
Stratumseind: the party center of Eindhoven
Stratumseind is the longest nightlife street in The Netherlands, with more than 50 bars. The types of these bars vary, but generally, the party-audience visiting these bars is quite young. On weekends, most bars are open until 4:00 am, on weekdays, they close at around 2:00 am. Conveniently, this is also where the Living Lab Stratumseind is located: a testbed where all kinds of new technologies, usually based on sensors, are applied in real-life. The Livings Labs currently hosts a number of experiment including decreasing human aggression through adjusting lighting, sound analysis algorithms that can detect things like breaking glass, and adjusting behavior through releasing specific odours.
The Livings Labs focuses on decreasing violence through various measures. Stratumseind is known for having a relatively high occurrence of violence, in part due to high levels of alcohol consumption. As such, Stratumseind provides for an excellent testbed for Oddity’s violence detection algorithm.
Three camera positions at Stratumseind
Camera setup
We were allowed to surveil three cameras, which were placed for this pilot on ‘hotspots’ of Stratumseind, as depicted in the picture above. The first camera is placed near the church, marking the beginning of the street. Camera 2 and 3 are placed give or take 100 meters up ahead. They are on top of each other, facing the opposite direction and surveilling the busiest part of the street. If you visit the street, you will also notice that these cameras cover the usually crowded area of Stratumseind. Existing data of violence within the area shows that this part of Stratumseind sees most violence.
Results
We finished our setup in August and have now been monitoring and analyzing footage for six months. During that time we analyzed over 8000 hours of video footage (more than 850 million frames!). We detected a number of violent incidents (a few of which were quite serious). The most important results (as of January 1st 2020) are:
- Crime and incident reports from Stratumseind show that we did not miss any incident.
- Our algorithm is more than 99.99% accurate (this number can be somewhat misleading as explained in our previous blog post about the Base Rate Fallacy).
- More importantly, Oddity’s algorithm produces around 0.3 false positives per camera per day, down from about 1.0 during the first month of the pilot.
- During this time, we have detected six violent incidents on two of the three cameras. Of these six, at least four were missed by the surveillance center.
Find a more detailed report of our results in Stratumseind in this infographic.
Even though Stratumseind is a challenging environment, Oddity’s algorithm performed well. It detected 100% of incidents and produces only one false positive per camera each three days. Based on our experience and the results of the pilot, Oddity believes the algorithm is ready to be used in an actual security setting.
And now…
Due to the success of the Startumseind pilot, it has been extended for at least another six months. We will continue to gather data and improve upon the algorithm during this time. Our technical team is working hard on the second iteration of the algorithm that brings substantial improvements in both accuracy and number of false positives. We hope to show off the improvements within the next few months on Stratumseind.
The primary aim of the Stratumseind pilot is technical validation. The results show that the algorithm operates within reasonable bounds of accuracy and FPR (false positive rate). The next step is implementing such a system into the pre-existing security infrastructure and processes of law enforcement organizations. Oddity is already preparing for a second pilot. This pilot will cover more cameras, but more importantly, it will focus primarily on how to integrate Oddity into camera observation centers and achieving better efficiency through human-machine collaboration.
We are taking advantage of being a first-mover regarding violence detection in public safety and security. It is a fact that none of the surveillance rooms in the Netherlands has a violence detection algorithm in place, so it makes sense to first conquer the national market. We’ve obtained interest from several cities and municipalities in the Netherlands, and we are eager to implement it in every surveillance center because we firmly believe that Oddity will bring public safety to a whole new level.
