Calirad is a system based on cameras that detect rally spectators who are at high risk of being struck by competing vehicles. Critical images from race cars are sent to a web application, where a safety officer can assess the level of threat and intervene if necessary.
Car and motorcycle races in high-speed motorsports traditionally attract a large number of spectators worldwide. Some competitions—usually circuit races such as Formula 1—take place on closed tracks in highly controlled environments. Others are often held on local roads (on-road) or off-road, and are therefore more difficult to control. Additionally, these competitions often span large geographical areas, making full security coverage and monitoring of the track practically impossible. Examples include various types of rally and raid competitions—from the World Rally Championship, through regional and national rallies, to raid events such as the well-known Paris–Dakar Rally. Unfortunately, the inability to secure sections where vehicles travel at high speeds often results in accidents causing serious injuries or even death to spectators. Many spectators try to get as close as possible to passing vehicles, which can be extremely dangerous in cases of uncontrolled skidding or a car leaving the track. Furthermore, careless spectators can also pose a threat to competitors, again potentially leading to serious accidents.
The problem Calirad aims to solve is the lack of complete monitoring of sections where high-speed car and motorcycle competitions take place. Since organizers are practically unable to secure every meter of the race route, it is necessary to develop a solution that complements the role of marshals or judges along the track. The first step is simply detecting spectators in unsafe positions along the track, after which a marshal can intervene, or the race can be temporarily or permanently halted. Additionally, depending on the track configuration, some zones may be extremely dangerous even for spectators who are seemingly at a safe distance. These zones are called “run-off zones,” and their danger is not always obvious to spectators—or even to less experienced marshals.
The solution Calirad offers is installing a specialized device in race cars to record the entire competition route and detect spectators who are too close to the track or positioned in potential run-off zones. The device’s primary goal is not to produce video recordings but rather to detect spectators in prohibited areas using machine-learning techniques. Moreover, most competitions that Calirad targets take place in relatively isolated and remote areas, where mobile or similar network coverage is very limited. The potentially large number of spectators can further strain connectivity. For this reason, it is unrealistic to rely on transmitting video and processing it in the cloud—instead, the focus is on on-device (edge) detection of unsafe spectator presence. Photos of detected spectators—being small in size—can then be sent to the cloud, where safety personnel are alerted and further action is decided.
In addition to detecting spectators in unsafe locations, the development of the proposed device will bring multiple benefits to race organizers. For example, the ability to estimate the number and density of spectators along the race route provides valuable and measurable data for marketing purposes. The ability to automatically generate short video clips in real time—depending on network connectivity—can also help organizers with promotional activities.
Team members
The Calirad team consists of two members who have been actively working since 2022 on developing and testing the proposed solution, which includes solving spectator-detection challenges, developing hardware, ensuring connectivity, and presenting detections in the cloud. The current system has been tested at several World Rally Championship events as well as numerous national competitions.
Marin Vuković is an Associate Professor at the Department of Telecommunications, Faculty of Electrical Engineering and Computing, University of Zagreb. His fields of expertise include information security and machine learning. He has published over 50 scientific and professional papers in conferences and journals. He actively participates in panels and round-table discussions and gives invited lectures promoting engineering and science. He is a co-inventor on a patent filed with the Croatian State Intellectual Property Office. He has taken part—both as leader and researcher—in numerous national scientific projects, EU-funded projects, and industry collaborations. He leads several industrial projects and one scientific project. He is the deputy head of the “Security and Privacy Laboratory (SPL)” and the “ICT-AAC Laboratory for Assistive Technologies and Augmentative Communication” at the University of Zagreb. He is a member of IEEE and KES International.
Juraj Peršić is a postdoctoral researcher in the LAMOR group at the Department of Control and Computer Engineering at the Faculty of Electrical Engineering and Computing, University of Zagreb, and the director of Calirad d.o.o. He is the author of 11 scientific papers (133 citations) in the field of mobile robotics, with focus on multisensor fusion and calibration as well as detection and tracking of multiple moving objects. He is the recipient of several academic and research awards, including the Lončar and Končar awards. Much of his research was conducted abroad, including stays in Denmark, Germany, Canada, and Singapore. In addition to working on various domestic and international scientific and industrial projects, he has worked for several companies and startups, including developing autonomous taxis at Motional (formerly MIT spin-off nuTonomy, now a collaboration between Aptiv and Hyundai Motor Group).
Motivation for Applying and Vision for the End of the Nuqleus Program
While developing a prototype of our solution in parallel with participating in the Nuqleus program, the specialized workshops, contacts, and experiences of other teams and industry experts helped us reflect on parts of our solution that we had not previously considered. In addition, we met experts and received advice from various domains—from design, to business-model development, to opportunities for different types of funding. In this regard, we believe that we still need additional guidance and support in developing our idea and identifying other potential areas of application.
Our vision for the end of the Nuqleus program is to gain a clearer understanding of the potential applications of our solution beyond its initial domain of car and motorcycle racing, to understand possible funding sources, and to gain access to domain experts who can help us further develop the proposed solution. We believe that without the experience provided by Nuqleus, it would have been much more difficult for us to understand the breadth of potential application areas, as well as to anticipate the path ahead toward the final product we have envisioned.
