Anomaly detection on data streams from vehicular networks

Abstract

Vehicular networks are characterized by high mobility nodes that are only active when the vehicle is moving, thus making the network unpredictable and in constant change. In such a dynamic scenario, detecting anomalies in the network is a challenging but crucial task. Veniam operates a vehicular network that ensures reliable connectivity through heterogeneous networks such as LTE, Wi-Fi and DSRC, connecting the vehicles to the Internet and to other devices spread throughout the city. Over time, nodes send data to the cloud either by real time technologies or by delay tolerant ones, increasing the network's dynamics. The aim of this dissertation is to propose and implement a method for detecting anomalies in a real-world vehicular network through means of an online analysis of the data streams that come from the vehicles to the cloud. The network's streams were explored in order to characterize the available data and select target use cases. The chosen datasets were submitted to different anomaly detection techniques, such as time series forecasting and density-based outlier detection, followed by the trade-offs' analysis to select the algorithms that best modeled the data characteristics. The proposed solution comprises two stages: a lightweight screening step, followed by a Nearest Neighbor classification. The developed system was implemented on Veniam's distributed cluster running Apache Spark, allowing a fast and scalable solution that classifies the data as soon as it reaches the Cloud. The performance of the method was evaluated by its precision, i.e., the percentage of true anomalies within the detected outliers, when it was submitted to datasets presenting artificial anomalies from different data sources, received either by real-time or delay tolerant technologies.