Framework for Offline Wireless Network Experimentation

Abstract

Projects related to communications systems often, if not always, require testing in a real testbed. This involves setting up a mostly temporary experiment, which may require the use of several resources. If the results from the real world experiments turn out to be different from those obtained from simulations, the project will need to be re-evaluated in a simulation environment before repeating the same, costly, real world experiments. This cycle of testing and validation can repeat itself an arbitrary number of times before a suitable solution is found, which in turn can amount to a considerable investment in terms of resources and time. This dissertation's project aims to minimize or even fully mitigate the amount of real world experimentation required to develop robust and reliable solutions by providing a system that allows its users to automatically create simulations whose conditions come very close to the ones found in the real world. Furthermore, the solution proposed in this dissertation would reduce the user's workload as well as minimize the possibility of human error in the process, by automating most of the tasks involved. This is achieved by storing layer one characteristics from previous real experiments, such as Signal-to-Noise Ratio (SNR) and node positions, and reproducing them in simulations instead of relying on the available Propagation Loss and Mobility models. By using this system, one can get more accurate results from a simulator, which means that when a project is finally tested in a real world environment, the probability that the obtained results differ significantly from those obtained in simulated environments is considerably lower, thus reducing the number of times the project needs to be tested in real testbeds. The solution proposed in this dissertation can also become a platform that enables wide and open access to network researchers data. This would simplify sharing of data and is useful in many cases, such as the validation of results published in scientific papers and those obtained from network projects in general. To validate the solution proposed in this document, a proof-of-concept version of the system was developed. The system is composed by three main components: 1) an information retrieval agent to be installed in the real testbed nodes; 2) a central storage system to store the layer one characteristics from real experiments; and 3) a web-based user interface to browse data from past experiments and download automatically generated trace-based simulations. The system was submitted to real usage, where its purpose was fulfilled by successfully recording a real laboratory testbed experiment and reproducing it as a trace-based simulation. Furthermore, a questionnaire was distributed among a group of researchers, the results of which showed a significant acceptance of the framework as an improvement upon the current methodology.