Assignment of critical missions to unmanned aerial vehicles (UAV) is bound to widen the grounds for adversarial intentions in the cyber domain, potentially ranging from disruption of command and control links to capture and use of airborne nodes for kinetic attacks. Ensuring the security of electronic and communications in multi-UAV systems is of paramount importance for their safe and reliable integration with military and civilian airspaces. Over the past decade, this active field of research has produced many notable studies and novel proposals for attacks and mitigation techniques in UAV networks. Yet, the generic modeling of such networks as typical MANETs and isolated systems has left various vulnerabilities out of the investigative focus of the research community. This paper aims to emphasize on some of the critical challenges in securing UAV networks against attacks targeting vulnerabilities specific to such systems and their cyber-physical aspects.