Insect-inspired vision for autonomous vehicles.

Flying insects are being studied these days as if they were agile micro air vehicles fitted with smart sensors, requiring very few brain resources. The findings obtained on these natural fliers have proved to be extremely valuable when it comes to designing compact low-weight artificial optical sensors capable of performing visual processing tasks robustly under various environmental conditions (light, clouds, contrast). Here, we review some outstanding bio-inspired visual sensors, which can be used for either detecting motion in the visible spectrum or controlling celestial navigation in the ultraviolet spectrum and for attitude stabilisation purposes. Biologically inspired visual sensors do not have to comprise a very large number of pixels: they are able to perform both short and long range navigation tasks surprisingly well with just a few pixels and a weak resolution.


Introduction
Insects are definitely an endless source of inspiration for scientists designing innovative sensors. Findings on the nocturnal butterfly's compound eye with its superimposed optics [3] have been used, for example, to design gradedindex (GRIN) lenses. Studies on the halteres forming the fly's rate gyros [29] have inspired what was probably the first micro rate gyroscope based on a vibrating part, which was subjected to the Coriolis force in order to measure the angular speed [13]. New photonic structures for energy-saving and healthcare applications have recently been developed based on studies on the black butterfly (Pachliopta aristolochiae) [51]. Arthropods' and insects' visual systems have also inspired dedicated visual sensors and small, highly efficient optical sensors with which to equip aerial robots [22]. In this review, it is proposed to describe some insect-inspired visual sensors which have been used in robotic applications, especially for performing autonomous navigation and flight stabilization tasks. Although we focus here on insects' vision, it is worth noting that many other studies [1,44], such as those on mantis shrimps in particular [11,63], have led to the production of some outstandingly efficient optical sensors.

Artificial compound eyes and motion sensors
In the field of artificial optical design, since the lens diameter is usually much larger than the light wavelength, the light diffraction is not taken into consideration. The inter pixel angle, defined as ∆φ = d f , where d is the pixels' pitch and f is the focal length, and ∆ρ is the acceptance angle given by the width of the Gaussian angular sensitivity at half height (∆ρ ≈ ∆φ), can be obtained by slightly defocusing the lens. Two artificial compound eyes have been developed, using either inorganic semiconductor photoreceptors comprising 630 pixels, where ∆φ = ∆ρ = 4.2 • [21],mimicking the Drosophila melanogaster 's compound eye (Fig. 1a), or organic photodiodes comprising 256 pixels, where ∆φ = 11 • and ∆ρ = 9.7 • [52], mimicking the fire ant's Solenopsis fugax compound eye (Fig. 1b).
The CurvACE artificial compound eye was developed in the framework of a European project (2009-2013; www.curvace.org, [21]).This functional prototype with its 630 pixels (forming 630 artificial ommatidia) gave a wide field of view (180 • x 60 • ) in a large range of lighting conditions and weights ∼ 2-gram [21]. A pair of CurvACE sensors was mounted on board an 80-gram micro flying robot called BeeRotor [19] for performing both ground and ceiling avoidance tasks. A minimalistic version of this artificial compound eye containing only three photoreceptors set under a common lens, weighing only 2 mg and detecting movements in two dimensions at rates of up to 300 Hz with an ultra low current consumption of only 0.444 mA, has also been constructed [43] (Fig. 2d).
A recent optic flow sensor based on the M 2 APix retina (M 2 APix stands for Michaelis-Menten Auto-adaptive Pixel [36], Fig. 1c) can auto-adapt in a 7-decade lighting range and responds appropriately to step changes of up to ±3 decades [37,56]. These pixels do not saturate thanks to the normalization process performed by Very Large Scale Integration (VLSI) transistors [36]; this advantage is due to the intrinsic properties of the Michaelis-Menten equation [40] on which these pixels are based. Comparisons between the characteristics of auto-adaptive Michaelis-Menten and Delbrück pixels [12] under identical lighting conditions (i.e., with the pixels integrated into the same retina) showed that the Michaelis-Menten pixels gave better performances in terms of their dynamic sensitivity and minimum contrast detection levels [36].
Algorithms of several kinds have been developed for computing local motion, which have resulted in various hardware implementations including templates, time-of-travel, feature tracking, edge counting, edge correlation devices and the Hassenstein-Reichardt correlator [5,39,62], as well as some software implementations [5,56]. However, analog VLSI motion sensors significantly reduce micro flying robots' power consumption and payload while increasing the bandwidth, thus improving both the precision and the accuracy of the onboard optic flow measurements.

Artificial ocelli
Ocelli are visual sensors which are present in the dorsal part of many insects's heads (Fig. 2a). The word ocellus means a little eye. One to three ocelli can be found, at different points on the top of insects' heads [31]. Ocelli usually consist of three elementary eyes having a lens with a relatively large diameter (ranging from 100µm to 500µm) and a small focal length. Because of the large aperture and the resulting f-number N of the lens (N = f D , where f is the focal length and D is the diameter of the aperture), ocelli can detect low light levels, but the lens is positioned in such a way that the image projected onto the photoreceptors is always under-focused. The exact function of the ocelli has not yet been elucidated, but behavioural experiments on flies have shown that the ocelli are involved in head stabilisation processes [28,48], and electrophisiological recordings on dragonflies have even shown that the second-order neuron (L-neuron) directly connected to the photoreceptors may be sensitive to preferred motion in ultraviolet light [2]. In short, the ocelli are closely involved in the stabilization reflexes responsible for controlling the fly's body rotation (on the pitch and roll axes) and the head orientation (in gaze control processes). Ocellar signals are fused with compound eye signals [42] but the ocelli are faster than the compound eyes because a much smaller number of neural processing layers are involved in the processing of the ocellar signals transmitted to the descending neurons [55]. On the basis of results obtained in biological studies, several artificial ocelli have been developed and tested. Chahl and Mizutani (2012) developed a biomimetic eye composed of 4 pairs of photodiodes which were sensitive to ultraviolet and green lights [4] (Fig. 2c). Artificial ocelli were used by the latter authors to stabilize the roll movements of a small unmanned aerial (fixed wing) vehicle [4]. Gremillion et al. (2012Gremillion et al. ( -2014 developed an analog ocellar sensor composed of four photodiodes with a response range of 350-1050 nm, which was used to estimate the pitch and roll movements of an aerial robotic platform [24,25] (Fig. 2b). Fuller et al. presented a method of stabilizing a micro flying insect by means of an ocelli-inspired sensor (Fig. 2b) to ob-tain angular velocity feedback (instead of attitude feedback) and used this method to stabilize an insect-scale robot in short take-off flights [23].

Celestial compass inspired by insects'dorsal rim area
To be able to travel long distances, insects have to keep a constant heading relative to a stable reference point [60]. Desert ants (Cataglyphis fortis) are able to home with a high level of accuracy, based on their knowledge of their heading acquired when exploring the environment [60]. Locusts [38], flies [59,61], bees [45,46], ants and dung beetles [10] are equipped with an outstandingly efficient optical compass, which is sensitive to the polarized light caused by the atmospheric scattering of the sunlight. The scattering of the sunlight within the Earth's atmosphere produces a polarization pattern across the sky. Solar radiations remain unpolarized until their entry into the atmosphere, where scattering interactions with atmospheric constituents induce the partially linear polarization of the skylight [9]. Insects possess photoreceptors in the dorsal region of their eye (the dorsal rim area, DRA) that are specialized in detecting the pattern of polarized skylight [32,60,61]. The first robotic application of the desert ants' DRA was presented in [34] (Fig. 3a). Chu et al. developed a smaller compass consisting of 6 photodiodes topped by linear polarizers set 60 • apart [6]. A pair of polarized-light sensors of this kind were tested under a clear sky (Fig. 3b): the accuracy of these polarized-light sensors was found to be within ±0.2 • [58]. A version of the polarization sensor shown in Fig. 3b, composed of just a single unit, which was recently mounted on board a quadrotor (Fig. 3d), gave promising performances: indoor accuracy 0.2 • , outdoor accuracy less than 2 • , and output refresh signal 10 Hz [64]. A polarization-based photodetector involving a bilayer nanowire was recently developed and tested under artificial blue lighting, giving a mean error of only ±0.1 • once a polynomial fitting process had been applied [7]. Other applications of wire grid micro-polarizers have been developed, some of which involve various angles of polarization (in regular steps of 45 • ) [30]. Bio-inspired approaches have also been adopted with cameras in the visible range [20] along with the polarization model developed in [35]. The brand new celestial sensor recently developed by the French Biorobotics group in Marseille is composed of two polarization units (POLunits) consisting of only two UV light sensors topped with actuated rotating linear sheet polarizers (Fig. 3c). This approach makes it possible to greatly reduce the number of pixels, but at the expense of the temporal resolution, which is limited by the mechanical rotational speed. This sensor mimicks the UV-sensitivity of the ant's photoreceptors and the log ratio of two orthogonal POL-units, as previously proposed in the Labhart's locust-based model [33]. In addition, this insect-based compass, which is highly reliable and suitable for performing navigation tasks in various meteorological contexts [15], is able to determine its heading angle with great precision: the median error recorded was only 2.9 • when the sky was slightly cloudy, and 1.9 • in the case of an overcast sky [16]. Several artificial celestial compass sensors have been produced for robotic and autonomous navigation purposes [17,34,35,64].

Fly-inspired hyperacute sensor
The accuracy (acuity) of a contrasting object's angular position measurements depends directly on the position sensor's optical resolution. The acuity of a compound eye might therefore be expected to be relatively poor in comparison with that of the human camerular eye, which contains several million photoreceptors (i.e., pixels). This would be true without the existence of the fly's active visual processing system based on retinal micro vibrations. Since the 1970s, several studies have shown that active retinal micro movements occur in flies, but the exact function of these micro movements and how they contribute to improving the visual acuity have not yet been established [57]. However, several authors have integrated micro movements of this kind into their artificial visual sensors, and the results obtained have shown that they were able to locate a contrasting target with much greater accuracy (up to 700-fold) than that achieved using optical systems alone [22,57]. Thanks to this hyperacuity, it was recently established using a biorobotic approach that a contrasting moving target (moving hands) could be located on a textured background and the distance travelled by a micro robot flying above a textured plane could be measured by processing the visual signals conveyed by the 40 ommatidia in a vibrating artificial compound eye [8]. The presence of hyperacuity has not yet been clearly determined in insects, but a highly counter-intuitive principle in which coarse and blurry vision (due to the Gaussian angular sensitivities of the photoreceptors) is combined with vibration seems to be able to give a more accurate perception of the world.

Insect-based visual guidance
Several ethological experiments have suggested that flying insects rely heavily on optic flow to avoid obstacles and directly control their flight speed in the presence of wind disturbances or changes in the configuration of the environment, as well as to achieve a smooth landing [18,49,53]. The magnitude of the translational optic flow component is inversely proportional to the distance, which enables flying insects to spontaneously sense the environmental configuration. Two main models have been developed to explain how insects detect the optic flow: the Hassenstein-Reichardt correlator [26] and the time-of-travel model [22]. Although the optic flow is not only used by flying insects [47,49], it has been frequently used in biologically inspired robots to equip them with insect-inspired vision for both short and long range navigation purposes [49,53,54].

Conclusion
The latest advances in the field of insect-like robots will enable roboticists to test biological hypotheses on the scale of insects in order to check whether their hypotheses hold true in the physical world [14,23,27,41,50]. The expanding set of roles for which robots have been designed means that they will be able to operate in the near future in many contexts such as cluttered and confined environments including buildings, warehouses, industrial plants, performance halls, urban canyons and forests. In these complex environments, autonomous vehicles will require redundant sensors to improve the reliability of their autopilots. The data transmitted by bio-inspired sensors could therefore usefully complement the existing measurements provided by classical sensors such as global positioning systems and magnetometers.