/philipp_memexhat.png)
Data from Image sensors is usually read out by scanning a frame, i.e pixel data is accessed row by row, and pixel by pixel within each row from left to right. Thus, pixel data appears for the entire frame pixel by pixel on a sensor output port, either as an analog voltage or a digital value.
Time-to-first-spike (TTFS) is an alternative encoding of pixel data. Pixel data will not occure in a given sequential scanning order. Instead, the digital output port will issue all pixel addresses, and their delay from the frame sync signal will encode their value, where the shortest delay indicates the highest pixel value. So we will still get one value per pixel on the output port, but now the 'strongest' pixels will convey their value first. (More geneally, encoding information with time information of pixel events is known as address event representation (AER), so we will implement TTFS using AER.)
If the sensor does not just encode pixel brightness as strong response, but somehow encodes a pixel's 'importance' (saliency) in relation to a specific application, that application will not need to wait for ALL pixel values, but can already start to work after having received pixel information from only a fraction of the most important pixels.
As a concrete example, when implementing object recognition, pixels with high contrast (as will likely occure along an object boundary) are more useful than those from uniform areas (maybe from an object surface or background). The object recognition algorithm will thus need only the first few pixel values and can abbort the data conveyance from the sensor (saving energy and data traffic) or reinitiate a new frame early (enahncing processing speed and reducing latency).
This project shall develop just such a TTFS sensor that encodes local contrast, providing a front end for object recognition applications. You will use the integrated layout design tool Cadence and design, produce and measure a real slicon chip, specifically an image sensor.