... | @@ -61,95 +61,6 @@ Image-Structure Computation. IEEE Trans. Circuits Syst. Video Techn. |
... | @@ -61,95 +61,6 @@ Image-Structure Computation. IEEE Trans. Circuits Syst. Video Techn. |
|
|
|
|
|
-----
|
|
-----
|
|
|
|
|
|
# Attention
|
|
|
|
|
|
|
|
This project presents a hardware architecture for the computation of
|
|
|
|
bottom-up inherent visual attention for FPGA. The bottom-up inherent
|
|
|
|
attention is generated including local energy, local orientation maps,
|
|
|
|
and red-green and blue-yellow color opponencies.
|
|
|
|
|
|
|
|
Visual information enters the visual cortex and is processed along two
|
|
|
|
parallel pathways: the ‘where’ pathway that involves spatial
|
|
|
|
localization (dorsal stream), and the ‘what’ pathway for object
|
|
|
|
recognition. Attention takes place in the first one and focuses on the
|
|
|
|
most relevant areas in the scene. Particularly, it allows searching by
|
|
|
|
processing the large amount of visual information that we perceive.
|
|
|
|
Perception mechanisms do not merely consist in information acquisition
|
|
|
|
but in active selection mechanisms of the most relevant information to
|
|
|
|
deal with real-world dynamic environments.
|
|
|
|
|
|
|
|
The implemented hardware model and architecture are based on \[1\]. More
|
|
|
|
details about the architecture can be found at
|
|
|
|
\[2\].
|
|
|
|
|
|
|
|
![](/uploads/0a418e4bd11db51af90a8851f50260bc/itti_model.png)
|
|
|
|
Fig 1. Itti model from \[1\].
|
|
|
|
|
|
|
|
As it can be seen in Fig 1, the original model combines features
|
|
|
|
estimated for different spatial resolutions. The maps for a specific
|
|
|
|
input example are shown in Fig
|
|
|
|
2.
|
|
|
|
|
|
|
|
![](/uploads/1e8b88854e472d807198bd55a95c7559/feature_maps.png)
|
|
|
|
Fig 2. Feature maps and sorted most salient locations (see \[2\]).
|
|
|
|
|
|
|
|
After several normalizations, they are integrated into a unique final
|
|
|
|
saliency map.
|
|
|
|
|
|
|
|
The final map can be estimated using more diverse cues. In \[2\] we also
|
|
|
|
propose the integration of [optical
|
|
|
|
flow](https://www.ohwr.org/project/optical-flow-cores/wiki) and [stereo
|
|
|
|
disparity](https://www.ohwr.org/project/stereo-disparity-cores/wiki).
|
|
|
|
The integration of the motion can be done straightforward as the other
|
|
|
|
features in the bottom-up system. Nevertheless, optical flow (as complex
|
|
|
|
information of speed and direction of the motion) and disparity can be
|
|
|
|
also integrated as top-down stream, in this case to modulate the final
|
|
|
|
saliency map.
|
|
|
|
|
|
|
|
Saliency can be used in many different fields such as robotics,
|
|
|
|
autonomous navigation, and aid devices for low-vision patients.
|
|
|
|
|
|
|
|
-----
|
|
|
|
|
|
|
|
## Contacts
|
|
|
|
|
|
|
|
- [Javier Diaz](mailto:jdiaz@ugr.es) [Seven
|
|
|
|
Solutions](https://www.ohwr.org/companies/seven-solutions)
|
|
|
|
- [Matteo Tomasi](mailto:matteo_tomasi@meei.harvard.edu) [Harvard
|
|
|
|
University](http://www.masseyeandear.org/)
|
|
|
|
- [Francisco Barranco](mailto:fbarranco@ugr.es) [University of
|
|
|
|
Maryland](http://www.umd.edu) and [University of
|
|
|
|
Granada](http://www.ugr.es)
|
|
|
|
|
|
|
|
-----
|
|
|
|
|
|
|
|
## Project Status
|
|
|
|
|
|
|
|
<table>
|
|
|
|
<tbody>
|
|
|
|
<tr class="odd">
|
|
|
|
<td><b> Date </b></td>
|
|
|
|
<td><b> Release </b></td>
|
|
|
|
</tr>
|
|
|
|
<tr class="even">
|
|
|
|
<td>??/02/2014</td>
|
|
|
|
<td>[v1.0 Release](https://www.ohwr.org/project/img-proc-core-lib/tree/master)</td>
|
|
|
|
</tr>
|
|
|
|
</tbody>
|
|
|
|
</table>
|
|
|
|
|
|
|
|
-----
|
|
|
|
|
|
|
|
## References
|
|
|
|
|
|
|
|
\[1\] L. Itti and C. Koch, Computational modelling of visual attention,
|
|
|
|
Nature Review Neuroscience, 2(3), pp. 194 – 203, 2001.
|
|
|
|
\[2\] F. Barranco, J. Diaz, B. Prieto, and E. Ros, Bottom-up visual
|
|
|
|
attention model based on fpga, in Electronics, Circuits and Systems
|
|
|
|
(ICECS), pp. 328 – 331, 2012.
|
|
|
|
|
|
|
|
-----
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
### Files
|
|
### Files
|
... | | ... | |