8D010: Front-End Vision and Multi-Scale Image Analysis

Autumn 2008

An introduction to modern multi-scale image analysis based on inspiration of biological vision.

A course given at the Department of Biomedical Engineering (Eindhoven University of Technology and University of Maastricht).

NB: This course is also open as "ASCI course a8" for the PhD Research Schools "ASCI" and "ImagO".

Total duration: 18 hours of 45 min. oral lectures and 20 hours hands-on training (5 blocks of 4 hours). Note: no courses in the week 8-12 September 2008.

Code TUE-BME: 8D010 (3 ECTS study points).
The lectures are recorded, and can be watched as streaming video on http://videocollege.tue.nl, course 8D010.

Goal:

Image analysis is the extraction of useful information from images. In this course we give a modern mathematical (and physics based) approach to multi-scale image analysis as a branch of computer vision. We give an intuitive introduction to multi-scale image analysis, trying to keep the analogy with stages in the human visual system as close as possible. The human visual system also widely exploits a diversity of multi-scale filters in its processing layers.

Among the topics covered are: filters to sample and analyze images, the use of filters in detecting edges and corners in images, multi-scale analysis of 2D and 3D shape and motion from image sequences, depth from stereo, orientation analysis, and the use of contemporary, well-understood mathematical tools in this field such as differential geometry and tensor analysis.

The majority of the examples discussed are from 2D, 3D and 4D (3D-time) medical imaging. We devote some time to the efficient numerical implementation of the different techniques. Hands-on experience is acquired in a computer lab, where experiments in Mathematica illustrate the theory and applications in practice.

Topics discussed:

• Notion of scale in physics and mathematics
• Physics of observation, apertures
• Axiomatic derivation of filters
• Differentiation, scale and regularization
• History of scale-space theory
• The human visual pathway, receptive field structure
• Retinal distribution of receptive fields, feedback in the visual system
• Information processing in the visual cortex
• Gaussian derivatives and the diffusion equation
• Some differential geometry on images in 2D, 3D, 2D-time for robust feature detection and shape analysis
• Geometric invariants, landmarks
• Applications in medical imaging: computer-aided detection
• Mathematical morphology, incl. the slope transform
• Nonlinear, geometry-driven diffusion (evolution of images by partial differential equations)
• Deep structure of images, watershed segmentation
• Orientation analysis
• Summary

A modern development is computer-aided diagnosis (CAD), where the computer assists in finding possible pathology in images, particularly in screening applications. We discuss some examples of this promising area: detection of stellate tumors in mammography, counting follicles for fertility related diagnosis with 3D ultrasound, and detecting polyps in 3D virtual colonoscopy.

We also discuss examples from molecular imaging. Special molecules can be designed with specific (ligand) binding locations, and with fluorescent markers making them visible with 3D fluorescence microscopy, with radioactive labels to detect them with SPECT and PET scanners, and/or contrast media to visualize them with MRI techniques. This leads to a high specificity for early cancer detection (and many other diseases). Computer vision plays an increasing role in the detection and recognition of structures, quantitative analysis, segmentation and visualization. 

Times:	Day Time Content Lecture material Room Assistant Monday 25 August 2008 13:30-14:15 Course Introduction   Introduction   Introduction (zipfile with all movies, 146 MB) Matrix 1.44     14:30-15:15 Course Notion of scale Powers of 10 Matrix 1.44     15:30-16:15 Course Axiomatics of multi-scale operators   Scale-space from entropy Matrix 1.44     16:30-17:15 Course The Gaussian kernel, regularization Gaussian kernel Matrix 1.44   Wednesday 27 August 2008 08:45-09:30 Computer lab Plenary lecture: Introduction to Mathematica 6.0 Video of the revolutionary interactivity in Mathematica 6 Tutorial Mathematica notebooks 8ZZ16: Cursus deel 1 van 3 (Dutch) Cursus deel 2 van 3 (Dutch) Cursus deel 3 van 3 (Dutch) Course part 1 of 3 (English) Course part 2 of 3 (English) Course part 3 of 3 (English)   BMIA MMA course WH 3A.06 Alessandro Becciu, Markus van Almsick   09:30-12:45 Computer lab Exercising with Mathematica 6.0 Download the FEV book MathVisionTools FEV6.nb, FEV6.m (store in directory C:\Documents and Settings\All Users\Application Data\Mathematica\Applications\FrontEndVision) MathVisionTools Exercises I Exercises II Some testimages to play with.   WH 3A.06 Alessandro Becciu, Markus van Almsick     Exercises with Mathematica 6.0 Study material: Eindhoven Tips Dictionary manipulations Often used commands FrondEnd Interactivity Demo active shape PackageNotebook.nb Wolfram Inc. Eduroam (network access through SurfNet)   Alessandro Becciu, Markus van Almsick Monday 1 September 2008 13:00-13:45 Course Gaussian derivatives, deblurring Gaussian derivatives Deblurring Deblurring Matrix 1.44     14:00-14:45 Course Limits on observations, numeric implementation of Gaussian derivatives Limits Limits Convolution01 Convolution02 Convolution03   Convolution animations Implementation   Matrix 1.44     15:00-15:45 Course Gauge coordinates Differential structure Matrix 1.44   Wednesday 3 September 2008 09:30-12:30 Computer Lab Exercising FEV lectures   WH 3A06 Alessandro Becciu, Markus van Almsick Monday 8 September 2008 No lectures         Wednesday 10 September 2008 09:30-12:45 Computer lab Exercising FEV lectures   WH3A.06 Alessandro Becciu, Markus van Almsick Monday 15 September 2008 13:30-14:15 Course Differential structure I Applications second order structure Matrix 1.44     14:30-15:15 Course Differential structure II Vesselness Matrix 1.44     15:30-16:15 Course Front-end visual system I Front-End Visual System Matrix 1.44     16:30-17:15 Course Front-end visual system II Visual illusions Spiral illusion Matrix 1.44   Wednesday 17 September 2008 09:30-12:30 Computer Lab Exercising FEV lectures   WH 3A06 Alessandro Becciu, Markus van Almsick Monday 22 September 2007 13:30-14:15 Course Geometry-driven diffusion Geometry-Driven Diffusion Geometry-Driven Diffusion PDF: Original paper: P. Perona, J. Malik, "Scale-space and edge detection using anisotropic diffusion", PAMI 12(7), pp. 629-639, 1990.  Matrix 1.44     14:30-15:15 Course Color Differential Structure Color differential structure Color differential structure Matrix 1.44     15:30-16:15 Course Scale-time: differential structure of time sequences Scale-time Scale-time Matrix 1.44   Wednesday 24 September 2008 08:45-09:30 Course Deep structure II: toppoints, image retrieval, ScaleSpaceViz demo Contextual operators Toppoints in image matching ScaleSpaceViz (VTK application) download Matrix 1.44     09:45-10:30 Conclusions, Q&A   Matrix 1.44   Wednesday 24 September 2008 13:30-17:00 Computer lab, Mathematica exercises Any topic of the course   WH 3A06 Alessandro Becciu, Markus van Almsick
Place:	*Lecture rooms:   see table above. Matrix: Matrix building WH: W-Hoog (Werktuigbouw-Hoog).      WH3A.06 is a room with desktop computers, available for exercising the course material with Mathematica 6 (and 5.2). See the campusmap of TU/e for directions.

Literature

Recommended reading:

• L. M. J. Florack: The Structure of Scalar Images. Computational Imaging and Vision Series, Dordrecht, Kluwer Academic Publishers, 1996.
• B. M. ter Haar Romeny, Ed.: Geometry-Driven Diffusion in Computer Vision. Dordrecht, Kluwer Academic Publishers, 1994.
• L. M. J. Florack: Visuele Perceptie en Digitale Beeldverwerking, Nieuw Archief voor Wiskunde, vol. 5/3, no. 1, pp. 34-41, maart 2002. PDF  
• B. M. ter Haar Romeny: Computer Vision and Mathematica. Computing and Visualization in Science, vol. 5, no.1, pp. 53-65, Springer, 2002. PDF (1.7MB), Mathematica 4 Notebook (3.1MB).
• T. Lindeberg: Scale-Space Theory in Computer Vision, Dordrecht, Kluwer Academic Publishers, 1994.
• J. Sporring, M. Nielsen, L. Florack and P. Johansen (Eds.): Gaussian Scale-Space, Dordrecht, Kluwer Academic Publishers, 1996.
• J. Weickert: Anisotropic Diffusion in Image Processing, ECMI Series, Teubner-Verlag, Stuttgart, Germany, 1998. 170 pages, hardcover. ISBN 3-519-02606-6.
• WebVision: the visual cortex.
• Wikipedia on scale-space.

Proceedings Scale-Space conferences

• B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, eds., Scale-Space '97: Proc. First Internat. Conf. on Scale-Space Theory in Computer Vision, Utrecht, vol. 1252 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 1997.
• M. Nielsen, P. Johansen, O.F. Olsen, J. Weickert: Scale-Space Theories in Computer Vision, second  International Conference, Scale-Space 1999, Corfu, Greece, September 26-27, 1999, vol. 1682 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 1999.
• M. Kerckhove: Scale-Space and Morphology in Computer Vision, Third International Conference, Scale-Space 2001, Vancouver, Canada, July 7-8, 2001, vol. 2106 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 2001.
• L.D. Griffin, M. Lillholm: Scale-Space Methods in Computer Vision, Fourth International Conference, Scale-Space 2003, Isle of Skye, UK, June 10-12, 2003, vol. 2695 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 2003.
• R. Kimmel, N. Sochen, and J. Weickert, Fifth International Conf. on Scale Space and PDE methods in Computer Vision, Hofgeismar, Germany, April 2005, vol. 3459 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 2005.

Other software:

• MathVisionTools, a toolbox for mathematical multi-scale computer vision operations. Written by Markus van Almsick, TU Eindhoven.
• ScaleSpaceViz, a package to visualize the deep structure of images. Written by Frans Kanters, TU Eindhoven.

Other:

• Visual illusions

Computer Laboratory:

On the Wednesday mornings computer laboratories will be organized to acquire hands-on experience with the discussed scaled differential invariant operators on a variety of 2D and 3D images. We use the program Mathematica 6.0 (http://www.wolfram.com/).

For TUE members: Mathematica 6.0 (and the previous version 5.2) can be downloaded from the TU/e campus software website.

Recommended tutorial books on Mathematica:

• N. Blachman & C.P. Williams: "Mathematica, a practical appraoch".Prentice Hall 1999, second edition. ISBN 0-13-259201-0.
• H. Ruskeepää: "Mathematica Navigator, graphics and methods of applied mathematics", Academic Press 1999, ISBN 0-12-603640-3 (paperback with CD-ROM).
• J. Glynn & Th. Gray: "The beginner's guide to Mathematica version 4", Cambridge University press, 2000, ISBN 0-521-77153-6 (hardback, 0-521-77769-0 (paperback).
• R. M. Maeder: "Computer Science with Mathematica 4. Theory and practice for science, mathematics and engineering", Cambridge University Press, 2000, ISBN 0-521-631172-6 (hardback), 0-521-66395-4 (paperback).

 Some useful notebooks:

• Introduction to the Fourier Transform
• Introduction to linear systems

The famous mathematics teaching files and resource online: MathWorld.

Examination:

For BME students: select any three (3) questions from this set of questions: Exam tasks BME 2008.

Write a small notebook per question (max. 1-2 A4 each) and send them within two weeks after the end of the course to B.M.terHaarRomeny@tue.nl.

Some examples

Low dose fluoroscopy image of an electro-physiology catheter in ther heart. The extra low dose is beneficial for the radiation dose, but leads to a deteriorated image quality.	Robust catheter detection with oriented filters and tensor voting.

Cerebral aneurysm from 3D CT angiography.	The same aneurysm cleaned with an adaptive edge-preserving 3D denoising filter.

Contact the tutor:

prof.dr.ir. Bart M. ter Haar Romeny, email: B.M.terHaarRomeny@tue.nl

Department of Biomedical Engineering (BME-BMT)
Group Biomedical Imaging Analysis BMIA
Eindhoven University of Technology
Den Dolech 2 - WH2.106
NL-5612 AZ Eindhoven
Tel. 040-2475537 (secr. Margret Philips).

FEV class of 2008 - BME course.