ASCI course a8:
FrontEnd Vision and MultiScale Image
Analysis
Autumn
2011

An introduction to modern multiscale image analysis based on
inspiration of biological vision. 
A PhD course given at the Department of
Biomedical Engineering (Eindhoven University of Technology and University of
Maastricht).
NB: This course is also
open for TU/e BME MSc students as course
8D010,
as "ASCI PhD course a8", and as course for the PhD Research School "ImagO".
Tutors: 
Prof. Bart M. ter Haar
Romeny, PhD, Eindhoven University of Technology / Maastricht University
Prof. Nicolai Petkov, PhD, University of Groningen 
Dates: 
2 fulltime weeks, with one 'recovery'
week in between without lectures.
Lectures in the morning from (09:4512:30) Monday 3 October 2011 till Friday
7 October 2011, and from Monday 17 October 2011 till
Friday 21 October 2011.
Computer laboratories in the afternoon (13:4516:30), using
Mathematica 8 to exercise the course material and tasks.
In the
week 1015 October 2011 there is no course. 
Registration:

Register through the ASCI website:
here.
Registration for course and exam is free for TU/e, ASCI, ImagO students, and employees of
industries officially collaborating with TU/e BME.
For registration as 'contractant' with TU/e to do an official exam for 3
ECTS as nonTU/e
student: see
STU registration form.
Costs: € 250 per course. Registration is free
when no exam is done.
Costs for industrial participants: € 1200 (invoice will be sent by ASCI
after registration). 
Evaluation: 
Form,
to be sent to ASCI. 
Total duration: 27 oral lectures of 45
minutes each, and 27 hours handson training.
Code ASCI: a8 (4 ECTS study points)
Code TUEBME:
8D010
(3 ECTS study points).
The lectures will be made available as
streaming video.
Goal:
Image analysis is the extraction of useful information from images. In this course we give a modern mathematical (and physics based) approach to
multiscale image analysis as a branch of computer vision. We give an intuitive
introduction to multiscale image analysis, trying to keep the analogy with
stages in the human visual system as close as possible. The human visual system also widely exploits a
diversity of multiscale filters in its processing layers.
Among the topics covered are: filters to sample and analyze images, the use
of filters in detecting edges and corners in images, multiscale analysis of 2D
and 3D shape and motion from image sequences, depth from stereo, orientation
analysis, and the use of contemporary, wellunderstood mathematical tools in
this field such as differential geometry and tensor analysis.
The majority of the examples discussed are from 2D, 3D and 4D (3Dtime)
medical imaging. We devote some time to the efficient numerical implementation
of the different techniques. Handson experience is acquired in a computer lab,
where experiments in Mathematica 8 illustrate the theory and applications in
practice.
Topics
discussed:
 Notion of scale in physics and mathematics
 Physics of observation, apertures
 Axiomatic derivation of filters
 Differentiation, scale and regularization
 History of scalespace theory
 The human visual pathway, receptive field structure
 Retinal distribution of receptive fields, feedback in
the visual system
 Information processing in the visual cortex
 Gaussian derivatives and the diffusion
equation
 Some differential geometry on images in 2D,
3D, 2Dtime for robust feature detection and shape analysis
 Geometric invariants, landmarks
 Applications in medical imaging: computeraided detection
 Mathematical morphology, incl. the slope
transform
 Nonlinear, geometrydriven diffusion (evolution of images by partial
differential equations)
 Deep structure of images, watershed
segmentation
 Orientation analysis
 Summary
Computer vision plays an increasing role in the
detection and recognition of structures, quantitative analysis, segmentation and
visualization.
A modern development is computeraided diagnosis
(CAD), where the computer assists in finding possible pathology in images,
particularly for screening applications.
We discuss several examples in detail:
 detection of stellate tumors in
mammography,
 counting follicles for fertility related diagnosis with 3D
ultrasound,
 detecting polyps in 3D virtual colonoscopy,
 detecting pulmonary emboli (narrowed vessels
in the lung),
 vesselness for vessel enhancement.
Detailed program and content:




Times: 
Day 
Time 
Content 
Lecture material 
Room
(to be announced) 
Monday
3 October 2011 
13:4514:30
Course 
Introduction 
Introduction
Introduction
(zipfile with all movies, 146 MB) 
GEMZ 3A10 
14:4515:30
Course 
Notion of scale 
Powers of 10 
GEMZ 3A10 
15:4516:30
Course 
Axiomatics of multiscale operators 
Scalespace from entropy 
GEMZ 3A10 
Tuesday
4 October 2011 
09:4511:30 Computer lab 
Introduction to Mathematica 8 
Tutorial Mathematica notebooks 8ZZ16:
Course
part 1 of 3 (English)
Course part 2 of 3 (English)
Course part 3 of 3 (English)
BMIA MMA
8
course 
GEMZ 3A06 
11:4512:30 Computer lab 
Exercises with
Mathematica
8 
Download the
FEV book
MathVisionTools:
www.mathvisiontools.net
(Run the command:
$AddOnsDirectory
in Mathematica, and store in this directory)
Images of the book & test images
(Data.zip,
31 MB, extract in
MathVisionTools\Data directory)
shortnotation.nb
Exercises I
Some testimages
to play with. 
GEMZ 3A06 

Exercises with Mathematica
8 
Study material:
Eindhoven Tips
Dictionary
manipulations
Often used commands
FrondEnd
Interactivity
Demo active shape
PackageNotebook.nb
Wolfram Inc.
Eduroam (network access through
SurfNet) 
GEMZ 3A06 
Tuesday
4 October 2011 
13:4514:30
Course 
The Gaussian kernel 
Gaussian
kernel 
Note:
Change of room!
STC 0.02
(Helix building = Chemistry), near the eleveators, to the right. 
14:4515:30
Course 
Gaussian derivatives 
Gaussian
derivatives 
STC 0.02 (Helix) 
15:4516:30
Course 
Deblurring
Numeric implementation of Gaussian
derivatives
Limits on observations 
Deblurring
Deblurring
Convolution01
Convolution02
Convolution03
Convolution animations
Implementation
Limits
Limits 
STC 0.02 (Helix) 
Wednesday
5 October 2011 
09:4510:30
Course 
Speed of Gaussian convolutions
Regularization 
Regularization 
STC 0.02 (Helix) 
10:4511:30
Course 
Gauge coordinates 
Differential structure 
STC 0.02 (Helix) 
11:4512:30
Course 
Differential structure I 

STC 0.02 (Helix) 
13:4516:30 Computer Lab 
Exercising FEV lectures 
Tasks 02 
GEMZ 3A06 
Thursday
6 October 2011 
09:4512:30 Computer Lab 
Exercising FEV lectures 

GEMZ 3A06 
13:4514:30
Course 
Second order differential structure 
Vesselness
Original paper: A. Frangi, W. Niessen, K. Vincken
and M. Viergever,
Multiscale vessel enhancement filtering. In: Proceedings of the
MICCAI’98Lecture Notes in Computer Science vol. 1496, SpringerVerlag,
Berlin (1998), pp. 130–137. 
GEMZ 3A08 
14:4515:30
Course 
Applications second order differential structure 
Applications second order structure 
GEMZ 3A08 
15:4516:30
Course 
Third order differential structure 

GEMZ 3A08 
Friday
7 October 2011 
09:4512:30 Computer Lab 
Exercising FEV lectures 
Tasks 03 
GEMZ 3A06 
13:4514:30
Course 
Frontend visual system I 
FrontEnd Visual
System 
AUD 07 
14:4515:30
Course 
Frontend visual system II 
Youtube: Hubel &
Wiesel  Cortical neuron V1 Youtube:
Hubel's research 
AUD 07 
15:4516:30 Course 
Frontend visual system III 
Visual illusions
Spiral illusion 
AUD 07 
Note: No lectures from 10 October
2011 till 15 October 2011. 

Monday
17 October 2011 
09:4512:30 Computer Lab 
Exercising FEV lectures 
Tasks 03 (with
some solutions)
Tasks 07 (with
some solutions) 
GEMZ 3A06 
13:4514:30
Course 
Gabor functions
and filters (as models of cortical areas V1/V2) 
Prof. Nicolai Petkov 
STC 0.02 (Helix) 
14:4515:30
Course 
Surround
suppression (or nonclassical receptive field inhibition in areas
V1/V2)
References:
Contour and boundary detection improved by surround
suppression of texture edges (pdf)
Contour detection based
on nonclassical receptive field inhibition (pdf)
Suppression of contour
perception by bandlimited noise and its relation to nonclassical
receptive field inhibition (pdf)

Prof. Nicolai Petkov
Lecture slides 01 (Gabor)
Lecture slides 02 (surround
suppression) 
STC 0.02 (Helix) 
15:4516:30
Course 
What comes after V2? Computational model of
shape selective neurons in area V4, with application to the
detection of blood vessel bifurcations in retinal funds images. 
Dr. George Azzopardi
(g.azzopardi@rug.nl,
www.cs.rug.nl/~george)
Lecture slides 
STC 0.02 (Helix) 
Tuesday
18 October 2011 
09:3012:30 Computer Lab 
Exercising FEV lectures 
Tasks  visual system 
GEMZ 3A06 
13:4514:30
Course 
Geometrydriven diffusion I 
GeometryDriven Diffusion
GeometryDriven Diffusion 
STC 0.02 (Helix) 
14:4515:30
Course 
Geometrydriven diffusion II 
PDF:
Original paper:
P. Perona, J. Malik, "Scalespace and edge detection using
anisotropic diffusion", PAMI 12(7), pp. 629639, 1990. 
STC 0.02 (Helix) 
15:4516:30
Course 
Scaletime: differential structure of time
sequences 
Scaletime
Scaletime 
STC 0.02 (Helix) 
Wednesday
19 October 2011 
09:4510:30
Course 
Color Differential Structure 
Color
differential structure
Color differential structure
Color
Invariants 
STC 0.02 (Helix) 
10:4511:30
Course 
Deep structure I:
Edge focusing, watershed segmentation,
ScaleSpaceViz demo 
Edge focusing, follicles
ScaleSpaceViz (VTK application) download 
STC 0.02 (Helix) 
11:4512:30
Course 
Deep structure II:
Follicle detection 
Edge focusing, follicles 
STC 0.02 (Helix) 
13:4516:30 Computer Lab 
Toppoints, image
retrieval 
Toppoints
in image matching
Toppoint detection 
GEMZ 3A06 
Thursday
20 October 2011 
10:0010:45 Course 
Steerable kernels 
Steerable
kernels 
GEMZ 1.03 
11:0011:45 Course 
Multiorientation
analysis, context 
Multiorientation
analysis 
GEMZ 1.03 
13:4516:30 Computer Lab 
Exercising FEV lectures,
exam tasks 

GEMZ 3A10 
Friday
21 October 2011 
09:3012:30 Computer Lab 
Exercising FEV lectures,
exam tasks 
Preparing the exam tasks. 
GEMZ 3A06 
13:4514:30 Course 
Rehearsal of the course, Questions & Answers, any topic of the course 
Canceled 
AUD 07 
14:4515:30 Course 
Rehearsal of the course, Questions & Answers, any topic of the course 
Canceled 
AUD 07 

Place:

*Lecture rooms:
MA: Matrix building
PAV: Paviljoen building
AUD: Auditorium
GEMZ: Gemini Zuid (Biomedical Engineering & Mechanical Engineering) building
GEMZ 3A.06 is a room with desktop computers, available for
exercising the course material with Mathematica 8.
See the campusmap
of TU/e for directions. 
Literature
The reader consists of the chapters of a the book: "FrontEnd Vision and Multiscale
Image Analysis", by Bart M. ter Haar Romeny. This book is written as a series of
Mathematica notebooks. It contains a CRROM
with all notebooks, which can be installed in the Mathematica Helpbrowser. The
Mathematica code is the
topic for the computer laboratories during the course. ISBN: 1402015070 (paperback), 1402015038 (hardcover). Springer,
Berlin.
Order the book with Springer or
Amazon.
NB: The book can also be downloaded with all
chapters, Mathematica notebooks and example images from
here (login with
your TU/e domain user/password). 

Recommended reading:
 L. M. J. Florack: The Structure of Scalar
Images. Computational Imaging and Vision Series, Dordrecht, Kluwer
Academic Publishers, 1996.
 B. M. ter Haar Romeny, Ed.: GeometryDriven
Diffusion in Computer Vision. Dordrecht, Kluwer Academic Publishers, 1994.
 L. M. J. Florack: Visuele Perceptie en Digitale Beeldverwerking, Nieuw
Archief voor Wiskunde, vol. 5/3, no. 1, pp. 3441, maart 2002.
PDF
 B. M. ter Haar Romeny: Computer Vision and Mathematica. Computing
and Visualization in Science, vol. 5, no.1, pp. 5365, Springer, 2002.
PDF
(1.7MB),
Mathematica
4 Notebook (3.1MB).
 T. Lindeberg: ScaleSpace Theory in
Computer Vision, Dordrecht, Kluwer Academic Publishers, 1994.
 J. Sporring, M. Nielsen, L. Florack and P.
Johansen (Eds.): Gaussian ScaleSpace, Dordrecht, Kluwer Academic
Publishers, 1996.
 J. Weickert: Anisotropic Diffusion in Image
Processing, ECMI Series, TeubnerVerlag, Stuttgart, Germany, 1998. 170
pages, hardcover. ISBN 3519026066.
 WebVision: the
visual cortex.
 Wikipedia on
scalespace.
Proceedings ScaleSpace conferences
 B. M. ter Haar Romeny, L. M. J. Florack, J. J. Koenderink, and M. A. Viergever, eds.,
ScaleSpace '97: Proc. First Internat. Conf. on ScaleSpace Theory in Computer Vision,
Utrecht, vol. 1252 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 1997.
 M. Nielsen, P. Johansen, O.F. Olsen, J. Weickert: ScaleSpace Theories in
Computer Vision, second
International Conference, ScaleSpace 1999, Corfu, Greece, September 2627,
1999, vol. 1682 of Lecture Notes in Computer Science. Berlin: Springer Verlag,
1999.
 M. Kerckhove: ScaleSpace and Morphology in Computer Vision, Third
International Conference, ScaleSpace 2001, Vancouver, Canada, July 78, 2001,
vol. 2106 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 2001.
 L.D. Griffin, M. Lillholm: ScaleSpace Methods in Computer Vision, Fourth
International Conference, ScaleSpace 2003, Isle of Skye, UK, June 1012, 2003,
vol. 2695 of Lecture Notes in Computer Science. Berlin: Springer Verlag, 2003.
 R. Kimmel, N. Sochen, and J. Weickert, Fifth International Conf. on
Scale Space and PDE methods in Computer Vision, Hofgeismar, Germany, April
2005, vol. 3459 of Lecture Notes in Computer Science. Berlin: Springer Verlag,
2005.
Other software:

MathVisionTools, a toolbox for
mathematical multiscale computer vision operations. Written by Markus van
Almsick, TU Eindhoven.

ScaleSpaceViz, a package to visualize the deep structure of images.
Written by Frans Kanters, TU Eindhoven.
Other:
Computer
Laboratory:
On the Wednesday mornings computer laboratories will be organized to
acquire handson experience with the discussed scaled differential invariant
operators on a variety of 2D and 3D images. We use the program Mathematica
8 (http://www.wolfram.com/).
For TUE members: Mathematica 8.0.1 for
Windows can be
downloaded from the
TU/e campus software website.
Mac OS x version: \\physstor\appl\macsoftware.
Linux version:
\\wtbfiler\Software\UnixSoftware.
Recommended tutorial books on Mathematica:
 N. Blachman & C.P. Williams: "Mathematica, a practical appraoch".Prentice
Hall 1999, second edition. ISBN
0132592010.
 H. Ruskeepää: "Mathematica Navigator, graphics and
methods of applied mathematics", Academic Press 1999, ISBN
0126036403 (paperback with CDROM.
Some useful notebooks:
The famous mathematics teaching files and resource online: MathWorld.
Examination:
Select any three (3) questions from this set of questions:
Exam tasks 2011.
Write a Mathematica 8 notebook per question and send them within two weeks
after the end of the course to B.M.terHaarRomeny@tue.nl.
Please explain the steps of your reasoning in detail, use Manipulate functions
if appropriate.
Make sure the notebook can run, so include your own images (store them in the
same directory as the notebook, and use SetDirectory[NotebookDirectory[]]), or
Import them from a web URL.
Some examples

The detection of ridges (midlines) for an Xray image of hands.
"Ridgeness" is a second order property. 


Low dose fluoroscopy image of an electrophysiology
catheter in ther heart. The extra low dose is beneficial for the
radiation dose, but leads to a deteriorated image quality. 
Robust catheter detection with oriented filters and tensor voting. 

Left: histological image of a fungus cell, paramecium caudatum.
Middle: edge detection with a detector optimized for yellowblue
differences. Right: edge detection with a detector optimized for redgreen
differences. 


Noisy 2photon microscopy
image of bone tissue. 
The same image enhanced with an
orientationscore
denoising filter. 
Contact the tutors:
prof.dr.ir. Bart M. ter Haar Romeny
Department of
Biomedical Engineering (BMEBMT)
Group Biomedical Imaging Analysis
BMIA Eindhoven University
of Technology Den Dolech 2  WH2.106 NL5612 AZ Eindhoven
Tel. 0031402475537 (secr. Margret Philips)
email: B.M.terHaarRomeny@tue.nl 
prof.dr. Nicolai Petkov
Department of Computing Science
Rijks Universiteit Groningen
Blauwborgje 3
NL9700 AV Groningen
Tel: 0031503633939 (secr.) / 3637129 / 3633931
Fax: 0031503633800
Email: N.Petkov@rug.nl

Class of 2011.