EU Marie Curie Initial Training Newtwork 
'Metric Analysis for Emergent Technologies' 

  

 

BME 8DM00 and ASCI course a8:
Front-End Vision
Multi-Scale and Multi-Orientation Image Analysis

Fall 2015


[Image of Scale-Space Cube]

An introduction to modern multi-scale and multi-orientation image analysis
inspired by biological vision, and applications in medical imaging

A PhD / MSc course given at the Department of Biomedical Engineering of Eindhoven University of Technology.

Tutors: Prof. Bart M. ter Haar Romeny, Eindhoven University of Technology / Northeastern University (Shenyang, China)
Prof. Nicolai Petkov, University of Groningen
Dates: 2 fulltime weeks.
Lectures (half day) from Monday 9 November 2015 till Friday 13 November 2015, and from Monday 16 November 2015 till Thursday 19 November 2015.
Computer laboratories (other half day) using Mathematica 10 to exercise the course material and tasks.

Registration:

Register through the ASCI website: here.
Non-ASCI members send an email to: R.A.v.Dijck.Postma@tue.nl.
TU/e BME students register through the regular TU/e OASIS page for 8DM00.


Registration for course and exam is free for TU/e, ASCI, NFBIA, ImagO students, and employees of industries officially collaborating with TU/e BME.
For registration as 'contractant' with TU/e to do an official exam for 3 ECTS as non-TU/e student: see STU registration form.
Costs: € 500 per course.
Costs for industrial participants: € 1200 (invoice will be sent by ASCI after registration).
Venue Campus of Eindhoven University of Technology, Eindhoven, the Netherlands.
Google Maps, TU/e campus map.
Hotel Suggestions (Booking.com)

Total duration: 27 oral lectures of 45 minutes each, and 27 hours hands-on training.

Code ASCI: a8 (4 ECTS study points).
Code TUE-BME: 8D010 (2.5 ECTS study points).
Code MANET: Training (4 ECTS study points).
 

Description:

Image analysis is the extraction of useful information from images. When we need to define a task on an image, in order to detect, enhance, register, recognize etc, in other words in order to translate the question of the clinical expert into an algorithm, we need a language for image analysis. In this course we give a modern mathematical (and physics based) approach to such a language: multi-scale differential geometry for (medical) image analysis as a branch of computer vision.

We give an intuitive introduction to multi-scale image analysis, trying to keep the analogy with stages in the human visual system as close as possible. The human visual system also widely exploits a diversity of multi-scale filters in its processing layers. We will treat the front-end visual system in depth, especially its receptive field structures in retina and primary visual cortex, and put emphasis on the Gaussian derivative and the Gabor model for simple cortical receptive fields.
We study in detail the regularized measurement of derivatives by an early vision system, and its inherent multi-scale structure, and why this is necessary. The discovery of the so-called pinwheel structure in the cortical columns inspired a well-founded multi-orientation image analysis, which gives deeper insight in contextual processing.

You will learn how geometric reasoning works and can be applied. We design image analysis algorithms by carefully studying the requirements, physical analogies, and in particular by looking how our visual system does it. After all, this is still the best performing recognition computer we know, even for noisy, partly missing (occluded) data, low contrast etc. Modern (often optical) brain imaging methods will be discussed (voltage sensitive dyes, opto-genetics, fMRI, DTI/HARDI) and recent discoveries of functional brain mechanisms in visual perception.

Among the topics covered are: high-order derivative filters for 2D and 3D images, detecting edges, ridges, corners, T-junctions etc. in images, multi-scale analysis of 2D and 3D shape and motion from image sequences, depth from stereo, multi-orientation analysis for contextual operations, and the use of contemporary, well-understood mathematical tools from differential geometry and tensor analysis (differential invariants, coordinate transformations, gauge coordinates).

The majority of the examples discussed are from 2D, 3D and 4D (3D-time) medical imaging. We devote some time to the efficient numerical implementation of the different techniques. Hands-on experience is acquired in a computer lab. We use Mathematica 10 as this software suite is eminently suited for this design process, and we experiment hands-on with virtually all aspects discussed in the course.

Topics discussed:

Computer vision plays an increasing role in the detection and recognition of structures, quantitative analysis, segmentation and visualization. 

A modern development is computer-aided diagnosis (CAD), where the computer assists in finding possible pathology in images, particularly for screening applications. We discuss several examples in detail:

Detailed program and content:

Times:
Day Time Content Lecture material Lecture hall
(see campus map)
Monday
9 November 2015
13:45-14:30 Course Introduction
Registration

Notion of scale

  Introduction  PDF
Powers of 10  Youtube
GEM-Z 3A13
14:45-15:30 Course Axiomatics of multi-scale operators   Gaussian aperture from entropy (NB: MMA 8)
Gaussian aperture from entropy

  Entropy.nb
(supplementary material)

GEM-Z 3A13
Tuesday
10 November 2015
09:45-10:30 Course The Gaussian kernel Gaussian kernel

Dither removal

GEM-Z 3A13
10:30-10:35 Group picture taken   Gemini-Z entrance hall
10:45-11:30 Course Gaussian derivatives Gaussian derivatives GEM-Z 3A13
11:45-12:30 Course Introduction to Mathematica 10 Wolfram: Hands-on start

Wolfram: learning Center

Tutorial Mathematica notebooks of TU/e course 8ZZ16:

Course part 1 of 3 (English)
Course part 2 of 3 (English)

Course part 3 of 3 (English)
 


MathVisionTools.zip

BMIA MMA 10 course

Image Processing with Mathematica 10 (by Markus van Almsick, Wolfram Research Inc.)

GEM-Z 3A13
13:45-14:30 Computer lab Exercises with Mathematica 10 Download the FEV book (will be updated soon)

MathVisionTools (containing an extensive package for Gaussian derivatives):  www.mathvisiontools.net

(Run the command:
$AddOnsDirectory
in Mathematica, and store in this directory)

Images of the book & test images
(Data.zip, 31 MB, extract in MathVisionTools\Data directory)

shortnotation.nb

Tasks I (and answers)

Some testimages to play with.

GEM-Z 3A13
14:45-15:30 Computer lab
 
Exercises with Mathematica 10 Additional study material:
Eindhoven Tips

Handy.nb
Dictionary manipulations
Often used commands

FrondEnd Interactivity
Demo active shape
Wolfram Inc.
Eduroam (WiFi network access at TU/e through SurfNet)

GEM-Z 3A13
15:45-16:30 Computer lab Exercises with Mathematica 10   GEM-Z 3A13
Tuesday
10 November 2015
15:45-16:30 Computer lab

 

(Rehearse from first year 8C120:

Convolution01
Convolution02
Convolution03

GEM-Z 3A13
Wednesday
11 November 2015
09:45-10:30 Course Deblurring Deblurring
Deblurring  PDF
GEM-Z 3A13
10:45-11:30 Course Gauge coordinates

Second order differential structure

Differential structure

Paper: B.M. ter Haar Romeny, A Geometric Model for the Functional Circuits of the Visual Front-End. Lecture Notes in Computer Science, vol. 8603, pp 35-50, 2014.

GEM-Z 3A13
11:45-12:30 Course Affine corner detection   GEM-Z 3A13
12:45-13:30 Colloquium BME Staff Lunch Colloquium

Open for FEV participants:

- bring your own chair
- bring your own lunch (from the BME canteen or elsewhere)

Prof. Bart ter Haar Romeny:

‘Brain-Inspired methods for Retinal Image Analysis’

Powerpoint

PDF

www.retinacheck.org

GEM-Z 4.24
13:45-16:30 Computer Lab Exercising FEV lectures Tasks 02

Some answers Tasks 02

MetaForum 14
Thursday
12 November 2015
09:45-12:30 Computer Lab Exercising FEV lectures   AUD 14
13:45-14:30 Course Limits on observations

Implementation of Gaussian convolutions

 

Limits
Limits   PDF

Implementation

GEM-Z 3A05
14:45-15:30 Course Regularization

Applications of second order differential structure

Regularization  PDF

Applications of second order structure

Vesselness

Original paper: A. Frangi, W. Niessen, K. Vincken and M. Viergever, Multiscale vessel enhancement filtering. In: Proceedings of the MICCAI’98Lecture Notes in Computer Science vol. 1496, Springer-Verlag, Berlin (1998), pp. 130–137. 

 

GEM-Z 3A10
15:45-16:30 Course Third order differential structure   GEM-Z 3A10
Friday
13 November 2015
09:45-12:30 Computer Lab Exercising FEV lectures Tasks 03

Tasks 03 (with solutions)

Tasks 04

Tasks 04 (with solutions)

Helix West 3.91
13:45-14:30 Course Front-end visual system I Front-End Vision.pptx

  Front-End-Vision.pdf

Front-End-Vision.zip
(pptx with movies)

GEM-Z 3A13
14:45-15:30 Course Front-end visual system II Youtube: Hubel & Wiesel - Cortical neuron V1

Youtube: Hubel's research

Youtube: Hubel: Brain & Perception

GEM-Z 3A13
15:45-16:30 Course Front-end visual system III Visual illusions
(Michael Bach)

Visual illusions
Spiral illusion (Win32) Spiral illusion (Youtube)

GEM-Z 3A13
Monday
16 November 2015
09:45-12:30 Computer Lab Exercising FEV lectures Tasks - visual system GEM-Z 3A10
13:45-14:30 Course Receptive fields from eigenpatches

Geometry-driven diffusion I

 

Eigenpatches

Brain development: the experiments of Blakemore and Cooper.

Geometry-Driven Diffusion
Geometry-Driven Diffusion

Helix Oost 4.91
14:45-15:30 Course Geometry-driven diffusion II PDF: Original paper:
P. Perona, J. Malik, "Scale-space and edge detection using anisotropic diffusion", PAMI 12(7), pp. 629-639, 1990. 
Helix Oost 4.91
15:45-16:30 Course Scale-time: differential structure of time sequences Scale-time
Scale-time

Reference:
J. J. Koenderink, Scale-time, Biological Cybernetics
November 1988, Volume 58, Issue 3, pp 159-162.

Helix Oost 4.91
Tuesday
17 November 2015
09:45-12:30 Computer Lab Exercising FEV lectures

Tasks 05

Tasks 05 (with some solutions)

GEM-Z 3A13
13:45-14:30 Course V1-inspired orientation selective filters for image processing and computer vision (Gabor and CORF)

References:
G. Azzopardi, N. Petkov: A CORF computational model of a simple cell that relies on LGN input outperforms the Gabor function model. Biological Cybernetics, 106 (3), 2012, 177-189 (pdf)

Azzopardi, G., Strisciuglio, N., Vento, M. and Petkov, N.: Trainable COSFIRE filters for vessel delineation with application to retinal images. Medical image analysis, 19 (1): 46-57, 2015. (pdf)

Azzopardi, G., Rodriguez-Sanchez, A., Piater, J., Petkov, N.: A Push-Pull CORF Model of a Simple Cell with Antiphase Inhibition Improves SNR and Contour Detection. Plos one, 9 (7), article nr. 98424, 13 pages, 2014.(pdf)

Azzopardi, G., Petkov, N: Ventral-stream-like shape representation: from pixel intensity values to trainable object-selective COSFIRE models. Frontiers in Computational Neuroscience, vol. 8, nr. 80, 1-9, 2014.(pdf)

Prof. Nicolai Petkov

Lecture slides 01 (Orientation selectivity)

GEM-Z 3A13
14:45-15:30 Course Surround suppression (or non-classical receptive field inhibition in areas V1/V2)

References:
C. Grigorescu, N. Petkov, M.A. Westenberg, Contour and boundary detection improved by surround suppression of texture edges, Image and Vision Computing 22, 609–622, 2004. (pdf)

C. Grigorescu, N. Petkov, M.A. Westenberg, Contour detection based on nonclassical receptive field inhibition, IEEE Tr. on Image Processing, vol. 12, 7, 729-739, 2003 (pdf)

N. Petkov, M.A. Westenberg, Suppression of contour perception by band-limited noise and its relation to non-classical receptive field inhibition, Biol. Cybern. 88, 236–246 (2003) (pdf)

Prof. Nicolai Petkov

 

Lecture slides 02 (surround suppression)

GEM-Z 3A13
15:45-16:30 Course What comes after V2? Computational model of shape selective neurons in area V4, with application to the detection of blood vessel bifurcations in retinal funds images.

Reference:
G. Azzopardi, N. Petkov: Trainable COSFIRE filters for keypoint detection and pattern recognition. IEEE Trans. on Pattern Analysis and Machine Intelligence, 35 (2), pp. 490-503, 2015. (pdf)

Prof. Nicolai Petkov

Lecture slides 03
(CORF keypoint detectors)

GEM-Z 3A13
Wednesday
18 November 2015
09:45-10:30 Course Color Differential Structure Color differential structure
Color differential structure

Color Invariants

Paper: Color RF

Helix West 3.91
10:45-11:30 Course Deep structure I:
Edge focusing, watershed segmentation,
ScaleSpaceViz demo
Edge focusing, follicles

ScaleSpaceViz (VTK application) download

Helix West 3.91
11:45-12:30

 Course

Deep structure II:

Toppoints, image retrieval
Follicle detection

Edge focusing, follicles

Toppoints in image retrieval

Topological numbers

Helix West 3.91
13:45-16:30 Computer Lab Exercising FEV lectures, exam tasks

 

HEO 4.91
Thursday
19 November 2015
09:45-12:30 Computer Lab Exercising FEV lectures, exam tasks   Auditorium 14
13:45-14:30 Course Steerable kernels, stellate tumor detection Steerable kernels GEM-Z 3A05
14:45-15:30 Course Multi-orientation analysis, context

Reference:

Franken, E.M., Duits, R., ter Haar Romenij, B.M., Nonlinear diffusion on the 2D Euclidean motion group. Lecture Notes in Computer Science, Vol. 4485, pp. 461-472, 2007.(pdf)

Multi-orientation analysis GEM-Z 3A05
15:45-16:30 Course Application of orientation analysis: RetinaCheck

Reference:

Bekkers, E.J., Duits, R., Berendschot, T.T.J.M. & ter Haar Romenij, B.M. (2014). A multi-orientation analysis approach to retinal vessel tracking. Journal of Mathematical Imaging and Vision, 49(3), 583-610.(pdf)

RetinaCheck (new)

www.retinacheck.org

GEM-Z 3A05
 
*Lecture rooms at TU/e campus, see map:

FLX: Flux building (Applied Physics & Electrical Engineering)
GEM-Z: Gemini South building (Biomedical Engineering & Mechanical Engineering)
GEM-N: Gemini North building (same entrance as Gemini-South).
HEO: Helix-East bulding (ST - Chemical Technology)
HEW: Helix-West building (ST - Chemical Technology)
Matrix: Matrix building.
 

Literature

The reader consists of the chapters of a the book: "Front-End Vision and Multi-scale Image Analysis", by Bart M. ter Haar Romeny. This book is written as a series of Mathematica notebooks. It contains a CR-ROM with all notebooks, which can be installed in the Mathematica Help-browser. The Mathematica code is the topic for the computer laboratories during the course.

ISBN: 1-4020-1507-0 (paperback), 1-4020-1503-8 (hardcover). Springer, Berlin. 
Order the book with Springer or Amazon.

Recommended reading (books):

Recommended reading (websites):

Proceedings of all "Scale-Space & Variational Methods" conferences:

Software:

Other:

Computer Laboratory:

Computer laboratories will be organized to acquire hands-on experience with all discussed topics on a variety of 2D and 3D images. We use the program Mathematica 10 (www.wolfram.com).

For TUE members: Mathematica 10 for Windows can be downloaded from the TU/e campus software website.
Mac OS x version:
 \\physstor\appl\mac-software.
Linux version:
\\wtbfiler\Software\Unix-Software.

Recommended tutorial books on Mathematica:

 Some useful notebooks:

The famous mathematics teaching files and resource online: MathWorld.


Examination:

Select any three (3) questions from this set of questions: ExamTasks_FEV2015.nb.    

Write a Mathematica 10 notebook per question and send them, preferably within two weeks after the end of the course, to B.M.terHaarRomeny@tue.nl.
Please explain the steps of your reasoning in detail, use Manipulate functions if appropriate.
Make sure the notebook can run, so include your own images (store them in the same directory as the notebook, and use SetDirectory[NotebookDirectory[ ]]), or Import them from a web URL.


Some examples

The detection of ridges (midlines) for an X-ray image of hands. "Ridgeness" is a second order property.

Low dose fluoroscopy image of an electro-physiology catheter in ther heart. The extra low dose is beneficial for the radiation dose, but leads to a deteriorated image quality. Robust catheter detection with oriented filters and tensor voting.

Left: histological image of a fungus cell, paramecium caudatum.
Middle: edge detection with a detector optimized for yellow-blue differences.
Right: edge detection with a detector optimized for red-green differences.

Noisy 2-photon microscopy image of bone tissue.

The same image enhanced with an orientation-score denoising filter.

Contact the tutors:

prof. Bart M. ter Haar Romeny, PhD
Department of Biomedical Engineering (BME-BMT)
Group Biomedical Imaging Analysis BMIA
Eindhoven University of Technology
De Rondom 70 - GEM-Z 2.106
NL-5612 AP Eindhoven, Netherlands
Tel. +31-40-2475537 (secr. Rina van Dijck)
email: B.M.terHaarRomeny@tue.nl
prof. Nicolai Petkov, PhD
Department of Computing Science
Rijks Universiteit Groningen
Blauwborgje 3
NL-9700 AV Groningen, Netherlands
Tel: 00-31-50-3633939 (secr.) / 3637129 / 3633931
Fax: +31-50-3633800
Email: N.Petkov@rug.nl 

Click for larger version.

Class of Fall 2015.