Robotica e Sensor Fusion per i Sistemi
Meccatronici
Object Detection with Superquadrics
Prof. Mariolino De Cecco, Dr. Ilya Afanasyev, Ing. Nicolo Biasi
Department of Structural Mechanical Engineering, University of Trento
Email: [email protected]
[email protected]
http://www.mariolinodececco.altervista.org/
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Object detection and localization
The block diagram of recognition task:
A-priori know:
- Dimensions;
- Features (shape,
curvature, etc.)
- Superquadrics,
etc.
- Optimization,
iterations, etc.;
- RANSAC;
Object
Measurement
system
Data
Math.
model
Preprocessing
- Camera, laser
- Kinect
- Multicamera system
- Point clouds
- Images, video
- Eliminating the
background;
- Changing the
contrast, size, etc.
Fitting Algorithm
- Criteria of quality.
Results
- Transformation matrix, angles,
distance, accuracy, etc.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
02/04/2
2/2
1
Examples of Superquadrics
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Examples of Superquadrics
[1]
Wireframes of Superquadrics
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Examples of Superquadrics
[1]
Examples of Superellipsoids
Superellipsoids can model spheres, cylinders, parallelepipeds
and shapes in between. Modeling capabilities can be
enhanced by tapering, bending and making cavities.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Examples of Superellipsoids
[4]
About Superquadrics
The term of Superquadrics was defined by Alan Barr in 1981
[2]. Superquadrics are a flexible family of 3D parametric
objects, useful for geometric modeling. By adjusting a
relatively few number of parameters, a large variety of
shapes may be obtained. A particularly attractive feature of
superquadrics is their simple mathematical representation.
Superquadrics are used as primitives for
shape representation and play the role
of prototypical parts and can be further
deformed and glued together into
realistic looking models.
[9]
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
About Superquadrics
Classification of Superquadrics
a)
[9]
d)
b)
a) Superellipsoids.
b) Superhyperboloids of one piece.
c) Superhyperboloids of two pieces.
d) Supertoroids.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Classification of Superquadrics [2,9]
c)
Definition of Spherical Products
For two 2x1 vectors [a b]’ and [c d]’ the spherical product is
3x1 vector for which:
Example. For two 2D curves: circle and parabola, the spherical product
is 3D paraboloid.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Definition of Spherical Products
Spherical Products
A 3D surface can be obtained by the spherical product of two 2D curves
[2]. The spherical product
is defined to operate on two 2D curves.
A unit sphere is produced by a spherical
product of a circle h(ω) horizontally and
a half circle m(η) vertically.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Spherical Products [9]
Spherical Products for Superellipsoids
The equation of Superellipse is
and in parametric form:
Superellipsoids can be obtained by a spherical products
of a pair of such superellipses:
The implicit equation is:
where
 1  2
- are parameters of shape squareness;
a 1 a 2 a 3
- parameters of Superellipsoid sizes.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Superellipsoids
Spherical Products for Superhyperboloids of one piece
Superhyperboloids of one piece can be obtained by a
spherical products of a hyperboloid and a superellipse:
The implicit equation is:
where
 1  2
- are parameters of shape squareness;
a 1 a 2 a 3
- parameters of Superellipsoid sizes.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Superhyperboloids of one piece
Spherical Products for Superhyperboloids of two pieces
Superhyperboloids of two pieces can be obtained by a
spherical products of a pair of such hyperboloids:
The implicit equation is:
where
 1  2
- are parameters of shape squareness;
a 1 a 2 a 3
- parameters of Superellipsoid sizes.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Superhyperboloids of two pieces
Spherical Products for Supertoroids
Supertoroids can be obtained by a spherical products of
the following surface vectors:
The implicit equation is:
where
 1  2
- are parameters of shape squareness;
- parameters of Superellipsoid sizes.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Supertoroids
Superquadrics in MATLAB
Modelling Superquadrics in MATLAB:
- Demo-function: xpquad - superquadrics plotting demonstration.
- ezmesh and ezsurf functions (see, Superquadrics_visualization.m).
%--------------------------------------------------------------% a1 = 4; a2 = 2; a3 = 1; eps1 = 1; eps2 = 1;
figure('Name','Superellipsoids')
ezmesh('4*sin(u)*cos(v)', '2*sin(u)*sin(v)', 'cos(u)', [0, 2*pi], [0, 2*pi], 20)
colormap([0 0 1])
hold on
rotate3d on
%---------------------------------------------------------------
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
02/04/2
15/
21
What a1, a2 and a3 mean?
a1, a2 and a3 – are parameters of
parallelepiped’s semi-sides.
If parallelepiped has dimensions:
20 x 30 x 10 cm, it means that
a1 = 10, a2 = 15 and a3 = 5.
The parameters of shape
squareness for parallelepiped are:
ε1 = ε1 = 0.1
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
16/20
What ε1 and ε2 mean?
ε1 and ε2 – are parameters of shape squareness.
ε2 = 0.1
ε2 = 1
ε2 = 2
ε1 = 0.1
[9]
ε1 = 1
ε1 = 2
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
17/20
Superellipsoids shapes varying from ε1, ε2
ε1 = 3
ε2 = 1
ε1 = 1
ε2 = 3
ε1 = 3
ε2 = 3
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
18/20
Plotting Superellipsoids in spherical coordinates
z
Vector r(η,ω) sweeps out
a closed surface in space
when η,ω change in the
given intervals:
a3
r(η,ω)
η
    
a2
y
a1
ω
x



 
2
2
 a cos( )  1 cos( )  2 
1


1
2
r (  )   a 2 cos( ) s in( ) 


1


a
s
in
(

)
3


η,ω – independent parameters (latitude and longitude angles)
of vector r(η,ω) expressed in spherical coordinates.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Superellipsoids
Plotting Superellipsoids in Cartesian coordinates
z
Use the implicit equation in
Cartesian coordinates, considering
-a1 ≤ x ≤ a1
f(x,y,z) = 1
-a2 ≤ y ≤ a2
z’
2
y=a2
x’
y’
x=a1
z=NaN
y


 x 
f ( x y z)   
 a 1 
2
2

2 1

2
y 

  
 a2  
z

 
 a3 
2
1
x,y – independent parameters (Cartesian
coordinates of SQ) are used to obtain z.
x
The implicit form is important for the recovery of Superquadrics and
testing for intersections, while the explicit form is more suitable for
scene reconstruction and rendering.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
20/20
Warning: complex numbers in SQ equation
1. If ε1 or ε2 < 1 and cos or sin of angles ω or η < 0, then vector r(η,ω) has
complex values. To escape them, it should be used signum-function of sin
or cos and absolute values of the vector components.



  
2
2
    
2. Analogically if x or y < 1 and ε1 > 1, the function f(x,y,z) willl have the
complex values of z. To overcome it, use the f(x,y,z) in power of
2
exponent ε1.


 x 
f ( x y z)   
 a 1 
2
2

2 1

2
 y 
  
 a2  
 z
 
 a3 
2
1
f(x,y,z)ε1 = 1
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
21/2
Rotation and translation of SQ
z
zW
xW
yW
Elevation
y
Azimuth
x
T – transformation matrix.
n – amounts of points in SQ surface.
SQ – coordinates of points of SQ surface.
Pw – coordinates of points of rotated SQ surface.
xW, yW, zW – world system of coordinates (with center in viewpoint).
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
22/20
Rotation and translation of SQ
z
zW
xW
px
yW
Elevation
y
Azimuth
T
SQ
x
az,el,px,py,pz,x,y,z – are given; xW,yW,zW – should be found.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
23/20
Pw
Applications with Superquadrics
Superquadrics have been employed in computer vision and robotics
problems related to object recognition. Superquadrics can be used for
1. Object recognition by fitting geometric shapes to 3D sensor data
obtained by a robot.
In order to fit a superquadric to a surface region, 11 parameters must
be determined: three extent parameters (a1, a2, a3), two shape
parameters (ε1 and ε2), three translation parameters, and three
rotation parameters.
2. Scene reconstruction and recognition. Rendering.
Shape reconstruction is a low level process where sensor data is
interpreted to regenerate objects in a scene making as few
assumptions as possible about the objects. Object recognition is a
higher level process whose goal is to abstract from the detailed data
in order to characterize objects in a scene.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Applications with Superquadrics
Figures from Superquadrics
[3]
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Definition of Superquadrics
Applications with SQ
Reconstruction of complex object [6]
Reconstruction of complex object [7]
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
Reconstruction of
complex
object [8]
26/2
Applications with SQ
Reconstruction of multiple objects [9]
Reconstruction of multiple objects [9]
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
05/04/2011
27/2
Task 1: To model Europallet in SQ
Dimensions of EuroPallet
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
02/04/2
28/
21
As an example of a Pallet model
 It
should
be
SQc
SQl
created
SQr
SQ model of pallet
z
z
z
d

y
Transformation: SQc – SQr
SQc
TSQr
1
0

0

0
0 0 d
1 0 0 
0 1 0

0 0 1
x

d
y
x

y
x
Transformation: SQc – SQl
SQc
TSQl
1
0

0

0
0 0  d
1 0 0 
0 1 0 

0 0 1 
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
02/04/2
29/
21
Task 2: To model objects by deformed SQ
Objects like an apple, a bottle, a chair, a tin of beer
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
02/04/2
30/
21
Links
1.
2.
3.
4.
5.
6.
7.
8.
9.
A. Skowronski, J. Feldman. Superquadrics. cs557 Project, McGill University.
http://www.skowronski.ca/andrew/school/557/start.html
Barr A.H. Superquadrics and Angle-Preserving Transformations. IEEE Computer Graphics
and Applications, 1, 11-22. 1981.
Chevalier L., etc. Segmentation and superquadric modeling of 3D objects. Journal of
WSCG, V.11 (1), 2003. ISSN 1213-6972.
Kindlmann G. Superquadric Tensor Glyphs. EUROGRAPHICS. IEEE TCVG Symposium on
Visualization (2004). Pages 8.
Solina F. and Bajcsy R. Recovery of parametric models from range images: The case for
superquadrics with global deformations. // IEEE Transactions on Pattern Analysis and
Machine Intelligence PAMI-12(2):131--147, 1990.
Chella A. and Pirrone R. A Neural Architecture for Segmentation and Modeling of Range
Data. // 10 pages.
Leonardis A., Jaklic A., and Solina F. Superquadrics for Segmenting and Modeling Range
Data. // IEEE Transactions On Pattern Analysis And Machine Intelligence, vol. 19, no. 11,
1997.
Bhabhrawala T., Krovi V., Mendel F. and Govindaraju V. Extended Superquadrics. //
Technical Report. New York, 2007. 93 pages.
Jaklic Ales, Leonardis Ales, Solina Franc. Segmentation and Recovery of Superquadrics. //
Computational imaging and vision 20, Kluwer, Dordrecht, 2000.
M. De Cecco - Lucidi del corso di Robotica e Sensor Fusion
Grazie per attenzione!!