Optical Design of a Variable View Imaging System with Combination of a Telecentric Scanner and Double Wedge Prisms

Xiaodong Tao 1, Hyungsuck Cho,1,* and Farrokh Janabi-Sharifi2

1

Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology, 373-1, Guseong-dong, Yuseong-dong, Daejeon, Korea

Department of Mechanical and Industrial Engineering, Ryerson University, 350 Victoria Street, Toronto,Canada *Corresponding author: hscho@kaist.ac.kr Recent developments in the micro-electro-mechanical system (MEMS) and biotechnology have presented an emerging need for observation of dynamic targets in three-dimensional space. Unfortunately, the conventional microscopes with fixed optical parameters are difficult to supply sufficient vision information because of occlusion, small field-of-view (FOV) and low depth resolution. This paper introduces the design of a variable view imaging system which can supply a flexible view with a relative large zenith angle and a simple kinematics. Due to several performance factors, a multiobjective optimization process is applied to achieve an appropriate design. A prototype system is developed and used to verify the proposed design.

OCIS codes: 110.1080, 220.1080, 120.4820, 150.3040, 170.0110, 170.0180, 220.4830.

1. Introduction
The small field-of-view (FOV) and fixed view direction of conventional microscope cannot supply sufficient visual information needs for observation of dynamic targets in threedimensional (3D) space, particularly in the fields of the micro-electro-mechanical systems (MEMS) and biotechnology applications. In the past, there have been several research efforts to solve this problem. Multiple fixed microscopes or moving stages are often applied in those applications [1,2]. Such approaches do not provide sufficient flexibility for observation of dynamic targets. A smart optical system with adjustable optical parameters becomes a promising solution. Potsaid has proposed an adaptive scanning optical microscope (ASOM) [3]. Through the integration of a scanning mirror and deformable mirror, the ASOM can obtain a large FOV with high resolution. A similar methodology is also applied in a scanning optical telescope [4]. Another system capable of achieving foveated viewing in a large FOV has been demonstrated using a spatial light modulator (SLM)[5]. In our previous research, a proposal was made for an active optical system which can change the view position and direction [6,7]. However, because of the coupling effect between the scanning mirror angle and view angle, the view angle is found to vary with scanning mirror, causing its zenith angle of view to decrease, making the kinematics complex. In order to solve those problems, a decoupled design of the system is proposed in this paper, which employs a telecentric lens. The basic concept of the system is presented in SPIE conference [8]. Since this design contains a number of parameters that affect the performance of te system, optimal election of these parameters is necessary. For this, an multiobjective optimization design process is described in this paper in detail for the first time. Experimental results are presented to show the validity of the proposed design.

2. Concept of the variable view imaging system

The objective of the design of the variable view imaging system is to change the view state by control of active optical components in the system. The view state includes view position and view angles defined as Eq. 1.
V = xw yw

(1)

where (xw,yw) is the view position in a world coordinate. and are the azimuth angle and the zenith angle of the view. In the robot area, by installing a camera on a robot, an active vision system can change the view state easily. However, considering the size and weight of vision system and speed of scanning, it is difficult to install a microscope system on a robot. In order to realize this function, the idea is to design the vision system with active components which mimic the function of robot. Fig. 1 shows the concept of the system. To steer the view angle (, ), double wedge prisms are applied because the compact size for deflection of light. The function of double wedge prisms is similar to a camera installed on a pan/tilt stage. It can steer the beam in any direction in a cone. To steer the view position (xw,yw), the telecentric scanner is applied. Its function is similar to a camera installed on the translation stage. Therefore by integration of these two active optical components, the proposed system can steer the both of view position (xw,yw) and orientation (, ). It can be considered as a camera installed on a robot with four degrees of freedom as shown in Fig. 1. In our previous research, a scanning mirror and double wedge prisms were used for view steering [6,7], where the scanning mirror angle was coupled with the zenith angle of the view. The zenith and azimuth angles changes with the change of the scanning mirror angle. There are several issues for the old design. The first issue is a small zenith angle. When the target locates near the

optical axis, the motion of scanning mirror decrease the zenith angle of view because the coupling effect between the scanning mirror angle and view orientation. The second issue is the complex kinematics relationship. Each variable in the view state is related to all variables in joint space as V=[fy(),fx(),f(),f()]T, where =[sx, sy, p1, p2] T is the vector of joint variables for scanning mirror and wedge prisms, respectively. fy(),fx(),f() and f() are the functions to calculate the four variables in the view state as shown in Eq. 1. The calculation of joint variables for a desired view state becomes difficult. And a precise alignment and calibration are required to obtain an accurate kinematics equations. To decouple the rotation angle of scanning mirror and view angles, a telecentric lens group is designed and introduced to the system. The system aperture is projected on the focal point of the telecentric lens. It makes the entrance pupil infinitely. Therefore in the object side, the chief rays are always parallel to the optical axis. The layout of the whole system is shown in Fig.2. The 3rd and 4th lenses integrate a deformable mirror into the system. The deformable mirror is used for correction of aberration induced by wedge prisms. The 4th and 5th lenses relay the active aperture of the deformable mirror to the system aperture. The system aperture is projected onto the scanning mirror, which is located at the focal point of the telecentric lens. It makes an entrance pupil infinitely. Therefore the rays are always parallel to the optical axis in the object side. The azimuth angel and zenith angle of view are only related to the rotation angle of the prisms. In order to determine view state given the joint variables, a rigorous analysis of the view position and angle is made by the ray tracing method. The forward kinematics of the system can be defined as follows,
V = f x ( sx , sy , p1 , p 2 ) f y ( sx , sy , p1 , p 2 ) f ( p1 , p 2 ) f ( p1 , p 2 )
T

(2)

where V(xw, yw, , ) is the vector for view state. And (sx, sy, p1,p2 ) includes the joint variables for scanning mirror and wedge prisms. The detail derivation can be found in [7]. The inverse kinematics can be achieved by using a damped least-squares (DLS) method, which can also be refer to [7].

3. Optimal design of the view steering part

The view steering part include scanning mirror, telecentric lens and wedge prisms. The performances of the system, such as scanning area, maximum zenith angle, wavefront error are all related to this part. In this section, the system performance is defined. Because several system performance factors are involved, a muliobjective optimization process is applied to give an appropriate design.

A.

System performance for the system design

System performance can be identified from the analysis of the functions of the system. The main function of the proposed system is to avoid the occlusion by changing the view angle. The effects of system parameters for the occlusion case can be shown in Fig. 3, where the image of a cubic with a depth D is captured by the proposed system. In the top view, the corner point B is occluded by the point A as shown in the figure. With a zenith angle , both point A and B can be observed in the image plane. Because of the telecentric lens group used in the system, the scaled orthographic projection model can be applied in the system. The distance between A and B in the image plane is defined as following equation.

d AB = Mdo

(3)

where M is the magnification of the system. d0 is projected length of A and B along optical axis, which is defined as

d0 = D sin( )

(4)

In order to avoid occlusion, the distance between A and B need to be observed in the image plane. Therefore the condition for the proposed system to avoid occlusion is as follows,

ro < D sin( )

(5)

where ro is the optical resolution of the system. We can find that with high resolution r and larger zenith angle , the system can easily avoid occlusion. Therefore the maximum zenith angle and the resolution are two important performance factors for occlusion avoidance. Another function of the proposed system is to increase the field of view by steering the view position. In the whole scanning area, the area A2 with full azimuth angle with maximum zenith angle is most important as shown in Fig. 4. In this area, the system can achieve the view with an azimuth angle from 0 to 2 and a zenith angle from 0 to maximum zenith angle. As an optical system, image quality is also an important performance factors. Because the wedge prisms are used in the system, a relative large wavefront error is induced into the system. In this design, a deformable mirror is applied to correct this aberration. However the wavefront error correction is limited due to the limited stroke of the actuator in the deformable mirror. In the proposed system, a micromachined membrane deformable mirror (37ch, D15mm, OKO) is applied. The limitation of wavefront correction for this mirror is 3.5 (wavelength =0.660 m) according to [10]. Therefore in system design, the wavefront error should be kept inside this limitation. The details calculation of these system performances are described as follows.

Maximum zenith angle, max

When two wedges have the same rotation angle, the zenith angle achieves its maximum value as shown in Fig. 5(a). Therefore for a given v, the maximum zenith angle is defined as

max = f (0, 0)

(6)
v

Therefore max is a function of the vertex angle of the wedge prisms defined as follows,

max = F ( v )
max

(7)

Fig. 5 (b) shows the final result of the relationship. Larger vertex angle of the wedge prisms can provide a larger zenith angle.

Radius of view area, r2

The configuration of the system for calculation of radius of A2 is shown in Fig. 6(a). The radius of the prism is defined as rp. The view will located at the boundary of the A2 when

xs = rp , ys = 0 , p1 = 0 , p 2 = 0 . Therefore the radius of A2 for a given v can be obtained

from following equation,

r2 = f x (rp , 0, 0 , 0 )

(8)

r2 is a function of the radius and vertex angle of wedge prism, defined as follows,

r2 = Fr2 (rp , v )

(9)

This relationship is shown in Fig. 6(b). Comparing to the vertex angle v of prisms , the radius of the prism gives more effects on the change of r2.

Optical Resolution

The Optical resolution is the ability of a microscope to allow one to distinguish between small objects. The optical resolution ro is depend on the numerical aperture as shown in Eq. 9.[3]

ro =

0.61 NA

(10)

where is the wavelength. NA is the numerical aperture, which characterizes the range of angles over which the system can accept. It can be defined as following equation.
NA = n sin( / 2)

(11)

where n is the refraction index, is the angle of the maximum cone of light that can enter the objective lens.

Peak-Valley Aberration, WPV

The aberration of the system is variable depending on the wedge prisms angle and scanning mirror angle. It can be corrected by a deformable mirror which was applied in the system. However, because the correction limit of the deformable mirror, the peak-valley wavefront error should small than the correction range of the deformable mirror. For a specific design, the peakvalley wavefront is depending on the configuration of the prisms and mirror. The worst configuration with the maximum peak-valley wavefront error can be obtained from Zemax software as shown in Fig. 7(a), where xs=rp, ys=0, p1=0,p2=180. The maximum peak-valley

wavefront error is a function of vertex angle of prisms, radius of the prism and numerical aperture defined as follows,
WPV = FW pv (rp , v , NA)

(12)

It is difficult to achieve this relationship analytically. From Zemax software, the warefront errors for different rp, v and NA can be obtained easily as shown in Fig. 7(b). However, this discrete result is not accurate enough for the system design. Therefore a Backpropagation (BP) networks are built for function approximation [11]. In this design, a standard BP networks with 2 hidden layers is applied. Each hidden layer has 10 neurons. The Tan-sigmoid transfer functions are applied in the networks. The data from the Zemax software was used for training data.

B.

Optimized Design of the system

The objective of the optimized design is to achieve best performance of the system. In this design, there are four system performance factors for optimization. The object functions can be formulated as follows,
max r2 , max max , max ro , min W pv
rp , v

NA

rp , v , NA

(13)

When the deformable mirror is applied the system, because the wavefront error inside a specific range can be corrected by the deformable mirror, the Wpv can be a boundary constraint in system design. Because the relationship between ro and NA defined in Eq. 9, ro can be replaced by NA to decrease the number of parameters for the design. Therefore the objectives of the system design can be simplified as following equation,

max r2 , max max , max NA

rp ,v rp ,v

(14)

Subject to Wpv (rp , v , NA) <

where is the correction limit of wavefront error for the deformable mirror. In this design, is set as 3.5 . As can be seen from Eq. 14, the optimal design becomes a multiobjective

optimization problem. The performance factors are conflicted with each other. For example, The increase of vertex angle v of the prisms will increase the maximum zenith angle max, and also increase the wavefront error and decrease the radius of view area, r2. The increase of rp increases the radius of view area, r2, and also increases the wavefront error. Therefore, we can not get a single optimized solution from Eq. 14. In order to solve this Multi-Objective optimization problem, -constraint method is applied in this design according to [12]. This method is to select one of the objective functions to be optimized and all the other objective functions become constraint. This method can simplify the multiobject optimization problem to single objective optimization. In this design, the boundary of the numerical aperture NA, radius of view area r2 can be defined based on the application. The objective of the design is to maximize the zenith angle max. Therefore the optimization problem can be simplified as

maximize max
NA , rp ,v

(15)

Subject to r2 K r , NA K NA , W pv < Kr and KNA are the boundary of the r2 and NA, which can be determined based on the application by the designer. In order to solve the inequality-constrained optimization, a Logarithmic barrier method is applied [34]. The boundary of view area is determined by the application. In the microassembly application, the Kr is set as 5 mm. KNA is 0.03 in this design. The design result is shown in Table 1. As can be seen, the maximum zenith angle, max is 22.29. In the real experiment, the in-stock components are selected with the most similar value achieved in the optimal design. The parameters for the real experiment setup are shown in Table 2. Based on the catalog of the wedge

10

prisms, the maximum vertex angle of the wedge prisms are 188, which gives a 19 degree of zenith angle. The zenith angle in the real experiment setup is only 3.3 less than the optimal design result. As discussed in the Section 3.1, the NA and max are two important factor for occlusion avoidance. By setting different KNA, the relationship between NA vs. max can be achieved from the optimal design solutions as shown in Fig. 8. As can be seen, when NA decrease to 0.02, 30 degree of zenith angle max can be achieved. When the NA increases to 0.1, only 4 degree of zenith angle max can be achieved. This tradeoff relationship between NA and max is caused by the limitation of wavefront error correction for the deformable mirror. The NA and max in the current experiment system is shown in the Fig. 8, which is near the optical design set.

4. Experiment results
In order to investigate the feasibility of the proposed optical system design, A prototype system was constructed according to the Table 3. The 37-channel MMDM was supplied by OKO Technologies. The detailed information for wavefront correction can be referred to [10]. The deviation angle of one wedge prism was 10. The wedge prisms were installed on the compact rotation stages. Four achromatic lenses with effective focal length of 150 mm were used in the system.

A. Experiment for the real object

The ability of the proposed system to change the view without moving the object was demonstrated by multiple views of micro gears. Three views with different configurations are shown in Fig. 9(a) ~ (c). Fig. 9 (a) shows the view with a zenith angle =18.9 and an azimuth angle =90, where the hole in the 2nd gear can be observed. Fig. 9 (b) shows the view with a

11

zenith angle =18.9 and an azimuth angle =0. The depth of the 2nd gear can be observed. Fig. 9 (c) shows the view with a zenith angle =18.9 and an azimuth angle =270, where the other side of the 2nd can be observed. In order to compare the imaging results with the previous system [7], the view state for the previous system is achieved in the proposed system. The max,2, for the previous system is determined from system kinematics, which equals to 13.8[7]. The three view states with zenith angle =13.8 is achieved as shown in Fig. 9(d)~(f). The interested features on the side of gears on the image plane are much smaller than the current system. The heights of gears, h1, h2, on the image plane for the previous system are 26 and 18 pixels respectively. In the proposed system, they are 36 and 25 pixels respectively, which is more than 35 percent larger than the coupled system. During online operation, the joint variables for a desired view state need be calculated in a short time. The numerical solution based on the damped least-squares (DLS) method is implemented, which is similar to the previous system [7]. Because of the simplified kinematics, only 23 iterations are need for calculation of joint variables, which is approximately 32ms in the current system. However, for the previous system, more than 350 iterations are needed, which will spend 490ms.

B. Experiment for the real application

In the microassembly experiment, a microassembly system including three translational motions was installed near the proposed system. The preliminary experiment setup was only intended to evaluate the performance of the system; therefore, the microassembly system was teleoperated without automation. The task is to insert the micro electrode into the micro slot as shown in Fig. 10 (a). The widths of slot and electrode are 300um and 250um respectively. The electrode is held by a gripper. As shown in Fig. 10 (b). At an initial state, the view angle is set as

12

=0 and =0. The captured image is shown in Fig. 10 (b). The micro electrode will align with

the slot in X direction as shown in Fig. 10 (c). However, the insertion part of the electrode cannot be observed because of occlusion. In order to align the electrode and slot in Y direction, the view angle is steered by the proposed system, and Fig. 10 (d) shows the image when a zenith angle is 19 and the azimuth angle is 0. With the visual information of the side surface, the electrode is aligned with the slot in Y direction as shown in Fig. 10 (e). The final assembly task is to insert the electrode into the slot. Fig. 10 (f) shows a final state of the assembly.

5. Conclusion
This paper presents the design of a variable view imaging system which can change the view angle and view position by actuating active optical components in the system. The proposed system with a telecentric lens group and double wedge prisms can supply a larger zenith angle of view and simpler kinematics relationship compare with the previous system [7]. During the system design, multi performance factors are identified. In order to obtain the best performance, a multiobjective design process is applied. The experimental results showed the validity of the proposed system to achieve a larger zenith angle at different azimuth angle. The focus of the future research will be on the automated operation of the system.

13

Reference

1. M. Probst, K. Vollmers, B. E. Kratochvil and B. J. Nelson, Design of an advanced microassembly system for the automated assembly of bio-Microrobots, presented at 5th International Workshop on Microfactories, Besancon, France, 2527 October 2006. 2. X. Hui, R. Weibin and S. Lining, A flexible experimental system for complex microassembly under microscale Force and vision-based Control, Int. J. Optomechatronics 1, 81102, 2007. 3. B. Potsaid, Y. Bellouard and J. Wen, Adaptive Scanning Optical Microscope (ASOM): a multidisciplinary optical microscope design for large field of view and high resolution imaging, Opt. Express 13, 65046518, 2005. 4. C. Scott, B. Potsaid and J. Wen, Off-Axis aberration correction for a wide field scanning telescope, in Proc. SPIE 7266, 72660Y, 2008 5. H. Hua and S. Liu, A dual-sensor foveated imaging system, Applied Optics 47, 317-27, 2008 6. X. Tao, D. H. Hong, H. S. Cho, The design of active vision system for variable view imaging of micro objects, in Proc. SPIE 6376, 637608112, 2006. 7. X. Tao, H. S. Cho and F. Janabi-Sharifi, Active optical system for variable view imaging of micro objects with emphasis on kinematic analysis, Appl. Opt. 47, 4121-4132, 2008 8. X. Tao, D. H. Hong, H. S. Cho, Variable view imaging system with decoupling design, International Symposium on Optomechatronic Technologies, Proc. of SPIE, 7266, 72661U-1 - 11, 2008. 9. J. R. Meyer-Arendt, Introduction to Classical and Modern Optics, Prentice Hall, 1995.

14

10. L. Zhu, P. C. Sun, D. U. Bartsch, W. R. Freeman and Y. Fainman, Wave-front generation of Zernike polynomial modes with a micromachined membrane deformable mirror, Appl. Opt. 38, 60196026, 1999. 11. S. Kumar, Neural Networks: A Classroom Approach, McGraw-Hill, 2005 12. K. Miettinen, Nonlinear Multiobjective Optimization, Kluwer Academic Publishers, 1999 13. H. S. Cho, Optomechatronic: Fusion of Optical and Mechatronic Engineering, CRC Press, 2005.

15

Fig. 1. Concept of the variable view imaging system Fig. 2. The layout of the proposed system. Fig. 3. Avoid occlusion in the proposed system Fig. 4. View area A2 with full azimuth angle with maximum zenith angle Fig. 5. (a) The configuration of the wedge prisms with maximum zenith angle (b) The relationship between the vertex angle v (degree) of the wedge prism and max (degree) Fig. 6. (a) The configuration of the system for calculation of r2 (b)The relationship between r2 , rp , v Fig. 7. (a) The worst configuration with the maximum peak-valley (b) The warefront errors for different rp, v and NA Fig. 8. NA vs. max in the optimized design Fig.9. Multiple views of micro gears: (a)~(c) captured images in three different azimuth angle with =18.9 and (d)~(f) =13.8 Fig. 10. Microassembly of a micro electrode and a slot, (a) microassembly task, and (b)~(f) captured images during the assembly. Table 1 The optimal design result Table 2 Specification of the experiment setup

16

Translation stage Scanning mirror Scanning lens Camera

Pan/tilt stage

Wedge prisms

Fig. 1. Concept of the variable view imaging system

camera 6th lens 5th lens 4th lens system aperture scanning mirror 2nd lens telecentric lens wedge prisms
Object plane

deformable mirror

3rd lens

view direction

Xw

Zv (x ,y Zw w w) Yv Ov Yw Xv Ow

Fig. 2. The layout of the proposed system.

17

Occlusion A Image plane Without occlusion di Telecentric lens group B A

A do D B

Fig. 3. Avoid occlusion in the proposed system

Zv

max,2
Ov

=0~2
Xv

Yv

Fig. 4. View area A2 with full azimuth angle with maximum zenith angle
90 80 70

max

60 50 40 30 20 10

max
(a)

0 0

10

20

30

40

50

60

v
(b)

18

Fig. 5. (a) The configuration of the wedge prisms with maximum zenith angle (b) The relationship between the vertex angle v (degree) of the wedge prism and max (degree)
r2(mm)
30 20 10 0 -10 30 20 10 10 15 0 5 25 20

rp pr i sm s

A2

r2

Fig. 6. (a) The configuration of the system for calculation of r2 (b)The relationship between r2 , rp , v

90 80 70

Wpv()

NA=0.1

60 50 40

NA=0.09

2nd wedge prism 1st wedge prism Object plane

30 20 10 0 25

NA=0.01
20 15

rp (mm)

10

0.1

0.2

0.3

0.4

0.5

v (degree)

(a)

(b)

Fig. 7. (a) The worst configuration with the maximum peak-valley (b) The warefront errors for different rp, v and NA

19

max

30

25

C ur r ent exper i m ent syst em

20

15

10

5 0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

NA Fig. 8. NA vs. max in the optimized design

2 nd gear

h2 h1 h2
500 m
(b) =18.9, =0

1 st

gear 500 m
(c) =18.9, =270

500 m
(a) =18.9, =90

h2 h1
500 m 500 m

h2
500 m

(d) =13.8, =90 (e) =13.8, =0

(f) =13.8, =270

Fig.9. Multiple views of micro gears: (a)~(c) captured images in three different azimuth angle with =18.9 and (d)~(f) =13.8

20

Gripper

Slot Electrode
(a) (b)

500um

500um
(c)

500um
(d)

500um
(e)

500um
(f)

Fig. 10. Microassembly of a micro electrode and a slot, (a) microassembly task, and (b)~(f) captured images during the assembly. Table 1 The optimal design result MaximumZenith angle, max Scanner area, r2 NA Wavefront error,WPV Vertex angle, v Radius of prisms, rp 22.29 5.56mm 0.0302 3.498, =685nm 24.3 15.15 mm

Table 2 Specification of the experiment setup Maximum Zenith angle, max Scanner area, r2 NA Wavefront error,WPV Vertex angle, v Radius of prisms, rp 19 4.35 mm 0.033 2.4, =685nm 18.13 12.5 mm

21