Professional Documents
Culture Documents
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
Hand Activity Level for a Load Transfer Task 299
0 2 4 6 8 10
Handle idle most Consistent, Slow, steady Steady Rapid, steady Rapid, steady
of the me; no conspicuous, long moon/exerons; moon/exeron; moon/exerons; moon/difficulty
regular exerons pauses; or very frequent brief infrequent pause no regular pauses keeping up or
slow moon pauses connuous
exeron
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
300 April 2013 - Human Factors
Figure 2. Laboratory repetitive motion task in which a participant gets loaded bottles
from the tray (left) and moves them onto the turntable (right).
the analyst uses human discretion to interac- Review Board of University of Wisconsin–
tively select a region of interest associated with Madison prior to participant recruitment, and
the repetitive task, such as a hand or arm, to all participants provided informed consent.
track rather than imposing an a priori model of
the tracked activity. Apparatus and Experimental
We propose an algorithm that operates by Procedures
identifying and tracing the pixel pattern that We implemented a laboratory mock-up of a
best resembles the selected region of interest as simple load transfer task to simulate repetitive
it changes location in successive video frames. motion activities typically performed in an
The velocity of this pixel area, relative to a sta- industrial setting (Figure 2). The loads were
tionary pixel region, is used to estimate the rate equally weighted plastic bottles filled with cali-
of repetition and pauses needed to calculate brated quantities of lead shot. The apparatus
HAL. Consequently, the analysis complexity is consisted of a 6-RPM turntable, driven by an
greatly reduced and may be more tolerable of electric motor, attached to a chute for the
the numerous variations encountered in field weighted bottles to drop into. The task involved
video recordings of occupational tasks. reaching with the right hand for a weighted
METHOD
bottle (2.2 N) in a dispenser located at the end
of a chute, grasping and moving it to the turn-
Participants table, and continuously repeating the operation
We recruited 12 young, healthy volunteers after the bottles returned to the chute. We paced
from within the University of Wisconsin– the repetition frequency and duty cycle to
Madison. Their ages ranged from 18 to 32 achieve various levels of hand activity.
years. There were 6 males and 6 females, all Each participant was required to perform
right-hand dominant, except 1 male. Before seven paced tasks in one single session.
each session was scheduled, we conducted a Participants were given a practice period to
short phone prescreening to help qualify candi- become familiar and proficient with the pace.
dates to ensure that they did not have any ongo- Each task contained 10 to 60 cycles and lasted
ing hand or elbow injuries that could prevent 60 s to 200 s, depending on the specific condi-
them from gripping and transferring weighted tion. We provided 2 min of rest in between
bottles for 1 hr. They were also asked to report experimental conditions to prevent fatigue. The
basic demographic information. The protocol seven paced tasks that were performed in the
was reviewed and approved by the Institutional laboratory varied in repetition frequency (Hz)
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
Hand Activity Level for a Load Transfer Task 301
TABLE 1: Frequency and Duty Cycle Settings for Each Hand Activity Level (HAL) Condition
Frequency (Hz) Duty Cycle (%) Paced HAL HAL by Equation (1)
0.125 25 1 0.1
0.250 10 2 2.1
0.250 20 2 2.4
0.500 20 4 3.7
0.500 30 4 4.1
0.500 50 5 4.7
1.000 50 5 5.4
and duty cycle (%) to produce HALs between 1 A cycle in the laboratory task was defined by
and 5. The frequency and duty cycle experi- the elements (a) reach for a bottle, (b) grasp and
mental conditions are described in Table 1. move bottle, (c) release bottle and return to the
A three-tone set of audio cues was used for starting position, and (d) rest (Figure 3). For this
pacing the task. The first tone signaled the par- particular task, cycle time was the time elapsed
ticipant to reach, grasp, and move the bottle between each repetition cycle (i.e., start of
from the dispenser to the turntable. The second Element 1, reach for a bottle). Hand exertion
tone signaled the participant to return the hand time was the time elapsed between grasping a
to the rest position. The third tone signaled the load and releasing it (i.e., Element 2, grasp and
start of a rest period, during which the partici- move bottle).
pant was instructed to remain at rest until the The HAL was estimated as a linear combina-
cycle repeated. Note that for the 50% duty cycle tion of the frequency of the movement time (1/f)
condition (HAL of 5), only two tones were used and the duty cycle (D) according to the
since there was no rest period in between cycles. equation
A JVC Everio GZ-MS230BUS video camera
was positioned on the right, 3 m away from the HAL = 4.31 – 0.637 × (1/f ) + 0.034 × D. (1)
participant. Each experiment was videoed and
stored as a single MPEG video file with 720 × If the equation returned a negative HAL, the
480 resolution at the frame rate of 30 frames per level was considered zero. The equivalent HAL
second. The videos were each processed by the values for corresponding combinations of fre-
video algorithm, and HAL was calculated on the quency and duty cycle according to the HAL
basis of the movement frequency and duty cycle. TLV (ACGIH, 2009) are included with the HAL
We also analyzed each video manually frame by values from Equation (1) in Table 1. We pro-
frame using Multimedia Video Task Analysis™ cessed all 84 videos using the automated video
(MVTA) software to measure the cycle time (i.e., analysis framework and manually using MVTA
1/frequency) and hand exertion time to compute to measure the actual frequency and duty cycle.
frequency, duty cycle, and HAL. To reflect the true algorithm performance, the
The automatic video algorithm measure- HAL values were not rounded to the nearest
ments were compared against similar observa- integer.
tionally measured properties. We timed the
motions observed in each video clip using Video-Tracking Algorithm
frame-by-frame analysis with the MVTA soft- An algorithm was programmed in C# with
ware tool. The analyst reviewed the videos and the open-source OpenCVSharp (.Net wrapper
manually measured repetition frequency and for the OpenCV) vision library. We initialized
duty cycle by annotating cyclic motions on a the cross-correlation-based tracking algorithm
frame-by-frame basis. by designating a rectangular region of interest
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
302 April 2013 - Human Factors
Figure 3. Selected video frames illustrating the task cycle: (a) reach for a bottle, Frame 207; (b) grasp
and move bottle, Frame 240; (c) release bottle and return to the starting position, Frame 257; and (d) rest,
Frame 293.
(ROI) on the upper limb to select the focal area vp,i = (pi + 1 – pi – 1) / 2∆
on the hand or arm. The template-matching (3)
tracking algorithm tracked the ROI motion tra- ap,i = (pi + 1 –2 × pi + pi – 1) / ∆2,
jectory across successive video frames. A cyclic
motion analysis algorithm estimated the move- where p ∈ {x, y}, i ∈ {2, 3, . . .}, and Δ = 1/30 s,
ment time on the basis of zero crossings of the which is the sample rate of the video. The video
ROI relative to a stationary gating region. velocity and acceleration magnitude were deter-
To elaborate, we denote ri to be a vector of mined by the following:
intensity values of all pixels within the ROI at
the ith frame and ri + 1(w) to be a candidate ROI
( vx ) + ( v y )
2 2
at the (i + 1)th frame with a displacement of w vxy , video =
∈Ω in a prespecified search area. Ω is deter- . (4)
( ax ) + ( a y )
2 2
mined on the basis of prior knowledge of the axy , video =
speed of arm movement (pixels per frame). The
cross-correlation between ri and ri + 1(w) is
defined as the angle between these two vectors An algorithm was developed to automatically
(||ri || is the magnitude of the vector ri, riT is the detect when the hand was loaded on the basis of
transpose of ri): velocity and acceleration of the ROI. It was
observed that the peak ROI acceleration was
R(w) = rTi ri + 1(w) / ||ri || ⋅ ||ri + 1(w)||. (2) greater when participants were moving the load
than when they were reaching and resting. This
The displacement w* = arg. max.WR(w) finding is illustrated by the histogram in Figure 4.
determines the updated position of the ROI at A representative single-cycle velocity and accel-
the (i + 1)th frame, ri + 1 = ri + 1(w*). The sequence eration curve from the video analysis is illustrated
of w* is recorded as the ROI displacement in Figure 5. The period between two consecutive
between these two frames, and the new position local minima points on the velocity curve was
of the ROI yields the motion trajectory at the considered loaded if more than one point on the
(i + 1)th frame. acceleration curve exceeded a threshold value.
In this design, our approach for computing The hand was considered still loaded if more than
the cycle time was to monitor the crossing of one acceleration point exceeded threshold for the
the moving ROI and a stationary gating region adjacent pair of local minima (see Figure 5). The
in the video frame. A cycle was defined as the duty cycle D was then calculated as the average
time elapsed between consecutive gating region time when hand was loaded divided by cycle
crossings from the same direction. The recorded time. In our experiment, the threshold value was
video data pixel location values xvid, yvid were arbitrarily set by trial and error to the 85th percen-
low-pass filtered with the use of a Butterworth tile acceleration when the tracked ROI was in
filter. The velocity (vx, vy) and acceleration (ax, motion (see Figure 5).
ay) vectors at each time stamp were calculated To further comprehend the algorithm used
as follows: to determine when the hand is loaded, refer to
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
Hand Activity Level for a Load Transfer Task 303
Figure 5. Representative single-cycle velocity and acceleration plots for the laboratory load
transfer task. The exertion time is the period in which the load is grasped, moved, and released,
and the cycle time is represented by the task period determined by the video-tracking algorithm.
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
304 April 2013 - Human Factors
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
Hand Activity Level for a Load Transfer Task 305
Paced HAL HAL by Equation (1) Predicted HAL Frequency (Hz) Duty Cycle (%)
When the hands reach beyond the field of view averaged together and can provide an estimate
of a camera, there is a great chance that the of the camera motion (Adams, Gelfand, & Pulli,
tracking system will fail. Camera motion causes 2008). Another way to estimate camera motion
undesired video input, such as a blurry image is to obtain help from devices such as a gyro-
and temporary disappearance of the ROI. scope or accelerometer, which provides the ori-
In practice, selection of a ROI that minimizes entation and motion information of the camera.
disappearance is highly dependent on the spe- The estimated camera motion information can
cific task, work layout, camera location, and later be used to correct the hand displacement
other workplace conditions. Our methodology quantity and to provide better estimates of the
requires the analyst to carefully identify the HAL value.
specific location of the ROI on the body— Different camera locations and angles can
whether it is hand, forearm, arm, or shoulder— impose further limitations on the current
by pinpointing a body part that is directly related method. One most obvious constraint is that
to the hand-arm motion and is visible to the when the participant is viewed from one side,
camera. In many cases in actual workplace con- the contralateral side is occluded by the body.
ditions, these measures by themselves may be Currently, our method has been tested in the
insufficient to fully eliminate disappearance. laboratory for a task that was limited in field-of-
The camera angle and location are additional view changes. We are investigating how the
considerations. In future research, we will study threshold selection affects the performance of
the magnitude of these limitations and strate- the HAL estimates and whether additional vari-
gies for minimizing their effects. ables, such as patch size, should be incorporated
To address these problems through image in the threshold selection process.
processing, it is crucial that the tracking system The measured velocity and acceleration in
make use of the past trajectory history. By the two-dimensional video may be affected by
incorporating a suitable statistical state transi- participant movements outside of the plane.
tion model, such as the Kalman filter, we may Different camera angles may also affect algo-
be able to predict the new ROI position. This rithm performance, including parallax, and this
predicted new position will be used as the refer- issue will be assessed and addressed in future
ence position for the next tracking cycle. Every studies. In future research, we will consider
iteration, the state model location is updated repetitive tasks that vary in body movements,
and corrected to maintain the accuracy of the directions, and exertions. The effect of ROI
prediction. selection on HAL estimates based on various
One way to compensate for camera motion is limb segments is another area of future research.
to track background patches or strong corner Future improvements to the algorithm will
points, in addition to tracking the ROI. The involve a systematic method for differentiating
motion of these patches or corners can be between the active and inactive portion of a
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
306 April 2013 - Human Factors
cycle. Currently, the threshold values were does not require attaching sensors to the bodies
determined by experience and trial and error of workers, which often interferes with the job
and were specific to the laboratory repetitive and, possibly, movement patterns and exertions.
motion task performed in this study. Furthermore, such an application might be ported
The ACGIH TLV involves measurements of to programmable, camera-enabled mobile devices,
normalized peak force in addition to HAL for which could lower the instrumentation barrier and
assessing exposure limits. HAL is strictly make routine analysis of upper-limb work-
related to frequency of motion and duty cycle. related occupational hazards more accessible to
Exertions, whether static or dynamic, are inde- general industry.
pendent of HAL and are considered separately
ACKNOWLEDGMENTS
in the TLV in the measurement of exertion lev-
els. The methodology is limited to coordinated This research was supported in part by a grant from
motions that are observed in the video record- the Wisconsin Alumni Research Foundation and mul-
ing, and hand exertions that may not be visible tiple donors. The authors wish to thank Steven Nelms
are not measured. The algorithm and the results for assistance building the repetitive-motion task
of this study were limited to one particular load apparatus and thank the reviewers and associate editor
transfer task described in this proof-of-concept Tom Armstrong for their insightful suggestions.
study. This study provides a starting point by KEY POINTS
which similar algorithms can be developed for
other tasks. The algorithm has not yet been •• The method is semiautomatic, whereby the human
tested in alternative tasks, such as using a com- analyst interactively selects a region of interest asso-
puter keyboard, or jobs that involve prolonged ciated with the repetitive task to track, rather than
static postures. The algorithm does not measure imposing an a priori model of the tracked activity.
static exertions, whereby forces are made with- •• The markerless video algorithm emphasizes tem-
out movements. For example, if an individual poral patterns, rather than high precision normally
gains control of an object and then continues to important for spatiotemporal motion tracking, for
grip it without motion, the action would extend measuring frequency of movements.
the work time, the cycle time, and the duty •• Exertion time is estimated from video by the iden-
cycle. In the future, researchers should address tification of changes in kinematic properties when
exertions and account for these types of a load is moved for duty cycle measurements.
activities. •• The approach is objective, unobtrusive, does not
For the task in this study, the peak ROI accel- require attaching sensors to the body of work-
eration was greater when moving the load than ers, and suitable for a real-time, direct-reading
when reaching and resting. This observation exposure assessment instrument for hand activity
was used to segment the cycle. It is likely that level.
other criteria might be needed (other than peak
acceleration) for different tasks. Algorithms for
detecting exertions based on ROI kinematics REFERENCES
need to be further evaluated and developed and Adams, A., Gelfand, N., & Pulli, K. (2008). Viewfinder alignment.
Eurographics, 27, 597–606.
will be the subject of future research. In the cur-
Allmen, M., & Dyer, C. R. (1990, June). Cyclic motion detec-
rent experiment, the threshold value was arbi- tion using spatiotemporal surfaces and curves. Paper pre-
trarily set to the 85th percentile acceleration sented at the International Conference on Pattern Recognition
when the tracked ROI was in motion. A general (ICPR’90), Atlantic City, NJ.
American Conference of Governmental Industrial Hygienists.
acceleration threshold algorithm should be (2009). Hand activity level TLVs® and BEIs® based on the
developed for duty cycle detection. The method documentation of the threshold limit values for chemical sub-
used in the current study sufficed for proof-of- stances and physical agents and biological exposure indices.
concept purposes. Cincinnati, OH: Author.
Bhattacharya, A., Warren, J., Teuschler, J., Dimov, M., Medve-
We conclude that the video-based direct dovic, M., & Lemasters, G. (1999). Development and evalu-
exposure assessment method shows promise. ation of a microprocessor-based ergonomic dosimeter for
The use of markerless video is unobtrusive and evaluating carpentry tasks. Applied Ergonomics, 30, 543–553.
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
Hand Activity Level for a Load Transfer Task 307
Buchholz, B., & Wellman, H. (1997). Practical operation of a biax- Marshall, M. M., Mozrall, J. R., & Shealy, J. E. (1999). The effects
ial goniometer at the wrist joint. Human Factors, 39, 119–129. of complex wrist and forearm posture on wrist range of motion.
Burt, S., & Punnett, L. (1999). Evaluation of interrater reliability Human Factors, 41, 205–213.
for posture observations in a field study. Applied Ergonomics, Paquet, V. L., Punnett, L., & Buchholz, B. (2001). Validity of fixed-
30, 121–135. interval observations for postural assessment in construction
Fan, Z. J., Silverstein, B. A., Bao, S., Bonauto, D. K., Howard, N. work. Applied Ergonomics, 32, 215–224.
L., Spielholz, P. O., . . . Viikari-Juntura, E. (2009). Quantitative Person, J. G., Hodgson, A. J., & Nagy, A. G. (2001). Automated
exposure-response relations between physical workload and high-frequency posture sampling for ergonomic assessment of
prevalence of lateral epicondylitis in a working population. laparoscopic surgery. Surgical Endoscopy and Other Interven-
American Journal of Industrial Medicine, 52, 479–490. tional Techniques, 15, 997–1003.
Frade, F. T., Marroquin, E. M., Perez, E. S., & Moreno, J. A. M. Radwin, R. G., & Lin, M. L. (1993). An analytical method for char-
(1997, September). Moving object detection and tracking acterizing repetitive motion and postural stress using spectral
system: A real-time implementation. Paper presented at the analysis. Ergonomics, 36, 379–389.
Symposium on Signal and Image Processing (GRETSI 97), Radwin, R. G., Lin, M. L., & Yen, T. Y. (1994). Exposure assess-
Grenoble, France. ment of biomechanical stress in repetitive manual work using
Garg, A., & Kapellusch, J. (2009, August). Consortium pooled frequency-weighted filters. Ergonomics, 37, 1984–1998.
data job physical exposure assessment. Paper presented at the Schoenmarklin, R. W., & Marras, W. S. (1993). Dynamic capa-
17th World Congress in Ergonomics Beijing, China. bilities of the wrist joint in industrial workers. International
Harris, C., Eisen, E., Goldberg, R., Krause, N., & Rempel, D. Journal of Industrial Ergonomics, 11, 207–224.
(2011). 1st place, PREMUS best paper competition: Workplace Sidenbladh, H., & Black, M. J. (2001, July). Learning image statis-
and individual factors in wrist tendinosis among blue-collar tics for Bayesian tracking. Paper presented at the International
workers. The San Francisco study. Scandinavian Journal of Conference on Computer Vision (ICCV 2001), Vancouver,
Work, Environment, and Health, 37, 85–98. Canada.
Hegmann, K. T., Thiese, M. S., Ott, U., Oostema, S., Garg, A., Silverstein, B., Fan, Z., Bonauto, D., Bao, S., Smith, C., Howard,
Kapellusch, J., . . . Foster, J. (2009, August). Prospective cohort N., & Viikari-Juntura, E. (2010). The natural course of carpal
study of upper extremity MSDs among 17 diverse employers. tunnel syndrome in a working population. Scandinavian Jour-
Paper presented at the 17th World Congress in Ergonomics nal of Work, Environment, and Health, 36, 384–393.
Beijing, China. Spielholz, P., Silverstein, B., Morgan, M., Checkoway, H., &
Homan, M. M., & Armstrong, T. J. (2003). Evaluation of three Kaufman, J. (2001). Comparison of self-report, video obser-
methodologies for assessing work activity during computer vation and direct measurement methods for upper extremity
use. Aiha Journal, 64, 48–55. musculoskeletal disorder physical risk factors. Ergonomics,
Jonsson, P., & Johnson, P. W. (2001). Comparison of measurement 44, 588–613.
accuracy between two types of wrist goniometer systems. Tsai, P.-S., Shah, M., Keiter, K., & Kasparis, T. (1993). Cyclic
Applied Ergonomics, 32, 599–607. motion detection. Orlando: University of Central Florida.
Juul-Kristensen, B., Hansson, G. A., Fallentin, N., Andersen, J. H., Yen, T. Y., & Radwin, R. G. (2000). Comparison between using
& Ekdahl, C. (2001). Assessment of work postures and move- spectral analysis of electrogoniometer data and observational
ments using a video-based observation method and direct tech- analysis to quantify repetitive motion and ergonomic changes
nical measurements. Applied Ergonomics, 32, 517–524. in cyclical industrial work. Ergonomics, 43, 106–132.
Ketola, R., Toivonen, R., & Viikari-Juntura, E. (2001). Interob-
server repeatability and validity of an observation method to
assess physical loads imposed on the upper extremities. Ergo-
Chia-Hsiung Chen is a PhD student in electrical and
nomics, 44, 119–131. computer engineering at the University of Wisconsin–
Latko, W. A., Armstrong, T. J., Foulke, J. A., Herrin, G. D., Madison and a research assistant in the Occupational
Rabourn, R. A., & Ulin, S. S. (1997). Development and evalu- Ergonomics and Biomechanics Laboratory. He earned
ation of an observational method for assessing repetition in
hand tasks. American Industrial Hygiene Association Journal,
his BS degree from the National Sun Yat-Sen
58, 278–285. University (Taiwan) and MS degree from the National
Lin, M. L., & Radwin, R. G. (1998a). Agreement between a Taiwan University (Taiwan).
frequency-weighted filter for continuous biomechanical mea-
surements of repetitive wrist flexion against a load and pub-
lished psychophysical data. Ergonomics, 41, 459–475. Yu Hen Hu is a professor at the University of
Lin, M. L., & Radwin, R. G. (1998b). Validation of a frequency- Wisconsin–Madison in electrical and computer engi-
weighted filter for continuous biomechanical stress in repetitive neering. He has a BS degree (1976) from the
wrist flexion tasks against a load. Ergonomics, 41, 476–484.
National Taiwan University (Taiwan) and an MS
Lin, M. L., Radwin, R. G., & Snook, S. H. (1997). A single metric
for quantifying biomechanical stress in repetitive motions and degree (1980) and PhD (1982) from the University
exertions. Ergonomics, 40, 543–558. of Southern California (Los Angeles).
Lowe, B. D. (2004). Accuracy and validity of observational esti-
mates of shoulder and elbow posture. Applied Ergonomics, 35,
Thomas Y. Yen is an associate instrumentation innova-
159–171.
Marras, W. S., & Schoenmarklin, R. W. (1993). Wrist motions in tor and instructor at the University of Wisconsin–
industry. Ergonomics, 36, 341–351. Madison in the Departments of Biomedical Engineering
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015
308 April 2013 - Human Factors
and Industrial and Systems Engineering. He has a BS and rehabilitation. He has a BS degree (1975) from
degree (1987) from Northwestern University (Evanston, New York University Polytechnic Institute, and he
Illinois) and an MS (1991) and PhD (1997) from the earned MS (1979) and PhD (1986) degrees from the
University of Wisconsin–Madison. University of Michigan (Ann Arbor).
Downloaded from hfs.sagepub.com at CALIFORNIA STATE UNIV FRESNO on July 24, 2015