Real-time movement biofeedback for walking gait modification in knee osteoarthritis

Tim V. Wrigley, Milena Simic, Michael A. Hunt, Rana S. Hinman, Kim L. Bennell
Centre for Health, Exercise and Sports Medicine (CHESM) School of Physiotherapy The University of Melbourne, Victoria, Australia timw@unimelb.edu.au, www.physioth.unimelb.edu.au/chesm/
Abstract A set of Matlab software tools is described for connecting to the real-time data stream from a Vicon MX system and generating movement biofeedback. The initial application is for patients with knee osteoarthritis, to aid in the teaching and learning of gait modification strategies to reduce knee loading. Keywords; real-time; motion capture; knee; osteoarthritis; biofeedback; Vicon; Matlab

Photogrammetric algorithms are used for the reconstruction of the markers 3D locations in real-time. While useful, the biofeedback capabilities offered by this real-time data display in the current generation of commercial motion capture systems do not generally have the flexibility, configurability, or ease of use to be readily tailored to a wide range of biofeedback requirements, for example in relation to the choice of visual display formats. Therefore, the aim of the work described here was to develop a set of software tools in Matlab (Mathworks Inc, Mass., USA) that would allow flexible and customizable access to real-time optical motion capture data, and the generation of different forms of movement biofeedback for patients. The initial aim for this system is to aid in teaching gait modification for knee load reduction to patients with knee OA. II. METHODS

I.

INTRODUCTION

A number of modifications to walking gait have been identified that reduce the load on the knee joint, which is of particular interest in the context of knee osteoarthritis (OA). There is no cure for OA, and treatment is largely limited to symptom (pain) management, with limited possibilities for slowing disease progression. However, because progression of the disease is related to knee loading [1], gait modification may be a productive strategy to slow disease progression [2]. But some of the relevant gait modifications are relatively subtle, and it is important during patient therapy that the magnitude of modification achieved by a patient can be immediately and accurately assessed. Real-time movement biofeedback has the potential to be particularly useful in this regard. The generation of real-time movement feedback from complex multi-joint human movements has traditionally presented considerable technical challenges. Researchers or clinicians wishing to explore such feedback for clinical application have had to adapt existing motion capture systems to a use for which they were not generally designed, or to develop custom systems from scratch. As a consequence of this difficulty in generating real-time movement biofeedback, its utility as a research and clinical tool is still relatively under-explored. Because of this, demonstrated evidence of its benefits is similarly limited. Relatively recently however, real-time data display has become available in the commercial systems commonly used for 3D human movement analysis, for example those typically used for analysis of walking gait in clinical and research contexts. These systems are predominantly camerabased ('optical' motion capture), relying on the detection of small retro-reflective spherical markers placed on anatomical landmarks on the patient's body using a network of cameras.

The system to be described utilizes real-time data streamed into Matlab from Vicon Nexus software (Vicon, Oxford, UK) connected to a Vicon MX 8-camera motion capture system, as well as three force platforms (AMTI, Mass., USA) embedded in a ten-meter walkway. The Nexus software offers several means of supplying data in real-time to a custom software application: (i) sending and receiving TCP/IP messages and data with Nexus directly using Vicons Tarsus Communication Protocol [3], or (ii) using the Vicon Real Time SDK / API's Dynamic Link Library (DLL) to communicate with Nexus via higher-level commands [4]. Both of these methods can theoretically be used from within Matlab, the first requiring its Instrument Control Toolbox for TCP/IP communications, or secondly simply calling the DLL routines using base Matlab. We have initially implemented the first of these options, with Matlab routines employing the Tarsus Protocol to request and receive data in real-time from Nexus. In order to fully customize real-time data access and display, one must usually write custom software, generally in C++. While compiled languages such as C++ will always yield the fastest performance, they lack in-built scientific and high-level graphics routines, as well as requiring not only standard C++ programming skills but also some specialist skills (e.g. sockets, real-time programming) which are not common amongst scientists. Matlab on the other hand is

978-1-4244-4189-1/09/$25.00 2009 IEEE

132

more widely used by scientists, and has in-built scientific and graphics routines. It is however significantly slower than compiled languages, which would normally preclude its use in demanding real-time applications. But depending on the application, the processing performance limitations of Matlab can be manageable, in return for which one has greater useability, flexibility, and customizability. Nexus makes available a range of data in the Tarsus realtime data stream, including 3D marker XYZ positions and body segment orientations (in angle-axis format), as well as analog signals (at camera frame rate), albeit undocumented. The choice of which of these, or measures derived from them, to use as biofeedback parameters is particularly important. Patients will not necessarily understand what some parameters 'mean'. Furthermore, humans cannot easily attend to, let alone respond to, rapidly changing movement information, especially since they are moving at the same time. It is critical that patients can 'engage' with the biofeedback system. Therefore, it is best to target simple, slowly changing parameters, repetitive parameters, or even defer parameter biofeedback until immediately after movement rather than during it. To this end, a flexible system of selecting, processing, and displaying real-time data was devised. The Matlab routines implement a scheme whereby just the data of interest is extracted from that sent by Nexus via the Tarsus protocol to Matlab. A standard lower limb marker

set consisting of 20 reflective markers can result in several hundred individual data items being sent from Nexus to Matlab for each frame (although a more minimal marker set can be used if necessary, with just those markers required to generate the specific data required for biofeedback). But just the items from the data stream required for a particular form of movement biofeedback become RT channels, which are then accessible within Matlab for immediate display, or to further routines that support calculation of derived parameters (RT calculations) and the determination of specific event conditions (RT events). A wide range of RT output formats are then available for these: charts (bar, 2D, 3D, vertical or horizontal strip chart), numeric displays, gauge, sound, digital / analog (ie for driving external devices, eg sound frequency generator, vibration feedback). Some outputs represent just the instantaneous value of a movement parameter, while others show the history of the patient's movements, so that they can tell if their movements are changing over time. These RT outputs can be delivered to the patient via visual (projection screen, see-through headmounted-display), auditory (speakers, headphones), or tactile means (eg mini vibrating pager motor). In order that specific goals can be set for a patient to aim for, it is also possible to set target levels (RT targets) for the visual output formats. These are displayed with the RT outputs so that the patient can see how their movement compares to the desired target. The overall scheme is depicted in Fig. 1.

Vicon Nexus

Vicon Tarsus channels (markers, segments, force plates, analog signals)

Matlab

RT channels
RT targets RT calculations RT events

RT outputs
Sound Bar chart 3D chart Numeric Gauge Analog Digital

Strip chart

2D chart

Delivery systems
Visual
Projection screen See-thru HMD

Auditory
Speakers Wireless headphones

Tactile
Vibration

Figure 1. Overall framework of Matlab routines for TCP/IP connection to the Tarsus real-time data stream from Vicon Nexus, and generation of 'RT outputs' for movement biofeedback to the patient.

133

A number of gait modifications have been associated with knee load reduction, which is generally assessed noninvasively as a reduction in the external knee adduction moment, closely linked to compressive load on the medial compartment of the knee where knee OA is most common [5]. While the external adduction moment itself is more difficult to measure in real-time, each of the gait modifications can be linked to simple kinematic movement biofeedback parameters: 'toe out' gait [6] Biofeedback parameter: 'foot progression' angle (angle of the foot in relation to the walking direction) lateral trunk lean [7] Biofeedback parameter: frontal plane trunk angle medial thrust gait (bringing the knees closer together) [8] Biofeedback parameters: distance between knees, knee deviation from mid-line of walking direction increased stance width [9] Biofeedback parameter: position of left and right foot placement walking with a cane [10] Biofeedback parameter: % body weight applied to the cane (using force-instrumented cane) Without real-time movement biofeedback, these gait modifications would be somewhat difficult to convey to patients, and it would be difficult to judge the magnitude of the modification to assess how well the patient achieved the modification. III. RESULTS

Significant latency is disconcerting to the patient (even more so if it is variable), and counter-productive to motor learning. We found that the latency between an actual movement and a change in the screen displays generated by Matlab, or output of a changing audio tone that varied with movement amplitude, was generally imperceptible. As the generation of more complex biofeedback parameters and delivery was employed, the frame rate could be decimated further while still retaining good latency. Fig. 2 shows an example of the use of the system for providing movement biofeedback on toe out angle during walking gait. IV. DISCUSSION

Biofeedback can be defined as 'the use of instrumentation to make covert physiological processes more overt' [11]. The mechanism by which movement biofeedback is provided to a patient via instrumentation can vary from the simple to the complex. But any form of biofeedback technology in rehabilitation must be more effective than the existing approaches used by therapists - both in terms of rehabilitation outcome and cost - if it is to be adopted. For example, while not strictly qualifying as 'instrumentation', therapists have been using mirrors in rehabilitation for decades to give patients visual feedback on their own performance. More recently, video has also been used. To be

We report here on the technical aspects and performance of the system. Research into its effectiveness for facilitating gait modification in knee OA patients is currently underway. While TCP/IP would normally be used to communicate between separate PCs - and Nexus real-time data streaming can operate in this way - we found that performance was markedly better when running both Nexus and Matlab on the same dual core Xeon 5150 PC, under Windows XP (32-bit), compared to two PCs connected via a gigabit switch. The Matlab routines demonstrated the ability to generate simple biofeedback measures and deliver them in visual or auditory form in real-time, with Nexus operating at 120 frames/second. A real-time loop in Matlab polls Nexus for the current frame. Processing time for each frame in Matlab typically results in at least several frames being dropped between processed frames (obviously CPU-dependent), but this was inconsequential. Indeed, while an 'adequate' frame rate is important, the key performance parameter for realtime biofeedback is latency that is, the delay between the time at which a movement occurs and the time at which data captured from that movement is fed back to the patient.

Figure 2. Use of the system for providing movement biofeedback on toe out during walking gait. The Vicon MX camera system is tracking the retroreflective markers 3D locations in real-time. Matlab is extracting the XYZ marker locations of the heel and toe markers via the Tarsus protocol, and calculating the angles of the left and right heel-toe vectors for display to the subject via a rear projection screen.

134

successful, movement biofeedback technology must lead to more effective rehabilitation outcomes than these simple, 'low-tech' approaches. So biofeedback may be employed in clinical contexts where a therapeutic movement goal has traditionally been addressed without such biofeedback technology. But the availability of advanced movement biofeedback technology could also stimulate the development of completely new therapeutic techniques. Therapeutic approaches that have not been contemplated before may become feasible with the availability of suitable movement biofeedback. For example, the complexity of a therapeutic movement goal might preclude traditional approaches to therapist-only instruction and feedback, where fine modifications in movement may be difficult to discern and modify without real-time measurement and feedback. The approach described here is clearly a 'high-tech' approach, although aimed at exploiting existing resources in motion capture laboratories. If we are able to determine which gait modifications for knee OA are the most effective, at what magnitude of the gait modification strategy, then the long-term aim would be to devise a minimal, sensor-based feedback system that could then be deployed to clinics that do not have expensive 3D motion capture systems. While a number of different aspects of human movement generation and control that are amenable to instrumented biofeedback have been studied and employed clinically, actual movement biofeedback has not been common, probably due to the technical challenges. Instead, simpler technologies have been the focus of many biofeedback studies. Early investigations of biofeedback in movement rehabilitation were largely focused on EMG [11]; the demonstrated benefits of this remain equivocal at best (e.g. [12]). Other attempts utilized force platforms for balance biofeedback; the proven benefits have been similarly limited (e.g. [13]). At the more complex end of the instrumentation spectrum, virtual reality environments have been investigated as a means of providing biofeedback to patients; it is not yet clear whether such technology-intensive approaches provide clear advantages over more traditional rehabilitation approaches [14]. However, the lack of more readily available real-time biofeedback delivery systems has clearly precluded wider study of even the simplest forms of movement biofeedback. Therefore, movement biofeedback efficacy studies would be

greatly facilitated by the availability of flexible biofeedback tools in conjunction with existing motion capture systems that are available in research labs and hospital gait analysis clinics, rather than requiring dedicated, often custom-built, and expensive biofeedback systems. The work described here shows that this is eminently feasible. REFERENCES
[1] T. Miyazaki, M. Wada, H. Kawahara, M. Sato, H. Baba, and S Shimada. "Dynamic load at baseline can predict radiographic disease progression in medial compartment knee osteoarthritis". Ann Rheum Dis, vol 61, pp:617-622, 2002. A. Chang, D. Hurwitz, D. Dunlop, J. Song, S. Cahue, K. Hayes, et al. "The relationship between toe-out angle during gait and progression of medial tibiofemoral osteoarthritis". Ann Rheum Dis, vol 66(10), pp 1271-1275, 2007. Vicon. Tarsus Communication Protocol. Oxford UK: Vicon. 2000. Vicon. Vicon Real Time SDK and API. Oxford UK: Vicon. 2002. D. Zhao S.A. Banks, K.H. Mitchell, D.D. D'Lima, C.W. Colwell, B.J. Fregly. "Correlation between the knee adduction torque and medial contact force for a variety of gait patterns". J Orthop Res. 25(6), pp. 789-97, 2007. C.J. Lin, K.A. Lai, Y.L. Chou, and C.S. Ho. "The effect of changing the foot progression angle on the knee adduction moment in normal teenagers". Gait Posture, vol 14(2), pp. 85-91, 2001. A. Mundermann, J.L. Asay, L. Mundermann, and T.P. Andriacchi. "Implications of increased medio-lateral trunk sway for ambulatory mechanics". J Biomech, vol 41(1), pp 165-170, 2008. B.J. Fregly, J.A. Reinbolt, K.L. Rooney, K.H. Mitchell, and T.L. Chmielewski. "Design of patient-specific gait modifications for knee osteoarthritis rehabilitation". IEEE Trans Biomed Eng, vol 54(9), pp 1687-95, 2007. B.J. Fregly. Computational assessment of combinations of gait modifications for knee osteoarthritis rehabilitation. IEEE Trans Biomed Eng. 55(8), pp 2104-6, 2008. G. Kemp, K.M. Crossley, T.V. Wrigley, B.R. Metcalf, and R.S. Hinman. "Reducing joint loading in medial knee osteoarthritis: Shoes and canes". Arthr Care & Res, vol 59(5), pp 609-614, 2008. H. Huang, S.L. Wolf, and J. He. "Recent developments in biofeedback for neuromotor rehabilitation", J Neuroengin & Rehab, vol 3, pp 11, 2006. H. Woodford, and C. Price. "EMG biofeedback for the recovery of motor function after stroke". Cochrane Database of Systematic Reviews, vol 2, CD004585, 2007. R. Barclay-Goddard, T. Stevenson, W. Poluha, M.E.K. Moffatt, and S.P. Taback. "Force platform feedback for standing balance training after stroke". Cochrane Database of Systematic Reviews, vol 4, CD004129, 2004. M.K. Holden. "Virtual environments for motor rehabilitation". Cyberpsychology & Behavior, vol 8, pp. 187-211, 2005.

[2]

[3] [4] [5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

135