You are on page 1of 5

THIS DOCUMENT IS PROTECTED BY U.S.

AND INTERNATIONAL COPYRIGHT It may not be reproduced, stored in a retrieval system, distributed or transmitted, in whole or in part, in any form or by any means. Downloaded from SAE International by University of Birmingham, Sunday, September 08, 2013 07:09:46 AM

Automation in Display Testing - A Breakthrough Approach

2013-01-0061 TSAE-13AP0061
Published 03/25/2013

Narasimhan Rajagopal
Caterpillar India Private Limited
Copyright 2013 SAE International and Copyright 2013 TSAE doi:10.4271/2013-01-0061

ABSTRACT
Display devices play a vital role as the man-machine interface in most embedded domains including the automotive industry. Display systems provide information regarding the health of various safety critical subsystems and the general status of the machine. Accuracy and precision of the information displayed is a key factor in proper machine operation. Hence display systems need to be tested meticulously before they are delivered to the customer.

switch on and off the display or switch off the display without switching off the power etc. Testing generally requires a person to be present to select and program various flash files and run the tests on all the flash files. Manual testing is not free from human errors, which can lead to safety concerns for both the machine and the operator. There are different methods available to solve this issue, like model-based automated testing and HIL (Hardware-in-loop) testing but they come with their own price tags and usually need dedicated and unique hardware setups to be implemented. These options will add to the product's cost and also complicate testing.

Current Scenario
The present generation display ECM's (Electronic Control Module) function is not restricted to displaying parameters like fuel level or speed; it also acts as the virtual master of the vehicle. Most display ECMs in off-highway vehicles present at least half a dozen screens to the operator, who selects the one he wants to view based on the need. Hence testing the display ECM becomes as complex as its design. The testing process covers various aspects like data link communication and conformance with protocol, button behavior and response, accurate displacement of gauges, the timely response of indicators and quick navigation between screens, sanity checks, flashing and compatibility checks. Most of the display ECM testing is done manually due to the fact that someone needs to physically press the buttons and view the contents on the screen. The tester also needs to verify the behavior of lights on the indicators and the displacement of gauges. Also certain parameters are available only on a particular screen and the tester needs to navigate to that screen and check them. He may also need to manually

DTA - New Automation Technique


In this paper, we will discuss a breakthrough approach - DTA (Display Test Automation) - which does not require any specific additional hardware or software. In fact, using the concept of DTA, we can develop our own custom software to suit specific purposes. In this paper, we will discuss how this method can reduce time and effort in the software life cycle and improve price-performance ratio. DTA aims to fully automate testing and to solve all the problems and limitations involved in display testing. Once testing of the display ECM is automated, we can extend the idea to all other ECMs and also perform integrated automated testing of the entire machine.

INTRODUCTION
With great advancement in technology, there is a growing need among users to get maximum information from their machine, and display devices act as the interface between the user and the machine. With advancements in mobile

THIS DOCUMENT IS PROTECTED BY U.S. AND INTERNATIONAL COPYRIGHT It may not be reproduced, stored in a retrieval system, distributed or transmitted, in whole or in part, in any form or by any means. Downloaded from SAE International by University of Birmingham, Sunday, September 08, 2013 07:09:46 AM

technology, user expectation has further increased and they now expect advanced features like touch screens and voice recognition in the display devices. As displays become more complex, it's testing becomes tedious. With manual testing it is quite difficult to cover all the extreme test cases and at times we need to repeat tests for every product release. Sometimes, even for small changes we need to run all the tests to make sure nothing else is broken. This leads to the following: 1. Majority of the tester's time is spent in testing working functions. 2. Less time is spent on testing new changes. 3. The overall test time is larger than the time taken to implement the change. Before solving these problems, it would be good to analyze how ECMs are tested.

together is also crucial. This type of integration testing becomes complex since all ECMs need to be tested simultaneously and multiple scenarios need to be simulated. Figure 1 highlights the overall testing process.

TESTING
Normally testing an ECM involves several stages. These are: 1. Sanity testing and hardware conformance testing. Includes testing the durability to temperature and voltage fluctuations. 2. Datalink communication testing and protocol compliance. 3. Sensor calibration and testing of input/outputs 4. Internal and external diagnostic testing 5. Happy path testing 6. Extreme cases 7. Regression testing. Figure 1. Testing process

NEW APPROACH - DISPLAY TEST AUTOMATION(DTA)


A majority of the problems in display testing can be solved by automating the testing process. As discussed earlier one of the major reasons for performing manual testing is the fact that an operator/user needs to be present in order to press the buttons or to check the indicators or verify the screen.

Simulating Key Presses


Simulating key presses solves the aforementioned problems and there are several ways to do it. We can either have external hardware that mechanically presses the keys, or hardware that sends voltages to the key ports. But any external hardware needs corresponding software to control it and the bundle tends to be costly and may not be suitable for all types of displays. The simplest and most efficient way to simulate a key press is to send a datalink message that triggers the same behavior as a key press. This simulation can cover several scenarios like: 1. Key press and release. 2. Key hold for a time period and release. 3. Multiple simultaneous key presses. In the datalink message we can set a certain byte as the key identifier and another byte as a status byte.The key identifier specifies which key is to be processed and the status byte

Display ECM Testing


Display ECMs are quite different from other types of ECMs. They include button inputs, multiple screen navigations and gauges, and may also be driving several indicators, lamps and alarms. Hence there are a sizeable number of features to be tested. The response of the display to key presses also needs to be tested since users expect very quick responses. Display testing usually involves an operator who needs to be physically present to press the buttons or to view the parameters on the screen.

Machine Testing
A machine might contain 6-10 ECMs. Some large machines, like autonomous trucks, mining and marine machines, contain more than a dozen ECMs and most of these ECMs directly communicate with the display ECM. Individual ECMs may work perfectly but when connected in a network several issues may come up. So testing multiple ECMs

THIS DOCUMENT IS PROTECTED BY U.S. AND INTERNATIONAL COPYRIGHT It may not be reproduced, stored in a retrieval system, distributed or transmitted, in whole or in part, in any form or by any means. Downloaded from SAE International by University of Birmingham, Sunday, September 08, 2013 07:09:46 AM

indicates whether it is pressed, released or held.In the same datalink message, we can specify the time period for which the key is held, the number of times it should be pressed, and also the interval between presses.

Figure 3. Machine key simulation Figure 3 depicts the DTA approach to Key-off (Shutdown) and Key-on (Startup) simulation of the Machine key. The time duration between key-off and key-on can be specified in the same message.

Figure 2. Key press simulation Figure 2 depicts the DTA approach to Key press simulation. The DTA approach co-exists with the physical key press without affecting the normal key behavior.

Non Volatile Memory


In machines, NVM (Non Volatile Memory) gets corrupted or erased in certain scenarios and the ECM is expected to gracefully recover. But since this is hard to simulate, it becomes very difficult to predict the display behavior. To solve this, we can use datalink message to simulate NVM erase and corruption at any point (at startup, normal operation, just before key-off). We can set certain bytes to identify the block of NVM, and another byte to specify the operation like erasure or corruption. Multiple blocks of NVM can also be selected.

Startup Related Functions


Another roadblock in display automation is that there are test cases in which the tester needs to switch on and off the display ECM several times. There are two distinct restart modes: 1. Power cycle: We remove power to the ECM abruptly and power it up again. 2. Key cycle: We turn off the key. And turn the key back on. This is usually the case in real machines. In order to simulate power cycle, we need to understand the processor specifications and pin details. Most processors have a configurable pin that can be used to reset them through software, and we can send a datalink message to set/reset this pin to cause an immediate reset. This successfully simulates the power cycle. During a typical key-off, the ECM performs certain critical processes like saving some information in the NVM before shutting down. Key-off is triggered by setting a pin to low/ high, and since voltage is required to set the pin low/high, it requires additional hardware. But there is a workaround. We can trace the point where the software accesses this physical pin value and map it to a datalink signal so that the signal also triggers the same. In the same datalink message we can also specify the duration between key-off and key-on. Using very small durations we can find the time required for the ECM to successfully perform the critical processes during key off. These kinds of scenarios cannot be tested manually.

Screen Navigation
During every release a lot of time is spent in testing the navigation between different screens. The tester has to understand the specification and manually check if the display moves from one screen to another on a particular key press. When a display contains 40-50 different screens this process might take several hours or even days if we include multiple languages. In software, we track the different screens by using a unique Identifier (ID or screen address). Since we have already automated key presses using DTA, we can use a similar process to obtain the screen ID. We can use a datalink message to request screen ID and the display will respond with the current screen's ID. The display can also respond with the IDs of previous screens if requested. With the screen ID, we can perform a simple comparison with the ID in the specification. There are several tools that are available that perform these calculations or we can develop a tool tailored to this need.

THIS DOCUMENT IS PROTECTED BY U.S. AND INTERNATIONAL COPYRIGHT It may not be reproduced, stored in a retrieval system, distributed or transmitted, in whole or in part, in any form or by any means. Downloaded from SAE International by University of Birmingham, Sunday, September 08, 2013 07:09:46 AM

displacement may vary for different gauges. We can specify the input and check if the expected displacement is achieved.

Development Effort
The framework for DTA can be implemented within a span of 2 weeks. With the initial framework in place, testers can immediately concentrate on developing automation test scripts. As test cases evolve, we can update the framework to incorporate complex logic. Table 1 provides the approximate estimates for the framework development. Table 1. Effort required to implement

Figure 4. Screen Navigation Testing Figure 4 depicts the DTA approach to screen navigation testing. We can simulate a key press and the DTA process will transmit a message that contains the current screen ID and previous screen ID. By comparing the screen ID with the expected screen ID we can test the screen navigation without the presence of a tester.

Indicators
Indicators provide visual feedback to the operator about the status of the machine. Unless we use a light detector or an IR sensor it is difficult to fully automate the testing of indicators. But at times, we miss the point. The display that the customer is going to use is not the same display that is used in testing. Only the functionality needs to be verified and not the actual glow of the indicator. We need to first understand the design of the indicators and locate the specific functions that trigger the indicators, and then verify whether these functions correctly trigger the indicators as per the design. Once this is verified we need not actually look at the indicator, we can just verify the data that is being passed to the triggering function. Similar to the technique used in screen navigation, we can use a datalink message to request the present and previous status of a particular indicator. There are cases where more than one input can trigger an indicator. We can also verify the priority of the inputs by enabling multiple inputs and checking which input actually drives the indicator. In case of blinking indicators we can also request the blinking rate and check if it is correct.

Software Tool
Any tool that is capable of sending datalink messages can be used. In order to develop and execute automated test scripts, we can either develop a simple custom tool that can send sequential messages or use any of the available datalink tools that have the same capabilities.

EXTENSION OF DTA: ADVANCED CONCEPTS String Checking


A major part of display testing is string checking. A display that contains 50 screens has a minimum of 2000 strings. If the display supports more than 10 languages, the number of strings is about 20000. It is almost impossible to check each and every string manually. Often, issues are reported regarding string truncation, overlapping etc. especially in foreign languages, which are hard to detect as the person may not be familiar with the language. Most displays have an LCD (Liquid Crystal Display) driver memory that stores the current snapshot of the display. We can have an interface to read this memory and obtain an image of the screen and compare it with the image generated from the specification. Though this process may sound far-fetched, it will not take a lot of time to implement once the LCD driver functionality is understood. An understanding of image processing is also needed to perform the comparison without human intervention.

Gauges
Gauges involve PWM (Pulse Width Modulation) inputs and the values that a gauge can take are not simple binary values like in indicators. Hence gauge testing is more difficult to automate. But we can follow a similar process like in indicators, and include additional bytes to support the entire range of values that the gauge can take. The angle of

THIS DOCUMENT IS PROTECTED BY U.S. AND INTERNATIONAL COPYRIGHT It may not be reproduced, stored in a retrieval system, distributed or transmitted, in whole or in part, in any form or by any means. Downloaded from SAE International by University of Birmingham, Sunday, September 08, 2013 07:09:46 AM

Geometric Objects and Bitmaps


The process mentioned for string checking can be extended to check bitmaps and other geometric objects like rectangles and lines.

DEFINITIONS/ABBREVIATIONS
DTA - Display Test Automation ECM - Electronic Control Module ID - Identifier LCD - Liquid Crystal Display NVM - Non Volatile Memory PWM - Pulse Width Modulation UI - User Interface

Remote Development and Testing


Since display is a visual interface, the tester or developer needs to have physical proximity to the display. But utilizing the previously explained concepts of DTA, we can overcome this problem by having UI (User Interface) that mimics the physical display hardware and communicates with the actual hardware that can be present anywhere. The developer or tester can perform the development/testing by accessing the UI rather than the actual hardware.

Other ECMs and Machine


Any ECM can be tested using this technique. Multiple ECMs can be connected and the entire machine can also be tested in. Many extreme cases and real time failures can be simulated using this approach. The DTA technique can also co-exist with any existing testing method.

SUMMARY/CONCLUSIONS
This technique saves a lot of time in testing and has the potential to remove the need for manual testing. Most of the repetitive testing during each release can be done without the need of a tester. The tester can work on developing the automation scripts for new changes rather than testing existing functionalities. Even complex test cases involving multiple ECMs can be automated. Since DTA is not developed with a specific datalink in mind, it is compatible with any. As datalinks evolve we just need to assign a particular message identifier (PGN in the case of SAE J1939) that can be used for this technique.

ACKNOWLEDGMENTS
1. L N S Sripada - For constant guidance and reminders 2. Aiswarya Bhavanishankar - For editing 3. Radhika Mahalingam - For suggesting the title

The Engineering Meetings Board has approved this paper for publication. It has successfully completed SAE's peer review process under the supervision of the session organizer. This process requires a minimum of three (3) reviews by industry experts. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of SAE. ISSN 0148-7191

Positions and opinions advanced in this paper are those of the author(s) and not necessarily those of SAE. The author is solely responsible for the content of the paper. SAE Customer Service: Tel: 877-606-7323 (inside USA and Canada) Tel: 724-776-4970 (outside USA) Fax: 724-776-0790 Email: CustomerService@sae.org SAE Web Address: http://www.sae.org Printed in USA

You might also like