by MR Yenchek · Cited by 3 — coal stockpiles, the majority being bulldozer operators [4]. msha/techsupp/techexchange/surgepile/surgepilesafety.pdf.
251 KB – 6 Pages
PAGE – 1 ============
____________________________ Figure 1. Dozer cover-up on coal stockpile. Remote Vision System for Dozers on Coal Stockpiles William H. Schiffbauer, Electronics Technician Michael R. Yenchek, Team Leader Electrical Safety Timothy Lutz P.E., Mechanical Engineer National Institute for Occupational Safety and Health Pittsburgh, PA Abstract ŠThe National Institute for Occupational Safety and Health (NIOSH) Pittsburgh Research Laboratory (PRL) participated in an experiment to facilitate the remote control of dozers used on coal stockpiles. The experiment consisted of supplementing the normal sensory cues an operator requires to manipulate the dozer from a remote location. NIOSH provided and tested a vision systems intended to give the operator the ability to operate the dozer safely and efficiently from a remote position. This paper highlights the genesis of the experiment, identifies the contributions of Consol Energy and Caterpillar, and details the NIOSH efforts and successful test results. Keywords-remote-control; vision system; stockpiles; safety I. INTRODUCTION Environmental conditions, coal compaction, and other factors can result in voids in coal stockpiles which can entrap bulldozers (Fig. 1) used to facilitate drawdown at the top of the piles [1, 2, 3]. Since 1980, there have been 18 fatalities at coal stockpiles, the majority being bulldozer operators [4]. Efforts by the Mine Safety and Health Administration (MSHA) and others have led to development of improved cab designs, high-strength windows, and communications which have proven to save operator lives during dozer cover-ups [5, 6, 7]. Providing remote control of the dozer has the potential to totally eliminate the danger to the operator by removing him/her from the machine. II. BACKGROUND AND APPROACH Hundreds of bulldozers in the U.S are used to facilitate drawdown at the top of coal stockpiles. Previous attempts in the U.S. to remotely control the dozer were never fully implemented due to a variety of issues. One drawback was that the operator, when removed from the dozer, lost the fifeelfl of the machine. This resulted in significant inefficiencies during remote operation as compared to on-board control. Disclaimer 1: fiThis information is distributed solely for the purpose o f pre-dissemination peer review under applicable information quality guidelines. It has not been formally disseminated by NIOSH. It does not represent and should not be construed to represent any agency determination or policy.fl Disclaimer 2: Mention of any company or product d oes not constitute endorsement by the National Institute for Occupational Safety and Health (NIOSH). In addition, citations to Web sites external to NIOSH do not constitute NIOSH endorsement of the sponsoring organizations or their programs or products. Furthermore, NIOSH is not responsible for the content of these Web sites. Consol Energy has been interested in addressing the problem of bulldozer cover-ups on coal stockpiles. MSHA had also been cognizant of the hazards associated with stockpiles and provided input during planning for the project. Caterpillar has been investigating the feasibility of incorporating remote- control technologies on its bulldozers. These parties informally collaborated to implement a remotely-operated dozer on a coal stockpile. Meetings were held with Consol, Beckwith Machinery, and Caterpillar at Consol™s Eighty-Four, PA office to formulate the approach and test plan. Consol Energy provided a test site and a field trailer (Fig. 2), at their Eighty Four Mine coal stockpile near Eighty Four, PA. The trailer was outfitted with power, heating, cooling, and water. A large window provided a wide-angle view of the stockpile. Installation and operational assistance as well as access to dozer operators, was provided as needed. Caterpillar of Peoria, IL provided a temporary loan of a D10T dozer (Fig. 3) and onsite technical support during system integration and testing. The dozer included a remote control system (Fig. 4) which interfaced to other dozer electronic systems through a CAN bus-based data network using the SAE Standard J1939 protocol. The dozer could be operated either via remote control or by an on-board operator. Other features included emergency stop (E-Stop), automatic blade control while the dozer carries a load (AutoCarry); automatic dangerous zone alerts using the GPS-based CAES
PAGE – 2 ============
Ultra system, remote diagnostics, and automatic braking. Further details are available at their web site (http://www.cat.com/cda/layout?m=71342&x=7). Figure 2. Consol Energy test site. Figure 3. Caterpillar D10T dozer. Figure 4. Dozer remote control. The intended outcome of this NIOSH research was to improve the safety of the operators of bulldozers on coal stockpiles by investigating the feasibility of sensor-enhanced remote control. The research project was conducted to answer the following: Ł Can a camera system substitute for line-of-sight operation of a remotely-controlled bulldozer? ŁWould it be necessary to enhance operator acceptance by using audio and/or motion cues? It was to be accomplished through four tasks: 1) study the operation of bulldozers on stockpiles, 2) select, acquire, and test technology that can supplement the normal sensory cues an operator requires to manipulate a dozer, 3) integrate the components on a dozer, and 4) test the system on a dozer on a coal mine stockpile. The NIOSH approach was to provide the operator of the remote-controlled dozer with sufficient sensory cues so that he/she could safely and efficiently operate the dozer from a distance. Dozer operations on a coal stockpile were observed to learn of basic operators™ needs. On-board control of dozers required the operator to constantly scan the work area, not only directly in front of the machine, but also side-to-side and to the rear. Visual feedback to a remote control site should consist of a field-of-view that is continuous (not discrete) and that would respond rapidly to the operator™s motion. Another perceived need would be remote stereo audio. This would enable the operator to be aware of subtle changes in dozer performance and production environment dynamics. Additionally, the operator would need to sense the pitch and roll of the dozer on the stockpile for safe operation. Most of the technologies required were available off-the -shelf. The performance of the equipment in a production scenario was a major concern. Gaining operator acceptance of the sensory technology was seen as one roadblock to success. Other envisioned obstacles were the operator™s ability to use the system under normal and foggy conditions, the ability of the hardware to be reliable in the harsh environment, and the efficiency of the operator using a remote-controlled system as compared to a conventional manned dozer. The answers to these concerns could only be determined through experimentation. III. DIGITAL VISION SYSTEM (DVS) To satisfy the visual sensor needs some type of camera/monitor system would be required. One basic requirement was to provide the operator with a 360-degree view of the dozer area with as close to the same field of view as if he/she were operating the machine in a conventional manner. The type of systems investigated included simulators, pan/tilt, omni-directional, stationary, stereo, and infrared cameras. Though stereo video was desired, NIOSH found that it was not available commercially.
PAGE – 3 ============
Pan and tilt cameras using operator head movements to control the cameras were investigated. The concept consisted of the operator wearing a pair of goggles, while operating the dozer via the remote control pendant. As the operator looked, up, down, right, or left, his head motion would control the dozer-mounted pan and tilt camera, thereby providing a 360 -degree view of the production area. The goggles could also provide stereo audio. A candidate vendor (Chatten (www.chattenassociates.com)) provided a demonstration of this technology at NIOSH and at the dozer stockpile site. The performance of the head-tracking system was considered acceptable. The controls for the remote-control pendant of the dozer and remote readouts of dozer gauges could be seen by looking down below the goggles. The stereo audio was not demonstrated, though the vendor could supply that option if required. Another candidate system consisted of a fixed camera focused on a parabolic mirror which reflects a 360-degree field of view around the machine (Remote Reality , http://www.remotereality.com/c ontent/view/63/115/). The camera would be mounted inside an appropriate housing. The panoramic view would constantly be transmitted to the remote site. Responding to the operator™s head movement, tracking software would excise the portion of the panoramic view of interest, process it into a planar image, which would then be transmitted to goggles. The vendor provided a demonstration of the technology at NIOSH. Stereo audio was not an option. The vendor indicated that the system could be used in combination with a large monitor to mirror what the operator saw in the goggles and also provide simultaneous views of other areas around the machine. A. Initial System Integration NIOSH and stakeholders reviewed the technology and decided to implement the Remote Reality camera system for the initial system. The selection was based on the clarity of the video images provided. The audio feedback technology would be addressed at a later time. Laboratory testing of the camera system was successful. The camera system components consisted of a PC, a 42-inch LCD monitor, a pair of electronic goggles with head tracker, a panoramic digital video camera, a two-node wireless link, and special software. Fundamentally the whole system was a wireless local-area-network (LAN), consisting of two WiFi g (also known as IEEE 802.11g) modems. The camera video was accessed by the PC using a static internet protocol (IP) address. The dozer operator could choose to use either the goggles to operate the dozer, or else use the LCD monitor. There were a number of viewing features available. All were software selectable. The LCD could echo the goggle view, or each could provide independent views. The LCD could show one wide view of the front of the dozer, or a wide front view and wide rear view, or one wide front view, and two or more auxiliary views around the dozer. B. DVS Field Tests The digital video system (DVS) was installed on the dozer (Fig. 5) and in the control trailer. The operators were given an opportunity to run the dozer using both the goggles (Fig. 6) and the LCD monitor for remote control. The general consensus of the operators was that the goggles were impractical as they limited their field of view, were generally uncomfortable, and were too heavy to wear for an 8-hour shift. Use of the LCD monitor was more acceptable. Establishing the desired views on the screen required many configuration changes. With the camera box attached on top of the cab, the operator had better overall views, but from a different perspective than when sitting in the cab. Here it was difficult for the operator to fully view the edges of the dozer blade. The decision was made to add more fixed cameras to the dozer because one camera was installed to view the left side of the dozer blade and tracks, a second to view the right side of Figure 5. Digital cameras on dozer cab. Figure 6. Goggles with head tracker.
PAGE – 4 ============
the dozer blade and tracks, the third was fixed forward, and the forth rearward (Fig. 7). While this configuration satisfied the operator requirements, the visual performance wasn™t acceptable. Latency, or the time delay for video transmission back to the trailer, was in the 0.5 to 1 second range. With the maximum dozer speed up to eight miles per hour the dozer could cover five feet before a given video image would update. To eliminate the time delays in the transmitted video a decision was made to pursue an analog-based video system. Figure 7. Digital video outputs at trailer. IV. ANALOG VISION SYSTEM (AVS) The basic requirement for the analog video system (AVS) was to provide a wireless, real-time, 360-degree image from the dozer. No vendor could be found to supply a system with all these characteristics as one product. However one vendor (DTC http://www.dtccom.com/) was found that could provide the wireless transmission system for four simultaneous video channels. The vendor could also supply a variety of cameras with different lenses. NIOSH specified wide-angle lenses that, when combined, would capture a 360-degree image. NIOSH obtained the technology and packaged the cameras in a rugged compact enclosure. The contents of the enclosure were; four cameras, four wide-angle lenses, a four-channel multiplexer, an analog transmitter, an antenna, and a battery (Fig. 8). The operator end of the system consisted of two antennas, a dual- diversity receiver, a four-channel de-multiplexer, and four 40 -inch LCD monitors. The LCD monitors were arranged in a circle, with a chair in the center for the operator. Video quality was acceptable, and there was no perceptible image latency. This hardware did provide remote monaural audio from one microphone. Figure 8. Analog video camera box. A. AVS Laboratory Tests A compact NIOSH-developed, battery-operated, radio- controlled, tracked, robotic vehicle (called a Multi-Purpose Tele-operated Tracked vehicle (MUTT)), was acquired from a previous NIOSH project for the purpose of testing the dozer video system. The camera housing of the AVS system was installed on the MUTT (Fig. 9). The receiver, demodulator, and four LCD monitors were placed in a small room. With very little training the operator could maneuver the MUTT solely via viewing the images on the LCD monitors (Fig. 10). The system was tested inside an unlit building, outside under heavy cloud cover, and outside during a bright sunny day. There were no perceptible latencies even when the MUTT was operated at full speed. Figure 9. MUTT test vehicle at PRL.
PAGE – 5 ============
Figure 10. Lab analog video camera tests. B. AVS Field Tests The AVS was taken to the Eighty Four Mine coal stockpile for further tests. The camera housing was placed on top of a non-remote-controlled D10R dozer (Fig. 11). The four LCD monitors and other equipment were installed in the field trailer (Fig. 12). The operator of the dozer was instructed to take the dozer around the entire periphery of the coal stockpile, through normal production areas as well as behind the stacker tubes. The video viewed in the field trailer was excellent. The whole stockpile could be covered with no loss or degradation to the video quality. A little noise in the video was noticed when the dozer went behind the stacker tubes. Since the system was not tested with a remote controlled dozer it is difficult to conclude the effectiveness or efficiency of a remote operator using the camera system. Future testing on a remote-controlled dozer is required. The audio system was not tested. The project concluded with these AVS field tests. Figure 11. Analog video camera box on dozer cab. Figure 12. Analog video camera field tests. V. SUMMARY AND CONCLUSIONS The purpose of the project was to improve the safety associated with the operation of bulldozers on coal stockpiles by investigating the feasibility of sensory-enhanced remote- control operation. The approach was to provide the operator with sufficient sensory cues so that he/she thought they were actually operating the dozer from on-board. A variety of tests were conducted with different camera systems. An analog- based video camera system provided excellent 360-degree real-time viewing coverage of the dynamic dozer production area. However, testing with a remote-controlled dozer is required to determine overall system efficiency and effectiveness. Echoing the pitch-and-roll motion of the dozer on the stockpile would be a desirable option that should be explored. Motion platforms are being used by the military to train vehicle operators (Army Guide, 2005, http://www.army -guide.com/eng/AG_Monthly.php). Developers of simulators also employ motion platforms. While most applications use the platforms in the simulator/trainer mode, the dozer application requires real-time operation. It was expected that some combination of off-the-shelf hardware with customization would be required to provide the real-time feedback desired. The concept envisions the use of a six -degree-of-freedom motion platform used for simulators, a wireless sensor package to acquire dozer motion, and custom software to acquire the data and drive the platform. A wide internet search for applicable technology identified one vendor™s hardware (Servo™s and Simulators Inc. http://www.servos.com/) which appears to have the requisite features. Efficient remote control of dozers on coal stock piles requires the operator to see and sense the environment around the dozer while off-board. NIOSH developed and tested a vision system which was fully capable of providing the
PAGE – 6 ============
operator excellent remote 360-degree views around the dozer. Future work is required to provide dozer motion cues to the operator. REFERENCES [1] W.J. Wiehagen, A.G. Mayton, J.S, Jaspal, and F.C. Turin, “An analysis of serious injuries to dozer operators in the U.S. Mining Industry,” Dept. Health & Human Services, NIOSH Information Circular 9455, Publication No. 2001-26. [2] J.W. Fredland, K.K. Wu, and D.W. Kirkwood, “Material instability hazards in mine-processing operations,” U.S. Dept. Labor, MSHA IR 1212, 1993, 10pp. [3] D. Zageer, “Discussion of dangers involved when transporting and storing unconsolidated materials,” Mining Engineering, January, 1987; pp 23-25. [4] Mine Safety and Health Administration, web page http:// www.msha.gov/techsupp/techexchange/surgepile/surgepilesafety.pdf [5] M. Skiles, “MSHA™s triangle of success helps safe lives,” Coal People Magazine, March 2003. [6] J.W. Fredland, “Stronger cab windows: a way to improve surge-pile safety,” U.S. Dept Labor, MSHA Status Report, December 1, 1999; 6pp. [7] R.A. Elam, “Fatalities resulting from hazards on coal stockpiles,” U.S. Dept. Labor, MSHA Program Information Bulletin P99-1, 1999.
251 KB – 6 Pages