ONBOARD WAVE MONITORING SYSTEM BY VIDEO IMAGE PROCESSING
Shigesuke Ishida1, Yoshifumi Takaishi, Nobuo Kiriya1, Iwao Watanabe1, Tsuyoshi Miyazaki2, Koichi Masuda3, Kohei Ohtsu4 and Kiyokazu Minami4
1National Maritime Research Institute, Mitaka, Tokyo, JAPAN
2Japan Marine Science and Technology Center, Yokosuka, Kanagawa, JAPAN
3Nihon University, Funabashi, Chiba, JAPAN
4Tokyo University of Mercantile Marine, Koto-ku, Tokyo, JAPAN
Information of ocean waves is important not only for safe and economical operation of ships but also for ocean environment survey. Recently remote-sensing devices such as altimeters on artificial satellites or radar systems are developed for measuring ocean waves. On the other hand visual observation from merchant ships is still widely carried out. Onboard wave monitoring will be continued because it can detect the waves around the ship immediately. However, to improve the quality of the wave information from merchant ships an onboard device with reasonable price is necessary. With this background a wave monitoring system for merchant ships was contrived, supported by the Program for Promoting Fundamental Transport Technology Research from the Corporation for Advanced Transport & Technology.
The major information of the proposed wave monitoring system is relative wave elevation in some positions along ship hull. The elevation is measured by processing video image taken by CCD cameras. The image processing is based on brightness in the analysis window. For removing the effect of flicker on sea surface, painting of the hull and so on, an ingenious algorithm is developed. For the measurement at night near infrared ray is used.
The relative wave elevation is converted to the absolute elevation in combination with vertical acceleration of the same location. And the wave field around the ship, i.e. wave height, period and direction, is estimated from the elevation of at least three points. The performance of the wave monitoring system was examined by experiments in laboratory and in real sea. In this paper the overview is described.
Wave information is indispensable for the safe and economical navigation of ships and is also important as the information on ocean environment. Recently ocean waves are observed from artificial satellites. Database of wave height measured by altimeters was published (Young et al. 1996) and the improved resolution and analysis algorithm of SAR (Synthetic Aperture Radar) images are making it possible to detect more detailed wave information then ever (Dankert et al., 2002) while the effort of improving the quality of onboard wave monitoring has been continued to have the real time wave information around ships and offshore structures. Measuring devices, making use of supersonic, radar and so on, are in practical use however, for replacing with visual observation and to get the wave information in wide area of ocean, a system with reasonable price and easy to use is demanded.
With this background an onboard wave monitoring system with a new concept has been developed. In this paper the overview is described. The framework of the research project is shown in Figure 1. The major information of the system is the relative wave motion along ship hull, which is measured by processing video image taken by CCD cameras. For estimating the absolute wave field around the ship, relative wave information from at least three cameras and vertical accelerations of the same positions are combined. The methods for estimating wave directional spectrum from ship motions and RAOs (response amplitude operators) of them are proposed by several researchers (Iseki et al., 1994; Hirayama et al., 1997; Saito et al., 2000; Yoshimoto et al., 1994). However the estimation error of RAOs and the ship motion caused by forces other than waves, for example wind, lead to a serious decline of accuracy. In the proposed wave monitoring system a simpler and more robust estimation method, just for wave height, period and direction, is adopted. Moreover, the proposals for making effective use of the wave monitoring system is within the scope of the research. In this paper the prototype of a navigation support system in heavy weather is shown.
Figure 1. Framework of the project
MEASUREMENT OF RELATIVE WAVE MOTION BY VIDEO IMAGE PROCESSING
Figure 2 shows the camera for extracting the intersection line of the ship hull and water surface. Small near-IR (infrared ray) illuminators are equipped. In general CCD has the sensitivity to the light from visible range to near-IR ray. And in order to have a natural picture for human eyes the latter range is eliminated by an optical filter. Here the filter is removed from the camera to make the monitoring system available even at night. In daytime the information by visible light is useful because much information, i.e. red, green, blue and brightness, is included. However, processing all the information is time consuming for the current personal computers. So, only the brightness information is used. The price of the camera is in the same order as the ones for domestic use.
Figure 2. CCD camera
Algorithm of Image Processing
The basic flow of the algorithm is shown in Figure 3. The intersection line between the ship hull and the wave surface is extracted by the change of brightness. This algorithm was confirmed to work well for the pictures taken in a model basin. However the pictures in real sea include some troublesome disturbances like the boundary of painting, change of brightness due to rolling and whitecapping. Figure 4 shows a sample picture of a ship's side. It is recognized that the boundary of painting appears very strongly.
Figure 3. Basic algorithm of video image processing
Figure 4. Brightness of a sample picture
The improved algorithm and the sample pictures are shown in Figures 5 and 6 respectively. At the first stage (pre-processing stage) the standard picture, called Σ-picture, is made by averaging the brightness for 15-20 seconds. Next, the picture at each time step of camera (1/30 seconds), called Δt-picture, is taken. In the analysis window (rectangular window, Fig. 6), the total brightness of Δt-picture is equalized to Σ-picture and the difference of brightness between the two is calculated. Through this processing the effect of brightness change at each instance is reduced and the no-moving parts, e.g. the boundary of painting, are removed. After that the brightness is smoothed to reduce noise. And the area with brightness over a threshold value is extracted as shown by the fourth picture in Figure 6. Finally the highest point of the area is judged to be the water surface. Along these processing the Σ-picture and the threshold value are revised for making them more appropriate. The time for processing one picture is 70 msec. When three cameras are equipped the sampling speed is 4 Hz, which is satisfactory for at-sea measurement.
Figure 5. Algorithm of the improved video image processing
Figure 6. Pictures of each step of image processing
Figure 7. An example of measured time history (left) and its error (right)
An example of measured time history of relative wave and its error are shown in Figure 7. In the left figure "True" value is estimated by human eyes on a computer display. The error is shown in the right as the difference between the two lines. The standard deviation of the error is 3.3 pixels. When the camera covers 8 meters, for example, in height of the ship's side wall, the accuracy of the location of water surface is 8[m]/640*3.3 = 4.1[cm] (640: the resolution of the camera).