Study on Combined Weather Numerical Model with Machine Learning on Big Data Computing Platform for Real-Time Typhoon Wind Velocity Prediction (II)

瀏覽統計 Email 通知 RSS Feed

簡歷

基本資料

Project title

Code/計畫編號

MOST107-2111-M019-003

Translated Name/計畫中文名

結合氣候數值模式與機器學習法於大數據運算平台即時預測颱風風速之研究(II)

Project Coordinator/計畫主持人

Chih-Chiang Wei

Funding Organization/主管機關

National Science and Technology Council

Department/Unit

Department of Marine Environmental Informatics

Website

https://www.grb.gov.tw/search/planDetail?id=12677821

Year

2018

Start date/計畫起

01-08-2018

Expected Completion/計畫迄

31-07-2019

Bugetid/研究經費

716千元

ResearchField/研究領域

大氣科學

本計畫將以機器學習法結合數值天氣預測模式並建構大數據平行分散式運算平台研發一套颱風時期即時預測地表或離岸風速風向預測模式整合系統。本計畫為期二年，上年度工作為「數值天氣預測模式模擬」和「機器學習模式預測」，目前已進行或部份完成相關工作；第二年度則以「結合機器學習和數值模式連結運算」和「大數據平行分散式運算平台建置」為主要工作。本計畫的數值天氣預測模式採用WRF數值模式結合機器學習法為一種混合數值與統計模式的概念，可用以結合統計和物理模式兩者之間的特性，便於解決短時距（如1小時）即時預測地表或離岸風速風向問題。一般而言，數值模式優點為所有參數都是由大氣動力學、數值方法計算而得，預測結果可由物理關係解釋，因此可用以求解高度非線性的氣象問題，然而，其模擬過程較為費時，可能無法提供即時性；相對的，統計模式在計算效率則較數值模式為高，其因是統計模式乃由過去長期觀測資料建立預測變數與其他屬性間之統計關係，利用合適的統計方法可快速地建立預測模式，統計模式之物理基礎較為薄弱，如兩參數統計關係很強但其物理上可能無因果關係可解釋。因此若能綜合兩者之優點或可提供一預測時的高時效性和準確性，因此衍生了本計畫之構想。本計畫以臺灣東北部和其他地區為研究區域，所發展的系統可用於在颱風期間地表或離岸任意位置和高度之風速風向預測功能以便有關單位防災時運用，預測長度為112小時（視誤差評估後而定）。本計畫發展模式過程中，首先，利用WRF數值模式模擬環流分布及網格點上風速風向，模組化主要過程中，利用WRF數值模式模擬多場颱風之環流分布，以便得到網格點風速風向數值解。其次，以機器學習法（如支援向量機、決策樹、隨機森林）建構風速風向預測模式。為了得到任意點之預測值，首先將利用有觀測資料測站（地面測站和海面浮標站）建立統計模式，以便評估統計解與觀測值的誤差，同時並比較數值解與統計解之優劣；經由驗證統計模式後，本計畫即可進行任意點之建模工作（即結合機器學習法和數值模式），建模中因任意點無風速風向觀測值，因此將以高精度解析下之數值解（視為真值）替代。最後，採用大數據最新技術Hadoop Spark建構運算平台，以達成1)同時接收來自各地遠端伺服器的資料來源；2)在主機端建構叢集運算框架和線上預測模式以預測任意點風速。本計畫完成整合系統後，將討論數值模式、機器學習與兩者結合預測能力。A two-year project is proposed here. A useful scheme for wind speed and direction forecasts during typhoons is highly desirable for Taiwan, because the powerful winds accompanying these severe typhoons drastically affect, for examples, the under-construction structures and wind turbines of the wind farm. A weather numerical model combined with machine learning algorithms is proposed for forecasting the wind speed and direction during typhoons. The developed combination model can be used to determine the wind speed and direction on a short time step (1 h or less) of a forecasting operation. We will employ the mostly used support vector machines for regressions, decision trees, and random forests to build the machine learning-based prediction model. The WRF (Weather Research and Forecasting) numerical model will be employed as the numerical simulation based model. The experimental area of Northeast Taiwan will be used for simulation. Regarding a complex typhoon system, the collected data comprise the typhoon tracks, FNL (Final) Operational Global Analysis Data for WRF model, the typhoon characteristics, the buoy atmospheric properties, and ground weather data. This study includes 40 typhoon events and more over the past 10 years (2007–present). Four criteria, including the relative mean absolute error, coefficient of variation of the root mean squared error, correlation coefficient, and efficiency coefficient, will be used to highlight the model cases capable of identifying the optimal performance level. This study will compare the WRF numerical model, the machine learning models, and the WRF combined with machine learning model, and observations.We also propose a Hadoop Spark Big Data platform applied to the real-time wind speed and direction predictions. There are various tools which can be used in Big Data management from data acquisition to data analysis. Hadoop brings the ability to cheaply process large amounts of data, regardless of its structure. Hadoop is made up of two core projects: Hadoop Distributed File System (HDFS) and MapReduce. Hadoop splits files into large blocks and distributes them across nodes in a cluster. To process data, Hadoop transfers packaged code for nodes to process in parallel based on the data that needs to be processed. In addition, Apache Spark provides programmers with an application programming interface centered on a data structure called the resilient distributed dataset, a read only multiset of data items distributed over a cluster of machines, that is maintained in a fault-tolerant way. It was developed in response to limitations in the MapReduce cluster computing paradigm, which forces a particular linear dataflow structure on distributed programs: MapReduce programs read input data from disk, map a function across the data, reduce the results of the map, and store reduction results on disk. Finally, several typhoons will be used as simulation typhoons for the real-time forecasts by using the Hadoop Spark Big Data platform. This study will demonstrate the feasibility of the proposed methodology and Big Data platform, and discuss the various model cases on the basis of increasing the accuracy and efficiency of the wind speed and direction predictions.

Keyword(s)

颱風
數值氣候模式
風速
機器學習
巨量資料
Typhoon
Numerical climate model
Wind speed
Machine learning
Big Data

DSpace CRIS

Study on Combined Weather Numerical Model with Machine Learning on Big Data Computing Platform for Real-Time Typhoon Wind Velocity Prediction (II)

基本資料

Description