Meteorological Administration Big Data Platform Project

 lUnified Management and Service of Full Data Sets: Leverages GBase UP to enable transparent access to heterogeneous databases, a unified data service interface, and unified data management; enables complete management of all meteorological data and provides a platform computing service for product processing and mining analysis beyond numerical weather prediction.

lReal-time Data Synchronization: Meteorological data enters transactional databases for real-time operational query access. Incremental data from the transactional database is synchronized to the GBase 8a analytical database in near real-time via Kafka. GBase 8a’s data storage, service, and analysis capabilities unlock the value of data, supporting weather forecasting, disaster prevention and mitigation, and other critical operations. 

lHigh-Performance Analysis: Harnesses the high-performance, distributed parallel computing power of GBase 8a clusters to significantly boost the performance of complex analytical workloads in meteorological operations.

Serving as a historical analytics database, GBase 8a MPP Cluster synchronizes existing meteorological observation data, geospatial information, and industry and social data into the GBase 8a database in real time via Kafka, enabling long-term time-series data storage and multi-dimensional analysis and computation.

It offers a unified database access interface, delivering transparent access across heterogeneous database engines to simplify unified data management, application migration, and development.

Cluster Deployment at the National Meteorological Center

• Using the 12 existing servers provided by the client, the GBase 8a cluster is deployed and installed. The cluster will be scaled out later based on the client’s requirements.

• Based on the existing hardware resources, the deployment adopts 3 management nodes + 10 data nodes, with one node co-locating management and data node roles.

Cluster Deployment at Provincial Meteorological Information Centers

• Additionally, 8 sets (8 nodes each) are deployed at regional centers, and 23 sets (6 nodes each) at provincial centers, totaling 32 sets and 219 nodes.

In accordance with the Meteorological Radar Development Special Plan (2017-2020) and its implementation guidelines, a high-efficiency, distributed, real-time and historical integrated data storage and management system must be established at both national and provincial levels to meet the access and analysis performance requirements for massive datasets. This project is a critical component of the radar data sharing platform, effectively accelerating the informatization and consolidation of national and provincial meteorological radar data sharing and operational applications.

The project aims to deploy mature, cost-effective distributed analytical databases at the national center and across 31 provincial nodes to store long-term historical time-series data, supporting the data services and analytics required by operational systems for climate monitoring, forecasting, and decision support.

The Meteorological Big Data Platform delivers unified management and services for the full dataset, providing platform-based computing services for processing and mining analysis of all types of meteorological data products (excluding numerical weather prediction). It directly supports applications, enables the comprehensive development of the "cloud + endpoint" business model, and forms an intensive, standardized, and open new ecosystem for meteorology.