Tibet Communications Administration Bureau Big Data Platform Project

Project Background

The Communications Administration serves as the competent authority for the telecommunications industry within each provincial-level administrative region, exercising centralized and unified supervision over the telecom sector. To enable security analysis of fundamental telecom networks and strengthen industry oversight, the Administration's big data analytics platform is designed to provide integrated management and analysis of threat intelligence. By centrally collecting massive, heterogeneous data and conducting focused threat analysis, the platform uncovers threats and surfaces relevant potential threat information.

Requirements

The project requires a big data analytics platform that can effectively support the processing and operation of diverse business information, deliver comprehensive online and offline data storage and processing capabilities, and ensure ease of management and scalability.
The platform must accommodate structured, semi-structured, and unstructured data. Through scalable distributed technologies, it should efficiently support data processing, retrieval, statistics, analysis, and deep mining, while optimally scheduling resources and tasks. Ultimately, it must provide upper-layer business systems with standard API-based services for data storage, retrieval, statistics, and analysis.

The platform is required to deliver the following capabilities:

• Data Ingestion and Storage

Leveraging an advanced big data storage and query architecture, it enables collection, preprocessing, and standardized ingestion of structured, semi-structured, and unstructured log data. This provides efficient integrated storage, rapid analysis of massive datasets, and real-time querying.

• Threat Correlation Discovery

Through feature extraction and built-in correlation analysis, the platform performs big data correlation discovery, cross-data perspective analysis, and contextual inference of threat behavior. This delivers the logical foundation for threat analysis.

• Deep Data Analysis and Mining

Drawing on aggregated data, the platform supports analytical determination and establishes a technical workflow that progresses from progressive analysis to deep mining. It identifies threat patterns and continuously enriches security knowledge bases—including the autonomous region vulnerability database, malware signature database, malicious program repository, IP/domain reputation database, and intrusion signature database. Systematic management enables rule updates and administration across these knowledge bases.

• Flexible Data Sharing and Access

This capability primarily delivers unified data sharing. The platform’s output interfaces for upper-layer services use standardized APIs for real-time data access, while non-real-time data is transmitted via standard log formats using FTP or similar protocols.

This project employs the GBase UP Big Data Fusion Platform to build a unified, standardized DaaS platform for data ingestion and data access. The platform includes a data ingestion layer, data storage layer, data computing layer, analytics service layer, and unified big data monitoring and management. The platform architecture 

Description:

• The data ingestion layer supports data ingestion, filtering, caching, and relay dispatching;

• The data storage layer ensures unified and reliable storage management for massive heterogeneous data, offering standardized access interfaces for structured, semi-structured, and unstructured data;

• The data computing layer, built on a distributed computing framework supporting multiple compute models, provides professional computing libraries for upper-layer business systems;

• The analytics service layer provides access interfaces supporting various protocols and standards, and implements cross-engine job scheduling within the big data platform;

• Unified big data monitoring and management handles platform operations and monitoring, offering both command-line and graphical management interfaces.

This project built a converged platform using MPP and Hadoop, deploying over a hundred nodes to meet project requirements.

Value Delivered

• Meet unified data management needs and accelerate service rollout

By adopting the GBase UP unified big data platform, the project addresses the need for unified multi-model data management and provides standard APIs for upper-layer applications, boosting development efficiency and speeding up time-to-market.

• Significantly reduce costs

The entire system is built on open-architecture hardware and software, drastically lowering construction costs.

• Fully domestic technology stack ensures system security

Given the high data confidentiality requirements of the communications regulatory authority project, a fully China-developed technology stack was adopted, keeping user data secure and under control.