Chexingyun big data application system is a comprehensive big data analysis and application system oriented to the public security traffic control departments, based on information data related to traffic information, vehicle and driver management information, illegal act and accident information, and integrating the passage monitoring, search control and practical action analysis.
Driven by data and supported by cloud computing and big data, the platform gathers, cleans, associates, collides and mines the business data of public security traffic control to improve the application level of the verification and comparison, query, analysis performance and management of massive data; gather, analyze, store and apply the wide-range, high-density, continuous and long-term road network traffic information, provide business applications close to the actual action, flexible tactics, intelligent analysis and judgment, and visual information display; carry out all-round, deep-level and multi-angle traffic management information analysis, explore traffic safety hazards and weak work links in an intelligent way and provide data support for the improvement of vehicle search and case detection efficiency.
The Chexingyun big data application system is a business application system and data analysis system widely used in the daily tasks of public security and traffic police.
Figure 2-1 shows the logical framework of Chexingyun big data application system. It is logically divided into four layers, namely data access layer, platform support layer, business application layer and user layer.
Figure 2-1 Logical Architecture of Chexingyun Big Data Application System
Data access layer: It mainly completes the convergence and access of data. The accessed data mainly includes public security six-in-one data, checkpoint passage data, electric alarm equipment data and ERI data.
Platform support layer: It conducts data access, processing, storage and modeling analysis through the intelligent traffic data platform, to provide functional service interface and data service interface for the upper business application layer and support the business application system at the upper layer.
Business application layer: With the use of modular design and focus on displaying human-computer interaction, it provides the business application functions of vehicle passage monitoring, search control and practical action analysis.
User layer: It provides comprehensive traffic information services for traffic control, public security, criminal investigation, technical investigation and other users to complete the daily business work.
3. Product Advantages
Distributed message queue system
The Chexingyun access data includes the structured data (e.g. passage data, illegal act evidence data, warning condition data and other types), unstructured data (e.g. video and image) and all other types. The functional modules of the backstage boast the strong interactivity, which requires the support of efficient and strong data exchange.
The Chexingyun adopts kafka, the distributed message queue of open source to address the access of various data sources and data exchange. Boasting the high throughput, kafka can solve the big data throughput effectively and enhance the efficiency of data integration.
Mixed storage architecture
The system includes the passage data, public security six-in-one data, related data of social resources, and other types of data. The approach of reasonable storage can resolve the practical problems such as timeliness, data analysis, etc.
Hence, for the purposes of different applications, the system adopts the mixed architecture for data storage. The system adopts the distributed file memory system (HDFS), image hosting (FASTDFS), real-time retrieval (elasticsearch), and structured achievement data storage (oracle/mysql), all of which meet the requirements of flexible data application for the offline analysis, image visiting, real-time query, result presentation, and other businesses.
Double computing engine
The computing engine includes: offline type and real-time type. Adopting the offline engine technology, the system deals with the data cleaning, preprocessing, eigenvalue analysis, modeling and algorithm implementation, instead of providing the data presentation after the analysis for products; in view of the attribute data (e.g. location information, license plate, checkpoint and speed) or video, and image data featuring dynamics, magnanimity, high dimension, timeliness, continuity, multisource and infinity, etc. and with the group record of its data flow varying along with time, the system adopts the real-time computing engine technology, accepts the real-time traffic data flow, and conducts the collision comparison, analysis of early warning, etc. on the eligible events within the designated time window on the basis of model evaluation.
The Chexingyun adopts the open source spark technology, which is a computing framework based on memory and whose performance can well meet the requirements of business under the scenarios of offline analysis and real-time computation.
The distributed storage, computing engine and others constitute a huge challenge to the operation and maintenance monitoring of machines. The automatic monitoring includes two parts: agent and server. The system collects the scheduled information about operating status of all machine elements through agent, such as the capacity factor of CPU, operation of services, connection of ports, occupancy of bandwidth traffic, etc. The server will store the collected information and vividly describe the operating status of the machine by the way of graphic display, so as to provide the operation and maintenance support for the operation and maintenance personnel.