Python script for reading different message patterns
I'm trying to make a flexible python script that reads and extracts some weather variables from a synop code.
This is the code:
import re
def extract_data_12_utc(message):
# pattern message
pattern = r'(\d{5}),(\d{4}),(\d{2}),(\d{2}),(\d{2}),(\d{2}),aaxx (\d{5}) (\d{5}) (\d{5}) (\d{5}) (1\d{4}) (2\d{4}) (3\d{4})? (4\d{4}) (6\d{4})? (7\d{4})? (8\d{4})? (\{3}) (2\d{4}) (5\d{4}) (7\d{4})'
matches = re.search(pattern, message)
# check if the match is successsful
if matches:
station = matches.group(1)
year = matches.group(2)
month = matches.group(3)
day = matches.group(4)
hour = matches.group(5)
min = matches.group(6)
# extracting variables
temp_air = float(matches.group(11)[2:]) / 10.0
temp_dew = float(matches.group(12)[2:]) / 10.0
pres_station = float(matches.group(13)[1:]) / 10.0 + 1000
pres_sealv = float(matches.group(14)[1:]) / 10.0 + 1000
prec_6h = float(matches.group(15)[2:4]) if matches.group(15) else none
wx = str(matches.group(16)[1:]) if matches.group(16) else none
cld = str(matches.group(17)[1:]) if matches.group(17) else none
temp_min = float(matches.group(19)[2:]) / 10.0 if matches.group(19) else none
pres_chg = float(matches.group(20)[2:]) / 10.0 if matches.group(20) else none
prec_24h = float(matches.group(21)[1:]) / 10.0 if matches.group(21) else none
# formatting results
formatted_data = [
station, year, month, day, hour, min,
f"{int(temp_air):02d}.{int((temp_air % 1) * 10):01d}",
f"{int(temp_dew):02d}.{int((temp_dew % 1) * 10):01d}",
f"{int(pres_station):04d}.{int((pres_station % 1) * 10):01d}",
f"{int(pres_sealv):04d}.{int((pres_sealv % 1) * 10):01d}",
f"{int(prec_6h):1d}" if prec_6h is not none else "none",
f"{int(wx):1d}" if wx is not none else "none",
f"{int(cld):1d}" if cld is not none else "none",
f"{int(temp_min):02d}.{int((temp_min % 1) * 10):01d}",
f"{int(pres_chg):1d}" if pres_chg is not none else "none",
f"{prec_24h:.1f}" if prec_24h is not none else "none"
]
# returns formatted data
return formatted_data
else:
# returns list if fails
return ["none"] * 16
# reading file
file_name = r"synop.txt"
with open(file_name, 'r') as file:
lines = file.readlines()
# list to store results
data_12_utc = []
# from 17th line
for line in lines:
data = extract_data_12_utc(line)
data_12_utc.append(data)
# show formatted data
for data in data_12_utc:
print(data)The input data is:
82145,2024,01,24,12,00,aaxx 24124 82145 32598 30502 10292 20250 30082 40124 83200 333 20231 58004= 82181,2024,01,24,12,00,aaxx 24124 82181 21498 73603 10257 20242 30008 40149 70262 84520 333 20246 59014 60084= 82184,2024,01,24,12,00,aaxx 24124 82184 21498 60502 10272 20252 30116 40124 70362 85520 333 20243 59014 69944= 82188,2024,01,24,12,00,aaxx 24124 82188 11560 53602 10264 20248 30128 40146 60214 72162 83260 333 58002 70210== 82191,2024,01,24,12,00,aaxx 24124 82191 12570 60501 10290 20262 30108 40114 60184 84250 333 20238 59014 70180== 82193,2024,01,24,12,00,aaxx 24124 82193 22470 30409 10289 20254 30106 40124 83100 333 20254 59016 60054= 82244,2024,01,24,12,00,aaxx 24124 82244 11470 70503 10269 20248 30061 40130 60024 70296 84220 333 20256 59002 70020== 82246,2024,01,24,12,00,aaxx 24124 82246 21596 83202 10252 20242 3//// 4//// 7036/ 887// 333 2//// 5//// 60254= 82263,2024,01,24,12,00,aaxx 24124 82263 11470 8//// 30118 69934 70352 887// 333 59013 70003== 82353,2024,01,24,12,00,aaxx 24124 82353 22497 63602 10264 20246 30002 40086 86400 333 20215 59014 60024= 82361,2024,01,24,12,00,aaxx 24124 82361 21497 63602 10276 20258 30088 40125 70265 86700 333 20269 59018 60024= 82444,2024,01,24,12,00,aaxx 24124 82444 12470 72703 10269 20252 30091 60624 85000 333 20270 58000 70620== 82445,2024,01,24,12,00,aaxx 24124 82445 22497 83202 10266 20254 30102 40154 8472/ 333 20243 58000 60314= 82562,2024,01,24,12,00,aaxx 24124 82562 32597 836// 1//// 2//// 3//// 4//// 8869/ 333 2//// 5////= 82861,2024,01,24,12,00,aaxx 24124 82861 21596 73202 1//// 2//// 39917 4//// 70360 8572/ 333 2//// 59027 60054=
However, it returns the following:
['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none'] ['none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none', 'none']
When I limit certain variables (i.e. until group 15) it returns:
['82145', '2024', '01', '24', '12', '00', '29.1', '25.0', '1008.2', '1012.3', 'None'] ['82181', '2024', '01', '24', '12', '00', '25.6', '24.1', '1000.7', '1014.8', 'None'] ['82184', '2024', '01', '24', '12', '00', '27.1', '25.1', '1011.6', '1012.3', 'None'] ['82188', '2024', '01', '24', '12', '00', '26.3', '24.8', '1012.7', '1014.6', '21'] ['82191', '2024', '01', '24', '12', '00', '29.0', '26.1', '1010.7', '1011.3', '18'] ['82193', '2024', '01', '24', '12', '00', '28.8', '25.3', '1010.6', '1012.3', 'None'] ['82244', '2024', '01', '24', '12', '00', '26.8', '24.8', '1006.1', '1013.0', '2'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None'] ['82353', '2024', '01', '24', '12', '00', '26.3', '24.6', '1000.2', '1008.6', 'None'] ['82361', '2024', '01', '24', '12', '00', '27.6', '25.8', '1008.7', '1012.5', 'None'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None'] ['82445', '2024', '01', '24', '12', '00', '26.6', '25.3', '1010.2', '1015.3', 'None'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None'] ['None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None', 'None']
How do I have a script that contains all types of schema messages?
Correct Answer
Even if only one variable is malformed, there may be reasons to reject the entire line (or replace it with a None string).
However, if you want to extract every correctly formatted variable, even if some variables in the line are malformed, you should split the line using re.split(', ', line) for a list of variables and convert/check each variable individually. Unfortunately, re matches the entire expression instead of each group
If you must use a flexible regular expression, you should consider potential formatting errors like (?:(4\d{4})|\d*[/] ) group.
Unfortunately, it increases the number of groups, so I use the non-capturing group operator :? to keep the group numbers the same. If you find it too unwieldy, another option is to use the more universal group expression (4[/\d]{4}), which allows missing values, but you will test the presence later Missing numeric symbol "/" or just catching an exception during conversion.
The above is the detailed content of Python script for reading different message patterns. For more information, please follow other related articles on the PHP Chinese website!
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
AI Hentai Generator
Generate AI Hentai for free.
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
Python script to create random jokes using pyjokes
Sep 13, 2023 pm 08:25 PM
Do you want to add some humor to your Python script or application? Whether you're building a chatbot, developing a command line tool, or just want to entertain yourself with random jokes, the pyjokes library can help. With pyjokes you can easily generate jokes in various categories and customize them to your liking. In this blog post, we will explore how to create random jokes in Python using the pyjokes library. We'll cover the installation process, generating different categories of jokes, customizing jokes, displaying them in a console application or web page, and handling any potential errors that may occur. Install pyjokes Before we start using pyjokes to create random jokes, we need
Do you know some reasons why crontab scheduled tasks are not executed?
Mar 09, 2024 am 09:49 AM
Summary of some reasons why crontab scheduled tasks are not executed. Update time: January 9, 2019 09:34:57 Author: Hope on the field. This article mainly summarizes and introduces to you some reasons why crontab scheduled tasks are not executed. For everyone Solutions are given for each of the possible triggers, which have certain reference and learning value for colleagues who encounter this problem. Students in need can follow the editor to learn together. Preface: I have encountered some problems at work recently. The crontab scheduled task was not executed. Later, when I searched on the Internet, I found that the Internet mainly mentioned these five incentives: 1. The crond service is not started. Crontab is not a function of the Linux kernel, but relies on a cron.
Exploring Orange3: Opening up a new world of data mining and machine learning!
Mar 04, 2024 pm 08:16 PM
Orange3 is a powerful open source data visualization and machine learning tool. It has rich data processing, analysis and modeling functions, providing users with simple and fast data mining and machine learning solutions. This article will briefly introduce the basic functions and usage of Orange3, and combine it with actual application scenarios and Python code cases to help readers better master the usage skills of Orange3. The basic functions of Orange3 include data loading, data preprocessing, feature selection, model establishment and evaluation, etc. Users can use the intuitive interface to drag and drop components to easily build data processes. At the same time, more complex data processing and modeling tasks can also be completed through Python scripts. Below we will go through a practical
Python script for monitoring website changes
Aug 29, 2023 pm 12:25 PM
In today's digital age, being aware of the latest changes on your website is crucial for a variety of purposes, such as tracking updates on your competitors' websites, monitoring product availability, or staying informed of important information. Manually checking your website for changes can be time-consuming and inefficient. This is where automation comes into play. In this blog post, we will explore how to create a Python script to monitor website changes. By leveraging the power of Python and some handy libraries, we can automate the process of retrieving website content, comparing it to previous versions, and notifying us of any changes. This allows us to remain proactive and react promptly to updates or modifications to the sites we monitor. Setting up the environment Before we start writing scripts to monitor website changes, we need to set up P
PyCharm Advanced Tutorial: Use PyInstaller to package code into EXE format
Feb 20, 2024 am 09:34 AM
PyCharm is a powerful Python integrated development environment that provides a wealth of functions and tools to help developers improve efficiency. Among them, PyInstaller is a commonly used tool that can package Python code into an executable file (EXE format) to facilitate running on machines without a Python environment. In this article, we will introduce how to use PyInstaller in PyCharm to package Python code into EXE format, and provide specific
Python script automatically refreshes Excel spreadsheet
Sep 09, 2023 pm 06:21 PM
Python and Excel are two powerful tools that when combined can open up a world of automation. Python has versatile libraries and user-friendly syntax that enable us to write scripts to perform various tasks efficiently. Excel, on the other hand, is a widely used spreadsheet program that provides a familiar interface for data analysis and manipulation. In this tutorial, we will explore how to leverage Python to automate the process of refreshing Excel spreadsheets, saving us time and effort. Do you find yourself spending valuable time manually refreshing your Excel spreadsheet with updated data? This is a repetitive and time-consuming task that can really kill productivity. In this article we will guide you through using Py
How to read excel data in pycharm
Apr 03, 2024 pm 08:42 PM
How to read Excel data using PyCharm? The steps are as follows: install the openpyxl library; import the openpyxl library; load the Excel workbook; access a specific worksheet; access cells in the worksheet; traverse rows and columns.
Flask installation and configuration tutorial: a tool to easily build Python web applications
Feb 20, 2024 pm 11:12 PM
Flask installation and configuration tutorial: A tool to easily build Python Web applications, specific code examples are required. Introduction: With the increasing popularity of Python, Web development has become one of the necessary skills for Python programmers. To carry out web development in Python, we need to choose a suitable web framework. Among the many Python Web frameworks, Flask is a simple, easy-to-use and flexible framework that is favored by developers. This article will introduce the installation of Flask framework,
