Skip to content
GitLab
Commit 59634a67 authored by Eric Duminil's avatar Eric Duminil
Browse files

Updated script and diagrams

parent e0609e3a
Show whitespace changes
Inline Side-by-side
python_scripts/get_dwd_precipitation/Parse Niederschlag.ipynb
View file @ 59634a67
%% Cell type:markdown id:d52465eb tags:
 
# Niederschlag from DWD
 
## Klimadaten
 
[stundenwerte_RR_05705_hist.zip](https://www.dwd.de/DE/leistungen/_config/leistungsteckbriefPublication.zip?view=nasPublication&nn=16102&imageFilePath=675349470684952943382999480164406894835507349402593906628786280354581421008571928705011438849092751258299962952312473511744392814375916459681379337607540679777573246835730326122056459559982440454207144420355774317891799102452512290249737221071767178890331035105331501334382788925&download=true)
 
from [Klimadaten Deutschland - Stundenwerte (Archiv)](https://www.dwd.de/DE/leistungen/klimadatendeutschland/klarchivstunden.html) "Würzburg, Niederschlag, Historisch"
 
## Climate data center
 
https://cdc.dwd.de/portal/
 
%% Cell type:code id:316a929e tags:
%% Cell type:code id:1cfa0188 tags:
 
``` python
import pandas as pd
```
 
%% Cell type:raw id:daf94ab6 tags:
%% Cell type:raw id:c566f571 tags:
 
# Old version
 
from datetime import datetime
dateparse = lambda x: datetime.strptime(x, '%Y%m%d%H')
 
df = pd.read_csv('produkt_rr_stunde_19950901_20211231_05705.txt',
sep=';',
skipinitialspace=True,
usecols = [1, 3],
names = ['datetime', 'precipitation'],
skiprows = 1,
parse_dates = ['datetime'],
date_parser=dateparse,
index_col = 'datetime'
)
df
 
%% Cell type:code id:f16392a3 tags:
%% Cell type:code id:fef5e23e tags:
 
``` python
df = pd.read_csv('data_OBS_DEU_PT1H_RR_5705.csv',
index_col=False,
parse_dates=['datetime'],
usecols = [2,3],
names=['datetime', 'precipitation'],
skiprows=1
)
df = df.set_index('datetime')
df
```
 
%% Output
 
precipitation
datetime
1995-09-01 00:00:00 0.0
1995-09-01 01:00:00 0.2
1995-09-01 02:00:00 0.2
1995-09-01 03:00:00 0.1
1995-09-01 04:00:00 0.0
... ...
2023-10-27 04:00:00 0.0
2023-10-27 05:00:00 0.0
2023-10-27 06:00:00 0.0
2023-10-27 07:00:00 0.0
2023-10-27 08:00:00 0.2
[245850 rows x 1 columns]
 
%% Cell type:code id:52f73e31 tags:
 
``` python
import matplotlib.pyplot as plt
plt.rcParams["figure.figsize"] = (20, 10)
```
 
%% Cell type:code id:672300ca tags:
 
``` python
df2 = df[df.precipitation >= 0]
```
 
%% Cell type:code id:0fdf4a6f tags:
 
``` python
df2.resample('d').sum().plot(title='Niederschlag in Würzburg [mm / Tag]');
```
 
%% Output
 
 
%% Cell type:code id:9a68733c tags:
 
``` python
df2.resample('M').sum().plot(title='Niederschlag in Würzburg [mm / Monat]', ylim=(0, None));
```
 
%% Output
 
 
%% Cell type:code id:929bacb6 tags:
 
``` python
yearly_values = df2[df2.index.year >= 1996].resample('Y').sum()
```
 
%% Cell type:code id:c4dd7535 tags:
 
``` python
yearly_values.describe()
df2[(df2.index.year >= 2010) & (df2.index.year < 2023)].resample('Y').sum().describe()
```
 
%% Output
 
precipitation
count 28.000000
mean 563.071429
std 96.674589
min 410.500000
25% 512.550000
50% 548.900000
75% 614.400000
max 807.100000
count 13.000000
mean 559.800000
std 93.541524
min 432.100000
25% 493.200000
50% 537.900000
75% 628.900000
max 744.400000
 
%% Cell type:code id:0a1cc458 tags:
 
``` python
yearly_values.plot.bar(title='Niederschlag in Würzburg [mm / Jahr]', ylim=(0, None));
plt.savefig('wuerzburg_yearly_precipitation.png', facecolor='w', bbox_inches='tight')
```
 
%% Output
 
 
%% Cell type:code id:c74631eb tags:
 
``` python
yearly_values.boxplot()
plt.title("Niederschlag in Würzburg, zwischen 1996 & 2022 [mm / Jahr]")
plt.savefig('wurzburg_yearly_precipitations_1996_2022.png', facecolor='w', bbox_inches='tight')
plt.savefig('wuerzburg_yearly_precipitations_1996_2022.png', facecolor='w', bbox_inches='tight')
```
 
%% Output
 
 
%% Cell type:code id:faf431eb tags:
 
``` python
df3 = df2[df2.index.year == 2022]
```
 
%% Cell type:code id:2509790f tags:
 
``` python
rain2022 = pd.pivot_table(df3, values='precipitation', index=df3.index.time, columns=df3.index.dayofyear)
rain2022
```
 
%% Output
 
datetime 1 2 3 4 5 6 7 8 9 10 ... 356 357 \
00:00:00 0.0 0.0 0.0 0.6 0.8 0.0 0.0 0.0 0.0 0.0 ... 0.0 1.2
01:00:00 0.0 0.0 0.0 1.1 0.4 0.0 0.0 0.0 0.3 0.0 ... 0.0 0.6
02:00:00 0.0 0.0 2.4 1.1 0.0 0.0 0.0 0.0 0.3 0.0 ... 0.1 0.2
03:00:00 0.0 0.0 0.1 0.6 0.6 0.0 0.0 0.3 0.4 0.0 ... 0.4 0.3
04:00:00 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.2 0.8 0.0 ... 0.0 0.2
05:00:00 0.0 0.0 0.0 1.6 0.0 0.0 0.0 1.0 0.9 0.0 ... 0.0 0.0
06:00:00 0.0 0.0 0.2 2.5 0.0 0.0 0.0 0.4 0.1 0.0 ... 0.1 0.1
07:00:00 0.0 0.0 1.2 1.6 0.0 0.0 0.0 0.1 0.2 0.0 ... 1.4 0.0
08:00:00 0.0 0.0 2.8 1.1 0.0 0.0 0.0 0.0 0.8 0.0 ... 0.3 0.0
09:00:00 0.0 0.0 0.1 1.2 0.0 0.1 0.0 0.0 0.2 0.0 ... 1.1 0.0
10:00:00 0.0 0.0 0.1 1.2 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.0
11:00:00 0.0 0.0 0.9 1.5 0.0 0.0 0.1 0.0 0.1 0.0 ... 0.0 0.0
12:00:00 0.0 0.0 0.0 3.2 0.0 0.0 0.2 0.0 0.0 0.0 ... 0.1 0.1
13:00:00 0.0 0.0 0.0 1.9 0.0 0.0 0.1 0.0 0.0 0.0 ... 0.0 0.5
14:00:00 0.0 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.2 1.5
15:00:00 0.0 0.0 0.2 0.6 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.4 2.3
16:00:00 0.0 0.0 0.1 0.8 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.3 2.3
17:00:00 0.0 0.0 0.0 0.4 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 1.1
18:00:00 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.2 1.4
19:00:00 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.2 1.8
20:00:00 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.0 0.6
21:00:00 0.0 0.0 0.3 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.1 1.5
22:00:00 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 0.7 0.1
23:00:00 0.0 0.0 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.0 ... 1.3 0.0
datetime 358 359 360 361 362 363 364 365
00:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
01:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
02:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
03:00:00 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0
04:00:00 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0
05:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
06:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
07:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
08:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
09:00:00 0.0 0.0 0.4 0.0 0.0 0.0 0.0 0.0
10:00:00 0.0 0.0 1.1 0.0 0.0 0.0 0.0 0.0
11:00:00 0.0 0.0 0.6 0.0 0.0 0.0 0.0 0.0
12:00:00 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0
13:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
14:00:00 0.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0
15:00:00 0.2 0.0 0.1 0.0 0.0 0.0 0.0 0.0
16:00:00 0.0 0.0 0.0 0.0 0.0 0.7 0.1 0.0
17:00:00 0.0 0.0 0.0 0.0 0.0 1.4 1.1 0.0
18:00:00 0.0 0.0 2.2 0.0 0.0 0.0 0.1 0.0
19:00:00 0.0 0.0 1.0 0.0 0.0 0.0 0.1 0.0
20:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
21:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
22:00:00 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0
23:00:00 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
[24 rows x 365 columns]
 
%% Cell type:code id:42ddd82d tags:
 
``` python
import seaborn as sns
```
 
%% Cell type:code id:f1b17ef2 tags:
 
``` python
sns.heatmap(rain2022, annot=False, cmap='viridis', vmax=5)
plt.title('mm/h Regen in Würzburg, 2022')
plt.show()
```
 
%% Output
 
 
%% Cell type:code id:29be9114 tags:
 
``` python
df4 = df2[(df2.index.year >= 2010)].resample('M').sum()
df4
```
 
%% Output
 
precipitation
datetime
2010-01-31 35.9
2010-02-28 28.5
2010-03-31 29.7
2010-04-30 23.7
2010-05-31 68.4
... ...
2023-06-30 21.3
2023-07-31 54.2
2023-08-31 69.9
2023-09-30 18.6
2023-10-31 52.7
[166 rows x 1 columns]
 
%% Cell type:code id:283cd890 tags:
 
``` python
import calendar
df4['Year'] = df4.index.year
df4['Month'] = df4.index.month
df4['Month'] = df4['Month'].apply(lambda x: calendar.month_abbr[x])
df4
```
 
%% Output
 
precipitation Year Month
datetime
2010-01-31 35.9 2010 Jan
2010-02-28 28.5 2010 Feb
2010-03-31 29.7 2010 Mar
2010-04-30 23.7 2010 Apr
2010-05-31 68.4 2010 May
... ... ... ...
2023-06-30 21.3 2023 Jun
2023-07-31 54.2 2023 Jul
2023-08-31 69.9 2023 Aug
2023-09-30 18.6 2023 Sep
2023-10-31 52.7 2023 Oct
[166 rows x 3 columns]
 
%% Cell type:code id:28a0fd61 tags:
 
``` python
df4 = df4.set_index(['Year', 'Month'])
df4
```
 
%% Output
 
precipitation
Year Month
2010 Jan 35.9
Feb 28.5
Mar 29.7
Apr 23.7
May 68.4
... ...
2023 Jun 21.3
Jul 54.2
Aug 69.9
Sep 18.6
Oct 52.7
[166 rows x 1 columns]
 
%% Cell type:code id:8e3bd7d1 tags:
 
``` python
df_precipitation = df4.reset_index().pivot_table(columns='Year',index='Month',values='precipitation', sort=False)
df_precipitation
```
 
%% Output
 
Year 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 \
Month
Jan 35.9 38.6 66.6 29.8 31.1 53.3 62.2 14.7 56.1 37.2 23.2
Feb 28.5 31.4 12.2 35.1 39.6 8.4 60.1 21.2 12.4 10.6 106.5
Mar 29.7 6.2 5.7 26.1 7.7 43.3 36.5 41.6 46.0 44.8 27.0
Apr 23.7 14.2 11.8 42.1 40.7 17.3 49.6 20.9 34.5 27.8 10.9
May 68.4 6.6 34.7 99.7 73.7 22.4 66.9 118.4 48.4 72.0 41.2
Jun 21.5 110.0 67.2 44.7 16.7 39.3 53.4 49.3 27.3 44.6 75.6
Jul 75.2 94.6 66.8 36.3 77.5 26.7 47.5 85.0 54.4 25.3 15.5
Aug 185.1 43.4 28.0 108.9 95.1 67.6 30.6 61.1 22.5 39.2 61.9
Sep 55.3 28.3 44.8 88.9 31.2 26.2 32.5 68.3 24.1 31.2 31.9
Oct 27.6 36.1 41.9 56.5 39.2 28.5 38.1 36.1 8.8 65.9 41.6
Nov 87.0 0.6 79.7 60.2 35.3 87.8 66.0 62.4 11.4 37.2 9.6
Dec 106.5 109.5 78.5 30.2 41.3 27.6 7.4 49.9 86.2 54.2 48.3
Year 2021 2022 2023
Month
Jan 54.6 50.7 41.8
Feb 48.6 51.0 20.5
Mar 35.8 18.4 72.5
Apr 17.6 87.8 49.3
May 69.8 28.9 21.9
Jun 100.9 23.2 21.3
Jul 138.7 17.5 54.2
Aug 82.0 40.2 69.9
Sep 6.0 109.1 18.6
Oct 42.5 43.9 52.7
Nov 26.3 43.7 NaN
Dec 60.5 46.9 NaN
 
%% Cell type:code id:fa12c1b9 tags:
 
``` python
ax=sns.heatmap(df_precipitation, cmap='viridis_r', annot=True, fmt=".0f")
ax.invert_yaxis()
plt.title("Precipitation in Würzburg [mm / month]");
plt.savefig('wurzburg_monthly_precipitations.png', facecolor='w', bbox_inches='tight')
plt.savefig('wuerzburg_monthly_precipitation.png', facecolor='w', bbox_inches='tight')
```
 
%% Output
 
 
%% Cell type:code id:470cf56e tags:
 
``` python
df_precipitation.sum().plot.bar(title='Niederschlag in Würzburg [mm / Jahr]', ylim=(0, None));
```
 
%% Output
 
 
%% Cell type:code id:40727a96 tags:
 
``` python
df_precipitation.mean(axis=1).plot.bar(title='Mittlerer Niederschlag in Würzburg\n1966 -> 2022\n[mm / Monat]',
ylim=(0, None));
```
 
%% Output
 
 
%% Cell type:code id:0a195120 tags:
 
``` python
df_precipitation.transpose().plot.box(showfliers=False, ylim=(0, None))
duration = "2010 -> 2022"
plt.title(f"Variation of monthly precipitation in Würzburg, {duration}")
plt.ylabel("[mm / month]");
plt.savefig('monthly_precipitation_variation_wuerzburg.png', facecolor="w", bbox_inches = 'tight')
plt.savefig('wuerzburg_monthly_precipitation_variation.png', facecolor="w", bbox_inches = 'tight')
```
 
%% Output
 
 
%% Cell type:code id:e5d682d5 tags:
 
``` python
# TODO: Compare 1995 -> 2021 with 2015 -> 2021
# TODO: Get duration from min max index
# TODO: Ridgeline for each month?
# TODO: apply to Feuerbach
# TODO: Push and document
# TODO: Show on map
```
 
%% Cell type:code id:803223b4 tags:
 
``` python
```
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment

Dies ist die Gitlab-Instanz des Transferportals der Hochschule für Technik Stuttgart. Hier geht es zurück zum Portal