Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add informative error message when using bind_into_vpts() on profiles with different altitude layers #341

Closed
pecard opened this issue Apr 9, 2020 · 4 comments
Milestone

Comments

@pecard
Copy link

pecard commented Apr 9, 2020

While trying to build a vertical profile for a specific radar, I realized that h5 files may be non-uniform, causing an error when binding several temporal profiles.

I came across this while trying to plot the vertical profile for all data from a single radar was throwing an error but it went just ok for part of the dataset.

I realized that binding the entire dataset was creating a vpts file with different data structure from the example_vpts data but with no error or warnings.

reproducible example:

library(bioRad)
library(tidyverse)
data(example_vpts)

download_vpfiles(
  '2019-01-01',
  '2020-04-09',
  radar = c("ptlis"),
  directory = here::here('data-raw'),
  overwrite = FALSE
)

h5f <- list.files(here::here('data-raw'), pattern = '.h5', recursive = T, full.names = T)

vpf <- lapply(h5f, read_vpfiles)

summary(example_vpts$data) # data type = numeric

# entire profile
vpts0 <- bind_into_vpts(vpf[c(1:141)])
is.vpts(vpts0) # TRUE
summary(vpts0$data) # data type = list and not numeric
integrate_profile(vpts0)
# Error in `rownames<-`(`*tmp*`, value = x$height) : 
#   attempt to set 'rownames' on an object with no dimensions

# but spliting the vp set
vpts1 <- bind_into_vpts(vpf[c(1:58)])
summary(vpts1$data) # data type = numeric
is.vpts(vpts1)
tsReg1 <- regularize_vpts(vpts1)
plot(tsReg1) # ok

vpts2 <- bind_into_vpts(vpf[c(59:141)])
summary(vpts2$data) # data type = numeric
is.vpts(vpts2)
tsReg2 <- regularize_vpts(vpts2)
plot(tsReg2) # ok

vpts3 <- bind_into_vpts(vpts1, vpts2)
# Error in bind_into_vpts.vpts(vpts1, vpts2) : 
#   Vertical profiles have non-aligning altitude layers

I'd say that bind_into_vpts is not dealing well with non-uniform data, particularly when height profile is not aligned among layers.

@adokter adokter changed the title trying to bind_into_vpts() error: Vertical profiles have non-aligning altitude layers add informative error message when trying to use bind_into_vpts() on profiles with different altitude layers Apr 9, 2020
@adokter
Copy link
Owner

adokter commented Apr 9, 2020

HI @pecard, thanks for this. Indeed bind_into_vpts() expects profiles that have the same altitude layer definition, i.e. the same number of height layers and the same width.

Adjusting it to so it can deal with different altitude layers definitions is something for the future, I've moved that part to issue #343

I'll try to add a fix relatively soon so at least bind_into_vpts() provides an informative error message that tells you where in the time series the altitude layer definition changes.

@adokter adokter changed the title add informative error message when trying to use bind_into_vpts() on profiles with different altitude layers add informative error message when using bind_into_vpts() on profiles with different altitude layers Apr 9, 2020
@adokter adokter added the bug label Apr 9, 2020
@adokter adokter added this to the 0.6.0 milestone Apr 9, 2020
@bart1
Copy link
Collaborator

bart1 commented Apr 21, 2020

I also just got bitten by this problem the vpts gets lists in the data instead of arrays, see example below. in this case the altitude layers are aligned so I guess missing data could be assigned for the missing bins

require(bioRad)
#> Loading required package: bioRad
#> Welcome to bioRad version 0.5.1.9344
#> Docker daemon running, Docker functionality enabled (vol2bird version 0.5.0)
(vp1<-read_vpfiles('/home/bart/bioRad_tmp_files/frmom//2019/11/25/12/frmom_vp_20191125T125500Z_0xb.h5'))
#>                Vertical profile (class vp)
#> 
#>        radar:  frmom 
#>       source:  NOD:frmom,PLC:Momuy,WMO:07606 
#> nominal time:  2019-11-25 12:45:00 
#> generated by:  vol2bird 0.3.20
(vp2<-read_vpfiles('/home/bart/bioRad_tmp_files/frmom//2019/09/01/10/frmom_vp_20190901T102500Z_0xb.h5'))
#>                Vertical profile (class vp)
#> 
#>        radar:  frmom 
#>       source:  NOD:frmom,PLC:Momuy,WMO:07606 
#> nominal time:  2019-09-01 10:15:00 
#> generated by:  vol2bird 0.3.20
vp2$attributes$where$maxheight
#> [1] 4000
vp1$attributes$where$maxheight
#> [1] 5000
vpts<-bind_into_vpts(vp1, vp2)
str(vpts$data$ff)
#> List of 2
#>  $ : num [1:20] NaN 1.38 NaN NaN NaN ...
#>  $ : num [1:25] NaN 3.13 4.42 6.45 NaN ...
integrate_profile(vpts)
#> Error in `rownames<-`(`*tmp*`, value = x$height): attempt to set 'rownames' on an object with no dimensions

Created on 2020-04-21 by the reprex package (v0.3.0)

@bart1
Copy link
Collaborator

bart1 commented Apr 21, 2020

One quick example for adding empty height bins (in this single case it does not seem to affect the integrated profile estimates):

require(bioRad)
#> Loading required package: bioRad
#> Welcome to bioRad version 0.5.1.9344
#> Docker daemon running, Docker functionality enabled (vol2bird version 0.5.0)
vp1<-read_vpfiles('/home/bart/bioRad_tmp_files/frmom//2019/11/25/12/frmom_vp_20191125T125500Z_0xb.h5')
vp2<-read_vpfiles('/home/bart/bioRad_tmp_files/frmom//2019/09/01/10/frmom_vp_20190901T102500Z_0xb.h5')
vp2$data<-rbind(vp2$data, vp2$data[1:5,])
vp2$data[21:25,]<-NA
vp2$data$height<-(0:24)*200
vpts<-bind_into_vpts(vp1, vp2)
str(vpts$data$ff)
#>  num [1:25, 1:2] NaN 1.38 NaN NaN NaN ...
integrate_profile(vpts)
#>              datetime       mtr        vid       vir       rtr        mt
#> 1 2019-09-01 10:15:00 0.2220192 0.04454692  1.578907  2.442211  226.7371
#> 2 2019-11-25 12:45:00 3.3966660 0.30158455 29.438553 37.363327 3695.5823
#>          rt       ff       dd          u         v height
#> 1  2494.108 1.384428 177.1907 0.06785376 -1.382764    300
#> 2 40651.406 3.128537 143.1474 1.87636542 -2.503397    300

Created on 2020-04-21 by the reprex package (v0.3.0)

@adokter adokter added the sprint label May 27, 2021
adokter added a commit that referenced this issue Jun 8, 2021
@adokter
Copy link
Owner

adokter commented Jun 8, 2021

closed by 15192fb and PR #447

@adokter adokter closed this as completed Jun 8, 2021
@adokter adokter mentioned this issue Jun 9, 2021
adokter added a commit that referenced this issue Jun 9, 2021
adokter added a commit that referenced this issue Jun 10, 2021
fix issue #341 did not work for vp list
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants