Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

split_part doesn't work when using a negative index on BigQuery #557

Closed
1 of 5 tasks
b-per opened this issue Apr 25, 2022 · 2 comments · Fixed by #559
Closed
1 of 5 tasks

split_part doesn't work when using a negative index on BigQuery #557

b-per opened this issue Apr 25, 2022 · 2 comments · Fixed by #559
Labels
bug Something isn't working

Comments

@b-per
Copy link
Contributor

b-per commented Apr 25, 2022

Describe the bug

In BigQuery, the macro split_part does not return results when the part_number is negative (e.g. when we want to get items from the end of the string and not the beginning)

Steps to reproduce

select {{ dbt_utils.split_part("'1/2/3/4.sql'", "'/'", -1) }}
which translates to select split('1/2/3/4.sql','/')[safe_offset(-1 -1)]

Expected results

4.sql

Actual results

Null

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

Are you interested in contributing the fix?

Happy to contribute or let another person do it.
The macro needs to be changed so we can add a check on whether part_number is negative, and if it is, the macro needs to return

split(
    {{ string_text }},
    {{ delimiter_text }}
)[safe_offset(
    array_length(
        split(
            {{ string_text }},
            {{ delimiter_text }}
        )
        {{ part_number}})
]
@b-per b-per added bug Something isn't working triage labels Apr 25, 2022
@b-per
Copy link
Contributor Author

b-per commented Apr 25, 2022

I just realized that Postgres doesn't support negative indexes either.
And reading Redshift docs it looks like the index needs to be positive as well.

@dbeatty10
Copy link
Contributor

Thank you for finding and reporting this bug, @b-per !

If you're will to do the following, that would be great:

  1. add an integration test that includes a negative part_number
  2. solve for BigQuery, Postgres, and Redshift

Just let me know if you'd like help for any of the above!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants