Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failures with ingesting MySQL databases with names with upper case characters #6708

Closed
daha opened this issue Dec 9, 2022 · 0 comments · Fixed by #6713
Closed

Failures with ingesting MySQL databases with names with upper case characters #6708

daha opened this issue Dec 9, 2022 · 0 comments · Fixed by #6713
Labels
bug Bug report

Comments

@daha
Copy link
Contributor

daha commented Dec 9, 2022

Describe the bug
I get failures, see below, with the MySQL source after updating to Datahub v0.9.2 from v0.8.43. I get these failures where the database name has upper case characters. The name of the database where I get the failure below is really amhDb. I suspect that error was introduced when introducing TwoTierSQLAlchemySource in the MySQLSource.

'failures': {'amhdb': ['Tables error: (pymysql.err.OperationalError) (1049, "Unknown database \'amhdb\'")\n[SQL: SHOW FULL TABLES FROM `amhdb`]\n(Background on this error at: https://sqlalche.me/e/14/e3q8)']},

To Reproduce
Ingest data with the mysql source where the database name has uppercase characters in the name.

Expected behavior
I do not expect any failure like in v0.8.43 when the database has a name with uppercase characters.

Desktop (please complete the following information):

  • Datahub version v0.9.2

Additional context
I suspect (I have not verified this) that the databases from inspector.get_schema_names() on line 84 are all in lower case and then when the new url is created on line 87 is defined with the name all in lower case and the error is introduced and I get failures when the inspector is used.

if self.config.database and self.config.database != "":
databases = [self.config.database]
else:
databases = inspector.get_schema_names()
for db in databases:
if self.config.database_pattern.allowed(db):
url = self.config.get_sql_alchemy_url(current_db=db)
inspector = inspect(
create_engine(url, **self.config.options).connect()
)
yield inspector

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug report
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant