AWS Athena unable to list schema (no tables found, check connection settings)

[slifin]

INFO [com.ultorg.database.SQLConnection]: Can't retrieve table list for catalog AwsDataCatalog; skipping (possibly a normal permissions issue)
com.simba.athena.amazonaws.services.glue.model.AccessDeniedException: User: XXXXXX is not authorized to perform: glue:GetTables on resource: arn:aws:glue:eu-west-1:XXXXXXX:catalog because no resource-based policy allows the glue:GetTables action (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: XXXXXX; Proxy: null)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1879)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1418)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1387)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1157)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:814)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:781)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:755)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:715)
	at com.simba.athena.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:697)
	at com.simba.athena.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:561)
	at com.simba.athena.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:541)
	at com.simba.athena.amazonaws.services.glue.AWSGlueClient.doInvoke(AWSGlueClient.java:12623)
	at com.simba.athena.amazonaws.services.glue.AWSGlueClient.invoke(AWSGlueClient.java:12590)
	at com.simba.athena.amazonaws.services.glue.AWSGlueClient.invoke(AWSGlueClient.java:12579)
	at com.simba.athena.amazonaws.services.glue.AWSGlueClient.executeGetTables(AWSGlueClient.java:7503)
	at com.simba.athena.amazonaws.services.glue.AWSGlueClient.getTables(AWSGlueClient.java:7472)
	at com.simba.athena.athena.api.AJClient.fetchTablesWithGlue(Unknown Source)
	at com.simba.athena.athena.api.AJClient.getTablesMetadata(Unknown Source)
	at com.simba.athena.athena.dataengine.metadata.AJTablesMetadataSource.initMetadata(Unknown Source)
	at com.simba.athena.athena.dataengine.metadata.AJTablesMetadataSource.<init>(Unknown Source)
	at com.simba.athena.athena.dataengine.AJDataEngine.makeNewMetadataSource(Unknown Source)
	at com.simba.athena.dsi.dataengine.impl.DSIDataEngine.makeNewMetadataResult(Unknown Source)
	at com.simba.athena.athena.dataengine.AJDataEngine.makeNewMetadataResult(Unknown Source)
	at com.simba.athena.jdbc.jdbc42.S42DatabaseMetaData.createMetaDataResult(Unknown Source)
	at com.simba.athena.jdbc.common.BaseDatabaseMetaData.getTables(Unknown Source)
	at com.ultorg.database.SQLConnection.a(SourceFile:500)
	at com.ultorg.database.SQLConnection.getDatabasesAndTables(SourceFile:614)
	at com.ultorg.project.model.n$1.run(SourceFile:1092)
	at com.ultorg.project.f.a(SourceFile:192)
	at com.ultorg.project.f.runTask(SourceFile:130)
	at com.ultorg.project.model.n.getTableList(SourceFile:87)
	at com.ultorg.project.d.b(SourceFile:462)
	at com.ultorg.project.d.c(SourceFile:441)
	at com.ultorg.project.j.a(SourceFile:709)
	at com.ultorg.project.j$a.b(SourceFile:1379)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1430)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:45)
	at org.openide.util.lookup.Lookups.executeWith(Lookups.java:287)
	at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:2045)
Caused: com.simba.athena.support.exceptions.GeneralException: [Simba][AthenaJDBC](100141) An error has been thrown from the AWS Glue client. User: arn:aws:sts::XXXXXX:assumed-role/Engineer/XXXXX is not authorized to perform: glue:GetTables on resource: arn:aws:glue:eu-west-1:XXXXXXX:catalog because no resource-based policy allows the glue:GetTables action
Caused: java.sql.SQLException: [Simba][AthenaJDBC](100141) An error has been thrown from the AWS Glue client. User: arn:aws:sts::XXXXXXX:assumed-role/Engineer/XXXXXXX is not authorized to perform: glue:GetTables on resource: arn:aws:glue:eu-west-1:XXXXXXXXX:catalog because no resource-based policy allows the glue:GetTables action
	at com.simba.athena.athena.api.AJClient.checkAndThrowException(Unknown Source)
	at com.simba.athena.athena.api.AJClient.fetchTablesWithGlue(Unknown Source)
	at com.simba.athena.athena.api.AJClient.getTablesMetadata(Unknown Source)
	at com.simba.athena.athena.dataengine.metadata.AJTablesMetadataSource.initMetadata(Unknown Source)
	at com.simba.athena.athena.dataengine.metadata.AJTablesMetadataSource.<init>(Unknown Source)
	at com.simba.athena.athena.dataengine.AJDataEngine.makeNewMetadataSource(Unknown Source)
	at com.simba.athena.dsi.dataengine.impl.DSIDataEngine.makeNewMetadataResult(Unknown Source)
	at com.simba.athena.athena.dataengine.AJDataEngine.makeNewMetadataResult(Unknown Source)
	at com.simba.athena.jdbc.jdbc42.S42DatabaseMetaData.createMetaDataResult(Unknown Source)
	at com.simba.athena.jdbc.common.BaseDatabaseMetaData.getTables(Unknown Source)
	at com.ultorg.database.SQLConnection.a(SourceFile:500)
	at com.ultorg.database.SQLConnection.getDatabasesAndTables(SourceFile:614)
	at com.ultorg.project.model.n$1.run(SourceFile:1092)
	at com.ultorg.project.f.a(SourceFile:192)
	at com.ultorg.project.f.runTask(SourceFile:130)
	at com.ultorg.project.model.n.getTableList(SourceFile:87)
	at com.ultorg.project.d.b(SourceFile:462)
	at com.ultorg.project.d.c(SourceFile:441)
	at com.ultorg.project.j.a(SourceFile:709)
	at com.ultorg.project.j$a.b(SourceFile:1379)
	at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1430)
	at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:45)
[catch] at org.openide.util.lookup.Lookups.executeWith(Lookups.java:287)

What's a bit weird is that I could see the list of databases initially, but not any more - I can continue to see them in Datagrip and Athena's web UI

[eirikbakke]

Thanks for the log exerpt! Another error is progress at least... hmm. Sorry about that!

What's a bit weird is that I could see the list of databases initially, but not any more - I can continue to see them in Datagrip and Athena's web UI

You mentioned in the other issue that there was a aws_session_token property in your ~/.aws/credentials file. Perhaps the session token timed out somehow? Could you perhaps refresh the list of tables in DataGrip, and see if the session token in the credentials file changes, and whether Ultorg is still not able to retrieve the list of tables?

(In Ultorg, you refresh the list of tables by right-clicking the data source icon and clicking Refresh Table Metadata. There is probably a similar action in DataGrip.)

[eirikbakke]

What's a bit weird is that I could see the list of databases initially, but not any more

And, just to confirm, you mean you could see the table list once in Ultorg, right?

[slifin]

Yes that's right, I saw the database list once in Ultorg from memory I then tried to click into loading the table list for one database and noticed in the error log it said security token expired

So I re-ran okta_aws I often have do this with Datagrip too, but never saw a database list after that (also closed Ultorg and re-opened it), I can confirm re-running okta_aws generates new AWS credentials, okta_aws is a company specific thing but I think its just giving me a new session token + access key + secret key after I give password/2 factor which I do once my current session expires feels like every 60 minutes or so

[slifin]

I've only just noticed I'm able to list the database when I use the Services tab my Athena is there under databases

But I'm unable to get tables from the database same as before

I'm going to try recreating the driver entry again just in case

[slifin]

There's no associated logs when no tables are found

[slifin]

For reference I was using the folders tab before to try access the databases

[eirikbakke]

So I re-ran okta_aws I often have do this with Datagrip too

OK, that sounds promising; it would explain why you saw a table list but then got the authorization error later. Though I'm wondering why it doesn't work when you restart Ultorg. Perhaps, if you have time for another experiment:

1. Close both Ultorg and DataGrip.
2. Re-run okta_aws.
3. Start both Ultorg and DataGrip again.
4. Refresh the table list in DataGrip, and make sure there are no errors. (Right-click "tables" in DataGrip's sidebar to the left and "Refresh"). If this works, proceed...
5. Then try refreshing the table list in Ultorg. (Right-click the data source icon and click "Refresh Table Metadata")
6. If it didn't work in Ultorg, check Ultorg's log for errors again. (Perhaps wait 2 minutes just in case there are timeouts involved.)

I've only just noticed I'm able to list the database when I use the Services tab my Athena is there under databases

That's interesting! I wonder if it's a timing issue... since you once saw it working before in the Folders tab. The test steps above might help reset things.

For reference I was using the folders tab before to try access the databases

Yeah, the Folders tab is the one to use, except for the purpose of installing the driver. The Services tab is a part of an underlying library that Ultorg is using, and it will be going away in a future release, once I have a better UI for installing JDBC drivers.