Queries fail with error socket: too many open files

[LalitLab]

Branch: https://github.com/turbot/steampipe-plugin-gcp/tree/issue-67

➜  config steampipe query 
Welcome to Steampipe v0.1.3
For more information, type .help

> select * from gcp_iam_role;
Error: pq: rpc error: code = Unknown desc = Get "https://iam.googleapis.com/v1/roles/container.admin?alt=json&prettyPrint=false": dial tcp 216.58.203.42:443: socket: too many open files

Partial results

> select * from gcp_iam_role;
+--------------------------------------------------+---------+-----------------------+----------------+-------+------------------------+--------------+-------------------------------------------------------------------------------------
| name                                             | deleted | role_id               | is_gcp_managed | stage | description            | etag         | included_permissions                                                                
+--------------------------------------------------+---------+-----------------------+----------------+-------+------------------------+--------------+-------------------------------------------------------------------------------------
| projects/niteowl-aaa/roles/gcp_metadata_mk5cnm9m | false   | gcp_metadata_mk5cnm9m | false          | BETA  |                        | BwWrGXTA3dQ= | ["datacatalog.categories.fineGrainedGet","datacatalog.categories.getIamPolicy","data
| projects/niteowl-aaa/roles/CustomRole2           | false   | CustomRole2           | false          |       | Created on: 2020-05-07 | BwWnFW9Fzhs= | ["accessapproval.requests.approve"]                                                 
| projects/niteowl-aaa/roles/gcp_owner_mk5cnm9m    | false   | gcp_owner_mk5cnm9m    | false          | BETA  |                        | BwWrGXTCbrk= | ["datacatalog.categories.fineGrainedGet","datacatalog.categories.getIamPolicy","data
|                                                  |         |                       |                |       |                        |              | omies.list","datacatalog.taxonomies.setIamPolicy","datacatalog.taxonomies.update","i
| projects/niteowl-aaa/roles/TestRole5             | false   | TestRole5             | false          |       | Created on: 2020-06-02 | BwWnFVa70Cg= | ["accessapproval.requests.approve"]                                                 
+--------------------------------------------------+---------+-----------------------+----------------+-------+------------------------+--------------+-------------------------------------------------------------------------------------

> .quit

Solution to the above issue on my system

The existing ulimit value on my system 256

➜  config ulimit -n
256

Changed ulimit value to 1024

config ulimit -n 1024

➜  config ulimit -n
1024

Now ran below queries and table works fine

➜  config steampipe query 
Welcome to Steampipe v0.1.3
For more information, type .help
> select count(*) from gcp_iam_role;
+-------+
| count |
+-------+
| 735   |
+-------+

> select * from gcp_iam_role;
+------------------------------------------------------------+---------+------------------------------------------------------+----------------+-
| name                                                       | deleted | role_id                                              | is_gcp_managed | 
+------------------------------------------------------------+---------+------------------------------------------------------+----------------+-
| projects/niteowl-aaa/roles/gcp_metadata_mk5cnm9m           | false   | gcp_metadata_mk5cnm9m                                | false          | 
| projects/niteowl-aaa/roles/gcp_owner_mk5cnm9m              | false   | gcp_owner_mk5cnm9m                                   | false          | 
|                                                            |         |                                                      |                | 
| roles/accesscontextmanager.policyReader                    | false   | accesscontextmanager.policyReader                    | true           | 
| roles/accesscontextmanager.policyAdmin                     | false   | accesscontextmanager.policyAdmin                     | true           | 
|                                                            |         |                                                      |                | 
| roles/anthosaudit.serviceAgent                             | false   | anthosaudit.serviceAgent                             | true           | 
| roles/apigee.analyticsEditor                               | false   | apigee.analyticsEditor                               | true           | 
| roles/androidmanagement.user                               | false   | androidmanagement.user                               | true           | 
| roles/anthos.serviceAgent                                  | false   | anthos.serviceAgent                                  | true           | 
| roles/accessapproval.approver                              | false   | accessapproval.approver                              | true           | 
| roles/accesscontextmanager.gcpAccessReader                 | false   | accesscontextmanager.gcpAccessReader                 | true           | 
| projects/niteowl-aaa/roles/TestRole5                       | false   | TestRole5                                            | false          | 
| roles/anthosservicemesh.serviceAgent                       | false   | anthosservicemesh.serviceAgent                       | true           | 
| roles/accesscontextmanager.vpcScTroubleshooterViewer       | false   | accesscontextmanager.vpcScTroubleshooterViewer       | true           | 
| roles/artifactregistry.admin                               | false   | artifactregistry.admin                               | true           | 
| projects/niteowl-aaa/roles/CustomRole2                     | false   | CustomRole2                                          | false          | 
| roles/actions.Admin                                        | false   | actions.Admin                                        | true           | 
| roles/apigee.synchronizerManager                           | false   | apigee.synchronizerManager                           | true           | 
| roles/appengine.serviceAdmin                               | false   | appengine.serviceAdmin                               | true           | 

Link I refered for ulimit solution
https://stackoverflow.com/questions/880557/socket-accept-too-many-open-files

[johnsmyth]

Closed in SDK fix - turbot/steampipe-plugin-sdk#45