Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new APIv2 endpoints that allow advanced searches and included pagination functionality #47

Merged
merged 30 commits into from
Jan 31, 2022
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
d198061
refactor: updated HttpSession class to be much more flexible
Jan 12, 2022
8386f54
feat: allow org-level access and sub-account to be modified dynamically
Jan 12, 2022
b585fce
refactor: created three base class types that endpoints should inherit
Jan 12, 2022
1c6f698
refactor: reorganized existing code and refactored to use new base cl…
Jan 12, 2022
40c66b6
feat: implemented all new APIv2 endpoints including pagination support
Jan 12, 2022
5145e53
chore: included flake8-quotes in dev dependencies
Jan 12, 2022
b884556
docs: small update to README.md for latest API endpoints
Jan 12, 2022
f4ceeb6
refactor: corrected docs and simplified `build_dict_from_items` method
Jan 19, 2022
8bc018a
fix: sanitizing access to 'query_data'
Jan 19, 2022
11f4fcc
fix: narrowing down exception handling for pagination iteration
Jan 19, 2022
48ea0d6
refactor: moved resource attribute to base of SeachEndpoint
Jan 22, 2022
af5f0ec
refactor: removed unnecessary override method
Jan 22, 2022
b6bbde2
docs: added docstrings for 'query_data' where needed
Jan 22, 2022
96b50b1
docs: added docstrings for methods which are passed
Jan 22, 2022
2397048
refactor: changed LaceworkException name to match conventions
Jan 22, 2022
689901e
tests: massively de-duped code for new search API tests
Jan 22, 2022
3e6a62e
chore: importing Retry directly from urllib
Jan 27, 2022
9525938
fix: passthrough Lacework response when maximum retries reached
Jan 27, 2022
861fbaa
fix: supply json to the query execute function
Jan 27, 2022
152bee4
docs: improved function documentation for the BaseEndpoint class
Jan 27, 2022
0817ad9
chore: removed references to `query_data` in favor of `json`
Jan 27, 2022
784f470
chore: removed redundant `search()` method overrides
Jan 27, 2022
7d123b8
fix: improved consistency of variable naming in AgentAccessTokensAPI
Jan 27, 2022
faf4ec4
chore: improved error message for JSONDecodeError
Jan 27, 2022
cf80f2d
docs: added additional class docstrings
Jan 27, 2022
384704b
fix: fixed bugs and consistency issues found in testing
Jan 28, 2022
7823cdc
refactor: simplified and modernized all APIv2 tests
Jan 28, 2022
83880bd
tests: fixed dependency issue with tests
Jan 28, 2022
157b036
fix: modified error logging for 'nextPage' parsing
Jan 31, 2022
1624ff8
refactor: changed LaceworksdkException to LaceworkSDKException
Jan 31, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/python-test-flaky.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ jobs:
python -m pip install --upgrade pip
python -m pip install flake8 flake8-quotes pytest pytest-rerunfailures
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt; fi
if [ -f jupyter/requirements.txt ]; then pip install -r jupyter/requirements.txt; fi

- name: Run setup.py
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/python-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.6", "3.7", "3.8", "3.9"]
python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]

steps:
- uses: actions/checkout@v2
Expand All @@ -25,8 +25,9 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install flake8 flake8-quotes pytest pytest-rerunfailures
python -m pip install flake8 flake8-quotes pytest pytest-lazy-fixture pytest-rerunfailures
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
if [ -f requirements-dev.txt ]; then pip install -r requirements-dev.txt; fi
if [ -f jupyter/requirements.txt ]; then pip install -r jupyter/requirements.txt; fi

- name: Lint with flake8
Expand Down
2 changes: 1 addition & 1 deletion examples/example_alert_channels.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
lacework_client.alert_channels.get()

# Search Alert Channels
lacework_client.alert_channels.search(query_data={
lacework_client.alert_channels.search(json={
"filters": [
{
"expression": "eq",
Expand Down
2 changes: 1 addition & 1 deletion examples/example_audit_logs.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
lacework_client.audit_logs.get(start_time=start_time, end_time=end_time)

# Search Audit Logs
lacework_client.audit_logs.search(query_data={
lacework_client.audit_logs.search(json={
"timeFilter": {
"startTime": start_time,
"endTime": end_time
Expand Down
2 changes: 1 addition & 1 deletion examples/example_cloudtrail.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
lacework_client.cloudtrail.get(start_time=start_time, end_time=end_time)

# Search CloudTrail
lacework_client.cloudtrail.search(query_data={
lacework_client.cloudtrail.search(json={
"timeFilter": {
"startTime": start_time,
"endTime": end_time
Expand Down
2 changes: 1 addition & 1 deletion laceworksdk/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import logging

from .api import LaceworkClient # noqa: F401
from .exceptions import ApiError, laceworksdkException # noqa: F401
from .exceptions import ApiError, LaceworksdkException # noqa: F401
alannix-lw marked this conversation as resolved.
Show resolved Hide resolved

# Initialize Package Logging
logger = logging.getLogger(__name__)
Expand Down
26 changes: 19 additions & 7 deletions laceworksdk/api/base_endpoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

class BaseEndpoint:
"""
Lacework BaseEndpoint Class.
A class used to implement base functionality for Lacework API Endpoints
"""

def __init__(self,
Expand All @@ -22,7 +22,10 @@ def __init__(self,

def build_dict_from_items(self, *dicts, **items):
"""
A method to build a dictionary based on inputs, pruning items that are None
A method to build a dictionary based on inputs, pruning items that are None.

:raises KeyError: In case there is a duplicate key name in the dictionary.
:returns: A single dict built from the input.
"""

dict_list = list(dicts)
Expand All @@ -32,17 +35,22 @@ def build_dict_from_items(self, *dicts, **items):
for d in dict_list:
for key, value in d.items():
camel_key = self._convert_lower_camel_case(key)
if value is not None:
if camel_key not in result.keys():
result[camel_key] = value
else:
raise KeyError(f"Attempted to insert duplicate key '{camel_key}'")
if value is None:
continue
if camel_key in result.keys():
raise KeyError(f"Attempted to insert duplicate key '{camel_key}'")

result[camel_key] = value

return result

def build_url(self, id=None, resource=None, action=None):
"""
Builds the URL to use based on the endpoint path, resource, type, and ID.

:param id: A string representing the ID of an object to use in the URL
:param resource: A string representing the type of resource to append to the URL
:param action: A string representing the type of action to append to the URL
"""

result = f"{self._endpoint_root}/{self._object_type}"
Expand All @@ -60,6 +68,10 @@ def build_url(self, id=None, resource=None, action=None):
def _convert_lower_camel_case(param_name):
"""
Convert a Pythonic variable name to lowerCamelCase.

This function will take an underscored parameter name like 'query_text' and convert it
to lowerCamelCase of 'queryText'. If a parameter with no underscores is provided, it will
assume that the value is already in lowerCamelCase format.
"""

words = param_name.split("_")
Expand Down
7 changes: 3 additions & 4 deletions laceworksdk/api/crud_endpoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,9 @@


class CrudEndpoint(BaseEndpoint):
"""
A class used to implement CRUD create/read/update/delete functionality for Lacework API Endpoints
"""

def __init__(self,
session,
Expand Down Expand Up @@ -63,10 +66,6 @@ def search(self, json=None, **kwargs):
:return response json
"""

# TODO: Remove this on v1.0 release - provided for back compat
if kwargs["query_data"]:
json = kwargs["query_data"]

response = self._session.post(self.build_url(action="search"), json=json)

return response.json()
Expand Down
9 changes: 9 additions & 0 deletions laceworksdk/api/search_endpoint.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@


class SearchEndpoint(BaseEndpoint):
"""
A class used to implement Search functionality for Lacework API Endpoints
"""

# If defined, this is the resource used in the URL path
RESOURCE = ""

def __init__(self,
session,
Expand All @@ -27,6 +33,9 @@ def search(self, json=None, resource=None, **kwargs):
:return a generator which yields a page of objects at a time as returned by the Lacework API.
"""

if not resource and self.RESOURCE:
resource = self.RESOURCE

response = self._session.post(self.build_url(resource=resource, action="search"), json=json)

while True:
Expand Down
137 changes: 44 additions & 93 deletions laceworksdk/api/v2/activities.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,25 @@


class ActivitiesAPI:
alannix-lw marked this conversation as resolved.
Show resolved Hide resolved
"""A class used to represent the Activities API endpoint.

The Activities API endpoint is simply a parent for different types of
activities that can be queried.

Attributes
----------
changed_files:
A ChangedFilesAPI instance.
connections:
A ConnectionsAPI instance.
dns:
A DnsAPI instance.
user_logins:
A UserLoginsAPI instance.
"""

def __init__(self, session):
"""
Initializes the ActivitiesAPI object.
"""Initializes the ActivitiesAPI object.

:param session: An instance of the HttpSession class

Expand All @@ -27,108 +42,44 @@ def __init__(self, session):


class ChangedFilesAPI(SearchEndpoint):
alannix-lw marked this conversation as resolved.
Show resolved Hide resolved
"""A class used to represent the Changed Files API endpoint.

def __init__(self, session, base_path):
"""
Initializes the ChangedFilesAPI object.

:param session: An instance of the HttpSession class

:return ChangedFilesAPI object.
"""

super().__init__(session, base_path)

def search(self,
json=None):
"""
A method to search Changed Files objects.

:param json: A dictionary containing the desired search parameters.
(timeFilter, filters, returns)

:return response json
"""

return super().search(resource="ChangedFiles", json=json)
Methods
-------
search(json=None)
A method to search ChangedFiles objects.
"""
RESOURCE = "ChangedFiles"


class ConnectionsAPI(SearchEndpoint):
"""A class used to represent the Connections API endpoint.

def __init__(self, session, base_path):
"""
Initializes the ConnectionsAPI object.

:param session: An instance of the HttpSession class

:return ConnectionsAPI object.
"""

super().__init__(session, base_path)

def search(self,
json=None):
"""
Methods
-------
search(json=None)
A method to search Connections objects.

:param json: A dictionary containing the desired search parameters.
(timeFilter, filters, returns)

:return response json
"""

return super().search(resource="Connections", json=json)
"""
RESOURCE = "Connections"


class DnsAPI(SearchEndpoint):
"""A class used to represent the DNS Lookup API endpoint.

def __init__(self, session, base_path):
"""
Initializes the DnsAPI object.

:param session: An instance of the HttpSession class

:return DnsAPI object.
"""

super().__init__(session, base_path)

def search(self,
json=None):
"""
Methods
-------
search(json=None)
A method to search DNS lookup objects.

:param json: A dictionary containing the desired search parameters.
(timeFilter, filters, returns)

:return response json
"""

return super().search(resource="DNSs", json=json)
"""
RESOURCE = "DNSs"


class UserLoginsAPI(SearchEndpoint):

def __init__(self, session, base_path):
"""
Initializes the UserLoginsAPI object.

:param session: An instance of the HttpSession class

:return UserLoginsAPI object.
"""

super().__init__(session, base_path)

def search(self,
json=None):
"""
A method to search User Logins objects.

:param json: A dictionary containing the desired search parameters.
(timeFilter, filters, returns)

:return response json
"""

return super().search(resource="UserLogins", json=json)
"""A class used to represent the UserLogins API endpoint.

Methods
-------
search(json=None)
A method to search UserLogins objects.
"""
RESOURCE = "UserLogins"
40 changes: 7 additions & 33 deletions laceworksdk/api/v2/agent_access_tokens.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,20 +41,6 @@ def create(self,
**request_params
)

def get(self,
id=None):
"""
A method to get AgentAccessTokens objects.

:param id: A string representing the object ID.

:return response json
"""

return super().get(
id=id
)

def get_by_id(self,
id):
"""
Expand All @@ -67,24 +53,9 @@ def get_by_id(self,

return self.get(id=id)

def search(self,
json=None,
query_data=None):
"""
A method to search AgentAccessTokens objects.

:param json: A dictionary containing the desired search parameters.
(filters, returns)

:return response json
"""

return super().search(json=json, query_data=query_data)

def update(self,
id,
alias=None,
enabled=None,
token_enabled=None,
**request_params):
"""
A method to update an AgentAccessTokens object.
Expand All @@ -101,10 +72,13 @@ def update(self,

return super().update(
id=id,
token_alias=alias,
token_enabled=int(bool(enabled)),
token_enabled=int(bool(token_enabled)),
**request_params
)

def delete(self):
alannix-lw marked this conversation as resolved.
Show resolved Hide resolved
alannix-lw marked this conversation as resolved.
Show resolved Hide resolved
pass
"""
A method to 'pass' when attempting to delete an AgentAccessToken object.

Lacework does not currently allow for agent access tokens to be deleted.
"""
Loading