From 3a8a71dd1551d23036d023a8153c7ced207a562c Mon Sep 17 00:00:00 2001 From: Danglewood <85772166+deeleeramone@users.noreply.github.com> Date: Wed, 22 May 2024 06:50:43 -0700 Subject: [PATCH] [Docs] Refactor Docs Pages For Diataxis (#6315) * This change creates a PR * start moving stuff around * data_models/_category_.json * reference/_category_.json * moving some stuff around * Introduction * Documentation updates * few broken links, clean up sidebar titles for how-to * codespell * quickstart - new provider extension * add a comment * map provider interface * quickstart for new router extension * [Docs] diataxis refactor refactor (#6425) * Start refactor * Edits * [DOCS] Reorganize and edit. Fix sidebar navigation (#6445) * Reorganize and edit. Fix sidebar navigation * Fix broken linkages * Edit --------- Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com> * Fix links --------- Co-authored-by: Theodore Aptekarev * Fix broken links * Reword CLA part in the Licensing FAQ * change titles and order suggestion - cc @IgorWounds @piiq --------- Co-authored-by: Igor Radovanovic <74266147+IgorWounds@users.noreply.github.com> Co-authored-by: James Maslek Co-authored-by: Theodore Aptekarev Co-authored-by: hjoaquim --- openbb_platform/CONTRIBUTING.md | 4 +- .../obbject_extensions}/charting/examples.md | 3 +- .../obbject_extensions}/charting/index.md | 3 +- .../charting/indicators.md | 0 .../charting/installation.md | 0 .../platform/data_models/_category_.json | 4 + .../platform/developer_guide/_category_.json | 4 + .../architecture_overview.mdx} | 109 +++++- .../developer_guide/command_coverage.mdx | 38 +++ .../commitment_of_traders.mdx} | 50 ++- .../contributing.mdx} | 8 +- .../dependency_management.mdx} | 6 +- .../deprecating_endpoints.mdx} | 4 +- .../documentation_development.mdx | 64 ++++ .../platform/developer_guide/extensions.mdx | 311 ++++++++++++++++++ .../platform/developer_guide/github.mdx | 66 ++++ .../http_requests.mdx} | 23 +- .../platform/developer_guide/index.mdx | 65 ++++ .../obbject.mdx} | 54 ++- .../developer_guide/standardization.mdx | 59 ++++ .../tests.md => developer_guide/tests.mdx} | 27 +- .../validators.mdx} | 81 +++-- .../platform/development/_category_.json | 4 - .../development/contributing/_category_.json | 4 - .../development/contributing/github.md | 43 --- .../content/platform/development/devtools.md | 48 --- .../development/how-to/_category_.json | 4 - website/content/platform/development/index.md | 68 ---- .../platform/development/standardization.md | 151 --------- .../platform/extensions/_category_.json | 5 - .../extensions/charting/_category_.json | 4 - .../platform/extensions/data_extensions.md | 99 ------ website/content/platform/extensions/index.md | 44 --- .../platform/extensions/toolkit_extensions.md | 175 ---------- website/content/platform/faqs/_category_.json | 2 +- .../{data_providers.md => data_providers.mdx} | 21 +- ...platform_vs_sdk.md => platform_vs_sdk.mdx} | 24 +- .../platform/getting_started/_category_.json | 4 + .../api_keys.mdx} | 9 +- .../platform/getting_started/api_requests.mdx | 95 ++++++ .../create_new_provider_extension.mdx | 176 ++++++++++ .../create_new_router_extension.mdx | 216 ++++++++++++ .../economic_indicators.mdx} | 51 ++- .../financial_statements.mdx} | 63 +++- .../find_symbols.mdx} | 32 +- .../historical_prices.mdx} | 97 +++++- .../index.md => getting_started/index.mdx} | 13 +- .../map_a_provider_to_a_route.mdx | 240 ++++++++++++++ .../market_calendars.mdx} | 38 ++- .../platform/getting_started/quickstart.mdx | 76 +++++ website/content/platform/index.md | 50 --- website/content/platform/index.mdx | 66 ++++ .../{installation.md => installation.mdx} | 94 +++--- .../platform/licensing/_category_.json | 2 +- website/content/platform/licensing/index.mdx | 2 +- .../platform/reference/_category_.json | 4 + .../content/platform/usage/_category_.json | 4 - .../content/platform/usage/basic_response.md | 138 -------- .../usage/explanation/_category_.json | 4 - .../platform/usage/explanation/index.md | 35 -- .../platform/usage/guides/_category_.json | 1 - website/content/platform/usage/index.md | 51 --- .../settings_and_environment_variables.md | 105 ------ .../platform/user_guides/_category_.json | 4 + .../add_command_examples.mdx} | 40 +-- .../add_data_provider_extension.mdx} | 31 +- .../add_data_to_existing_endpoint.mdx} | 181 +++++----- .../add_endpoint_to_existing_provider.mdx} | 21 +- .../add_obbject_extension.mdx} | 32 +- .../add_toolkit_extension.mdx} | 124 ++----- .../platform/user_guides/basic_response.mdx | 66 ++++ .../basic_syntax.mdx} | 97 +----- .../user_guides/dynamic_command_execution.mdx | 47 +++ .../content/platform/user_guides/index.mdx | 24 ++ .../content/platform/user_guides/llm_mode.mdx | 90 +++++ .../rest_api.md => user_guides/rest_api.mdx} | 28 +- .../settings_and_environment_variables.mdx | 106 ++++++ 77 files changed, 2630 insertions(+), 1606 deletions(-) rename {website/content/platform/extensions => openbb_platform/obbject_extensions}/charting/examples.md (99%) rename {website/content/platform/extensions => openbb_platform/obbject_extensions}/charting/index.md (99%) rename {website/content/platform/extensions => openbb_platform/obbject_extensions}/charting/indicators.md (100%) rename {website/content/platform/extensions => openbb_platform/obbject_extensions}/charting/installation.md (100%) create mode 100644 website/content/platform/data_models/_category_.json create mode 100644 website/content/platform/developer_guide/_category_.json rename website/content/platform/{development/architecture_overview.md => developer_guide/architecture_overview.mdx} (82%) create mode 100644 website/content/platform/developer_guide/command_coverage.mdx rename website/content/platform/{usage/explanation/commitment_of_traders.md => developer_guide/commitment_of_traders.mdx} (93%) rename website/content/platform/{development/contributing/index.md => developer_guide/contributing.mdx} (90%) rename website/content/platform/{development/dependency_management.md => developer_guide/dependency_management.mdx} (93%) rename website/content/platform/{development/contributing/deprecating_endpoints.md => developer_guide/deprecating_endpoints.mdx} (93%) create mode 100644 website/content/platform/developer_guide/documentation_development.mdx create mode 100644 website/content/platform/developer_guide/extensions.mdx create mode 100644 website/content/platform/developer_guide/github.mdx rename website/content/platform/{development/contributing/http_requests.md => developer_guide/http_requests.mdx} (96%) create mode 100644 website/content/platform/developer_guide/index.mdx rename website/content/platform/{development/obbject.md => developer_guide/obbject.mdx} (79%) create mode 100644 website/content/platform/developer_guide/standardization.mdx rename website/content/platform/{development/tests.md => developer_guide/tests.mdx} (92%) rename website/content/platform/{development/contributing/validators.md => developer_guide/validators.mdx} (62%) delete mode 100644 website/content/platform/development/_category_.json delete mode 100644 website/content/platform/development/contributing/_category_.json delete mode 100644 website/content/platform/development/contributing/github.md delete mode 100644 website/content/platform/development/devtools.md delete mode 100644 website/content/platform/development/how-to/_category_.json delete mode 100644 website/content/platform/development/index.md delete mode 100644 website/content/platform/development/standardization.md delete mode 100644 website/content/platform/extensions/_category_.json delete mode 100644 website/content/platform/extensions/charting/_category_.json delete mode 100644 website/content/platform/extensions/data_extensions.md delete mode 100644 website/content/platform/extensions/index.md delete mode 100644 website/content/platform/extensions/toolkit_extensions.md rename website/content/platform/faqs/{data_providers.md => data_providers.mdx} (75%) rename website/content/platform/faqs/{platform_vs_sdk.md => platform_vs_sdk.mdx} (74%) create mode 100644 website/content/platform/getting_started/_category_.json rename website/content/platform/{usage/api_keys.md => getting_started/api_keys.mdx} (96%) create mode 100644 website/content/platform/getting_started/api_requests.mdx create mode 100644 website/content/platform/getting_started/create_new_provider_extension.mdx create mode 100644 website/content/platform/getting_started/create_new_router_extension.mdx rename website/content/platform/{usage/economic_indicators.md => getting_started/economic_indicators.mdx} (97%) rename website/content/platform/{usage/explanation/financial_statements.md => getting_started/financial_statements.mdx} (85%) rename website/content/platform/{usage/explanation/find_symbols.md => getting_started/find_symbols.mdx} (98%) rename website/content/platform/{usage/explanation/historical_prices.md => getting_started/historical_prices.mdx} (87%) rename website/content/platform/{development/how-to/index.md => getting_started/index.mdx} (55%) create mode 100644 website/content/platform/getting_started/map_a_provider_to_a_route.mdx rename website/content/platform/{usage/explanation/market_calendars.md => getting_started/market_calendars.mdx} (94%) create mode 100644 website/content/platform/getting_started/quickstart.mdx delete mode 100644 website/content/platform/index.md create mode 100644 website/content/platform/index.mdx rename website/content/platform/{installation.md => installation.mdx} (78%) create mode 100644 website/content/platform/reference/_category_.json delete mode 100644 website/content/platform/usage/_category_.json delete mode 100644 website/content/platform/usage/basic_response.md delete mode 100644 website/content/platform/usage/explanation/_category_.json delete mode 100644 website/content/platform/usage/explanation/index.md delete mode 100644 website/content/platform/usage/guides/_category_.json delete mode 100644 website/content/platform/usage/index.md delete mode 100644 website/content/platform/usage/settings_and_environment_variables.md create mode 100644 website/content/platform/user_guides/_category_.json rename website/content/platform/{development/contributing/function_examples.md => user_guides/add_command_examples.mdx} (75%) rename website/content/platform/{development/how-to/add_data_provider_extension.md => user_guides/add_data_provider_extension.mdx} (94%) rename website/content/platform/{development/how-to/add_data_to_existing_endpoint.md => user_guides/add_data_to_existing_endpoint.mdx} (85%) rename website/content/platform/{development/how-to/add_endpoint_to_existing_provider.md => user_guides/add_endpoint_to_existing_provider.mdx} (98%) rename website/content/platform/{development/how-to/add_obbject_extension.md => user_guides/add_obbject_extension.mdx} (83%) rename website/content/platform/{development/how-to/add_toolkit_extension.md => user_guides/add_toolkit_extension.mdx} (55%) create mode 100644 website/content/platform/user_guides/basic_response.mdx rename website/content/platform/{usage/basic_syntax.md => user_guides/basic_syntax.mdx} (63%) create mode 100644 website/content/platform/user_guides/dynamic_command_execution.mdx create mode 100644 website/content/platform/user_guides/index.mdx create mode 100644 website/content/platform/user_guides/llm_mode.mdx rename website/content/platform/{usage/rest_api.md => user_guides/rest_api.mdx} (94%) create mode 100644 website/content/platform/user_guides/settings_and_environment_variables.mdx diff --git a/openbb_platform/CONTRIBUTING.md b/openbb_platform/CONTRIBUTING.md index 62a2806e600d..1f041d103e92 100644 --- a/openbb_platform/CONTRIBUTING.md +++ b/openbb_platform/CONTRIBUTING.md @@ -64,7 +64,7 @@ This document provides guidelines for contributing to the OpenBB Platform. Throughout this document, we will be differentiating between two types of contributors: Developers and Contributors. 1. **Developers**: Those who are building new features or extensions for the OpenBB Platform or leveraging the OpenBB Platform. -2. **Contributors**: Those who contribute to the existing codebase, by opening a [Pull Request](#how-to-create-a-pr) thus giving back to the community. +2. **Contributors**: Those who contribute to the existing codebase, by opening a [Pull Request](#getting_started-create-a-pr) thus giving back to the community. **Why is this distinction important?** @@ -676,7 +676,7 @@ When using the OpenBB Platform on a API Interface, the types are a bit more limi The Contributor Guidelines are intended to be a continuation of the [Developer Guidelines](#developer-guidelines). They are not a replacement, but rather an expansion, focusing specifically on those who seek to directly enhance the OpenBB Platform's codebase. It's crucial for Contributors to be familiar with both sets of guidelines to ensure a harmonious and productive engagement with the OpenBB Platform. -There are many ways to contribute to the OpenBB Platform. You can add a [new data point](#how-to-add-a-new-data-point), add a [new command](#openbb-platform-commands), add a [new visualization](/openbb_platform/extensions/charting/README.md), add a [new extension](#how-to-build-openbb-extensions), fix a bug, improve or create documentation, etc. +There are many ways to contribute to the OpenBB Platform. You can add a [new data point](#getting_started-add-a-new-data-point), add a [new command](#openbb-platform-commands), add a [new visualization](/openbb_platform/extensions/charting/README.md), add a [new extension](#getting_started-build-openbb-extensions), fix a bug, improve or create documentation, etc. ### Expectations for Contributors diff --git a/website/content/platform/extensions/charting/examples.md b/openbb_platform/obbject_extensions/charting/examples.md similarity index 99% rename from website/content/platform/extensions/charting/examples.md rename to openbb_platform/obbject_extensions/charting/examples.md index 87b9918ceeda..34a57a8d0c4f 100644 --- a/website/content/platform/extensions/charting/examples.md +++ b/openbb_platform/obbject_extensions/charting/examples.md @@ -44,7 +44,7 @@ By default, more than three symbols will draw the chart as cumulative returns fr The tickers below are a collection of State Street Global Advisors SPDR funds, representing S&P 500 components. The data is looking back five years. -``` +```python SPDRS = [ "SPY", "XLE", @@ -162,4 +162,5 @@ create_bar_chart( title="S&P Energy Sector YTD Turnover Rate", ) ``` + ![S&P 500 Energy Sector Turnover Rate](https://github.com/OpenBB-finance/OpenBBTerminal/assets/85772166/4f59eb36-e637-4a54-87ab-e730e43baf8d) diff --git a/website/content/platform/extensions/charting/index.md b/openbb_platform/obbject_extensions/charting/index.md similarity index 99% rename from website/content/platform/extensions/charting/index.md rename to openbb_platform/obbject_extensions/charting/index.md index f55bbe6a8fed..75f6ff107619 100644 --- a/website/content/platform/extensions/charting/index.md +++ b/openbb_platform/obbject_extensions/charting/index.md @@ -339,11 +339,12 @@ The `openbb-charting` extension is equipped with interactive tables, utilizing t data = obb.equity.price.quote("AAPL,MSFT,GOOGL,META,TSLA,AMZN", provider="yfinance") data.charting.table() ``` + ![Interactive Tables](https://github.com/OpenBB-finance/OpenBBTerminal/assets/85772166/77f5f812-b933-4ced-929c-c1e39b2a3eed) External data can also be supplied, providing an opportunity to filter or apply Pandas operations before display. -``` +```python new_df = df.to_df().T new_df.index.name="metric" new_df.columns = new_df.loc["symbol"] diff --git a/website/content/platform/extensions/charting/indicators.md b/openbb_platform/obbject_extensions/charting/indicators.md similarity index 100% rename from website/content/platform/extensions/charting/indicators.md rename to openbb_platform/obbject_extensions/charting/indicators.md diff --git a/website/content/platform/extensions/charting/installation.md b/openbb_platform/obbject_extensions/charting/installation.md similarity index 100% rename from website/content/platform/extensions/charting/installation.md rename to openbb_platform/obbject_extensions/charting/installation.md diff --git a/website/content/platform/data_models/_category_.json b/website/content/platform/data_models/_category_.json new file mode 100644 index 000000000000..75b4946489d2 --- /dev/null +++ b/website/content/platform/data_models/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Data Models", + "position": 5 +} diff --git a/website/content/platform/developer_guide/_category_.json b/website/content/platform/developer_guide/_category_.json new file mode 100644 index 000000000000..e4606db18143 --- /dev/null +++ b/website/content/platform/developer_guide/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Developer Guide", + "position": 3 + } diff --git a/website/content/platform/development/architecture_overview.md b/website/content/platform/developer_guide/architecture_overview.mdx similarity index 82% rename from website/content/platform/development/architecture_overview.md rename to website/content/platform/developer_guide/architecture_overview.mdx index c17ff7833461..5dedccf93629 100644 --- a/website/content/platform/development/architecture_overview.md +++ b/website/content/platform/developer_guide/architecture_overview.mdx @@ -1,6 +1,6 @@ --- title: Architecture Overview -sidebar_position: 2 +sidebar_position: 1 description: This guide provides insights into the architecture and components of the OpenBB Platform. It covers the key classes, import statements, and the TET pattern used in building the Fetcher classes. keywords: - OpenBB Platform Architecture @@ -30,13 +30,13 @@ The OpenBB Platform core relies on a set of carefully selected Python libraries. - Pandas for data manipulation and analysis. - Pydantic for data validation and serialization using Python type annotations. - Requests/AIOHTTP for making HTTP requests. -- Websockets for handling WebSocket connections. +- WebSockets for handling WebSocket connections. These dependencies are specified in the `pyproject.toml` files. ### Importance Of A Lean Core -Keeping the OpenBB Platform core as lean as possible is crucial for maintaining the platform's performance, ease of use, and flexibility. A lean core means faster installation times, less memory usage, and overall better performance. It also reduces the risk of conflicts between dependencies and makes the platform easier to maintain and update. +Keeping the OpenBB Platform core as lean as possible is crucial for maintaining the its performance, ease of use, and flexibility. A lean core means faster installation times, less memory usage, and overall better performance. It also reduces the risk of conflicts between dependencies and makes the platform easier to maintain and update. Moreover, a lean core allows for greater flexibility. Users of the platform can add additional functionality through extensions without being burdened by unnecessary core dependencies. This makes the OpenBB Platform adaptable to a wide range of use cases and requirements. @@ -77,15 +77,67 @@ When using the OpenBB Platform via an API Interface, the types are a bit more co ] ``` -## `QueryParams` Class +## QueryParams Class The QueryParams class is a standardized model for handling query input parameters in the OpenBB platform. It extends the BaseModel from the Pydantic library, which provides runtime data validation and serialization. -The class includes a dictionary, `__alias_dict__`, which can be used to map the original parameter names to aliases. This can be useful when dealing with different data providers that may use different naming conventions for similar parameters. +Below is the [EquityHistorical](https://docs.openbb.co/platform/data_models/EquityHistorical) standard model. -The `__json_schema_extra__` dictionary can be used to define whether multiple items are accepted by a parameter. +```python +"""Equity Historical Price Standard Model.""" + +from datetime import ( + date as dateType, + datetime, +) +from typing import Optional, Union +from dateutil import parser +from pydantic import Field, field_validator + +from openbb_core.provider.abstract.data import Data +from openbb_core.provider.abstract.query_params import QueryParams +from openbb_core.provider.utils.descriptions import ( + DATA_DESCRIPTIONS, + QUERY_DESCRIPTIONS, +) + +class EquityHistoricalQueryParams(QueryParams): + """Equity Historical Price Query.""" + + symbol: str = Field(description=QUERY_DESCRIPTIONS.get("symbol", "")) + interval: Optional[str] = Field( + default="1d", + description=QUERY_DESCRIPTIONS.get("interval", ""), + ) + start_date: Optional[dateType] = Field( + default=None, + description=QUERY_DESCRIPTIONS.get("start_date", ""), + ) + end_date: Optional[dateType] = Field( + default=None, + description=QUERY_DESCRIPTIONS.get("end_date", ""), + ) + + @field_validator("symbol", mode="before", check_fields=False) + @classmethod + def to_upper(cls, v: str) -> str: + """Convert field to uppercase.""" + return v.upper() ``` + +:::info +Note that not all possible parameters are defined here, and can be further refined in the provider-specific model. + +For example, `interval` is defined as a string in the standard model, but is defined again as a `Literal` with explicit choices specific to that source. Here, it would not be possible to define all valid choices in a way that is compatible with all providers. +::: + + +The class can have a dictionary, `__alias_dict__`, which can be used to map the original parameter names to aliases. This can be useful when dealing with different data providers that may use different naming conventions for similar parameters. + +The `__json_schema_extra__` dictionary can be used to define whether multiple items are accepted by a parameter. + +``` python __json_schema_extra__ = { "symbol": ["multiple_items_allowed"], "category": ["multiple_items_allowed"], @@ -98,8 +150,7 @@ The `model_config` attribute is a `ConfigDict` instance that allows extra fields The `model_dump` method is used to serialize the model into a dictionary. If the `__alias_dict__` is not empty, it will use the aliases defined in it for the keys in the returned dictionary. If the `__alias_dict__` is empty, it will return the original serialized model. - -## `Data` Class +## Data Class The OpenBB Standardized Data Model. @@ -108,7 +159,9 @@ for OpenBB's data processing pipeline as it's structured to support dynamic fiel The model leverages Pydantic's powerful validation features to ensure data integrity while providing the flexibility to handle extra fields that are not explicitly defined in the model's -schema. This makes the `Data` class ideal for working with datasets that may have varying +schema. + +This makes the `Data` class ideal for working with datasets that may have varying structures or come from heterogeneous sources. Key Features: @@ -141,7 +194,35 @@ The class is highly extensible and can be subclassed to create more specific mod particular datasets or domains, while still benefiting from the base functionality provided by the `Data` class. -## `Fetcher` Class +The `Data` model for the `EquityHistorical` standard model is shown below. + +```python +class EquityHistoricalData(Data): + """Equity Historical Price Data.""" + + date: Union[dateType, datetime] = Field( + description=DATA_DESCRIPTIONS.get("date", "") + ) + open: float = Field(description=DATA_DESCRIPTIONS.get("open", "")) + high: float = Field(description=DATA_DESCRIPTIONS.get("high", "")) + low: float = Field(description=DATA_DESCRIPTIONS.get("low", "")) + close: float = Field(description=DATA_DESCRIPTIONS.get("close", "")) + volume: Optional[Union[float, int]] = Field( + default=None, description=DATA_DESCRIPTIONS.get("volume", "") + ) + vwap: Optional[float] = Field( + default=None, description=DATA_DESCRIPTIONS.get("vwap", "") + ) + + @field_validator("date", mode="before", check_fields=False) + def date_validate(cls, v): + """Return formatted datetime.""" + if ":" in str(v): + return parser.isoparse(str(v)) + return parser.parse(str(v)).date() +``` + +## Fetcher Class The `Fetcher` class is an abstract base class designed to provide a structured way to fetch data from various providers. It uses generics to allow for flexibility in the types of queries, data, and return values it handles. @@ -164,7 +245,7 @@ The `Fetcher` class implementation is based on the [TET pattern](/platform/devel ::: -## `OBBject` Class +## OBBject Class The OBBject class is a generic class in the OpenBB platform that represents a standardized object for handling and manipulating data fetched from various providers. It extends the `Tagged` class and uses Python's generics to allow flexibility in the type of results it can handle. @@ -174,7 +255,7 @@ The class provides several methods for converting the fetched data into differen The class also includes a `__repr__` method for a human-readable representation of the object, as well as the familiar Pydantic BaseModel methods like `model_dump()` and `model_json_schema()`. -## `Router` Class +## Router Class The `Router` class in the OpenBB platform is responsible for managing and routing API requests. It uses the `APIRouter` from the FastAPI library to handle routing. @@ -272,7 +353,9 @@ AVEquityHistoricalData(date=2023-11-03 00:00:00, open=174.24, high=176.82, low=1 > The `AVEquityHistoricalData` class, is a child class of the `Data` class. -Note how we've indexed to get only the first element of the `results` list (which represents a single row, if we want to think about it as a tabular output). This simply means that we are getting a `List` of `AVEquityHistoricalData` from the `obb.equity.price.historical` command. Or, we can also say that that's equivalent to `List[Data]`! +Note how we've indexed to get only the first element of the `results` list (which represents a single row, if we want to think about it as a tabular output). + +This simply means that we are getting a `List` of `AVEquityHistoricalData` from the `obb.equity.price.historical` command. Or, we can also say that that's equivalent to `List[Data]`! This is very powerful, as we can now apply any data processing command to the `results` list, without worrying about the underlying data structure. diff --git a/website/content/platform/developer_guide/command_coverage.mdx b/website/content/platform/developer_guide/command_coverage.mdx new file mode 100644 index 000000000000..2138fa75c56b --- /dev/null +++ b/website/content/platform/developer_guide/command_coverage.mdx @@ -0,0 +1,38 @@ +--- +title: Command Coverage +sidebar_position: 6 +description: This page details the instructions for determining the command coverage provided by the installed extensions. +keywords: +- tutorial +- standardized output +- OBBject +- provider +- results +- warnings +- chart +- extra +- command coverage +--- + +## Commands and Provider Coverage + +The installed commands and data providers are found under, `obb.coverage`. + +```python +obb.coverage +``` + +```console +/coverage + + providers + commands + command_model + command_schemas +``` + +`obb.coverage.providers` is a dictionary of the installed provider extensions, each with its own list of available commands. + +`obb.coverage.commands` is a dictionary of commands, each with its own list of available providers for the data. + +`obb.coverage.command_model` is a dictionary where the keys are the command paths and the values is a nested dictionary of QueryParams and Data models associated with that function. diff --git a/website/content/platform/usage/explanation/commitment_of_traders.md b/website/content/platform/developer_guide/commitment_of_traders.mdx similarity index 93% rename from website/content/platform/usage/explanation/commitment_of_traders.md rename to website/content/platform/developer_guide/commitment_of_traders.mdx index 085b0207b5bd..5268b3131bbe 100644 --- a/website/content/platform/usage/explanation/commitment_of_traders.md +++ b/website/content/platform/developer_guide/commitment_of_traders.mdx @@ -1,9 +1,7 @@ --- title: Commitment of Traders -sidebar_position: 7 -description: This page provides details on the accessing Commitment of Traders reports with - the OpenBB Platform, published by the CFTC weekly. This guide provides examples for - using the combinations of parameters used to get aspects of the report's data. +sidebar_position: 22 +description: This page provides details on the accessing Commitment of Traders reports with the OpenBB Platform, published by the CFTC weekly. This guide provides examples for using the combinations of parameters used to get aspects of the report's data. keywords: - futures - commodities @@ -38,7 +36,8 @@ The `obb.regulators` module contains data published by industry regulators and a While the data is public and available directly from the CFTC website, [Nasdaq Data Link](https://data.nasdaq.com/databases/CFTC) provides access to a parsed database with the complete history, of current and obsolete reports, that the `openbb-nasdaq` data provider connects to. -### COT Search +
+COT Search The `obb.regulators.cftc.cot_search()` end point is a curated list of the 100 most common reports. The list can be searched by fuzzy query - i.e., "commodities" - and they are classified under categories and subcategories. Get the whole list with an empty query. @@ -52,7 +51,10 @@ The major US indices - S&P 500, Nasdaq 100, Dow Jones Industrial Average, Russel reports[reports["category"] == "Index"] ``` -### COT Reports +
+ +
+COT Reports There is a default report, Two-Year Treasury Note Futures, which has the code: `042601`. Where available, like the two-year note, the futures continuation symbol (ZT=F) can be used instead of the code. The name can also be used: @@ -79,7 +81,10 @@ zt.iloc[-1] Look up reports not listed under `obb.regulators.cftc.cot_search()` by using the Nasdaq Data Link code for the series. Refer to their documentation for a complete list. ::: -### Parameters +
+ +
+Parameters There are parameters that will alter the type of report returned. @@ -87,7 +92,8 @@ There are parameters that will alter the type of report returned. Not every combination of parameters is valid for all reports. An error will be raised when parameters are invalid. ::: -#### `data_type` +
+`data_type` The `data_type` changes what is returned, futures or futures and options. @@ -126,7 +132,10 @@ zc_cits.iloc[-1] | positions_long_cit | 353355 | | positions_short_cit | 123216 | -#### `legacy_format` +
+ +
+`legacy_format` When `True`, reports are broken down by exchange. These reports have a futures only report and a combined futures and options report. Legacy reports break down the reportable open interest positions into two classifications: non-commercial and commercial traders. @@ -156,7 +165,10 @@ legacy_zt.iloc[-1] | non_reportable_longs | 267944 | | non_reportable_shorts | 136298 | -#### `report_type` +
+ +
+`report_type` The `report_type` parameter has four choices: @@ -169,7 +181,10 @@ For selected commodities where there is a well-defined marketing season or crop For the "old" and "other" figures, spreading is calculated for equal long and short positions within a crop year. If a non-commercial trader holds a long position in an "old" crop-year future and an equal short position in an "other" crop-year future, the long position will be classified as "long-only" in the "old" crop year and the short position will be classified as "short-only" in the "other" crop year. In this example, in the "all" category, which considers each trader's positions without regard to crop year, that trader's positions will be classified as "spreading." For this reason, summing the "old" and "other" figures for long-only, for short-only, or for spreading will not necessarily equal the corresponding figure shown for "all" futures. Any differences result from traders that spread from an "old" crop-year future to an "other" crop-year future. -#### Major Markets for Which the COT Data Is Shown by Crop Year +
+ +
+Major Market - COT by Crop Year |Market| Commodity | First Future| Last Future| |:-------|:---------:|:-----:|-----:| @@ -189,7 +204,10 @@ For the "old" and "other" figures, spreading is calculated for equal long and sh |NYBT| Cotton No.2 | October | July| |NYBT| Frozen Conc Orange Juice | January | November| -#### `measure` +
+ +
+`measure` The `measure` parameter has four choices, with `None` as the default state. @@ -204,7 +222,10 @@ Based on the information contained in the report of futures-and-options combined All of these traders—whether coming from the noncommercial or commercial categories—are generally replicating a commodity index by establishing long futures positions in the component markets and then rolling those positions forward from future to future using a fixed methodology. Some traders assigned to the Index Traders category are engaged in other futures activity that could not be disaggregated. As a result, the Index Traders category, which is typically made up of traders with long-only futures positions replicating an index, will include some long and short positions where traders have multi-dimensional trading activities, the preponderance of which is index trading. Likewise, the Index Traders category will not include some traders who are engaged in index trading, but for whom it does not represent a substantial part of their overall trading activity. -#### `transform` +
+ +
+`transform` The `transform` parameter modifies the requested report to show the values as the week-over-week difference, rate of change, cumulative, or normalized. These are standard parameters for all Nasdaq Data Link queries. @@ -216,3 +237,6 @@ The `transform` parameter modifies the requested report to show the values as th :::info Explanations in this page are from the explanatory notes on the CFTC's [website](https://www.cftc.gov/MarketReports/CommitmentsofTraders/ExplanatoryNotes/index.htm) ::: + +
+
diff --git a/website/content/platform/development/contributing/index.md b/website/content/platform/developer_guide/contributing.mdx similarity index 90% rename from website/content/platform/development/contributing/index.md rename to website/content/platform/developer_guide/contributing.mdx index d1087cbbb7d2..2396d6f36af7 100644 --- a/website/content/platform/development/contributing/index.md +++ b/website/content/platform/developer_guide/contributing.mdx @@ -1,6 +1,6 @@ --- -title: Expectations For Contributors -sidebar_position: 1 +title: Contributing +sidebar_position: 6 description: This guide outlines the expectations for contributors to the OpenBB Platform. It covers aspects such as use cases, documentation, code quality, testing, performance, and collaboration. Whether you're enhancing functionality, building extensions, or contributing code. keywords: - OpenBB Platform @@ -19,7 +19,7 @@ keywords: import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - + Thank you for taking the time to engage as a contributor. There is no contribution that is too small. whether it is a spelling error in the documentation, or contributing code and participating in the @@ -34,7 +34,7 @@ The list below is intended to provide some guidance on what the general expectat 2. Documentation: - All code contributions should come with relevant documentation, including the purpose of the contribution, how it works, and any changes it makes to existing functionalities. - Update any existing documentation if your contribution alters the behavior of the OpenBB Platform. - - New router functions must have usage [examples](contributing/function_examples) defined. + - New router functions must have usage [examples](/platform/user_guides/add_command_examples) defined. 3. Code Quality: - Your code should adhere strictly to the OpenBB Platform's coding standards and [conventions](architecture_overview). diff --git a/website/content/platform/development/dependency_management.md b/website/content/platform/developer_guide/dependency_management.mdx similarity index 93% rename from website/content/platform/development/dependency_management.md rename to website/content/platform/developer_guide/dependency_management.mdx index 508ae81fd651..b606718d6a47 100644 --- a/website/content/platform/development/dependency_management.md +++ b/website/content/platform/developer_guide/dependency_management.mdx @@ -1,9 +1,7 @@ --- title: Dependency Management -sidebar_position: 9 -description: Dealing with dependencies when developing with the OpenBB Platform. Learn - how to add new dependencies to the OpenBB Platform and how to add new dependencies - to your custom extension. +sidebar_position: 5 +description: Dealing with dependencies when developing with the OpenBB Platform. Learn how to add new dependencies to the OpenBB Platform and how to add new dependencies to your custom extension. keywords: - OpenBB Platform - Open source diff --git a/website/content/platform/development/contributing/deprecating_endpoints.md b/website/content/platform/developer_guide/deprecating_endpoints.mdx similarity index 93% rename from website/content/platform/development/contributing/deprecating_endpoints.md rename to website/content/platform/developer_guide/deprecating_endpoints.mdx index c0f1a797f23f..cda58a8de4c5 100644 --- a/website/content/platform/development/contributing/deprecating_endpoints.md +++ b/website/content/platform/developer_guide/deprecating_endpoints.mdx @@ -1,6 +1,6 @@ --- title: Deprecating Endpoints -sidebar_position: 4 +sidebar_position: 11 description: This guide provides detailed instructions on how to deprecate an endpoint in the OpenBB Platform. keywords: - OpenBB community @@ -14,7 +14,7 @@ keywords: import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - + Deprecating commands is essential to maintaining the OpenBB Platform. This guide outlines the process for deprecating an endpoint. diff --git a/website/content/platform/developer_guide/documentation_development.mdx b/website/content/platform/developer_guide/documentation_development.mdx new file mode 100644 index 000000000000..16d91de5240a --- /dev/null +++ b/website/content/platform/developer_guide/documentation_development.mdx @@ -0,0 +1,64 @@ +--- +title: Documentation - Development +sidebar_position: 30 +description: Building the OpenBB Platform documentation and packages. +keywords: +- OpenBB Platform +- Open source +- Documentation +- Development +- Markdown +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + + +The documentation and packages are kept in the `/website` folder, at the base of the repository. Navigate there to install the dependencies and start the development server. + +### Generate Markdown Files + +Markdown files for the Reference and Data Models pages need to be generated. +This will generate content based on what is actually installed and contained locally, so it may appear different than what is on this website. + +Open a terminal command line to the `/OpenBBTerminal/website` folder, then enter: + +```bash +python generate_platform_v4_markdown.py +``` + +### Node.js + +- [Node.js](https://nodejs.org/en/) >= 16.13.0 + To check if Node.js installed, run this command: + +```bash +node --version # should be v16.13.0 or higher +``` + +#### Install Dependencies + +```bash +npm install +``` + +#### Start Development Server + +```bash +npm start +``` + +This starts a local development server at: [http://localhost:3000](http://localhost:3000) + +Most changes are reflected live without having to restart the server. + +#### Build + +```bash +npm run build +``` + +This command generates static content into the `build` directory and can be served using any static contents hosting service. + +OpenBB uses Github Pages to host our website, it's deployed in the `gh-pages` branch. diff --git a/website/content/platform/developer_guide/extensions.mdx b/website/content/platform/developer_guide/extensions.mdx new file mode 100644 index 000000000000..28acc772175e --- /dev/null +++ b/website/content/platform/developer_guide/extensions.mdx @@ -0,0 +1,311 @@ +--- +title: Extensions +sidebar_position: 3 +description: This page describes the toolkit extensions available for the OpenBB Platform. +keywords: +- OpenBB Platform +- Python client +- Fast API +- getting started +- extensions +- data providers +- data extensions +- toolkit extensions +- toolkits +- endpoints +- community +- technical analysis +- quantitative analysis +- charting libraries +- Plotly +- OpenBBFigure +- PyWry +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + +The extension framework allows individual pieces to be installed and removed seamlessly within the environment, using only the desired data and toolkit extensions. + +There are two primary types of extensions for the OpenBB Platform: + +- Data +- Toolkits + +The OpenBB Core installation does not include any toolkit extensions. Install the OpenBB Platform with all data and toolkit extensions from PyPI with: + +```python +pip install openbb[all] +``` + +When installing from source, navigate into the `openbb_platform` folder from the root of the project and enter: + +```console +python dev_install.py -e +``` + +This installs all extensions in editable mode, and the Python interface is compiled in, `/openbb_platform/openbb/package`, instead of the environment's `site-packages` folder. The tables in the next pages lists extensions as either, Core or Community. The Core extensions are installed by default. + +A couple of notable differences between data and toolkit extension are: + +- In the OpenBB GitHub repo, extensions are all located under: + + ```console + ~/OpenBBTerminal/openbb_platform/extensions + ``` + +- An additional folder housing integration tests, with the `tests` folder staying empty. +- There is a `router` file, and there can be sub-folders with additional routers. +- Utility functions don't need their own sub-folder. +- `__init__.py` files are all empty. + +## Data Extensions + +
+ + +Data extensions will expand the breadth and coverage of the data available in the OpenBB Platform. Each source (provider) is its own independent extension, even if there is only one endpoint accessible. This allows every data source to be inserted or removed, at any time, without disturbing the operation of the Core components. + +Functions will appear in the Python Interface and Fast API only if a supported provider, for that specific endpoint, is installed. Additional Python libraries will be installed, where required, by the extension. + +## Provider Coverage + +The total installed coverage can be determined through the Python interface, as a dictionary. + +```python +from openbb import obb +obb.coverage.providers +``` + +## Installation + +All data extensions are installed with similar syntax. Published data extensions will have names beginning with `openbb`. For example, yFinance. + +```console +pip install openbb-yfinance +``` + +Additions and removals update the router automatically to reflect the changes when the Python interpreter is refreshed. Below is a list of data provider extensions. + +Uninstall any extension with `pip uninstall`. + +```console +pip uninstall openbb-yfinance +``` + +## Available Data Extensions + +### Core `openbb` Providers + +These packages are what will be installed when `pip install openbb` is run + +| Extension Name | Description | Installation Command | Minimum Subscription Type Required | +|----------------|-------------|----------------------|------------------------------------| +| openbb-benzinga | [Benzinga](https://www.benzinga.com/apis/en-ca/) data connector | pip install openbb-benzinga | Paid | +| openbb-fmp | [FMP](https://site.financialmodelingprep.com/developer/) data connector | pip install openbb-fmp | Free | +| openbb-fred | [FRED](https://fred.stlouisfed.org/) data connector | pip install openbb-fred | Free | +| openbb-intrinio | [Intrinio](https://intrinio.com/pricing) data connector | pip install openbb-intrinio | Paid | +| openbb-oecd | [OECD](https://data.oecd.org/) data connector | pip install openbb-oecd | Free | +| openbb-polygon | [Polygon](https://polygon.io/) data connector | pip install openbb-polygon | Free | +| openbb-sec | [SEC](https://www.sec.gov/edgar/sec-api-documentation) data connector | pip install openbb-sec | None | +| openbb-tiingo | [Tiingo](https://www.tiingo.com/about/pricing) data connector | pip install openbb-tiingo | Free | +| openbb-tradingeconomics | [TradingEconomics](https://tradingeconomics.com/api) data connector | pip install openbb-tradingeconomics | Paid | +| openbb-yfinance | [Yahoo Finance](https://finance.yahoo.com/) data connector | pip install openbb-yfinance | None | + +### Community Providers + +These packages are not installed when `pip install openbb` is run. They are available for installation separately or by running `pip install openbb[all]` + +| Extension Name | Description | Installation Command | Minimum Subscription Type Required | +|----------------|-------------|----------------------|------------------------------------| +| openbb-alpha-vantage | [Alpha Vantage](https://www.alphavantage.co/) data connector | pip install openbb-alpha-vantage | Free | +| openbb-biztoc | [Biztoc](https://api.biztoc.com/#biztoc-default) News data connector | pip install openbb-biztoc | Free | +| openbb-cboe | [Cboe](https://www.cboe.com/delayed_quotes/) data connector | pip install openbb-cboe | None | +| openbb-ecb | [ECB](https://data.ecb.europa.eu/) data connector | pip install openbb-ecb | None | +| openbb-federal-reserve | [Federal Reserve](https://www.federalreserve.gov/) data connector | pip install openbb-federal-reserve | None | +| openbb-finra | [FINRA](https://www.finra.org/finra-data) data connector | pip install openbb-finra | None / Free | +| openbb-finviz | [Finviz](https://finviz.com) data connector | pip install openbb-finviz | None | +| openbb-government-us | [US Government](https://data.gov) data connector | pip install openbb-us-government | None | +| openbb-nasdaq | [Nasdaq Data Link](https://data.nasdaq.com/) connector | pip install openbb-nasdaq | None / Free | +| openbb-seeking-alpha | [Seeking Alpha](https://seekingalpha.com/) data connector | pip install openbb-seeking-alpha | None | +| openbb-stockgrid | [Stockgrid](https://stockgrid.io) data connector | pip install openbb-stockgrid | None | +| openbb-tmx | [TMX](https://money.tmx.com) data connector | pip install openbb-tmx | None | +| openbb-tradier | [Tradier](https://tradier.com) data connector | pip install openbb-tradier | None | +| openbb-wsj | [Wall Street Journal](https://www.wsj.com/) data connector | pip install openbb-wsj | None | + +Have you published a data provider extension and want it featured on this list? Tell us about it! Open a pull request on [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal/) to submit an extension for inclusion. Code contributions, for new and existing, data providers are always welcome. + +Search [PyPI](https://pypi.org/search/?q=openbb-) to find more extensions. + + +
+ +## Toolkit Extensions +
+ +OpenBB Toolkit Extensions expand the Platform with functions for manipulating data and preparing it for display. The Core Platform installation does not install any toolkit extensions. The table below is the current list of toolkit extensions. + +| Extension Name | Description | Installation Command | Core/Community | Router Path | +|:-----------------|:-----------:|:-------------------:|:------------------:|-------------:| +| openbb-charting | Rest API charting service and Plotly library. | pip install openbb-charting | Community | N/A | +| openbb-devtools | Aggregates dependencies that facilitate a nice development experience for OpenBB. | pip install openbb-devtools | N/A | +| openbb-econometrics | Econometrics models for the Python interface only. | pip install openbb-econometrics | Community | obb.econometrics | +| openbb-quantitative | Functions for performing quantitative analysis. | pip install openbb-quantitative | Community | obb.quantitative | +| openbb-technical | Functions for performing technical analysis. | pip install openbb-technical | Community | obb.technical | + +The sections below outline any specific installation considerations for the extension. + +## Charting + +The OpenBB Charting Extension supplies charting infrastructure and services to the OpenBB Platform. Figure objects are served via REST API or Python Client. It utilizes [PyWry](https://github.com/OpenBB-finance/pywry) for handling the display of interactive charts and tables in a separate window, with a Plotly library. + +Functions with dedicated views return figures to the `chart` attribute of the `OBBject` response object. They are displayed with the class method, `show()`. + +:::tip +The `openbb-charting` is an [`OBBject` extension](platform/getting_started/add_obbject_extension), which means the general functionality is exposed in every command result. +::: + +The following packages are dependencies of the `openbb-charting` extension: + +- scipy +- plotly +- statsmodels +- reportlab +- pywry +- svglib +- nbformat +- pandas-ta + +### Installation + +Install from PyPI with: + +```console +pip install openbb-charting +``` + +For more information check the documentation of the openbb-charting extension. + +## Devtools + +Please refer to the following PyPI distributed [package](https://pypi.org/project/openbb-devtools/). + +This Python package, `openbb-devtools`, is designed for OpenBB Platform Developers and contains a range of dependencies essential for robust and efficient software development. + +These dependencies cater to various aspects like code formatting, security analysis, type checking, testing, and kernel management. + +The inclusion of these packages ensures that the development process is streamlined, the code quality is maintained, and the software is secure and reliable. + +Included dependencies: + +- `ruff`: A fast Python linter focused on performance and simplicity. +- `pylint`: A tool that checks for errors in Python code, enforces a coding standard, and looks for code smells. +- `mypy`: A static type checker for Python, helping catch type errors during development. +- `pydocstyle`: A linter for Python docstrings to ensure they meet certain style requirements. +- `black`: An uncompromising Python code formatter, ensuring consistent code style. +- `bandit`: A tool designed to find common security issues in Python code. +- `pre-commit`: Manages and maintains pre-commit hooks that run checks before each commit, ensuring code quality. +- `nox`: A generic virtualenv management and test command line tool for running tests in isolated environments. +- `pytest`: A mature full-featured Python testing tool that helps in writing better programs. +- `pytest-cov`: A plugin for pytest that measures code coverage during testing. +- `ipykernel`: A package that provides the IPython kernel for Jupyter. +- `types-python-dateutil`: Type stubs for python-dateutil, aiding in static type checking. +- `types-toml`: Type stubs for TOML, useful for static type checking in TOML parsing. +- `poetry`: A tool for dependency management and packaging in Python. + +Each dependency plays a critical role in ensuring the code is clean, efficient, and functional, ultimately leading to the development of high-quality software. + +While developing code for the OpenBB Platform, one should always install the DevTools packages so that the above development tooling is available out-of-the-box. + +### Installation + +Install from PyPI with: + +```console +pip install openbb-devtools +``` + +:::info +When setting up the environment using the `openbb_platform/dev_install.py` script, the DevTools will also be installed. +::: + +## Econometrics + +The `openbb-econometrics` extension installs a new router path (`obb.econometrics`) and additional Python libraries: + +- scipy +- statsmodels +- arch +- linearmodels + +:::note + +Statsmodels requires a C compiler be present on the system. Follow the instructions [here](https://cython.readthedocs.io/en/latest/src/quickstart/install.html) for system-specific methods. + +This extension is not accessible via REST API because `statsmodels` is not serializable. +::: + +### Installation + +Install from PyPI with: + +```console +pip install openbb-econometrics +``` + +To install from source in editable mode, navigate into the folder, `~/openbb_platform/extensions/econometrics`, and enter: + +```console +pip install -e . +``` + +After installation, the Python interface will automatically rebuild on initialization. + +## Quantitative + +The `openbb-quantitative` extension installs a new router path (`obb.quantitative`) and a few additional Python libraries: + +- pandas-ta +- scipy +- statsmodels + +### Installation + +Install from PyPI with: + +```console +pip install openbb-quantitative +``` + +To install from source in editable mode, navigate into the folder, `~/openbb_platform/extensions/quantitative`, and enter: + +```console +pip install -e . +``` + +After installation, the Python interface will automatically rebuild on initialization. + +## Technical + +The `openbb-technical` extension is for performing technical analysis on time series data. It installs a new router path (`obb.technical`) and some additional Python libraries: + +- pandas-ta +- scikit-learn +- scipy +- statsmodels + +### Installation + +Install from PyPI with: + +```console +pip install openbb-technical +``` + +To install from source in editable mode, navigate into the folder, `~/openbb_platform/extensions/technical`, and enter: + +```console +pip install -e . +``` + +After installation, the Python interface will automatically rebuild on initialization. +
diff --git a/website/content/platform/developer_guide/github.mdx b/website/content/platform/developer_guide/github.mdx new file mode 100644 index 000000000000..ff341170cbf2 --- /dev/null +++ b/website/content/platform/developer_guide/github.mdx @@ -0,0 +1,66 @@ +--- +title: GitHub +sidebar_position: 10 +description: This page outlines working with GitHub, including branch naming conventions and information related to creating a pull request. +keywords: +- Commit changes +- PR creation +- Branch naming conventions +- Commit guidelines +- GitHub +- contributing +- submission +- cheat sheet +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +This page covers working with the GitHub repository, and contributing code via a pull request. + +## GitHub Cheat Sheet + +See the page [here](https://training.github.com/downloads/github-git-cheat-sheet/) for a GitHub cheat sheet. + +## Merge Conflicts + +If you have installed the OpenBB Platform in editable mode (i.e, with `dev_install.py`), you may encounter merge conflicts when switching branches. They will most likely originate from the static assets automatically generated for the Python interface, and are in the folders: + +- `/openbb_platform/openbb/assets` +- `/openbb_platform/openbb/package` + +Discard any changes within and they will be regenerated upon the next initialization. + +Alternatively, `git stash` will do the trick. + +## Contributing + +Code contributions are made by creating a pull request on GitHub. Branches should typically be created from the `/develop` branch, and they should be named according to the conventions described next section below. + +### Branch Naming Conventions + +Before creating a new branch, switch to the `develop` branch and update your local cloned repo. + +```console +git fetch +git checkout develop +``` + +If there are conflicting changes with the develop branch, `stash` the local changes first. + +To submit a PR, a local branch or fork must be named according to the naming conventions: + +- `feature/feature-name` +- `bugfix/bugfix-name` +- `docs/docs-name` + +These branches can only have PRs pointing to the `develop` branch. + +### Create Pull Request + +A pull request should contain descriptions and details of all proposed changes, with any details maintainers and testers will need to know. + +Please use one of the supplied Pull Request Templates. + +Linting errors should be cleared, and any tests related to the changed files should be passing. diff --git a/website/content/platform/development/contributing/http_requests.md b/website/content/platform/developer_guide/http_requests.mdx similarity index 96% rename from website/content/platform/development/contributing/http_requests.md rename to website/content/platform/developer_guide/http_requests.mdx index cdf944980d5a..e62e31496da1 100644 --- a/website/content/platform/development/contributing/http_requests.md +++ b/website/content/platform/developer_guide/http_requests.mdx @@ -1,6 +1,6 @@ --- title: HTTP Requests -sidebar_position: 1 +sidebar_position: 6 description: This guide outlines OpenBB processes for making HTTP requests synchronously and asynchronously. Using the helpers will keep the codebase leaner and easier to maintain by eliminating duplicate processes. Anyone can build effective and efficient data fetchers, this guide outlines how to import and implement either type of request into any fetcher. keywords: - OpenBB Platform @@ -89,8 +89,9 @@ Some data providers allow for bulk downloading from a list of symbols, while man Ultimately, the choice is at the discretion of the developer. OpenBB has made the implementation of both methods easy and fast, the next sections will elaborate. -### Synchronous - Requests +### Synchronous - Requests +
```python from openbb_core.provider.utils import make_request ``` @@ -120,9 +121,11 @@ All parameters of `requests.get` or `requests.post`are accessible and passed thr ValueError If invalid method is passed ``` +
### Asynchronous - AIOHTTP +
Single-URL requests can be made asynchronously. The name of the function now starts with, `a`. ```python @@ -164,8 +167,10 @@ Absent `await`, the response is a coroutine - a task waiting to be executed. ::: -### Multi-URL Requests +
+### Multi-URL Requests +
The helper function becomes plural, `amake_requests`, when fetching for a list of URLs. Under the hood, it is using `asyncio.gather` to perform the tasks concurrently. The same default callback function from `amake_request` exists, only here it appends the expected `json` output to a `List[Dict]`. ```python @@ -191,9 +196,10 @@ from openbb_core.provider.utils.helpers import amake_requests Union[dict, List[dict]] Response json ``` +
### Custom Callback - +
Customize the response parsing by creating a specific callback function. The example below is a method for converting CSV data to a dictionary and appending it to a list. ```python @@ -209,9 +215,11 @@ async def response_callback(response, _: Any): data = DataFrame(StringIO(response), skiprows=2) results.append(data.to_dict("records")) ``` +
-### Asynchronous Fetchers +### Asynchronous Fetchers +
When a Fetcher is asynchronous, the `extract_data` static method needs to be defined accordingly - `aextract_data` instead of `extract_data`. ```python @@ -223,4 +231,7 @@ When a Fetcher is asynchronous, the `extract_data` static method needs to be def ) -> List[Dict]: ``` -These helper functions simplify and standardize the majority of HTTP requests. They are starting points for building or modifying data provider extensions, and they can also be imported as a standalone utility within any Python session. +These helper functions simplify and standardize the majority of HTTP requests. + +They are starting points for building or modifying data provider extensions, and they can also be imported as a standalone utility within any Python session. +
diff --git a/website/content/platform/developer_guide/index.mdx b/website/content/platform/developer_guide/index.mdx new file mode 100644 index 000000000000..70b8518ec01b --- /dev/null +++ b/website/content/platform/developer_guide/index.mdx @@ -0,0 +1,65 @@ +--- +title: Developer Guide +sidebar_position: 1 +description: The OpenBB Platform supplies core architecture and services for connecting data providers and extensions, consumable through the Python client or the Fast API +keywords: +- developer_guide +- OpenBB Platform +- Python client +- Fast API +- getting started +- authorization +- data providers +- OpenBB Hub +- local environment +- environment variables +- user settings +- command execution +- API response +- logging +- proxy networks +- data cleaning +- technical analysis +- quantitative analysis +- charting libraries +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +## What is the OpenBB Platform? + +Starting with V4, we have completely restructured the previous version of the OpenBB SDK. + +Instead of a single monolithic SDK, that comes with dependency nightmares and compatibility issues with things you may not need, we have morphed into the OpenBB Platform, which serves as a collection of building blocks to be used for your own need. + +We have broken the Platform into two main components: + +- OpenBB Core +- OpenBB Extensions + - OpenBB Providers + - OpenBB Toolkits + +The OpenBB Core provides all of the groundwork for building custom applications. It is a lightweight package providing connections with data providers in a standardized way, and the ability to build additional toolkits on top. Additionally, the `openbb-core` library contains the infrastructure for Fast API deployments. + +OpenBB Extensions are the pieces that can be added to the core for building on top of. We have grouped them categorically as, Providers and Toolkits. + +As the name suggests, Providers are the interface to obtaining any raw data from any data source. The data providers are accessed through asset class extensions, such as `openbb-equity`. OpenBB toolkits, such as `openbb-technical`, provide additional functionalities on top of the raw data access. + +When you install the Platform (openbb), we provide a default set of extensions that give access to a wide range of functionalities and data. Additional data and processing tools are available to be installed and used as desired. The reason we are doing this is to avoid an overcomplicated environment with many dependencies that can cause issues. The goal is that OpenBB can be used as a drop-in within any environment for building and extending applications. + +At its base, the OpenBB Platform supplies core architecture and services for connecting data providers and extensions, consumable through the Python client or the Fast API. + +The extension framework provides interoperability between as many, or few, services required. + +Optional extras are not included with the base installation, and these include: + +- Charting libraries and views +- Data processing and calculations +- Technical/Quantitative Analysis +- Community data providers +- Toolkit extensions +- CLI Terminal + +Refer to [Extensions](/platform/developer_guide/extensions) for the list of extensions hosted in the OpenBB GitHub repository that can be installed independently. diff --git a/website/content/platform/development/obbject.md b/website/content/platform/developer_guide/obbject.mdx similarity index 79% rename from website/content/platform/development/obbject.md rename to website/content/platform/developer_guide/obbject.mdx index 3d0b57a4390d..1a737868ac37 100644 --- a/website/content/platform/development/obbject.md +++ b/website/content/platform/developer_guide/obbject.mdx @@ -1,6 +1,6 @@ --- title: OBBject -sidebar_position: 3 +sidebar_position: 4 description: This page provides information about the OBBject class in the OpenBB Platform. This class provides the interface to interact with commands keywords: - OBBject @@ -20,7 +20,9 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -The OBBject (OpenBB Object) is at the heart of developing around the OpenBB Platform. Every command will return this class as the command output. This class contains the following attributes: +The OBBject (OpenBB Object) is at the heart of developing around the OpenBB Platform. Every command will return this class as the command output. + +This class contains the following attributes: - results - This contains serializable data that was obtained by running the command. @@ -89,9 +91,12 @@ In general, this will not be how you want to ingest or modify your data, so to h We understand that there is no one size fits all solution for data ingestion and manipulation, so we have provided a few methods to help with this and handle the conversion from Obbject to your preferred data type. These methods are accessed by running `obbject.`. -### to_dataframe() +
+to_dataframe() + +One of the most popular libraries for python data manipulation, this method will return a pandas dataframe with the results. -One of the most popular libraries for python data manipulation, this method will return a pandas dataframe with the results. We have designed this function to work with various data structures we have encountered. Given the time series nature of the data, we set the index to be `date` if that column appears in the models. +We have designed this function to work with various data structures we have encountered. Given the time series nature of the data, we set the index to be `date` if that column appears in the models. This also has the shorter alias `to_df()`. @@ -103,7 +108,10 @@ type(obb.equity.price.historical("AAPL", provider="fmp").to_dataframe()) ``` -### to_dict(orient) +
+ +
+to_dict() A very common data structure is to return the data in a dictionary. This model provides an interface to do so, with options to return the dictionary in different orientations, just as in the pandas `to_dict()` method. The default method will return a dictionary that allows you to index to obtain data: @@ -137,24 +145,44 @@ obb.equity.price.historical("AAPL", provider="fmp").to_dict("records")[0] 'change_over_time': 0.0002600949} ``` -### Others - -In addition to these two highly used methods, we have also provided the following methods: +
-#### to_numpy() +
+to_numpy() Returns numpy arrays of the results. Note that this loses the column names, so you will need to index the array to obtain the data. -#### to_polars() +```python +to_numpy = obb.equity.price.historical("AAPL", provider="fmp").to_numpy()[:1] +``` -Growing in popularity, polars is a blazingly fast dataframe library. This method will return a polars dataframe. Note that we do not include polars in the core dependency tree, so it needs to be installed separately. +```console +[ + [datetime.date(2023, 5, 19), 176.39, 176.39, 174.94, 175.16, + 55809475, 175.72, 174.23, 55809475.0, -1.23, -0.0069732] +], +... +``` + +
+ +
+to_polars() -#### show() +Growing in popularity, polars is a blazingly fast dataframe library. This method will return a polars dataframe. + +Note that we do not include polars in the core dependency tree, so it needs to be installed separately. +
+ +
+show() When a charting extension is installed, this method will display the chart if it was computed during the execution of the command, by setting the `chart` parameter to `True`. +
+ ## OBBject Extensions OBBject extensions registers an accessor in each OBBject that is returned when a command is executed. -This [page](how-to/add_obbject_extension) details the process for extending the OBBject class. +This [page](platform/user_guide/add_obbject_extension) details the process for extending the OBBject class. diff --git a/website/content/platform/developer_guide/standardization.mdx b/website/content/platform/developer_guide/standardization.mdx new file mode 100644 index 000000000000..9105389cec05 --- /dev/null +++ b/website/content/platform/developer_guide/standardization.mdx @@ -0,0 +1,59 @@ +--- +title: Standardization +sidebar_position: 2 +description: Learn about the OpenBB Platform, an open-source solution built by the community. Understand its use via Python interface and REST API, and acquaint yourself with how to build a custom extension or contribute directly to the platform +keywords: +- OpenBB Platform +- Open source +- Python interface +- REST API +- Data integration +- Data standardization +- OpenBB extensions +- openbb-core +- Python package +- High-Level Architecture +- Custom extension +- Contribution +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +## What Is The Standardization Framework? + +The Standardization Framework is a set of tools and guidelines that enable the user to query and obtain data in a consistent way across multiple providers. + +Each provider data model should inherit from an already defined [standard](https://docs.openbb.co/platform/data_models) model. All standard models are created and maintained by the OpenBB team. + +If a standard model needs to be created, please open a pull request and detail its use. + +Standardizing provider query parameters and response data enhances the user experience by overcoming things like: + +- Consistent query parameters across all data sources for a function, or type of function. +- Output data that has conformed types, is validated, and will be JSON serializable. + - `NaN`, `NaT`, `"None"`, empty strings, are always returned as `NoneType` (null). +- Transparently defined schemas for the data and query parameters. +- Outputs from multiple sources are comparable with each other and easily interchanged. + +The standard models are all defined in the `/OpenBBTerminal/openbb_platform/core/openbb_core/provider/standard_models/` [directory](https://github.com/OpenBB-finance/OpenBBTerminal/tree/main/openbb_platform/core/openbb_core/provider/standard_models). + +### What Is A Standard Model? + +Every standard model consists of two classes, with each being a Pydantic model. + +- [`QueryParams`](https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/openbb_platform/core/openbb_core/provider/abstract/query_params.py) +- [`Data`](https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/openbb_platform/core/openbb_core/provider/abstract/data.py) + +Any parameter or field can be assigned a custom `field_validator`, or the entire model can be passed through a `model_validator` on creation. + +### Caveats + +The standardization framework is a very powerful tool, but it has some caveats that you should be aware of: + +- We standardize fields and parameters that are shared between multiple providers. + - In some cases, it can be undesirable to define common items in the standard model. In this event, we still want consistent names and descriptions. +- When mapping the column names from a provider-specific model to the standard model, the CamelCase to snake_case conversion is done automatically. If the column names are not the same, you'll need to manually map them. + - e.g., `__alias_dict__ = {"o": "open"}` +- The standard models are created and maintained by the OpenBB team. If you want to add or modify a field within a standard model, you'll need to open a PR to the OpenBB Platform. diff --git a/website/content/platform/development/tests.md b/website/content/platform/developer_guide/tests.mdx similarity index 92% rename from website/content/platform/development/tests.md rename to website/content/platform/developer_guide/tests.mdx index 74a3c7f0fbf7..b403a90431f7 100644 --- a/website/content/platform/development/tests.md +++ b/website/content/platform/developer_guide/tests.mdx @@ -1,6 +1,6 @@ --- title: Tests -sidebar_position: 6 +sidebar_position: 11 description: This section provides an in-depth look at the Quality Assurance (QA) process in the OpenBB Platform. It covers the use of QA tools for testing extensions, creation of unit and integration tests, and the importance of maintaining a short import time for the package. keywords: - OpenBB QA process @@ -14,7 +14,9 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -We are strong believers in the Quality Assurance (QA) process and we want to make sure that all the extensions that are added to the OpenBB Platform are of high quality. To ensure this, we have a set of QA tools that you can use to test your work. +We are strong believers in the Quality Assurance (QA) process and we want to make sure that all the extensions that are added to the OpenBB Platform are of high quality. + +To ensure this, we have a set of QA tools that you can use to test your work. Primarily, we have tools that semi-automate the creation of unit and integration tests. @@ -22,6 +24,7 @@ Primarily, we have tools that semi-automate the creation of unit and integration ## Unit tests +
Each `Fetcher` comes equipped with a `test` method that will ensure it is implemented correctly, that it is returning the expected data, that all types are correct, and that the data is valid. To create unit tests for your Fetchers, you can run the following command: @@ -43,9 +46,11 @@ pytest --record=all :::note Sometimes manual intervention is needed. For example, adjusting out-of-top level imports or adding specific arguments for a given fetcher. ::: +
## Integration tests +
The integration tests are a bit more complex than the unit tests, as we want to test both the Python interface and the API interface. For this, we have two scripts that will help you generate the integration tests. To generate the integration tests for the Python interface, you can run the following command: @@ -88,8 +93,12 @@ pytest openbb_platform -m integration pytest openbb_platform ``` +
+ ## Import time +
+ We aim to have a short import time for the package. To measure that we use `tuna`. - [https://pypi.org/project/tuna/](https://pypi.org/project/tuna/) @@ -103,8 +112,12 @@ python -X importtime openbb/__init__.py 2> import.log tuna import.log ``` +
+ ## Known caveats +
+ When using the OpenBB QA Framework it is important to be aware of the following caveats: - The tests are semi-automated and might require manual intervention. For example, adjusting out-of-top level imports or changing specific arguments for a given payload. @@ -126,11 +139,13 @@ When using the OpenBB QA Framework it is important to be aware of the following on your local machine. You will know that this is the case if your tests pass locally, but fail on the CI. To fix this, you can delete the cache file from your local machine and re-record the tests. - > Note that the cache is likely located here: - > Windows: `C:\Users\user\AppData\Local\` - > Linux: `/home/user/.cache/` - > Mac: `/Users/user/Library/Caches` + > Note that the cache is likely located here: + > Windows: `C:\Users\user\AppData\Local\` + > Linux: `/home/user/.cache/` + > Mac: `/Users/user/Library/Caches` - Some providers (we are aware only of YFinance so far) do an additional request when used from the US region. As our CI is running from the US region, this might cause the tests to fail. A workaround for this is to use a VPN to record the tests from a different region. + +
diff --git a/website/content/platform/development/contributing/validators.md b/website/content/platform/developer_guide/validators.mdx similarity index 62% rename from website/content/platform/development/contributing/validators.md rename to website/content/platform/developer_guide/validators.mdx index 8048bf5233f1..71a9c601b079 100644 --- a/website/content/platform/development/contributing/validators.md +++ b/website/content/platform/developer_guide/validators.mdx @@ -1,6 +1,6 @@ --- title: Validators -sidebar_position: 2 +sidebar_position: 9 description: This guide provides detailed instructions on how and where validators should be used. keywords: - OpenBB Platform @@ -21,6 +21,7 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; Both QueryParams and Data models can benefit from the tactical use of Pydantic validators. This page will outline some of the key scenarios where they are deployed. + Overall, they assist with enforcing Fast API compliance for both inputs and outputs, and they work in the final stage of transformation immediately before output. @@ -42,63 +43,77 @@ The items to import from the Pydantic library are: from pydantic import field_validator, model_validator ``` -### Parsing Dates +
+Parsing Dates Providers will format dates in a number of ways. OpenBB uses YYYY-MM-DD as the standard convention, for both inputs and outputs. + Outputs are a `datetime` object or JSON serlialized string. Validators are used to parse the date from the specific format. This example is used within a provider's `Data` model. ```python - @field_validator("last_trade_timestamp", mode="before", check_fields=False) - @classmethod - def parse_timestamp(cls, v): - """Parse a Unix timestamp.""" - return datetime.fromtimestamp(v) +@field_validator("last_trade_timestamp", mode="before", check_fields=False) +@classmethod +def parse_timestamp(cls, v): + """Parse a Unix timestamp.""" + return datetime.fromtimestamp(v) ``` -### Normalize Percent Values +
+ +
+Normalize Percent Values At the provider level, we want to standardize the way values representing a percent are returned. + It is our intention to ensure those values are ready-to-consume by formulas without conversion. This example would be used within a provider's `Data` model. ```python - @field_validator("change_percent", mode="before", check_fields=False) - @classmethod - def normalize_percent(cls, v): - """Normalize the percent.""" - return v / 100 if v else None +@field_validator("change_percent", mode="before", check_fields=False) +@classmethod +def normalize_percent(cls, v): + """Normalize the percent.""" + return v / 100 if v else None ``` -### Dynamic Default Date +
+ +
+Dynamic Default Date It might be desirable to have a default date parameter that is not static. To allow this, we must set the default parameter value as `None`, and use the `model_validator`. This example is for the `QueryParams`. ```python - @model_validator(mode="before") - @classmethod - def validate_dates(cls, values) -> dict: - """Validate the query parameters.""" - if values.get("start_date") is None: - values["start_date"] = (datetime.now() - timedelta(days=90)).date() - if values.get("end_date") is None: - values["end_date"] = datetime.now().date() - return values +@model_validator(mode="before") +@classmethod +def validate_dates(cls, values) -> dict: + """Validate the query parameters.""" + if values.get("start_date") is None: + values["start_date"] = (datetime.now() - timedelta(days=90)).date() + if values.get("end_date") is None: + values["end_date"] = datetime.now().date() + return values ``` -### Replace 0s With None +
+ +
+Replace 0s With None Sometimes values are returned as a `0` when they should really be a `null`. This example looks at the entire `Data` model, but could easily be adapted to use on individual fields. ```python - @model_validator(mode="before") - @classmethod - def replace_zero(cls, values): - """Check for zero values and replace with None.""" - return ( - {k: None if v == 0 else v for k, v in values.items()} - if isinstance(values, dict) - else values - ) +@model_validator(mode="before") +@classmethod +def replace_zero(cls, values): + """Check for zero values and replace with None.""" + return ( + {k: None if v == 0 else v for k, v in values.items()} + if isinstance(values, dict) + else values + ) ``` + +
diff --git a/website/content/platform/development/_category_.json b/website/content/platform/development/_category_.json deleted file mode 100644 index 7a45206fc82e..000000000000 --- a/website/content/platform/development/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "Development", - "position": 7 -} \ No newline at end of file diff --git a/website/content/platform/development/contributing/_category_.json b/website/content/platform/development/contributing/_category_.json deleted file mode 100644 index b75320d67bb7..000000000000 --- a/website/content/platform/development/contributing/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "Contributing", - "position": 5 - } \ No newline at end of file diff --git a/website/content/platform/development/contributing/github.md b/website/content/platform/development/contributing/github.md deleted file mode 100644 index ffd1d450acd2..000000000000 --- a/website/content/platform/development/contributing/github.md +++ /dev/null @@ -1,43 +0,0 @@ ---- -title: GitHub -sidebar_position: 5 -description: This page outlines the GitHub branch naming conventions and information related to creating a pull request. -keywords: -- Commit changes -- PR creation -- Branch naming conventions -- Commit guidelines -- GitHub -- contributing -- submission ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -## Branch Naming Conventions - -Before creating a new branch, switch to the `develop` branch and update your local cloned repo. - -```console -git fetch -git checkout develop -``` - -If there are conflicting changes with the develop branch, `stash` the local changes first. - -To submit a PR, a local branch or fork must be named according to the naming conventions: - -- `feature/feature-name` -- `bugfix/bugfix-name` -- `docs/docs-name` - -These branches can only have PRs pointing to the `develop` branch. - -## Create Pull Request - -A pull request should contain descriptions and details of all proposed changes, with any details maintainers and testers will need to know. -Please use one of the supplied Pull Request Templates. - -Linting errors should be cleared, and any tests related to the changed files should be passing. diff --git a/website/content/platform/development/devtools.md b/website/content/platform/development/devtools.md deleted file mode 100644 index 1487c796e557..000000000000 --- a/website/content/platform/development/devtools.md +++ /dev/null @@ -1,48 +0,0 @@ ---- -title: DevTools -sidebar_position: 10 -description: This page provides information about the `openbb-devtools` package, which includes a range of dependencies essential for robust and efficient software development on the OpenBB Platform. -keywords: -- DevTools -- Python -- Development -- OpenBB Platform -- Dependencies ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -Please refer to the following PyPI distributed package: https://pypi.org/project/openbb-devtools/ - -This Python package, `openbb-devtools`, is designed for OpenBB Platform Developers and contains a range of dependencies essential for robust and efficient software development. - -These dependencies cater to various aspects like code formatting, security analysis, type checking, testing, and kernel management. - -The inclusion of these packages ensures that the development process is streamlined, the code quality is maintained, and the software is secure and reliable. - -Included dependencies: - -- `ruff`: A fast Python linter focused on performance and simplicity. -- `pylint`: A tool that checks for errors in Python code, enforces a coding standard, and looks for code smells. -- `mypy`: A static type checker for Python, helping catch type errors during development. -- `pydocstyle`: A linter for Python docstrings to ensure they meet certain style requirements. -- `black`: An uncompromising Python code formatter, ensuring consistent code style. -- `bandit`: A tool designed to find common security issues in Python code. -- `pre-commit`: Manages and maintains pre-commit hooks that run checks before each commit, ensuring code quality. -- `nox`: A generic virtualenv management and test command line tool for running tests in isolated environments. -- `pytest`: A mature full-featured Python testing tool that helps in writing better programs. -- `pytest-cov`: A plugin for pytest that measures code coverage during testing. -- `ipykernel`: A package that provides the IPython kernel for Jupyter. -- `types-python-dateutil`: Type stubs for python-dateutil, aiding in static type checking. -- `types-toml`: Type stubs for TOML, useful for static type checking in TOML parsing. -- `poetry`: A tool for dependency management and packaging in Python. - -Each dependency plays a critical role in ensuring the code is clean, efficient, and functional, ultimately leading to the development of high-quality software. - -While developing code for the OpenBB Platform, one should always install the DevTools packages so that the above development tooling is available out-of-the-box. - -:::info -When setting up the environment using the `openbb_platform/dev_install.py` script, the DevTools will also be installed. -::: diff --git a/website/content/platform/development/how-to/_category_.json b/website/content/platform/development/how-to/_category_.json deleted file mode 100644 index 6e862ca763d2..000000000000 --- a/website/content/platform/development/how-to/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "How-To Guides", - "position": 4 -} \ No newline at end of file diff --git a/website/content/platform/development/index.md b/website/content/platform/development/index.md deleted file mode 100644 index 23b9003d530e..000000000000 --- a/website/content/platform/development/index.md +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Considerations -sidebar_position: 1 -description: Learn about the OpenBB Platform, an open-source solution built by the - community. Understand its use via Python interface and REST API, and acquaint yourself - with how to build a custom extension or contribute directly to the platform -keywords: -- OpenBB Platform -- Open source -- Python interface -- REST API -- Data integration -- Data standardization -- OpenBB extensions -- openbb-core -- Python package -- High-Level Architecture -- Custom extension -- Contribution ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -These sections provide guidelines for developing with, and contributing to, the OpenBB Platform. -There are comprehensive guides on how to build extensions, add new data points, contribute to the code base, and more. - -We generalize between two distinct types of users: - -1. **Developers**: Those who are building on top of, and extending, the OpenBB Platform for their own purposes and have no intention of contributing the code directly to the GitHub repository. This includes those independently publishing extensions to PyPI or other package managers. -2. **Contributors**: Those who contribute to the existing codebase, by opening a Pull Request, thus giving back to the community. This can include bug fixes, enhancements, documentation, and more. - -**Why Is This Distinction Important?** - -The OpenBB Platform has been designed as a foundation for further development of investment research applications. We anticipate a wide range of creative use cases. - -Some of them may be highly specific, or detail-oriented, solving particular problems that may not necessarily fit within the OpenBB Platform Github repository. This is entirely acceptable, even encouraged. Regardless of intention, OpenBB is a proponent of building in public and sharing. We love seeing what people are building, so don't be shy about it! - - -## Before Beginning - -- Familiarize yourself with the codebase, architecture, and components. -- Set clear goals with defined outcomes - i.e, I want to create a technical indicator that uses multiple data points and sources where the output is a chart. - -Below is a high-level overview of the OpenBB Platform architecture. - - - - OpenBB Platform High-Level Architecture - - -Cloning the [GitHub repo](https://github.com/OpenBB-finance/OpenBBTerminal) will be the best way to inspect and play around with the code. - -## What Is An Extension? - -An extension is an installable component adding functionality to the OpenBB Platform. It can be a new data source, a new command, a new visualization, or anything imaginable. They can generally be classified as one of: - -- Data Provider - - The individual sources of data. -- Toolkit - - Data processing, router modules, visualizations. -- OBBject - - Extending the OBBject class itself. - -The extensions within the OpenBB GitHub repository are maintained by the OpenBB Team. The pages under [how-to](how-to/add_obbject_extension) detail getting started building with the OpenBB Platform. - -We welcome contributions, and anyone is free to publish their own OpenBB extension to PyPI, or elsewhere. If you do, please name the package beginning with, "openbb-". We love seeing what you build! diff --git a/website/content/platform/development/standardization.md b/website/content/platform/development/standardization.md deleted file mode 100644 index d25d93f0c88a..000000000000 --- a/website/content/platform/development/standardization.md +++ /dev/null @@ -1,151 +0,0 @@ ---- -title: Standardization -sidebar_position: 1 -description: Learn about the OpenBB Platform, an open-source solution built by the - community. Understand its use via Python interface and REST API, and acquaint yourself - with how to build a custom extension or contribute directly to the platform -keywords: -- OpenBB Platform -- Open source -- Python interface -- REST API -- Data integration -- Data standardization -- OpenBB extensions -- openbb-core -- Python package -- High-Level Architecture -- Custom extension -- Contribution ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -## What Is The Standardization Framework? - -The Standardization Framework is a set of tools and guidelines that enable the user to query and obtain data in a consistent way across multiple providers. - -Each provider data model should inherit from an already defined [standard](https://docs.openbb.co/platform/data_models) model. All standard models are created and maintained by the OpenBB team. If a standard model needs to be created, please open a pull request and detail its use. - -Standardizing provider query parameters and response data enhances the user experience by overcoming things like: - -- Consistent query parameters across all data sources for a function, or type of function. -- Output data that has conformed types, is validated, and will be JSON serializable. - - `NaN`, `NaT`, `"None"`, empty strings, are always returned as `NoneType` (null). -- Transparently defined schemas for the data and query parameters. -- Outputs from multiple sources are comparable with each other and easily interchanged. - -The standard models are all defined in the `/OpenBBTerminal/openbb_platform/core/openbb_core/provider/standard_models/` [directory](https://github.com/OpenBB-finance/OpenBBTerminal/tree/main/openbb_platform/core/openbb_core/provider/standard_models). - -### What Is A Standard Model? - -Every standard model consists of two classes, with each being a Pydantic model. - -- [`QueryParams`](https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/openbb_platform/core/openbb_core/provider/abstract/query_params.py) -- [`Data`](https://raw.githubusercontent.com/OpenBB-finance/OpenBBTerminal/main/openbb_platform/core/openbb_core/provider/abstract/data.py) - -Any parameter or field can be assigned a custom `field_validator`, or the entire model can be passed through a `model_validator` on creation. - -### Caveats - -The standardization framework is a very powerful tool, but it has some caveats that you should be aware of: - -- We standardize fields and parameters that are shared between multiple providers. - - In some cases, it can be undesirable to define common items in the standard model. In this event, we still want consistent names and descriptions. -- When mapping the column names from a provider-specific model to the standard model, the CamelCase to snake_case conversion is done automatically. If the column names are not the same, you'll need to manually map them. - - e.g., `__alias_dict__ = {"o": "open"}` -- The standard models are created and maintained by the OpenBB team. If you want to add or modify a field within a standard model, you'll need to open a PR to the OpenBB Platform. - - -### QueryParams - -The `QueryParams` is an abstract class that defines what parameters will be needed to make a query to a data source. Below is the [EquityHistorical](https://docs.openbb.co/platform/data_models/EquityHistorical) standard model. - -```python -"""Equity Historical Price Standard Model.""" - -from datetime import ( - date as dateType, - datetime, -) -from typing import Optional, Union - -from dateutil import parser -from pydantic import Field, field_validator - -from openbb_core.provider.abstract.data import Data -from openbb_core.provider.abstract.query_params import QueryParams -from openbb_core.provider.utils.descriptions import ( - DATA_DESCRIPTIONS, - QUERY_DESCRIPTIONS, -) - -class EquityHistoricalQueryParams(QueryParams): - """Equity Historical Price Query.""" - - symbol: str = Field(description=QUERY_DESCRIPTIONS.get("symbol", "")) - interval: Optional[str] = Field( - default="1d", - description=QUERY_DESCRIPTIONS.get("interval", ""), - ) - start_date: Optional[dateType] = Field( - default=None, - description=QUERY_DESCRIPTIONS.get("start_date", ""), - ) - end_date: Optional[dateType] = Field( - default=None, - description=QUERY_DESCRIPTIONS.get("end_date", ""), - ) - - @field_validator("symbol", mode="before", check_fields=False) - @classmethod - def to_upper(cls, v: str) -> str: - """Convert field to uppercase.""" - return v.upper() -``` - -:::info -Note that not all possible parameters are defined here, and can be further refined in the provider-specific model. - -For example, `interval` is defined as a string in the standard model, but is defined again as a `Literal` with explicit choices specific to that source. Here, it would not be possible to define all valid choices in a way that is compatible with all providers. -::: - -### Data - -The `Data` model for the `EquityHistorical` standard model is shown below, which is a continuation of the code block above. - -```python -class EquityHistoricalData(Data): - """Equity Historical Price Data.""" - - date: Union[dateType, datetime] = Field( - description=DATA_DESCRIPTIONS.get("date", "") - ) - open: float = Field(description=DATA_DESCRIPTIONS.get("open", "")) - high: float = Field(description=DATA_DESCRIPTIONS.get("high", "")) - low: float = Field(description=DATA_DESCRIPTIONS.get("low", "")) - close: float = Field(description=DATA_DESCRIPTIONS.get("close", "")) - volume: Optional[Union[float, int]] = Field( - default=None, description=DATA_DESCRIPTIONS.get("volume", "") - ) - vwap: Optional[float] = Field( - default=None, description=DATA_DESCRIPTIONS.get("vwap", "") - ) - - @field_validator("date", mode="before", check_fields=False) - def date_validate(cls, v): - """Return formatted datetime.""" - if ":" in str(v): - return parser.isoparse(str(v)) - return parser.parse(str(v)).date() -``` - -The `Data` class is an abstract class that tells us the expected output data. - -We can see that the `volume` and `vwap` fields are `Optional`. This is because not all providers return this field, but is common between several of them. - -We can also note that `date` is a field name, but because this is a `datetime` class, we import `datetime.datetime.date` as `dateType` to avoid conflicts and linting errors. - -The date validator is there to parse an ISO date string, with a caveat to only encode time values when they exist. This avoids dates from being inadvertently encoded with timezone information that can influence the actual date value via localization. Dates should always be the same date for all users globally. If a date is on a Monday, it will always be Monday. diff --git a/website/content/platform/extensions/_category_.json b/website/content/platform/extensions/_category_.json deleted file mode 100644 index 88bcd7867285..000000000000 --- a/website/content/platform/extensions/_category_.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "label": "Extensions", - "position": 2 - } - \ No newline at end of file diff --git a/website/content/platform/extensions/charting/_category_.json b/website/content/platform/extensions/charting/_category_.json deleted file mode 100644 index bbcd6d4e0839..000000000000 --- a/website/content/platform/extensions/charting/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "Charting", - "position": 3 -} \ No newline at end of file diff --git a/website/content/platform/extensions/data_extensions.md b/website/content/platform/extensions/data_extensions.md deleted file mode 100644 index 2016afa5373c..000000000000 --- a/website/content/platform/extensions/data_extensions.md +++ /dev/null @@ -1,99 +0,0 @@ ---- -title: Data Extensions -sidebar_position: 1 -description: Learn about the OpenBB Platform and its extension framework that allows - seamless integration of modules like 'openbb-yfinance'. Discover how installations - and removals automatically update the router when the Python interpreter is refreshed. - This page lists the data provider extensions available. -keywords: -- OpenBB Platform -- extension framework -- yFinance -- install openbb-yfinance -- Python interpreter -- PyPI -- openbb-qa -- data -- vendors -- providers -- install -- free -- subscription ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -Data extensions will expand the breadth and coverage of the data available in the OpenBB Platform. Each source (provider) is its own independent extension, even if there is only one endpoint accessible. This allows every data source to be inserted or removed, at any time, without disturbing the operation of the Core components. - -Functions will appear in the Python Interface and Fast API only if a supported provider, for that specific endpoint, is installed. Additional Python libraries will be installed, where required, by the extension. - -## Provider Coverage - -The total installed coverage can be determined through the Python interface, as a dictionary. - -```python -from openbb import obb -obb.coverage.providers -``` - -## Installation - -All data extensions are installed with similar syntax. Published data extensions will have names beginning with `openbb`. For example, yFinance. - -```console -pip install openbb-yfinance -``` - -Additions and removals update the router automatically to reflect the changes when the Python interpreter is refreshed. Below is a list of data provider extensions. - -Uninstall any extension with `pip uninstall`. - -```console -pip uninstall openbb-yfinance -``` - -## Available Data Extensions - -### Core `openbb` Providers - -These packages are what will be installed when `pip install openbb` is run - -| Extension Name | Description | Installation Command | Minimum Subscription Type Required | -|----------------|-------------|----------------------|------------------------------------| -| openbb-benzinga | [Benzinga](https://www.benzinga.com/apis/en-ca/) data connector | pip install openbb-benzinga | Paid | -| openbb-fmp | [FMP](https://site.financialmodelingprep.com/developer/) data connector | pip install openbb-fmp | Free | -| openbb-fred | [FRED](https://fred.stlouisfed.org/) data connector | pip install openbb-fred | Free | -| openbb-intrinio | [Intrinio](https://intrinio.com/pricing) data connector | pip install openbb-intrinio | Paid | -| openbb-oecd | [OECD](https://data.oecd.org/) data connector | pip install openbb-oecd | Free | -| openbb-polygon | [Polygon](https://polygon.io/) data connector | pip install openbb-polygon | Free | -| openbb-sec | [SEC](https://www.sec.gov/edgar/sec-api-documentation) data connector | pip install openbb-sec | None | -| openbb-tiingo | [Tiingo](https://www.tiingo.com/about/pricing) data connector | pip install openbb-tiingo | Free | -| openbb-tradingeconomics | [TradingEconomics](https://tradingeconomics.com/api) data connector | pip install openbb-tradingeconomics | Paid | -| openbb-yfinance | [Yahoo Finance](https://finance.yahoo.com/) data connector | pip install openbb-yfinance | None | - -### Community Providers - -These packages are not installed when `pip install openbb` is run. They are available for installation separately or by running `pip install openbb[all]` - -| Extension Name | Description | Installation Command | Minimum Subscription Type Required | -|----------------|-------------|----------------------|------------------------------------| -| openbb-alpha-vantage | [Alpha Vantage](https://www.alphavantage.co/) data connector | pip install openbb-alpha-vantage | Free | -| openbb-biztoc | [Biztoc](https://api.biztoc.com/#biztoc-default) News data connector | pip install openbb-biztoc | Free | -| openbb-cboe | [Cboe](https://www.cboe.com/delayed_quotes/) data connector | pip install openbb-cboe | None | -| openbb-ecb | [ECB](https://data.ecb.europa.eu/) data connector | pip install openbb-ecb | None | -| openbb-federal-reserve | [Federal Reserve](https://www.federalreserve.gov/) data connector | pip install openbb-federal-reserve | None | -| openbb-finra | [FINRA](https://www.finra.org/finra-data) data connector | pip install openbb-finra | None / Free | -| openbb-finviz | [Finviz](https://finviz.com) data connector | pip install openbb-finviz | None | -| openbb-government-us | [US Government](https://data.gov) data connector | pip install openbb-us-government | None | -| openbb-nasdaq | [Nasdaq Data Link](https://data.nasdaq.com/) connector | pip install openbb-nasdaq | None / Free | -| openbb-seeking-alpha | [Seeking Alpha](https://seekingalpha.com/) data connector | pip install openbb-seeking-alpha | None | -| openbb-stockgrid | [Stockgrid](https://stockgrid.io) data connector | pip install openbb-stockgrid | None | -| openbb-tmx | [TMX](https://money.tmx.com) data connector | pip install openbb-tmx | None | -| openbb-tradier | [Tradier](https://tradier.com) data connector | pip install openbb-tradier | None | -| openbb-wsj | [Wall Street Journal](https://www.wsj.com/) data connector | pip install openbb-wsj | None | - -Have you published a data provider extension and want it featured on this list? Tell us about it! Open a pull request on [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal/) to submit an extension for inclusion. Code contributions, for new and existing, data providers are always welcome. - -Search [PyPI](https://pypi.org/search/?q=openbb-) to find more extensions. diff --git a/website/content/platform/extensions/index.md b/website/content/platform/extensions/index.md deleted file mode 100644 index 77060f9e8b3e..000000000000 --- a/website/content/platform/extensions/index.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: OpenBB Platform Extensions -sidebar_position: 1 -description: This page describes the types of extensions available for the OpenBB Platform. -keywords: -- OpenBB Platform -- Python client -- Fast API -- getting started -- extensions -- data providers -- data extensions -- function extensions -- endpoints -- community -- technical analysis -- quantitative analysis -- charting libraries ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -When the core OpenBB Platform package is installed, familiar components might be missing. This is to keep the core components as light as possible. The extension framework allows individual pieces to be installed and removed seamlessly within the environment, using only the desired data and toolkit extensions. - -There are two primary types of extensions for the OpenBB Platform: - -- [Data](/platform/extensions/data_extensions) -- [Toolkits](/platform/extensions/toolkit_extensions) - -The OpenBB Core installation does not include any toolkit extensions. Install the OpenBB Platform with all data and toolkit extensions from PyPI with: - -```python -pip install openbb[all] -``` - -When installing from source, navigate into the `openbb_platform` folder from the root of the project and enter: - -```console -python dev_install.py -e -``` - -This installs all extensions in editable mode, and the Python interface is compiled in, `/openbb_platform/openbb/package`, instead of the environment's `site-packages` folder. The tables in the next pages lists extensions as either, Core or Community. The Core extensions are installed by default. diff --git a/website/content/platform/extensions/toolkit_extensions.md b/website/content/platform/extensions/toolkit_extensions.md deleted file mode 100644 index 00ce133e8327..000000000000 --- a/website/content/platform/extensions/toolkit_extensions.md +++ /dev/null @@ -1,175 +0,0 @@ ---- -title: Toolkits -sidebar_position: 1 -description: This page describes the toolkit extensios available for the OpenBB Platform. -keywords: -- OpenBB Platform -- Python client -- Fast API -- getting started -- extensions -- data providers -- data extensions -- toolkit extensions -- toolkits -- endpoints -- community -- technical analysis -- quantitative analysis -- charting libraries -- Plotly -- OpenBBFigure -- PyWry ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -OpenBB Toolkit Extensions expand the Platform with functions for manipulating data and preparing it for display. The Core Platform installation does not install any toolkit extensions. The table below is the current list of toolkit extensions. - -| Extension Name | Description | Installation Command | Core/Community | Router Path | -|:-----------------|:-----------:|:-------------------:|:------------------:|-------------:| -| openbb-charting | Rest API charting service and Plotly library. | pip install openbb-charting | Community | N/A | -| openbb-devtools | Aggregates dependencies that facilitate a nice development experience for OpenBB. | pip install openbb-devtools | N/A | -| openbb-econometrics | Econometrics models for the Python interface only. | pip install openbb-econometrics | Community | obb.econometrics | -| openbb-quantitative | Functions for performing quantitative analysis. | pip install openbb-quantitative | Community | obb.quantitative | -| openbb-technical | Functions for performing technical analysis. | pip install openbb-technical | Community | obb.technical | - -The sections below outline any specific installation considerations for the extension. - -## Charting - -The OpenBB Charting Extension supplies charting infrastructure and services to the OpenBB Platform. Figure objects are served via REST API or Python Client. It utilizes [PyWry](https://github.com/OpenBB-finance/pywry) for handling the display of interactive charts and tables in a separate window, with a Plotly library. - -Functions with dedicated views return figures to the `chart` attribute of the `OBBject` response object. They are displayed with the class method, `show()`. - -:::tip -The `openbb-charting` is an [`OBBject` extension](platform/development/how-to/add_obbject_extension.md), which means the general functionality is exposed in every command result. -::: - -The following packages are dependencies of the `openbb-charting` extension: - -- scipy -- plotly -- statsmodels -- reportlab -- pywry -- svglib -- nbformat -- pandas-ta - -### Installation - -See the [charting](charting) section for more details and [installation](charting/installation) instructions. - -## Devtools - -This extension aggregates the dependencies that facilitate a nice development experience -for OpenBB. It does not contain any code itself, but rather pulls in the following dependencies: - -- bandit -- black -- ipykernel -- mypy -- poetry -- pre-commit -- pydocstyle -- pylint -- pytest -- pytest-cov -- ruff -- tox -- types-python-dateutil -- types-toml - -### Installation - -The extension is included in the `dev_install.py` script. - -Standalone installation: - -```console -pip install openbb-devtools -``` - -## Econometrics - -The `openbb-econometrics` extension installs a new router path (`obb.econometrics`) and additional Python libraries: - -- scipy -- statsmodels -- arch -- linearmodels - -:::note - -Statsmodels requires a C compiler be present on the system. Follow the instructions [here](https://cython.readthedocs.io/en/latest/src/quickstart/install.html) for system-specific methods. - -This extension is not accessible via REST API because `statsmodels` is not serializable. -::: - -### Installation - -Install from PyPI with: - -```console -pip install openbb-econometrics -``` - -To install from source in editable mode, navigate into the folder, `~/openbb_platform/extensions/econometrics`, and enter: - -```console -pip install -e . -``` - -After installation, the Python interface will automatically rebuild on initialization. - -## Quantitative - -The `openbb-quantitative` extension installs a new router path (`obb.quantitative`) and a few additional Python libraries: - -- pandas-ta -- scipy -- statsmodels - -### Installation - -Install from PyPI with: - -```console -pip install openbb-quantitative -``` - -To install from source in editable mode, navigate into the folder, `~/openbb_platform/extensions/quantitative`, and enter: - -```console -pip install -e . -``` - -After installation, the Python interface will automatically rebuild on initialization. - -## Technical - -The `openbb-technical` extension is for performing technical analysis on time series data. It installs a new router path (`obb.technical`) and some additional Python libraries: - -- pandas-ta -- scikit-learn -- scipy -- statsmodels - -### Installation - -Install from PyPI with: - -```console -pip install openbb-technical -``` - -To install from source in editable mode, navigate into the folder, `~/openbb_platform/extensions/technical`, and enter: - -```console -pip install -e . -``` - -After installation, the Python interface will automatically rebuild on initialization. diff --git a/website/content/platform/faqs/_category_.json b/website/content/platform/faqs/_category_.json index e4157c1e437d..9164f47b5376 100644 --- a/website/content/platform/faqs/_category_.json +++ b/website/content/platform/faqs/_category_.json @@ -1,4 +1,4 @@ { "label": "FAQs", - "position": 8 + "position": 6 } diff --git a/website/content/platform/faqs/data_providers.md b/website/content/platform/faqs/data_providers.mdx similarity index 75% rename from website/content/platform/faqs/data_providers.md rename to website/content/platform/faqs/data_providers.mdx index f890d123418a..14e884f9e8ff 100644 --- a/website/content/platform/faqs/data_providers.md +++ b/website/content/platform/faqs/data_providers.mdx @@ -1,8 +1,7 @@ --- title: Data and Data Providers sidebar_position: 2 -description: This page contains some frequently asked questions about OpenBB - data and providers. +description: This page contains some frequently asked questions about OpenBB data and providers. keywords: - provider - data @@ -22,14 +21,18 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; Equity market coverage will vary by provider and subscription status with them. It is common for free tiers to be US-listings only. -You can find all data models [here](/platform/data_models), or the [Reference](/platform/reference) page of endpoints. If the type of data you are looking for is not listed there, send us a [feature request](https://openbb.co/request-a-feature) telling us about your use case. +You can find all data models [here](/platform/data_models), or the [Reference](/platform/reference) page of endpoints. + +If the type of data you are looking for is not listed there, send us a [feature request](https://openbb.co/request-a-feature) telling us about your use case.
The router appears to be missing functions. -The router populates itself from the installed extensions. For example, if the Technical Analysis extension is not installed, the `obb.technical` router path will not be present. +The router populates itself from the installed extensions. + +For example, if the Technical Analysis extension is not installed, the `obb.technical` router path will not be present. The same applies to data extensions. If a provider module is not installed, it will not be displayed as a choice. @@ -56,7 +59,7 @@ python dev_install.py -e The nightly PyPI distribution is another way to install everything, and to be on the bleeding edge of development: -```bashe +```bash pip install openbb-nightly ``` @@ -69,7 +72,11 @@ pip install openbb-nightly The provider may not have data from the requested period, in which case the data will be what they return. For example, `provider='yfinance'` at one-minute intervals will not return beyond one week ago. -Another reason could be the data entitlements of your API key. Check the provider's website to know what data coverage to expect. If there is technical problem with a provider or function, please check [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal/issues/new/choose) and raise an issue if one does not already exist. Or, send us an [email](mailto:support@openbb.co) with the details, your system configuration, the syntax used, and any error messages that are raised. +Another reason could be the data entitlements of your API key. Check the provider's website to know what data coverage to expect. + +If there is technical problem with a provider or function, please check [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal/issues/new/choose) and raise an issue if one does not already exist. + +Or, send us an [email](mailto:support@openbb.co) with the details, your system configuration, the syntax used, and any error messages that are raised.
@@ -97,7 +104,7 @@ Please [request a feature](https://openbb.co/request-a-feature), tell us about y
Can I contribute my own data provider extension? -Yes! Please take a look at our [Development](/platform/development) pages for more information. +Yes! Please take a look at our [Development](/platform/developer_guide/contributing) pages for more information.
diff --git a/website/content/platform/faqs/platform_vs_sdk.md b/website/content/platform/faqs/platform_vs_sdk.mdx similarity index 74% rename from website/content/platform/faqs/platform_vs_sdk.md rename to website/content/platform/faqs/platform_vs_sdk.mdx index d3692e9605fb..eb6629f8c90a 100644 --- a/website/content/platform/faqs/platform_vs_sdk.md +++ b/website/content/platform/faqs/platform_vs_sdk.mdx @@ -1,9 +1,7 @@ --- title: Platform vs SDK sidebar_position: 1 -description: The OpenBB SDK has evolved to become the OpenBB Platform. - This page describes some of the key differences between the legacy version - and the new architecture. +description: The OpenBB SDK has evolved to become the OpenBB Platform. This page describes some of the key differences between the legacy version and the new architecture. keywords: - what's new - v3 @@ -18,7 +16,11 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -If you're already an OpenBB user, you may be familiar with some of the legacy pain points. As [this](https://openbb.co/blog/celebrating-the-openbb-platform-v4-beta) blog post highlights, there were many challenges with maintaining the existing codebase. We needed to refresh the architecture to make it modular, resilient, and scalable. The core components have been trimmed substantially to be lean and efficient - the number of dependencies has reduced from nearly four-hundred down to about twenty. The result is a much smoother installation procedure, with the tradeoff being some breaking changes for those transitioning from V3 SDK to the V4 Platform. The major differences are described below. +If you're already an OpenBB user, you may be familiar with some of the legacy pain points. As [this](https://openbb.co/blog/celebrating-the-openbb-platform-v4-beta) blog post highlights, there were many challenges with maintaining the existing codebase. + +We needed to refresh the architecture to make it modular, resilient, and scalable. The core components have been trimmed substantially to be lean and efficient - the number of dependencies has reduced from nearly four-hundred down to about twenty. + +The result is a much smoother installation procedure, with the tradeoff being some breaking changes for those transitioning from V3 SDK to the V4 Platform. The major differences are described below. ### Terminal Application @@ -54,7 +56,9 @@ uvicorn openbb_core.api.rest_api:app ### Verbose Namespaces -After careful consideration, the decision was made to name functions with more verbosity. This adds clarity to the functions and lets the user better understand its purpose. It also improves the performance of AI tooling built on top of the Platform. +After careful consideration, the decision was made to name functions with more verbosity. This adds clarity to the functions and lets the user better understand its purpose. + +It also improves the performance of AI tooling built on top of the Platform. ```python obb.equity.fundamental.employee_count("AAPL") @@ -99,7 +103,9 @@ obb.account.login() ### Function Outputs -The default output format can be selected by the user, and all outputs are Pydantic models. If you are transitioning from V3 SDK and like working with Pandas DataFrames, set the preference to "dataframe" to get a V3-like response. +The default output format can be selected by the user, and all outputs are Pydantic models. + +If you are transitioning from V3 SDK and like working with Pandas DataFrames, set the preference to "dataframe" to get a V3-like response. ```python from openbb import obb @@ -129,7 +135,9 @@ pip install jupyter-lab ### Views -Most of the development has been on the core architecture and data providers. Most views from the V3 SDK and Terminal have yet to be ported over to the V4 Platform, although some charts are already available with the `openbb-charting` toolkit extension - which includes PyWry for window creation. +Most of the development has been on the core architecture and data providers. + +Most views from the V3 SDK and Terminal have yet to be ported over to the V4 Platform, although some charts are already available with the `openbb-charting` toolkit extension - which includes PyWry for window creation. Install the charting extension with: @@ -143,4 +151,4 @@ More views to come soon! ### Getting Started -See the [usage](/platform/usage) page for examples on getting started using the OpenBB Platform. +See the [getting started](/platform/getting_started) page for examples on getting started using the OpenBB Platform. diff --git a/website/content/platform/getting_started/_category_.json b/website/content/platform/getting_started/_category_.json new file mode 100644 index 000000000000..3825718cdfdd --- /dev/null +++ b/website/content/platform/getting_started/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Getting Started", + "position": 2 +} diff --git a/website/content/platform/usage/api_keys.md b/website/content/platform/getting_started/api_keys.mdx similarity index 96% rename from website/content/platform/usage/api_keys.md rename to website/content/platform/getting_started/api_keys.mdx index bcb05dccf40c..4c38085ef5d4 100644 --- a/website/content/platform/usage/api_keys.md +++ b/website/content/platform/getting_started/api_keys.mdx @@ -1,8 +1,9 @@ --- -title: API Keys -sidebar_position: 1 +title: Authorization and API Keys +sidebar_position: 2 description: An overview for setting up the OpenBB Platform Python client and Fast API with data provider API keys. keywords: +- tutorial - OpenBB Platform - Python client - Fast API @@ -18,7 +19,7 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -By default, authorization is not required to initialize and use the core services. Most data providers, however, require an API key to access their data. Keys can be stored locally and they can also be securely saved to your OpenBB Hub [account](https://my.openbb.co) for convenient remote access. +By default, authorization is not required to initialize and use the core services. Most data providers, however, require an API key to access their data. Keys can be stored locally and they can also be securely saved to your OpenBB Hub [account](https://my.openbb.co/app/hub) for convenient remote access. ### OpenBB Hub @@ -26,7 +27,6 @@ By default, authorization is not required to initialize and use the core service The OpenBB Hub is only accessible via the Python Interface. For REST API, store credentials and preferences in the `user_settings.json` file [local](api_keys#local-environment) to the deployment. ::: - Data provider credentials and user preferences can be securely stored on the OpenBB Hub and accessed in Python using a revokable Personal Access Token (PAT). Login to the [Hub](https://my.openbb.co/) to manage this method of remote authorization. The OpenBB Hub is a convenient solution for accessing data in temporary Python environments, like Google Colab ([example notebook](https://github.com/OpenBB-finance/OpenBBTerminal/blob/develop/examples/googleColab.ipynb)). Login with: @@ -55,7 +55,6 @@ obb.account.logout() Set `remember_me` as `False` to discard all credentials at the end of the session. - :::tip With `remember_me=True`, credentials will be permanently stored in the environment. Wrapping this sequence before deploying an API server is one (insecure) way to authorize data providers for remote access. diff --git a/website/content/platform/getting_started/api_requests.mdx b/website/content/platform/getting_started/api_requests.mdx new file mode 100644 index 000000000000..487b46dc8a0e --- /dev/null +++ b/website/content/platform/getting_started/api_requests.mdx @@ -0,0 +1,95 @@ +--- +title: REST API Requests +sidebar_position: 8 +description: How to send requests to the OpenBB Platform REST API. +keywords: +- tutorial +- OpenBB Platform +- Python client +- Fast API +- getting started +- requests +- data providers +--- + + +Most REST API endpoints are for data retrieval and are defined as `GET` requests, but some [toolkit extensions](/platform/user_guides/extensions) require data to pass through the function. In these instances, it must be a `POST` request. + +### Example + +This example will use a GET request to fetch daily VIX data from the Cboe data extension, and then make a POST request which passes through the data to a technical analysis function. + +First, start a development server. + +```bash +uvicorn openbb_core.api.rest_api:app --reload +``` + +This example will use Python and the `requests` library. + +#### Fetch Some Data + +```python +import requests + +get_url = "http://127.0.0.1:8000/api/v1/index/price/historical?provider=cboe&symbol=vix&interval=1d" +get_response = requests.get(get_url) +data_results = get_response.json()["results"] + +data_results[-1] +``` + +```json +{'date': '2023-11-17T00:00:00', + 'open': 14.18, + 'high': 14.19, + 'low': 13.67, + 'close': 13.79, + 'volume': 0, + 'calls_volume': None, + 'puts_volume': None, + 'total_options_volume': None} +``` + +#### Send a POST Request + +Next, pass the `data_results` to a function, using the `json` field in the POST headers. + +For this example, realized volatility cones, the default parameters assume the time series data is daily and that volatility should be annualized over 252 trading days. + +```python +import pandas as pd + +post_url = "http://localhost:8000/api/v1/technical/cones" +post_response = requests.post(post_url, json=data_results) +ta_results = post_response.json()["results"] + +pd.DataFrame.from_records(ta_results).head() +``` + +| window | realized | min | lower_25% | median | upper_75% | max | +|---------:|-----------:|-----------:|------------:|---------:|------------:|--------:| +| 3 | 0.396165 | 0.00701638 | 0.444709 | 0.72414 | 1.11563 | 8.47636 | +| 10 | 0.623199 | 0.190188 | 0.665584 | 0.852915 | 1.15491 | 4.83264 | +| 30 | 0.988435 | 0.332913 | 0.750007 | 0.921482 | 1.17072 | 2.98404 | +| 60 | 0.932594 | 0.47639 | 0.792548 | 0.964617 | 1.20171 | 2.35563 | +| 90 | 0.915137 | 0.551011 | 0.815229 | 0.965553 | 1.2128 | 2.04104 | + +The output from the Fast API is a serialized version of this object, and these methods are lost on conversion. OBBject can be reconstructed to recover the helpers by importing the model and validating the data. + +```python +import requests +from openbb_core.app.model.obbject import OBBject + +data = [] +symbol="SPY" +url = f"http://127.0.0.1:8000/api/v1/equity/price/historical?provider=polygon&symbol={symbol}" +headers = {"accept": "application/json"} + +response = requests.get(url, headers=headers, timeout=3) + +if response.status_code == 200: + data = OBBject.model_validate(response.json()) + +data.to_df() +``` diff --git a/website/content/platform/getting_started/create_new_provider_extension.mdx b/website/content/platform/getting_started/create_new_provider_extension.mdx new file mode 100644 index 000000000000..3018c366586d --- /dev/null +++ b/website/content/platform/getting_started/create_new_provider_extension.mdx @@ -0,0 +1,176 @@ +--- +title: Create New Provider Extension +sidebar_position: 9 +description: This page will walk through creating a new OpenBB provider extension from scratch. By the end, you will have the shell structure for holding models that connect to a router through the provider interface. +keywords: +- OpenBB Platform +- Open source +- community +- code +- provider +- openbb-provider +- data +- extension +- how-to +- new +- template +- quickstart +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +This page will walk through creating a new OpenBB Provider Extension from scratch. By the end, you will have the shell structure for holding models that connect to the Router through the Provider Interface. + + +## Preparation + +### Create Project Folder + +Create a folder for the project. For this example, we will name the folder, `empty_provider`. + +### Create `__init__.py` File + +- Inside the new folder, create a new file called, `__init__.py`. +- Open the file and add a docstring in the first line, and leave an empty line below it. + +```python +"""Empty OpenBB Provider.""" +``` + +### Create `README.md` File + +- In the same location, create a new file called, `README.md`. +- Open the file, then add a title and any other high-level information about the extension. + +```markdown +# Empty OpenBB Provider Extension + +An example Provider extension for the OpenBB Platform. +``` + +### Create `pyproject.toml` File + +- In the same location, create a new file called, `pyproject.toml`. + +```toml +[tool.poetry] +name = "openbb-empty-provider" +version = "0.0.1" +description = "Empty provider extension for OpenBB" +authors = ["OpenBB Team "] +readme = "README.md" +packages = [{ include = "openbb_empty_provider" }] + +[tool.poetry.dependencies] +python = ">=3.8,<3.12" +openbb = "^4.1.7" + +[build-system] +requires = ["poetry-core"] +build-backend = "poetry.core.masonry.api" + +[tool.poetry.plugins."openbb_provider_extension"] +empty = "openbb_empty_provider:empty_provider" +``` + +### Create Sub-Folder For Code + +- Create a sub-folder that begins with `openbb` and is followed by the name of the project folder, `openbb_empty_provider`. + +### Create `__init__.py` File + +- In the new sub-folder, create a new file called, `__init__.py`. This is where all models are mapped to the Provider interface. + +```python +"""Empty Provider Module.""" + +from openbb_core.provider.abstract.provider import Provider + +empty_provider = Provider( + name="empty", # This should be the same as the assigned name at the bottom of the pyproject.toml file + website="http://empty.io", + description="""The empty provider is a supplier of promises.""", + #credentials=["api_key"], # uncomment to require credentials + fetcher_dict={ + # "SomeModel" : EmptySomeModelFetcher # Map Fetchers to the Router here. + }, +) +``` + +### Create `models` Sub-Folder + +- In the same location as the file just saved, create a new folder called, `models`. + +### Create `__init__.py` File + +- In the new `models` folder, make a new file called, `__init__.py`. +- Open the file and add a doctstring to the top line, followed by an empty line. + +```python +"""Empty Provider Models.""" + +``` + +## Build Lock File + +- The Python environment should have `toml` and `poetry` installed as packages from PyPI. + +```console +pip install toml poetry +``` + +- Navigate into the main folder, and with the environment active, enter: + +```console +poetry lock +``` + +## Install Extension In Editable Mode + +```console +pip install -e . +``` + +The installation can be verified, and it should display a path similar to the one below. +Everything else is installed under the `site-packages` of the environment. + +```console +pip list | grep openbb +``` + +```console +openbb 4.1.7 +openbb-benzinga 1.1.5 +openbb-commodity 1.0.4 +openbb-core 1.1.6 +openbb-crypto 1.1.5 +openbb-currency 1.1.5 +openbb-derivatives 1.1.5 +openbb-economy 1.1.5 +openbb-empty-provider 0.0.1 /Users/username/path_to_created_folder/empty_provider +... +``` + +## Build Static Assets + +```console +python -c "import openbb; openbb.build()" +``` + +The installation can be verified by inspecting `obb.reference`. Start a Python session and import the OpenBB Platform. + +```python +from openbb import obb + +obb.reference["info"]["extensions"]["openbb_provider_extension"] +``` + +The list should include the newly created and installed extension, `empty@0.0.1`. + +## Conclusion + +This process created, built, and installed a new OpenBB Platform Provider extension from scratch. + +The next step is to create the models for connecting with the Provider Interface and Router. diff --git a/website/content/platform/getting_started/create_new_router_extension.mdx b/website/content/platform/getting_started/create_new_router_extension.mdx new file mode 100644 index 000000000000..d323d787e1bd --- /dev/null +++ b/website/content/platform/getting_started/create_new_router_extension.mdx @@ -0,0 +1,216 @@ +--- +title: Create New Router Extension +sidebar_position: 8 +description: This page will walk through creating a new OpenBB Router Extension from scratch. By the end, you will have the code structure for a new router path and an api endpoint. +keywords: +- OpenBB Platform +- Open source +- community +- code +- router +- openbb-core +- data +- extension +- how-to +- new +- endpoint +- template +- quickstart +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +This page will walk through creating a new OpenBB router extension from scratch. By the end, you will have the shell structure for adding commands to the OpenBB Platform's interfaces. + +## Preparation + +### Create Project Folder + +Create a folder for the project. For this example, we will name the folder, `empty_router`. + +### Create `pyproject.toml` File + +- Open the folder and create a new file called, `pyproject.toml`. + +```toml +[tool.poetry] +name = "openbb-empty-router" +version = "0.0.1" +description = "An empty OpenBB Router extension" +authors = ["OpenBB Team "] +readme = "README.md" +packages = [{ include = "openbb_empty_router" }] + +[tool.poetry.dependencies] +python = "^3.8,<3.12" +openbb = "^4.2.0" + +[build-system] +requires = ["poetry-core"] +build-backend = "poetry.core.masonry.api" + +[tool.poetry.plugins."openbb_core_extension"] +empty = "openbb_empty_router.empty_router:router" +``` + +### Create `README.md` File + +- In the same location, create a new file called, `README.md`. +- Open the file, then add a title and any other high-level information about the extension. + +```markdown +# Empty OpenBB Router Extension + +An example Router extension for the OpenBB Platform. +``` + +### Create Sub-Folder For Code + +- Create a sub-folder that begins with `openbb` and is followed by the name of the project folder, `openbb_empty_router`. + +### Create `__init__.py` File + +- In the new sub-folder, create a new file called, `__init__.py`. +- Add a docstring on the first line, followed by an empty line. + +```python +"""Empty Router Module.""" + +``` + +### Create Router File + +- In the same location as the file just saved, create a new file called, `empty_router.py`. + +:::info +This is the file mapped at the bottom of the `pyproject.toml` file. + +```toml +[tool.poetry.plugins."openbb_core_extension"] +empty = "openbb_empty_router.empty_router:router" +``` + +::: + +- Insert code for our command route. + +```python +"""Empty Router""" + +from typing import Any, Dict, List, Literal, Optional +from openbb_core.app.model.command_context import CommandContext +from openbb_core.app.model.obbject import OBBject +from openbb_core.app.provider_interface import ( + ExtraParams, + ProviderChoices, + StandardParams, +) +from openbb_core.provider.abstract.data import Data +from openbb_core.app.query import Query +from openbb_core.app.router import Router +from openbb_core.app.model.obbject import OBBject + +router = Router(prefix="", description="An Empty OpenBB Router Extension.") + +@router.command(methods=["GET"]) +async def hello() -> OBBject[str]: # The output of every router command must be an instance of `OBBject`. + """OpenBB Hello World.""" + return OBBject(results="Hello from the Empty Router extension!") + +# This is a clone of `obb.equity.price.historical`. +# The model can be replaced with a a different model from the Provider Interface. +@router.command( + model="EquityHistorical", +) +async def empty_function( + cc: CommandContext, + provider_choices: ProviderChoices, + standard_params: StandardParams, + extra_params: ExtraParams, +) -> OBBject[Data]: + """An empty function using the Provider Interface.""" + return await OBBject.from_query(Query(**locals())) +``` + +## Build Lock File + +- The Python environment should have `toml` and `poetry` installed as packages from PyPI. + +```console +pip install toml poetry +``` + +- Navigate into the main folder, and with the environment active, enter: + +```console +poetry lock +``` + +## Install Extension In Editable Mode + +```console +pip install -e . +``` + +The installation can be verified, and it should display a path similar to the one below. +Everything else is installed under the `site-packages` of the environment. + +```console +pip list | grep openbb +``` + +```console +openbb 4.1.7 +openbb-benzinga 1.1.5 +openbb-commodity 1.0.4 +openbb-core 1.1.6 +openbb-crypto 1.1.5 +openbb-currency 1.1.5 +openbb-derivatives 1.1.5 +openbb-economy 1.1.5 +openbb-empty-router 0.0.1 /Users/username/path_to_created_folder/empty_router +... +``` + +## Build Static Assets + +```console +python -c "import openbb; openbb.build()" +``` + +The installation can be verified by inspecting `obb.reference`. Start a Python session and import the OpenBB Platform. + +```python +from openbb import obb + +obb.reference["info"]["extensions"]["openbb_core_extension"] +``` + +The list should include the newly created and installed extension, `empty@0.0.1`. + +## Smoke Test + +Run the added commands. + +```python +obb.empty.hello() +``` + +```console +OBBject + +id: 0663956f-a01d-751b-8000-b44c72e60664 +results: Hello from the Empty Router extension! +provider: None +warnings: None +chart: None +extra: {'metadata': {'arguments': {'provider_choices': {}, 'standard_params': {}, '... +``` + +## Conclusion + +This process created, built, and installed a new OpenBB router extension from scratch. + +The next step is to map Provider Interface models, and/or create custom GET/POST request functions to import directly for use in the router. diff --git a/website/content/platform/usage/economic_indicators.md b/website/content/platform/getting_started/economic_indicators.mdx similarity index 97% rename from website/content/platform/usage/economic_indicators.md rename to website/content/platform/getting_started/economic_indicators.mdx index 7affd0f8fc2e..ba163c18d9fe 100644 --- a/website/content/platform/usage/economic_indicators.md +++ b/website/content/platform/getting_started/economic_indicators.mdx @@ -1,6 +1,6 @@ --- title: Economic Indicators -sidebar_position: 8 +sidebar_position: 7 description: This page provides a tutorial for getting started using the `obb.economy.indicators` endpoint, with the `openbb-econdb` provider extension. The command provides access to over 100 standardized indicator symbols, covering countries around the world. @@ -20,7 +20,7 @@ keywords: import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - + This page provides a tutorial for getting started using the `obb.economy.indicators` endpoint, with the `openbb-econdb` provider extension. The command provides access to over 100 standardized indicator symbols, covering countries around the world. @@ -122,7 +122,9 @@ data.extra["results_metadata"] 'DATA_TYPE_FM:Financial market data type': 'LEV:Level'}}} ``` -### Basic Descriptions +
+Basic Descriptions + Basic descriptions of the base symbols can be imported from within the extension module helpers. @@ -310,8 +312,13 @@ This list should not be considered as the absolute source of truth. The metadata | WAGEMAN | Hourly wage manufacturing | | Y10YD | Long term yield | +
+ ## Countries + +
+ The `country` parameter will accept the ISO country code, or the country name. Regional groups listed below are also valid: - all @@ -357,9 +364,13 @@ from openbb_econdb.utils.helpers import get_indicator_countries get_indicator_countries("GDPPC") # returns a list of two-letter ISO country codes ``` +
+ ## How To Enter Symbols +
+ The three parameters - symbol, country, transform - all work together. - Symbol (base symbol) @@ -380,7 +391,8 @@ The `transform` will apply to all combinations of `symbol` and `country`. If you attempt to pass a base symbol (excluding commodity and world indicators) with no country, it will raise an error. ::: -### Example - One Indicator & Country +
+Example - Onde Indicator & Country M3 Money Supply @@ -423,8 +435,10 @@ data.extra.get("results_metadata") 'SERIES_NAME:Series name (FRB)': 'M2_N.M:M2_N.M', 'UNIT:Units': 'CURRENCY:Currency'}}} ``` +
-### Example - One Indicator & Country With Transform +
+Example - One Indicator & Country With Transform US PPI - Change from one year ago. @@ -468,8 +482,10 @@ data.extra.get("results_metadata") 'FREQ:Frequency': 'M:Monthly', 'UNIT_MULT:Unit multiplier': '0:Units'}}} ``` +
-### Example - Commodity Indicator +
+Example - Commodity Indicator Values are always in USD. @@ -504,8 +520,10 @@ lead.extra["results_metadata"] 'UNIT_MEASURE:Unit of Measure': 'USD:US Dollars', 'UNIT_MULT:Scale': '0:Units'}}} ``` +
-### Example - Multiple Indicators & Countries With Transform +
+Example - Multiple Indicators & Countries With Transform ```python params = {"symbol": "core,cpi", "country": "us,de,jp", "transform": "toya"} @@ -532,9 +550,10 @@ df | 2024-03-01 | CORE | COREUS~TOYA | United States | 0.03797 | | 2024-03-01 | CPI | CPIDE~TOYA | Germany | 0.021533 | | 2024-03-01 | CPI | CPIUS~TOYA | United States | 0.03475 | +
- -### Example - All Countries +
+Example - All Countries Setting the country to "all" will retrieve data for all available countries. @@ -590,12 +609,18 @@ data.extra.get("results_metadata")["NYSE~TUSD"] 'CURRENCY:Currency': 'MIO_NAC:Million units of national currency', 'BOP_ITEM:BOP_item': 'IN1:Primary income'}} ``` +
+ +
## Advanced Symbols +
+ The grouping behaviour can be overridden. This will allow multiple transformations, or for a specific symbol to ignore the supplied `country` and `transform` parameters. -### Example - Bypass Group Parameters +
+Example - Bypass Group Parameters The "~" character is used to separate the base symbol + 2-letter ISO country code, and the transformation. It works as a flag to exclude from the other parameters. @@ -633,8 +658,10 @@ obb.economy.indicators(symbol=["CPIUS~","CPI"], country="fr", transform="toya"). | 2024-03-01 | CPI | CPIFR~TOYA | France | 0.022947 | | 2024-03-01 | CPI | CPIUS | United States | 312.2 | +
-### Example - Non-Standard Symbols +
+Example - Non-Standard Symbols The `symbol` parameter can also be used to access non-standard series. These are specific to reporting entities, like the Ministry of Finance, Japan. These symbols are not searchable, but the structure will be familiar if you have worked with the particular source before. @@ -705,3 +732,5 @@ data.extra["results_metadata"].get("MFJP_IR.5Y.D.JP") 'additional_info': {'3:Indicator': '8:Japanese Government Bonds - 5Y yield', 'GEO:None': '107:None'}} ``` +
+
diff --git a/website/content/platform/usage/explanation/financial_statements.md b/website/content/platform/getting_started/financial_statements.mdx similarity index 85% rename from website/content/platform/usage/explanation/financial_statements.md rename to website/content/platform/getting_started/financial_statements.mdx index dd4a121f6583..89183ebbb1e6 100644 --- a/website/content/platform/usage/explanation/financial_statements.md +++ b/website/content/platform/getting_started/financial_statements.mdx @@ -1,9 +1,7 @@ --- -title: Financial Statements -sidebar_position: 6 -description: This page provides an introduction to financial statement data available in the OpenBB - Platform. This includes quarterly and annual reports, along with metrics and ratios by company. - This guide provides examples for using the variety of sources. +title: Introduction to Financial Statements +sidebar_position: 5 +description: This page provides an introduction to financial statement data available in the OpenBB Platform. This includes quarterly and annual reports, along with metrics and ratios by company. This guide provides examples for using the variety of sources. keywords: - stocks - companies @@ -29,7 +27,11 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -OpenBB Platform data extensions provide access to financial statements as quarterly or annual. There are also endpoints for ratios and other common non-GAAP metrics. Most data providers require a subscription to access all data, refer to the website of a specific provider for details on entitlements and coverage. +OpenBB Platform data extensions provide access to financial statements as quarterly or annual. + +There are also endpoints for ratios and other common non-GAAP metrics. + +Most data providers require a subscription to access all data, refer to the website of a specific provider for details on entitlements and coverage. Financial statement functions are grouped under the `obb.equity.fundamental` module. @@ -56,7 +58,8 @@ The main parameters are: - `period`: 'annual' or 'quarter'. Default is 'annual'. - `limit`: Limit the number of results returned, from the latest. Default is 5. For perspective, 150 will go back to 1985. The amount of historical records varies by provider. -### Field Names +
+Field Names :::info @@ -64,9 +67,10 @@ The main parameters are: - Items within each statement will vary by source and by the type of company reporting. - Names of line items will vary by source. - "Date" values may differ because they are from the period starting/ending or date of reporting. + ::: -This example highlights how different providers will have different labels for compnay facts. +This example highlights how different providers will have different labels for company facts. ```python import pandas as pd @@ -103,13 +107,20 @@ df | 2 | 53811000000 | 53811000000 | 53811000000 | 53811000000 | | 3 | 53335000000 | 53335000000 | 53335000000 | 53335000000 | -### Weighted Average Shares Outstanding +
+ +
+Weighted Average Shares Outstanding + +This key metric will be found under the income statement. It might also be called, 'basic', and the numbers do not include authorized but unissued shares. -This key metric will be found under the income statement. It might also be called, 'basic', and the numbers do not include authorized but unissued shares. A declining count over time is a sign that the company is returning capital to shareholders in the form of buy backs. Under ideal circumstances, it is more capital-efficient, for both company and shareholders, because distributions are double-taxed. The company pays income tax on dividends paid, and the beneficiary pays income tax again on receipt. +A declining count over time is a sign that the company is returning capital to shareholders in the form of buy backs. Under ideal circumstances, it is more capital-efficient, for both company and shareholders, because distributions are double-taxed. + +The company pays income tax on dividends paid, and the beneficiary pays income tax again on receipt. A company will disclose how many shares are outstanding at the end of the period as a weighted average over the reporting period - three months. -Let's take a look at Target. To make the numbers easier to read, we'll divide the entire column by one million. +Let's take a look at Target. To make the numbers easier to read, we'll divide the entire column by one million. ```python data = ( @@ -140,7 +151,7 @@ shares.tail(1) |:--------------------|--------------------------------------:| | 2023-10-31 | 461.6 | -Thirty-seven years later, the share count is approaching a two-thirds reduction. 12.2% over the past five years. +Thirty-seven years later, the share count is approaching a two-thirds reduction. That is 12.2% over the past five years. ```python shares.pct_change(20).iloc[-1] @@ -174,8 +185,10 @@ round((price["close"].mean()*1300000)/1000000, 2) ```console 187.55 ``` +
-### Dividends Paid +
+Dividends Paid Dividends paid is in the cash flow statement. We can calculate the amount-per-share with the reported data. @@ -222,10 +235,14 @@ data.set_index("ex_dividend_date").loc["2023-08-15": "2022-11-15"] | 2023-08-15 | 1.1 | The numbers check out, and the $2B paid to investors over four quarters is more than ten times the $190M returned through share buy backs. +
+ +
+Financial Attributes -### Financial Attributes +The `openbb-intrinio` data extension has an endpoint for extracting a single fact from financial statements. -The `openbb-intrinio` data extension has an endpoint for extracting a single fact from financial statements. There is a helper function for looking up the correct `tag`. +There is a helper function for looking up the correct `tag`. #### Search Financial Attributes @@ -270,14 +287,19 @@ marketcap.index = marketcap.index.astype(str) ```console -0.24 ``` +
## Ratios and Other Metrics -Other valuation functions are derivatives of the financial statements, but the data provider does the math. Values are typically ratios between line items, on a per-share basis, or as a percent growth. +
+Other valuation functions are derivatives of the financial statements, but the data provider does the math. + +Values are typically ratios between line items, on a per-share basis, or as a percent growth. This data set is where you can find EPS, FCF, P/B, EBIT, quick ratio, etc. -### Quick Ratio +
+Quick Ratio Target's quick ratio could be one reason why its share price is losing traction against the market. Its ability to pay current obligations is not optimistically reflected in a 0.27 score, approximately 50% below the historical median. @@ -295,8 +317,10 @@ display(f"Median Quick Ratio: {ratios['quick_ratio'].median()}") Current Quick Ratio: 0.27 Median Quick Ratio: 0.58 ``` +
-### Free Cash Flow Yield +
+Free Cash Flow Yield The `metrics` endpoint, with the `openbb-fmp` data extension, has a field for free cash flow yield. It is calculated by taking the free cash flow per share divided by the current share price. We could arrive at this answer by writing some code, but these types of endpoints do the work so we don't have to. This is part of the value-add that API data distributors provide, they allow you to get straight to work with data. @@ -337,3 +361,6 @@ fcf_yield.transpose() | The TJX Companies, Inc. | 0.0271588 | 0.0234975 | 0.0517687 | 0.0401668 | 0.0488266 | 0.0399352 | 0.0536965 | 0.0433279 | 0.0464416 | 0.0406432 | Explore the rest of the `fundamental` module under the [Reference](/platform/reference/equity/fundamental) section. +
+ +
diff --git a/website/content/platform/usage/explanation/find_symbols.md b/website/content/platform/getting_started/find_symbols.mdx similarity index 98% rename from website/content/platform/usage/explanation/find_symbols.md rename to website/content/platform/getting_started/find_symbols.mdx index ab71c8da32e6..503442ca6b2a 100644 --- a/website/content/platform/usage/explanation/find_symbols.md +++ b/website/content/platform/getting_started/find_symbols.mdx @@ -1,9 +1,7 @@ --- -title: Finding Symbols +title: Finding Ticker Symbols sidebar_position: 3 -description: This page provides comprehensive information about finding stocks in the - with the OpenBB Platform. Search companies from different sources, and filter results. - This guide is intended to introduce some methods for searching, screening, and discovery. +description: This page provides comprehensive information about finding stocks in the with the OpenBB Platform. Search companies from different sources, and filter results. This guide is intended to introduce some methods for searching, screening, and discovery. keywords: - stocks - companies @@ -23,7 +21,7 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -Finding the ticker symbol, security identifier, the sector, and other metadata is easy if you know where to look. This guide is intended to introduce some methods for searching, screening, and discovery. +Finding the ticker symbol, security identifier, the sector, and other metadata is easy if you know where to look. This guide is intended to introduce some methods for searching, screening, and discovery. :::note For maximum coverage and functionality, install OpenBB with `[all]` packages. @@ -41,6 +39,8 @@ The simplest way to find tickers is with a basic text query. ## Search Nasdaq +
+ ```python obb.equity.search("JPMorgan", provider="nasdaq").to_df().head(3) ``` @@ -51,9 +51,12 @@ obb.equity.search("JPMorgan", provider="nasdaq").to_df().head(3) | 1 | BBAG | JPMorgan BetaBuilders U.S. Aggregate Bond ETF | Y | P | | Y | 100 | N | | BBAG | BBAG | N | | 2 | BBAX | JPMorgan BetaBuilders Developed Asia Pacific-ex Japan ETF | Y | Z | | Y | 100 | N | | BBAX | BBAX | N | +
## Search Cboe +
+ ```python obb.index.search("SPX", provider="cboe").to_df().tail(5) ``` @@ -66,8 +69,13 @@ obb.index.search("SPX", provider="cboe").to_df().tail(5) | 35 | WPUT | Cboe S&P 500 One-Week PutWrite Index | Tracks the value of a portfolio that overlays a short weekly SPX put on one-month Treasury bills | 15 | USD | America/Chicago | 08:00:00 | 16:00:00 | MonToFri | C | Regular | | 36 | XSPAM | Mini SPX Index (AM Settlement) | Mini SPX Index (AM Settlement) | 15 | USD | America/Chicago | 08:00:00 | 16:00:00 | MonToFri | C | Regular | +
+ ## Search ETFs + +
+ ```python obb.etf.search("gold", provider="tmx").to_df().iloc[-5:] ``` @@ -110,9 +118,13 @@ obb.etf.search("gold", provider="tmx").to_df().iloc[-5:] | beta_20y | nan | nan | 0.560996 | nan | nan | +
## Search the SEC + +
+ Use an empty string, `""`, to return the complete list - over 10,000. ```python @@ -216,8 +228,13 @@ obb.equity.fundamental.filings("AAPL", type="4", provider="sec").to_df().iloc[-1 | complete_submission_url | https://www.sec.gov/Archives/edgar/data/0000320193/0000320193-23-000109.txt | | filing_detail_url | https://www.sec.gov/Archives/edgar/data/0000320193/0000320193-23-000109-index.htm | +
+ ## Screen Markets + +
+ Screeners provide a targeted search, a tool for comparison and discovery. Find stocks from around the world with the screener endpoint, and the `openbb-fmp` provider. ### Find Stocks From India @@ -328,8 +345,12 @@ obb.equity.screener( | MRK | Merck & Co., Inc. | 258192673024 | Healthcare | Drug Manufacturers—General | 0.375 | 101.75 | 2.92 | 6760568 | NYSE | New York Stock Exchange | US | False | True | | VZ | Verizon Communications Inc. | 152314546478 | Communication Services | Telecom Services | 0.391 | 36.23 | 2.66 | 14960968 | NYSE | New York Stock Exchange | US | False | True | +
+ ## Get Available Indices +
+ List all indices from a source with: ```python @@ -382,3 +403,4 @@ With the `openbb-yfinance` extension, index time series can be loaded using the The examples above show demonstrate the most basic ways to find ticker symbols with the OpenBB Platform. Create your own custom scripts for discovery by combining these with other methods. +
diff --git a/website/content/platform/usage/explanation/historical_prices.md b/website/content/platform/getting_started/historical_prices.mdx similarity index 87% rename from website/content/platform/usage/explanation/historical_prices.md rename to website/content/platform/getting_started/historical_prices.mdx index 5244f2955dd0..c42dc73c1d9e 100644 --- a/website/content/platform/usage/explanation/historical_prices.md +++ b/website/content/platform/getting_started/historical_prices.mdx @@ -1,9 +1,7 @@ --- -title: Historical Prices +title: Loading Historical Price Data sidebar_position: 4 -description: This page provides an introduction to financial statement data available in the OpenBB - Platform. This includes quarterly and annual reports, along with metrics and ratios by company. - This guide provides examples for using the variety of sources. +description: This page provides an introduction to historical prices, including how to access and use them in the OpenBB Platform. keywords: - stocks - companies @@ -19,7 +17,9 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -Historical market prices typically come in the form of OHLC+V - open, high, low, close, volume. There may be additional fields returned by a provider, but those are the expected columns. Granularity and amount of historical data will vary by provider and subscription status. Visit their websites to understand what your entitlements are. +Historical market prices typically come in the form of OHLC+V - open, high, low, close, volume. There may be additional fields returned by a provider, but those are the expected columns. + +Granularity and amount of historical data will vary by provider and subscription status. Visit their websites to understand what your entitlements are. :::info These examples will assume that the OpenBB Platform is initialized in a Python session. @@ -33,6 +33,9 @@ import pandas as pd ## Historical OHLC + +
+ The `historical` function is located under a submodule for each asset type. In the `openbb-equity` module. ```python @@ -76,7 +79,10 @@ df_daily.head(1) |:--------------|-------:|-------:|------:|--------:|-----------:|------------:|---------------:|----------------:| | 1993-01-29 | 43.97 | 43.97 | 43.75 | 43.94 | 1003200 | 0 | 0 | 0 | -### Intervals + +
+Intervals + The intervals are entered according to this pattern: @@ -101,7 +107,12 @@ df_monthly.tail(2) | 2023-10-01 | 426.62 | 438.14 | 409.21 | 418.2 | 1999149700 | 0 | 0 | 0 | | 2023-11-01 | 419.2 | 456.38 | 418.65 | 455.02 | 1161239576 | 0 | 0 | 0 | -### Resample a Time Series + +
+ + +
+Resample a Time Series `yfinance` returns the monthly data for the first day of each month. Let's resample it to take from the last, using the daily information captured in the previous cells. @@ -134,7 +145,13 @@ df_daily.loc["2023-11-01":].sum()["volume"] 1210484176 ``` -### Differences Between Sources +
+ + + +
+Differences Between Sources + To demonstrate the difference between sources, let's compare values for daily volume from several sources. @@ -169,11 +186,20 @@ compare | 2023-11-21 | 49244639 | 49244639 | 49244639 | 49244600 | | 2023-11-22 | 59446573 | 59313820 | 58205780 | 59394900 | +
+ +
+ ## Other Types of Symbols + +
+ Other types of assets and ticker symbols can be loaded from `obb.equity.price.historical()`, below are some examples but not an exhaustive list. -### Share Classes +
+Share Classes + Some sources use `-` as the distinction between a share class, e.g., `BRK-A` and `BRK-B`. Other formats include: @@ -189,15 +215,33 @@ obb.equity.price.historical("brk.b", provider="polygon") obb.equity.price.historical("brk-b", provider="fmp") ``` -While some providers handle the different formats on their end, others do not. This is something to consider when no results are returned from one source. Some may even use a combination, or accept multiple variations. Sometimes there is no real logic behind the additional characters, `GOOGL` vs. `GOOG`. These are known unknown variables of ticker symbology, what's good for one source may return errors from another. +While some providers handle the different formats on their end, others do not. + +This is something to consider when no results are returned from one source. -### Regional Identifiers +Some may even use a combination, or accept multiple variations. Sometimes there is no real logic behind the additional characters, `GOOGL` vs. `GOOG`. -With providers supporting market data from multiple jurisdictions, the most common method for requesting data outside of US-listings is to append a suffix to the ticker symbol (e.g., `RELIANCE.NS`). Formats may be unique to a provider, so it is best to review the source's documentation for an overview of their specific conventions. [This page](https://help.yahoo.com/kb/SLN2310.html) on Yahoo describes how they format symbols, which many others follow to some degree. +These are known unknown variables of ticker symbology, what's good for one source may return errors from another. + +
+ + +
+Regional Identifiers + + +With providers supporting market data from multiple jurisdictions, the most common method for requesting data outside of US-listings is to append a suffix to the ticker symbol (e.g., `RELIANCE.NS`). + +Formats may be unique to a provider, so it is best to review the source's documentation for an overview of their specific conventions. + +[This page](https://help.yahoo.com/kb/SLN2310.html) on Yahoo describes how they format symbols, which many others follow to some degree. `openbb-tmx` follows the composite convention, "SPY:US". When the symbol is for its domestic Canadian market, "CNQ", no identifier is required. -### Indices +
+ +
+Indices Sources will have their own treatment of these symbols, some examples are: @@ -224,7 +268,10 @@ obb.equity.price.historical("^RUT", provider="fmp").to_df().tail(1) **For an endpoint geared more specifically towards indices, try `obb.index.price.historical()`** ::: -### Currencies +
+ +
+Currencies FX symbols face the same dilemma as share classes, there are several variations of the same symbol. @@ -252,7 +299,12 @@ obb.equity.price.historical("C:EURUSD", provider="polygon").to_df().tail(1) |:--------------|--------:|-------:|-------:|--------:|---------:|-------:|---------------:| | 2023-11-21 | 1.09168 | 1.0923 | 1.0851 | 1.0888 | 155827 | 1.0893 | 155827 | -### Crypto +
+ + +
+Crypto + Similar, but different to FX tickers. @@ -282,7 +334,11 @@ obb.crypto.price.historical("BTCUSD", provider="polygon").to_df().tail(1) |:--------------|-------:|-------:|------:|--------:|---------:|--------:|---------------:| | 2023-11-21 | 35756 | 37900 | 35633 | 37433.8 | 30411.4 | 36841.5 | 464907 | -### Futures +
+ +
+Futures + Historical prices for the continuation chart, can be fetched by the `fmp` or `yfinance` data extensions. Individual active contracts are returned by `yfinance`. @@ -308,7 +364,10 @@ obb.equity.price.historical("CLZ24.NYM", provider="yfinance").to_df().tail(1) |:--------------|-------:|-------:|------:|--------:|---------:|------------:|---------------:| | 2023-11-22 | 74.07 | 74.07 | 73.41 | 73.46 | 610 | 0 | 0 | -### Options +
+ +
+Options Individual options contracts are also loadable from `openbb.equity.price.historical()`. @@ -330,3 +389,7 @@ obb.equity.price.historical("O:SPY241220P00400000", provider="polygon").to_df(). | date | open | high | low | close | volume | vwap | transactions | |:--------------|-------:|-------:|------:|--------:|---------:|--------:|---------------:| | 2023-11-20 | 10.9 | 10.95 | 10.75 | 10.75 | 17 | 10.8376 | 10 | + +
+ +
diff --git a/website/content/platform/development/how-to/index.md b/website/content/platform/getting_started/index.mdx similarity index 55% rename from website/content/platform/development/how-to/index.md rename to website/content/platform/getting_started/index.mdx index 488c0d74aa97..22221f79f3b6 100644 --- a/website/content/platform/development/how-to/index.md +++ b/website/content/platform/getting_started/index.mdx @@ -1,6 +1,5 @@ --- -title: How To Guides -sidebar_position: 1 +title: Getting started description: The pages in this section outline different processes for building with the OpenBB Platform. Guides cover adding data, toolkit and OBBject extensions. keywords: - OpenBB Platform @@ -28,12 +27,6 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -The pages in this section provide practical examples for getting started building with, or contributing to, the OpenBB Platform. +The pages in this section provide practical examples for getting started with the OpenBB Platform. -Topics covered include: - -- [Creating a new data provider extension](how-to/add_data_provider_extension) -- [Extending the OBBject class](how-to/add_obbject_extension) -- [Add new data to an existing endpoint](how-to/add_data_to_existing_endpoint) -- [Build a new router path](how-to/add_toolkit_extension) -- [How to make a new standard model](how-to/add_endpoint_to_existing_provider#how-to-build-a-standard-model) +Jupyter Notebook examples are also hosted in the OpenBB GitHub repository [here](https://github.com/OpenBB-finance/OpenBBTerminal/tree/develop/examples) diff --git a/website/content/platform/getting_started/map_a_provider_to_a_route.mdx b/website/content/platform/getting_started/map_a_provider_to_a_route.mdx new file mode 100644 index 000000000000..8ef9df71d89d --- /dev/null +++ b/website/content/platform/getting_started/map_a_provider_to_a_route.mdx @@ -0,0 +1,240 @@ +--- +title: Fetch Data From a New Provider +sidebar_position: 10 +description: This page demonstrates how to get started mapping Provider extension models to the Provider Interface and Router. By the end, you will have mapped a new Provider extension to World News standard model. +keywords: + - OpenBB Platform + - Open source + - community + - code + - provider + - openbb-provider + - data + - extension + - how-to + - new + - template + - quick start + - quickstart + - provider interface +--- + +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; + + + +This page demonstrates how to get started mapping Provider extension models to the Provider Interface and Router. + +It continues the example provided [here](create_new_provider_extension). By the end, you will have mapped a new Provider extension to the World News standard model. This will connect it to the existing command and make it accessible through the `provider` parameter. + +All Provider models are made of three elements: + +- QueryParams +- Data +- Fetcher + +## Start Mapping + +### Create Model File + +- In the `/models` folder of the extension - `/Users/username/path_to_created_folder/empty_provider/openbb_empty_provider/models` - create a new file called, "world_news.py". +- Start the file with a docstring, then a new line. + +```python +"""Empty World News Model.""" + +``` + +### Import Statements + +The import schema will follow a pattern similar to this, depending on the specific requirements of the function. + +```python +from typing import Any, Dict, List, Literal, Optional + +from openbb_core.provider.abstract.fetcher import Fetcher +from openbb_core.provider.standard_models.world_news import WorldNewsQueryParams, WorldNewsData + +from pydantic import Field +``` + +### Create a QueryParams Model + +- Create a class that inherits from the standard model. + +```python +class EmptyWorldNewsQueryParams(WorldNewsQueryParams): + """Empty World News Query Params.""" + + some_param: Optional[str] = Field( + default=None, + description="Some Empty Parameter.", + ) +``` + +Field and model validators are added here, if required. + +### Create a Data Model + +- Create a class that inherits from the standard model. + +```python +class EmptyWorldNewsData(WorldNewsData): + """Empty World News Data""" + + some_param: Optional[str] = Field( + default=None, + description="Some Empty Data Field." + ) +``` + +Field and model validators are added here, if required. + +### Build Fetcher + +The Fetcher consists of three stages: + +- Transform Query +- Extract Data +- Transform Data + +Each stage is a static method within the Fetcher class that inherits the QueryParams and Data models. + +The structure of the methods within should always be the same, and input/output types should match throughout. + +```python +class EmptyWorldNewsFetcher( + Fetcher[ + EmptyWorldNewsQueryParams, + List[EmptyWorldNewsData], + ] +): + """Empty World News Fetcher.""" + + @staticmethod + def transform_query(params: Dict[str, Any]) -> EmptyWorldNewsQueryParams: + """Transform query params.""" + new_params = params + if params.get("some_param"): + new_params["some_param"] = "do_something" + return EmptyWorldNewsQueryParams(**new_params) + + @staticmethod + async def aextract_data( # The function does not have to be asynchronous. If not, remove the leading 'a', 'extract_data'. + query: EmptyWorldNewsQueryParams, + credentials: Optional[Dict[str, str]], + **kwargs: Any, + ) -> List[Dict]: + """Extract data. Put HTTP Requests here. This should return the closest form of raw data possible.""" + results = { + "date": datetime.now().date(), + "title": "Hello from the Empty Provider extension!", + "some_param": query.some_param, + } + return [results] + + @staticmethod + def transform_data( + query: EmptyWorldNewsQueryParams, + data: List[Dict], + **kwargs: Any, + ) -> List[EmptyWorldNewsData]: + """Transform data. Put parsing logic here, if required. This should return validated data models.""" + return [EmptyWorldNewsData.model_validate(d) for d in data] +``` + +## 2. Map Model To Provider Interface + +- In the `__init__.py` where the Provider class was defined, import the Fetcher from the model file and map it to the router. + +```python +# /Users/username/path_to_created_folder/empty_provider/openbb_empty_provider/__init__.py +"""Empty Provider Module.""" + +from openbb_empty_provider.models.world_news import EmptyWorldNewsFetcher +from openbb_core.provider.abstract.provider import Provider + +empty_provider = Provider( + name="empty", + website="http://empty.io", + description="""The empty provider is a supplier of promises.""", + # credentials=["api_key"], + fetcher_dict={ + "WorldNews": EmptyWorldNewsFetcher + }, +) +``` + +## 3. Rebuild Static Assets + +- The Python Interface and static assets need to be rebuilt after these changes. + +```bash +python -c "import openbb; openbb.build()" +``` + +## 4. Smoke Test + +The function's docstring should list the new provider, and the command should return an `OBBject` instance. + +```console + Parameters + ---------- + limit : int + The number of data entries to return. The number of articles to return. + start_date : Union[datetime.date, None, str] + Start date of the data, in YYYY-MM-DD format. + end_date : Union[datetime.date, None, str] + End date of the data, in YYYY-MM-DD format. + provider : Optional[Literal['benzinga', 'empty', 'fmp', 'intrinio', 'tiingo'... + The provider to use for the query, by default None. +``` + +```python +from openbb import obb +obb.news.world(provider="empty") +``` + +```console +OBBject + +id: 066356fc-9661-7448-8000-5885e5120efb +results: [{'date': datetime.datetime(2024, 5, 3, 0, 0), 'title': 'Hello from the Em... +provider: empty +warnings: None +chart: None +extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'empty'}, 'stan... +``` + +The parameters supplied are captured under `results.extra["metadata"].arguments`. + +```python +response = obb.news.world(provider="empty", some_param="adjustment") +response.extra["metadata"].arguments +``` + +```console +{ + 'provider_choices': {'provider': 'empty'}, 'standard_params': {'limit': 2500, 'start_date': None, 'end_date': None}, + 'extra_params': {'some_param': 'adjustment'} +} +``` + +The transformed parameters are passed through to the function. + +```python +response.results +``` + +```console +>>> response.results +[EmptyWorldNewsData(date=2024-05-03 00:00:00, title=Hello from the Empty Provider extension!, images=None, text=None, url=None, some_param=do_something)] +``` + +## Conclusion + +This process created, built, and mapped a new OpenBB Platform Provider model to an existing router endpoint and standard model. + +The output was validated and returned as an instance of `OBBject`. + +Create a new model for each endpoint needing to be mapped. diff --git a/website/content/platform/usage/explanation/market_calendars.md b/website/content/platform/getting_started/market_calendars.mdx similarity index 94% rename from website/content/platform/usage/explanation/market_calendars.md rename to website/content/platform/getting_started/market_calendars.mdx index f129838fc677..656b7a960da9 100644 --- a/website/content/platform/usage/explanation/market_calendars.md +++ b/website/content/platform/getting_started/market_calendars.mdx @@ -1,9 +1,7 @@ --- title: Market Calendars -sidebar_position: 5 -description: This page provides details on the market calendars available in the OpenBB - Platform. Equity and economic calendars keep investors abreast of market activity and events. - This guide provides examples for using the variety of calendars, and differences between sources. +sidebar_position: 6 +description: This page provides details on the market calendars available in the OpenBB Platform. Equity and economic calendars keep investors abreast of market activity and events. This guide provides examples for using the variety of calendars, and differences between sources. keywords: - stocks - companies @@ -55,6 +53,9 @@ import pandas as pd ## Economic Calendar +
+ + The economic calendar aggregates global central bank and macroeconomic releases, it is located within the `obb.economy` module. :::tip @@ -65,7 +66,7 @@ Do not rely on the economic calendar for real-time updates. Times posted are sch There are subtle differences between providers, the main consideration will be the timestamp. FMP and TradingEconomics both return the calendar as UTC-0, while Nasdaq posts events in US/Eastern time. Of the three, only TradingEconomics provides a TZ-aware timestamp. The differences can be reconciled with a few lines of code. -To identify the issue, let's look at one event. First, from FMP: +To identify the issue, let's look at one event. First, from FMP: ```python fmp_df = obb.economy.calendar(provider="fmp", start_date="2023-11-19", end_date="2023-11-20").to_df() @@ -105,7 +106,7 @@ fmp_df[fmp_df["event"].str.contains("20-Year Bond Auction")] |:--------------------------|:----------|:---------------------|-----------:|------------:|:-------------|:-----------|---------:|-----------------:| | 2023-11-20 13:00:00-05:00 | US | 20-Year Bond Auction | 5.245 | nan | Low | USD | nan | 0 | -Timestamps can be a factor with start/end dates because the calendar day will roll over at midnight, moving the date. Converting the timestamp will overcome this, but be aware of when the time is `00:00:00`, signifying an all-day event like a holiday. +Timestamps can be a factor with start/end dates because the calendar day will roll over at midnight, moving the date. Converting the timestamp will overcome this, but be aware of when the time is `00:00:00`, signifying an all-day event like a holiday. An exception was added in the code above to maintain the time where applicable, instead of rolling it back five hours. @@ -151,8 +152,12 @@ events[events["event"].str.contains("ISM Manufacturing New Orders")] | 2023-11-01 14:00:00 | US | ISM Manufacturing New Orders (Oct) | 45.5 | 49.2 | nan | Low | USD | -3.7 | -7.52 | | 2023-12-01 15:00:00 | US | ISM Manufacturing New Orders (Nov) | nan | 45.5 | nan | Low | USD | nan | 0 | +
+ ## Earnings Calendar +
+ The earnings calendar works in a similar way. For companies outside of the US, try the `openbb-fmp` provider. ```python @@ -193,9 +198,15 @@ This returned 1,234 results, but let's filter it down to those companies with an EPS values are reported in the currency of the exchange listing price, direct comparisons are not viable across domiciles without a conversion factor. ::: +
+ ## Dividend Calendar -The dividend calendar uses start/end dates that reflect the ex-dividend date - the date when it begins trading without dividend rights. Aside from the notable dates, the information returned tells you only the amount paid. Calculating the yield requires more data. +
+ +The dividend calendar uses start/end dates that reflect the ex-dividend date - the date when it begins trading without dividend rights. + +Aside from the notable dates, the information returned tells you only the amount paid. Calculating the yield requires more data. :::note @@ -204,11 +215,14 @@ The dividend calendar uses start/end dates that reflect the ex-dividend date - t - The `openbb-nasdaq` provider has US-only data for this endpoint. - The same markets covered by FMP's earnings calendar are included in their dividend calendar. + ::: ### Calculate Dividend Yield -The ten highest-payments going ex-div between November 20-24 are shown below. With T+2 settlement, a purchase needs to occur two days prior to the record date for payment eligibility. The dividend yield is the current payment annualized as a percent of the asset's price. +The ten highest-payments going ex-div between November 20-24 are shown below. + +With T+2 settlement, a purchase needs to occur two days prior to the record date for payment eligibility. The dividend yield is the current payment annualized as a percent of the asset's price. ```python dividends = ( @@ -255,8 +269,13 @@ dividends["yield"] = ( | CBCYB | 2023-11-24 | 2023-12-01 | 3.75 | 8 | 647 | 1.2365 | | CBCY | 2023-11-24 | 2023-12-01 | 3.75 | 8 | 660 | 1.2121 | + +
+ ## IPO Calendar +
+ The IPO calendar shows events based on their status - `["upcoming", "priced", "withdrawn"]` - and Intrinio provides a filter for the min/max dollar amount offered. :::note @@ -302,3 +321,6 @@ obb.equity.calendar.ipo(provider="nasdaq", is_spo=True).to_df().tail(1) | symbol | ipo_date | name | offer_amount | share_count | deal_status | id | exchange | share_price | |:---------|:-----------|:----------------------------------------|---------------:|--------------:|:--------------|:--------------|:---------------------|--------------:| | SKWD | 2023-11-16 | Skyward Specialty Insurance Group, Inc. | 152500000.0 | 5000000 | Priced | 854131-108262 | NASDAQ Global Select | 30.5 | + + +
diff --git a/website/content/platform/getting_started/quickstart.mdx b/website/content/platform/getting_started/quickstart.mdx new file mode 100644 index 000000000000..47c105dee3da --- /dev/null +++ b/website/content/platform/getting_started/quickstart.mdx @@ -0,0 +1,76 @@ +--- +title: Quickstart +sidebar_position: 1 +description: Get started with the OpenBB Platform by following this quickstart guide. +keywords: +- OpenBB Platform +- investment research infrastructure +- data connectors +- financial reports +- OpenBB team +- quickstart +- getting started +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +To get started with the OpenBB Platform, all you need to do is to import `obb` and start querying away. + +```python +from openbb import obb + +# Get the price of a stock +quote_data = obb.equity.price.quote(symbol="AAPL", provider="yfinance") +quote_data +``` + +Out output will look like this: + +```console +OBBject + +id: 06649f4e-896c-7b31-8000-52242b1605f2 +results: [{'symbol': 'AAPL', 'asset_type': 'EQUITY', 'name': 'Apple Inc.', 'exchang... +provider: yfinance +warnings: None +chart: None +extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'yfinance'}, 's... +``` + +To view the output as a dataframe, you can use the `to_df()` method. + +```python +quote_data.to_df() +``` + +Let's try another example. This time, we'll get the historical price of a stock. + +```python +obb.equity.price.historical(symbol="AAPL", provider="yfinance").to_df() +``` + +To view all the available commands, routers and extensions, you can do: + +```python +obb +``` + +You can also keep exploring by accessing each route like this: + +```python +obb.equity +``` + +If you see a command you're interested in, to get help on how to use it, you can do: + +```python +help(obb.equity.price.historical) +``` + +Visit our [reference](/platform/reference) documentation to see all the available commands and their parameters. + +And that's it! You're now ready to start using the OpenBB Platform. + +![Platform Docs pic](https://github.com/OpenBB-finance/OpenBBTerminal/assets/85772166/74520441-5e95-4ba6-9d16-6a2d5c966cf9) diff --git a/website/content/platform/index.md b/website/content/platform/index.md deleted file mode 100644 index d60117ce5ada..000000000000 --- a/website/content/platform/index.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -title: Introduction -sidebar_position: 0 -description: Introduction to OpenBB Platform; a convenient and powerful tool that - provides pre-built data connectors and libraries to design and build financial reports - and applications. Learn more about contributing to the platform. -keywords: -- OpenBB Platform -- investment research infrastructure -- data connectors -- financial reports -- OpenBB team -- third-party data providers -- CONTRIBUTING GUIDELINES ---- - - - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -The OpenBB Platform has been created and is currently maintained by the OpenBB team together with the contributions from hundreds of community members. This gives us an unrivaled speed of development and the ability to maintain stable integrations with numerous third-party data providers. - -Developing and maintaining a full-blown investment research infrastructure from the ground up takes a lot of time and effort. However, it does not have to be this way. By taking advantage of OpenBB Platform with its out-of-the-box data connectors and library of extensions, you can focus on designing and building your financial reports and applications. - -![Platform Docs pic](https://github.com/OpenBB-finance/OpenBBTerminal/assets/85772166/74520441-5e95-4ba6-9d16-6a2d5c966cf9) - -## What is the OpenBB Platform? - -Starting with V4, we have completely restructured the previous version of the OpenBB SDK. -Instead of a single monolithic SDK, that comes with dependency nightmares and compatibility issues with things you may not need, we have morphed into the OpenBB Platform, which serves as a collection of building blocks to be used for your own need. - -We have broken the Platform into two main components: - -- OpenBB Core -- OpenBB Extensions - - OpenBB Providers - - OpenBB Toolkits - -The OpenBB Core provides all of the groundwork for building custom applications. It is a lightweight package providing connections with data providers in a standardized way, and the ability to build additional toolkits on top. Additionally, the `openbb-core` library contains the infrastructure for Fast API deployments. - -OpenBB Extensions are the pieces that can be added to the core for building on top of. We have grouped them categorically as, Providers and Toolkits. -As the name suggests, Providers are the interface to obtaining any raw data from any data source. The data providers are accessed through asset class extensions, such as `openbb-equity`. OpenBB toolkits, such as `openbb-technical`, provide additional functionalities on top of the raw data access. - -When you install the Platform (openbb), we provide a default set of extensions that give access to a wide range of functionalities and data. Additional data and processing tools are available to be installed and used as desired. The reason we are doing this is to avoid an overcomplicated environment with many dependencies that can cause issues. The goal is that OpenBB can be used as a drop-in within any environment for building and extending applications. - ---- - -Want to contribute? Check out our [Development section](/platform/development). diff --git a/website/content/platform/index.mdx b/website/content/platform/index.mdx new file mode 100644 index 000000000000..9ade1a95cb3e --- /dev/null +++ b/website/content/platform/index.mdx @@ -0,0 +1,66 @@ +--- +title: Introduction +sidebar_position: 0 +description: Introduction to OpenBB Platform; a convenient and powerful tool that + provides pre-built data connectors and libraries to design and build financial reports + and applications. Learn more about contributing to the platform. +keywords: +- OpenBB Platform +- investment research infrastructure +- data connectors +- financial reports +- OpenBB team +- third-party data providers +- CONTRIBUTING GUIDELINES +--- + +{/* markdownlint-disable MD012 MD031 MD033 */} + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import NewReferenceCard from "@site/src/components/General/NewReferenceCard"; + + + +The OpenBB Platform has been created and is currently maintained by the OpenBB team together with the contributions from hundreds of community members. + +With its ready-to-use data connectors and a wealth of extensions, it lets you concentrate on creating outstanding financial reports and applications quickly and easily. + +## Documentation Structure + +
    + + + + + + + +
diff --git a/website/content/platform/installation.md b/website/content/platform/installation.mdx similarity index 78% rename from website/content/platform/installation.md rename to website/content/platform/installation.mdx index 66cae1748b4a..f0a5be3f4f01 100644 --- a/website/content/platform/installation.md +++ b/website/content/platform/installation.mdx @@ -40,7 +40,7 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; ## General System Requirements -Most systems capable of running Python 3.8-3.11 will be compatible with the OpenBB Platform. A modern processor (five years or less), running an up-to-date operating system, with at least 4GB of RAM, is recommended. Maintaining the system with current patches ensures maximum compatibility. At a minimum, Windows and macOS should be: +Most systems capable of running Python 3.9-3.11 will be compatible with the OpenBB Platform. A modern processor (five years or less), running an up-to-date operating system, with at least 4GB of RAM, is recommended. Maintaining the system with current patches ensures maximum compatibility. At a minimum, Windows and macOS should be: - Windows 10 - Mac OS Big Sur @@ -49,7 +49,7 @@ Linux users should run the command line update for the package manager, prior to ## Supported Environments -The OpenBB Platform is installed within a Python virtual environment. It is compatible with versions of Python between 3.8 and 3.11, inclusively. The method for creating the environment will be a matter of user preference, from the command line - [Conda](https://docs.conda.io/projects/miniconda/en/latest/miniconda-install.html), [venv](https://docs.python.org/3/library/venv.html), etc. - or in a code editor and IDE - [VS Code](https://code.visualstudio.com/docs/languages/python), [PyCharm](https://www.jetbrains.com/pycharm/), [Jupyter](https://jupyter.org/). +The OpenBB Platform is installed within a Python virtual environment. It is compatible with versions of Python between 3.9 and 3.11, inclusively. The method for creating the environment will be a matter of user preference, from the command line - [Conda](https://docs.conda.io/projects/miniconda/en/latest/miniconda-install.html), [venv](https://docs.python.org/3/library/venv.html), etc. - or in a code editor and IDE - [VS Code](https://code.visualstudio.com/docs/languages/python), [PyCharm](https://www.jetbrains.com/pycharm/), [Jupyter](https://jupyter.org/). If you're interested in using the [Docker](/platform/installation#docker) container, skip ahead to the specific section [below](/platform/installation#docker). @@ -75,6 +75,7 @@ conda activate openbb ### PyPI +
Install from PyPI with: ```console @@ -123,6 +124,15 @@ From your python interpreter, import the OpenBB Platform: from openbb import obb ``` +:::warning +This import statement is required due to the statefulness of the obb package. There is currently no support for imports such as: + +```console +from openbb.obb.equity import * +``` + +::: + When the package is imported, any installed extensions will be discovered, imported and available for use. :::note @@ -134,8 +144,29 @@ pip install openbb-core && pip install openbb --no-deps ::: +To update the package: + +```console +pip install --upgrade openbb +``` + +To update all extensions and providers: + +```console +pip install --upgrade openbb[all] +``` + +If you want to uninstall the package and all its dependencies: + +```console +pip uninstall openbb[all] +``` + +
+ ### Docker +
OpenBB provides a `.dockerfile` on [GitHub](https://github.com/OpenBB-finance/OpenBBTerminal). Run the following command from the repo root to build the image: @@ -151,9 +182,11 @@ docker run --rm -p 8000:8000 -v ~/.openbb_platform:/root/.openbb_platform openbb ``` This will mount the local `~/.openbb_platform` directory into the Docker container to use with the API keys and preferences from there, and it will expose the API on port `8000`. +
### Source +
To build the OpenBB Platform from the source code, first install `git`: ```console @@ -193,6 +226,7 @@ python dev_install.py :::note To install all extensions and providers, run: `python dev_install.py -e` ::: +
### Devcontainer @@ -207,11 +241,13 @@ The OpenBB Platform can be run in a [Devcontainer](https://code.visualstudio.com ## Post-Installation -With a fresh installation, and upon installing or uninstalling extensions, the Python interface needs to be built. This is done automatically, but can be manually triggered if required. Start a Python session and then `import openbb`: +With a fresh installation, and upon installing or uninstalling extensions, the Python interface needs to be built. This is done automatically, but can be manually triggered if required. Start a Python session and import openbb: ```console python +``` +```python from openbb import obb exit() @@ -236,11 +272,11 @@ Start the REST API with: uvicorn openbb_core.api.rest_api:app --host 0.0.0.0 --port 8000 --reload ``` -See more information about using the REST API in the [usage section](/platform/usage/rest_api) +See more information about using the REST API in the [usage section](/platform/user_guides/rest_api) ## Hub Synchronization -Once you have installed the OpenBB Platform with the desired providers and extensions, you can synchronize with the [OpenBB Hub](my.openbb.co). The main benefit of this is that you can use your single login to access your saved credentials and preferences from any instance. To login, you can use the `login` method, either using your email and password: +Once you have installed the OpenBB Platform with the desired providers and extensions, you can synchronize with the [OpenBB Hub](https://my.openbb.co/app/hub). The main benefit of this is that you can use your single login to access your saved credentials and preferences from any instance. To login, you can use the `login` method, either using your email and password: ```python obb.account.login(email='my_email_here', password='my_password_here') @@ -251,51 +287,3 @@ Or using your personal access token: ```python obb.account.login(pat='my_pat_here') ``` - -## Documentation - -The documentation and packages are kept in the `/website` folder, at the base of the repository. Navigate there to install the dependencies and start the development server. - -### Generate Markdown Files - -Markdown files for the Reference and Data Models pages need to be generated. -This will generate content based on what is actually installed and contained locally, so it may appear different than what is on this website. - -Open a terminal command line to the `/OpenBBTerminal/website` folder, then enter: - -```bash -python generate_platform_v4_markdown.py -``` - -### Node.js - -- [Node.js](https://nodejs.org/en/) >= 16.13.0 - To check if Node.js installed, run this command: - -```bash -node --version # should be v16.13.0 or higher -``` - -#### Install Dependencies - -```bash -npm install -``` - -#### Start Development Server - -```bash -npm start -``` - -This starts a local development server at: [http://localhost:3000](http://localhost:3000) - -Most changes are reflected live without having to restart the server. - -#### Build - -```bash -npm run build -``` - -This command generates static content into the `build` directory and can be served using any static contents hosting service. OpenBB uses Github Pages to host our website, it's deployed in the `gh-pages` branch. diff --git a/website/content/platform/licensing/_category_.json b/website/content/platform/licensing/_category_.json index bdef9d44818e..cac9b372c312 100644 --- a/website/content/platform/licensing/_category_.json +++ b/website/content/platform/licensing/_category_.json @@ -1,4 +1,4 @@ { "label": "Licensing", - "position": 9 + "position": 7 } diff --git a/website/content/platform/licensing/index.mdx b/website/content/platform/licensing/index.mdx index a1155089c580..8c9bc66a8cf7 100644 --- a/website/content/platform/licensing/index.mdx +++ b/website/content/platform/licensing/index.mdx @@ -73,7 +73,7 @@ A: If you modify the OpenBB Platform and do not distribute your modified version
Q: We want to contribute to the OpenBB project. How does the licensing affect our contributions? -A: This doesn’t change. Contributions to the OpenBB project are very welcome under our existing [Contributor License Agreement (CLA)](https://cla-assistant.io/OpenBB-finance/OpenBBTerminal). This means any contributions you make will also be licensed under the AGPL, ensuring they remain free and open. +A: This doesn’t change. Contributions to the OpenBB project are very welcome. Contributions to the [main GitHub repository](https://github.com/OpenBB-finance/OpenBBTerminal) are accepted under our existing [Contributor License Agreement (CLA)](https://cla-assistant.io/OpenBB-finance/OpenBBTerminal). This means any contributions that are accepted into the main repository will be re-licensed by us under the AGPL, ensuring they remain free and open.
diff --git a/website/content/platform/reference/_category_.json b/website/content/platform/reference/_category_.json new file mode 100644 index 000000000000..7805441d8282 --- /dev/null +++ b/website/content/platform/reference/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Reference", + "position": 4 +} diff --git a/website/content/platform/usage/_category_.json b/website/content/platform/usage/_category_.json deleted file mode 100644 index 0365f6c34e4f..000000000000 --- a/website/content/platform/usage/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "Usage", - "position": 3 -} diff --git a/website/content/platform/usage/basic_response.md b/website/content/platform/usage/basic_response.md deleted file mode 100644 index 42623fe110a5..000000000000 --- a/website/content/platform/usage/basic_response.md +++ /dev/null @@ -1,138 +0,0 @@ ---- -title: Basic Response & Command Coverage -sidebar_position: 4 -description: This page details the basic response and output that can be expected to be received from the the OpenBB Platform, as well as instructions for determining the command coverage provided by the installed extensions. -keywords: -- standardized output -- OBBject -- basic response -- provider -- results -- warnings -- chart -- extra -- command coverage ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -The output of every command is an object which contains the results of the request, along with additional information. It is a custom class, `OBBject`, and always returns with the fields listed below: - -```console -id: ... # UUID Tag -results: ... # Serializable results. -provider: ... # Provider name. -warnings: ... # List of warnings. -chart: ... # Chart object. -extra: ... # Extra info. -``` - -```python -from openbb import obb - -data = obb.equity.price.historical("SPY", provider="polygon") - -data -``` - -```console -OBBject - -id: 06520558-d54a-7e53-8000-7aafc8a42694 -results: [{'date': datetime.datetime(2022, 10, 5, 0, 0), 'open': 375.62, 'high': 37... -provider: polygon -warnings: None -chart: None -extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'polygon'}, 'st... -``` - -Additional class methods are helpers for converting the results to a variety of formats. - -- `to_dict()`: converts to a dictionary, accepting all standard "orientation" parameters, i.e., "records" -- `to_df()` / `to_dataframe()`: converts to a Pandas DataFrame. -- `to_numpy()`: converts to a Numpy array. -- `to_polars()`: converts to a Polars table. - -The output from the Fast API is a serialized version of this object, and these methods are lost on conversion. OBBject can be reconstructed to recover the helpers by importing the model and validating the data. - -```python -import requests -from openbb_core.app.model.obbject import OBBject - -data = [] -symbol="SPY" -url = f"http://127.0.0.1:8000/api/v1/equity/price/historical?provider=polygon&symbol={symbol}" -headers = {"accept": "application/json"} - -response = requests.get(url, headers=headers, timeout=3) - -if response.status_code == 200: - data = OBBject.model_validate(response.json()) - -data.to_df() -``` - -:::info -The preferred output type can be set with a user preference. - -```python -obb.user.preferences.output_type="dataframe" -``` -::: - -## Dynamic Command Execution - -Dynamic execution provides an alternate entry point to functions. This method requires formatting the query as demonstrated below. - -```python -from openbb_core.app.command_runner import CommandRunner -runner = CommandRunner() -output = await runner.run( - "/equity/fundamental/ratios", - provider_choices={ - "provider": "fmp", - }, - standard_params={ - "symbol" : "TSLA", - "period" : "quarter", - }, - extra_params={} -) -``` - -```console ->>> output -OBBject - -id: 065241b7-bd9d-7313-8000-9406d8afab75 -results: [{'symbol': 'TSLA', 'date': '2023-06-30', 'period': 'Q2', 'current_ratio':... -provider: fmp -warnings: None -chart: None -extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'fmp'}, 'standa... -``` - -## Commands and Provider Coverage - -The installed commands and data providers are found under, `obb.coverage`. - -```python -obb.coverage -``` - -```console -/coverage - - providers - commands - command_model - command_schemas -``` - -`obb.coverage.providers` is a dictionary of the installed provider extensions, each with its own list of available commands. - -`obb.coverage.commands` is a dictionary of commands, each with its own list of available providers for the data. - -`obb.coverage.command_model` is a dictionary where the keys are the command paths and the values is a nested dictionary of QueryParams and Data models associated with that function. diff --git a/website/content/platform/usage/explanation/_category_.json b/website/content/platform/usage/explanation/_category_.json deleted file mode 100644 index 43f047c48ddc..000000000000 --- a/website/content/platform/usage/explanation/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "Explanation", - "position": 6 - } diff --git a/website/content/platform/usage/explanation/index.md b/website/content/platform/usage/explanation/index.md deleted file mode 100644 index 5dbc13c7d7e8..000000000000 --- a/website/content/platform/usage/explanation/index.md +++ /dev/null @@ -1,35 +0,0 @@ ---- -title: Explanation -sidebar_position: 1 -description: This section provides explanations of concepts related to usage and getting started with the OpenBB Platform. - The pages include example syntax and outline what you can expect while working with the OpenBB Python interface and REST API. -keywords: -- getting started -- explanation -- guide -- syntax -- examples -- symbols -- calendars -- commitment of traders -- financial statements -- time series -- historical prices ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -This section provides explanations for getting started using the OpenBB Platform. -Pages outline things to expect and include example syntax for use. - -Subjects covered include: - -- [Finding Ticker Symbols](explanation/find_symbols) -- [Loading Historical Price Data](explanation/historical_prices) -- [Market Calendars](explanation/market_calendars) -- [Financial Statements](explanations/financial_statements) -- [Commitment of Traders](explanations/commitment_of_traders) - -Jupyter Notebook examples are also hosted in the OpenBB GitHub repository [here](https://github.com/OpenBB-finance/OpenBBTerminal/tree/develop/examples) diff --git a/website/content/platform/usage/guides/_category_.json b/website/content/platform/usage/guides/_category_.json deleted file mode 100644 index 0967ef424bce..000000000000 --- a/website/content/platform/usage/guides/_category_.json +++ /dev/null @@ -1 +0,0 @@ -{} diff --git a/website/content/platform/usage/index.md b/website/content/platform/usage/index.md deleted file mode 100644 index ee0ddcfba88e..000000000000 --- a/website/content/platform/usage/index.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -title: Overview -sidebar_position: 1 -description: An overview for getting started with the OpenBB Platform Python client and Fast API, details on authorization, data providers, settings, responses, commands, logging, and features such as dynamic command execution. -keywords: -- OpenBB Platform -- Python client -- Fast API -- getting started -- authorization -- data providers -- OpenBB Hub -- local environment -- environment variables -- user settings -- command execution -- API response -- logging -- proxy networks -- data cleaning -- technical analysis -- quantitative analysis -- charting libraries ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -At its base, the OpenBB Platform supplies core architecture and services for connecting data providers and extensions, consumable through the Python client or the Fast API. -The extension framework provides interoperability between as many, or few, services required. - -Optional extras are not included with the base installation, and these include: - -- Charting libraries and views -- Data processing and calculations -- Technical/Quantitative Analysis -- Community data providers -- Toolkit extensions -- CLI Terminal - -Additional functionality and data providers can be installed or removed independently. Refer to [Extensions](extensions) for the list of extensions hosted in the OpenBB GitHub repository. - -The pages in this section cover topics to get started using the OpenBB Platform, including: - -- [Authorization and API Keys](usage/api_keys) -- [Launching the REST API](usage/rest_api) -- [User Settings and Environment Variables](usage/settings_and_environment_variables) -- [Basic Response](usage/basic_response) -- [Basic Syntax](usage/basic_syntax) -- [Explanations](usage/explanation) diff --git a/website/content/platform/usage/settings_and_environment_variables.md b/website/content/platform/usage/settings_and_environment_variables.md deleted file mode 100644 index 75d059e6d96d..000000000000 --- a/website/content/platform/usage/settings_and_environment_variables.md +++ /dev/null @@ -1,105 +0,0 @@ ---- -title: User Settings & Environment Variables -sidebar_position: 3 -description: This section details configuring the OpenBB Platform settings and environment variables. -keywords: -- OpenBB Platform -- Python client -- getting started -- OpenBB Hub -- local environment -- environment variables ---- - -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; - - - -This page outlines configuring the OpenBB Platform with user settings and environment variables. - -## User Settings - -User preferences are stored locally, `~/.openbb_platform/`, as a JSON file, `user_settings.json`. It is read upon initializing the Python client, or when the Fast API is authorized. If the file does not exist, it will be created on the first run. - -| **Preference** | **Default** | **Options** | **Description** | -|-----------------------|----------------------------------|------------------------|---------------| -| data_directory | /home/OpenBBUserData | Any path. | When launching the application for the first time this directory will be created. It serves as the default location where the application stores usage artifacts such as logs and exports. | -| export_directory | /home/OpenBBUserData/exports | Any path. | The OpenBB Charting Extension provides the capability to export images in various formats. This is the directory where it attempts to save such exports. | -| cache_directory | /home/OpenBBUserData/cache | Any path. | The directory where http requests and database caches are stored, for functions with caching. | -| user_styles_directory | /home/OpenBBUserData/styles/user | Any path. | The OpenBB Charting Extension supports custom stylization. This directory is the location where it looks for user-defined styles. If no user styles are found in this directory the application will proceed with the default styles. | -| charting_extension | openbb_charting | ["openbb_charting"] | Name of the charting extension to be used with the application. | -| chart_style | dark | ["dark", "light"] | The default color style to use with the OpenBB Charting Extension plots. Options include "dark" and "light". | -| plot_enable_pywry | True | [True, False] | Whether the application should enable PyWry. If PyWry is disabled the image will open in your default browser otherwise it will be displayed within your editor or in a separate PyWry window. | -| plot_pywry_width | 1400 | Any positive integer. | PyWry window width. | -| plot_pywry_height | 762 | Any positive integer. | PyWry window height. | -| plot_open_export | False | [True, False] | Controls whether the "Save As" window should pop up as soon as the image is displayed." | -| table_style | dark | ["dark", "light"] | "The default color style to use with the OpenBB Charting Extension tables. Options are "dark" and "light"" | -| request_timeout | 15 | Any positive integer. | Specifies the timeout duration for HTTP requests. | -| metadata | True | [True, False] | Enables or disables the collection of metadata which provides information about operations including arguments duration route and timestamp. Disabling this feature may improve performance in cases where contextual information is not needed or when the additional computation time and storage space are a concern. | -| output_type | OBBject | ["OBBject", "dataframe", "numpy", "dict", "chart", "polars"] | Specifies the type of data the application will output when a command or endpoint is accessed. Note that choosing data formats only available in Python such as `dataframe`, `numpy` or `polars` will render the application's API non-functional. | -| show_warnings | True | [True, False] | Enables or disables the display of warnings. | - -### Notes on Preferences - -:::note - -- If a `OpenBBUserData` folder in not in the home directory, the application will create one on first run. The user preferences with paths all default to this folder, be it exports, styles or data - this can be changed at any time to suit. -- The `OpenBBUserData` will still be created even if preferences are not pointing to it, this is because the application needs a place to store logs and other artifacts. -- One way to export files or images from the OpenBB Platform is to leverage that functionality from the OpenBB Charting Extension. The `export_directory` preference is the location where the extension will attempt to save CSV and image files. -::: - -```json -{ - "preferences": { - "data_directory": "~/OpenBBUserData", - "export_directory": "~/OpenBBUserData/exports", - "cache_directory": "~/OpenBBUserData/cache", - "user_styles_directory": "~/OpenBBUserData/styles/user", - "charting_extension": "openbb_charting", - "chart_style": "dark", - "plot_enable_pywry": true, - "plot_pywry_width": 1400, - "plot_pywry_height": 762, - "plot_open_export": false, - "table_style": "dark", - "request_timeout": 15, - "metadata": true, - "output_type": "OBBject" -} -} - - -## Environment Variables - -Environment variables are defined in a `.env` file. If this file does not exist, create it inside the same folder `user_settings.json` is located. - -- `OPENBB_DEBUG_MODE`: enables verbosity while running the program -- `OPENBB_DEVELOP_MODE`: points hub service to .co or .dev -- `OPENBB_AUTO_BUILD`: enables automatic SDK package build on import -- `OPENBB_CHARTING_EXTENSION`: specifies which charting extension to use -- `OPENBB_API_AUTH_EXTENSION`: specifies which authentication extension to use -- `OPENBB_API_AUTH`: enables API authentication for command endpoints -- `OPENBB_API_USERNAME`: sets API username -- `OPENBB_API_PASSWORD`: sets API password - -Variables can be defined for current session only. - -```python -import os -os.environ["OPENBB_DEBUG_MODE"] = "True" -from openbb import obb -``` - -### Proxy Networks - -An environment variable can be set, in the `.env` file, to direct the Requests library to a specific address and port. - -```env -HTTP_PROXY="
" or HTTPS_PROXY="
” -``` - -For example: - -```env -HTTP_PROXY="http://10.10.10.10:8000" -``` \ No newline at end of file diff --git a/website/content/platform/user_guides/_category_.json b/website/content/platform/user_guides/_category_.json new file mode 100644 index 000000000000..dd71614dedc0 --- /dev/null +++ b/website/content/platform/user_guides/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "User Guides", + "position": 2 +} diff --git a/website/content/platform/development/contributing/function_examples.md b/website/content/platform/user_guides/add_command_examples.mdx similarity index 75% rename from website/content/platform/development/contributing/function_examples.md rename to website/content/platform/user_guides/add_command_examples.mdx index f46b639cfef3..5043d617f750 100644 --- a/website/content/platform/development/contributing/function_examples.md +++ b/website/content/platform/user_guides/add_command_examples.mdx @@ -1,28 +1,27 @@ --- -title: Function Examples -sidebar_position: 3 -description: This guide provides detailed instructions for including function examples in the router endpoints of the OpenBB Platform. +title: Add Examples to Commands +sidebar_position: 12 +description: This guide provides detailed instructions for including command examples in the router endpoints of the OpenBB Platform. keywords: -- OpenBB community -- OpenBB Platform -- Custom commands -- API -- Python Interface -- Examples -- Usage -- Parameters -- contributing -- requirements + - OpenBB community + - OpenBB Platform + - Custom commands + - API + - Python Interface + - Examples + - Usage + - Parameters + - contributing + - requirements --- -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; - + -Usage examples are defined in the router and are expected to provide working syntax, -with descriptions for complex functions requiring many parameters. +Usage examples are defined in the router and are expected to provide working syntax, with descriptions for complex functions requiring many parameters. -> When a given provider is not installed, its example will be excluded from `openapi.json` and Python docstrings +> When a provider is not installed, its example will be excluded from `openapi.json` and Python docstrings ## Requirements @@ -40,10 +39,11 @@ All router endpoints must have examples. - Descriptions are mandatory. - ## Example Models -There are two models for defining examples, `APIEx` and `PythonEx`. `APIEx` is more structured aiming to be language agnostic - provide less freedom - while `PythonEx` gives more freedom to create complex examples. +There are two models for defining examples, `APIEx` and `PythonEx`. ` + +`APIEx` is more structured aiming to be language agnostic - provides less freedom - while `PythonEx` gives more freedom to create complex examples. ```python from openbb_core.app.model.example import APIEx, PythonEx diff --git a/website/content/platform/development/how-to/add_data_provider_extension.md b/website/content/platform/user_guides/add_data_provider_extension.mdx similarity index 94% rename from website/content/platform/development/how-to/add_data_provider_extension.md rename to website/content/platform/user_guides/add_data_provider_extension.mdx index a9ae9233ad07..72fd33666243 100644 --- a/website/content/platform/development/how-to/add_data_provider_extension.md +++ b/website/content/platform/user_guides/add_data_provider_extension.mdx @@ -1,6 +1,6 @@ --- -title: How To Add Data Provider Extensions -sidebar_position: 3 +title: Build New Provider Extension +sidebar_position: 7 description: This guide outlines the process for adding a new data provider extension to the OpenBB Platform. keywords: - OpenBB Platform @@ -24,16 +24,16 @@ keywords: - template --- -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; - + This page will walk through adding a new data provider extension to the OpenBB Platform. By the end, you will have an extension that is ready to be published, or submitted as a pull request. ## Template For Getting Started -If you've already been through some of the other data provider tasks - [How To Add Data To An Existing Endpoint](add_data_to_existing_endpoint) & [How To Add A New Data Endpoint With An Existing Provider](add_endpoint_to_existing_provider) - these steps will simply tag on to the beginning. Instead of editing the `__init__.py` file, we will create it. If you haven't been through either of those guides, review them first before jumping in here. +If you've already been through some of the other data provider tasks - [How To Add Data To An Existing Endpoint](add_data_to_existing_endpoint) & [How To Add A New Data Endpoint With An Existing Provider](add_endpoint_to_existing_provider) - these steps will simply tag on to the beginning. Instead of editing the `__init__.py` file, we will create it. An easy way to get started is to copy and paste something existing. In the OpenBB GitHub repository, provider extensions are located [here](https://github.com/OpenBB-finance/OpenBBTerminal/tree/develop/openbb_platform/providers). @@ -67,27 +67,36 @@ The `__init__.py` file where models are mapped to the router is under, `/openbb_ To get started: - Unpack the downloaded [zip](ttps://github.com/OpenBB-finance/OpenBBTerminal/files/14519701/provider_extension_template.zip) file. + - If working with a cloned GitHub repo, the folder is: + ```console ~/OpenBBTerminal/openbb_platform/providers ``` + - Rename everything, "template", to suit. File names, models, import statements, docstrings. - Add any provider-specific package requirements in the `pyproject.toml` file. - Update the Provider information in the `__init__.py` file. + - If credentials are required, add a line to the Provider class initialization. + ```python credentials=["api_key", "account_type"], # account_type is either "sandbox" or "live" ``` + - From a terminal command line, navigate into the folder where the extension is, then install the empty blank package in "editable" mode. + ```console poetry lock pip install -e . ``` + - Start creating data models using the steps outlined [here](add_data_to_existing_endpoint) ### Cookiecutter In order to speed up the process of building an extension, we have created a [**Cookiecutter**](https://github.com/OpenBB-finance/openbb-cookiecutter) template. + It serves as a jumpstart for your extension development, and can be used instead of the template ZIP referenced earlier. Instructions are located on the [GitHub page](https://github.com/OpenBB-finance/openbb-cookiecutter). :::note @@ -102,9 +111,10 @@ The `pyproject.toml` file defines the package itself. - Before adding any dependency, ensure it aligns with the Platform's existing dependencies. - If possible, use loose versioning. + ::: -``` +```toml [tool.poetry] name = "openbb-template" version = "1.0.0" @@ -114,8 +124,8 @@ readme = "README.md" packages = [{ include = "openbb_template" }] [tool.poetry.dependencies] -python = "^3.8" -openbb-core = "^1.1.2" +python = ">=3.8,<3.12" +openbb = "^4.1.7" [build-system] requires = ["poetry-core"] @@ -125,7 +135,7 @@ build-backend = "poetry.core.masonry.api" template = "openbb_template:template_provider" ``` -The last line (poetry.plugins) maps to the provider defined in the **init**.py file. +The last line (poetry.plugins) maps to the provider defined in the `__init__.py` file. Additionally, for local extensions, you can add this line in the `LOCAL_DEPS` variable in the `dev_install.py` file, located in `~/OpenBBTerminal/openbb_platform/`: @@ -139,8 +149,7 @@ Now you can use the `python dev_install.py [-e]` command to install the local ex ## Authorization Credentials -Access to most data sources is authorized with an API key, issued by the source. Sometimes there are multiple authorization fields, -and other times there may be a need to change the base URL depending on the type of account. +Access to most data sources is authorized with an API key, issued by the source. Sometimes there are multiple authorization fields, and other times there may be a need to change the base URL depending on the type of account. > If no authorization is required, leave out the 'credentials' parameter. diff --git a/website/content/platform/development/how-to/add_data_to_existing_endpoint.md b/website/content/platform/user_guides/add_data_to_existing_endpoint.mdx similarity index 85% rename from website/content/platform/development/how-to/add_data_to_existing_endpoint.md rename to website/content/platform/user_guides/add_data_to_existing_endpoint.mdx index 2b1137eca8d4..ea5bb7b54e65 100644 --- a/website/content/platform/development/how-to/add_data_to_existing_endpoint.md +++ b/website/content/platform/user_guides/add_data_to_existing_endpoint.mdx @@ -1,30 +1,30 @@ --- -title: How To Add A Data Provider To An Existing Endpoint -sidebar_position: 1 +title: Add Provider To An Existing Command +sidebar_position: 10 description: This guide outlines the process for adding an endpoint to an existing data provider and router endpoint. keywords: -- OpenBB Platform -- Open source -- Python interface -- REST API -- contribution -- contributing -- documentation -- code -- provider -- data -- endpoint -- existing -- OpenBB extensions -- OpenBB provider -- standard model -- provider model -- how to + - OpenBB Platform + - Open source + - Python interface + - REST API + - contribution + - contributing + - documentation + - code + - provider + - data + - endpoint + - existing + - OpenBB extensions + - OpenBB provider + - standard model + - provider model + - how to --- -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; - + This page will walk through adding a data provider to an existing endpoint, using a standard model. At a high level, the process will look something like: @@ -44,8 +44,8 @@ Before getting started, get a few housekeeping items in order: - Clone the GitHub repo and navigate into the project's folder. - If you have already done this, update your local branch: - - `git fetch` - - `git pull origin develop` + - `git fetch` + - `git pull origin develop` - Install the OpenBB Platform in "editable" mode. - `cd openbb_platform` - `python dev_install.py -e` @@ -72,7 +72,7 @@ Quarterly data also includes analyst estimates and surprise metrics. ### Provider API Documentation -The documentation for this endpoint is, [https://www.alphavantage.co/documentation/#earnings](https://www.alphavantage.co/documentation/#earnings). This link will be added to the query parameters model docstring. +The documentation for this endpoint is, [https://www.alphavantage.co/documentation/#earnings](https://www.alphavantage.co/documentation/#earnings). This link will be added to the query parameters model docstring. ### Base URL @@ -96,35 +96,35 @@ They provide a sample JSON output, returning both annual and quarterly data in t ```json { - "symbol": "IBM", - "annualEarnings": [ - { - "fiscalDateEnding": "2023-12-31", - "reportedEPS": "9.61" - }, - { - "fiscalDateEnding": "2022-12-31", - "reportedEPS": "9.12" - }, - ], - "quarterlyEarnings": [ - { - "fiscalDateEnding": "2023-12-31", - "reportedDate": "2024-01-24", - "reportedEPS": "3.87", - "estimatedEPS": "3.78", - "surprise": "0.09", - "surprisePercentage": "2.381" - }, - { - "fiscalDateEnding": "2023-09-30", - "reportedDate": "2023-10-25", - "reportedEPS": "2.2", - "estimatedEPS": "2.13", - "surprise": "0.07", - "surprisePercentage": "3.2864" - }, - ], + "symbol": "IBM", + "annualEarnings": [ + { + "fiscalDateEnding": "2023-12-31", + "reportedEPS": "9.61" + }, + { + "fiscalDateEnding": "2022-12-31", + "reportedEPS": "9.12" + } + ], + "quarterlyEarnings": [ + { + "fiscalDateEnding": "2023-12-31", + "reportedDate": "2024-01-24", + "reportedEPS": "3.87", + "estimatedEPS": "3.78", + "surprise": "0.09", + "surprisePercentage": "2.381" + }, + { + "fiscalDateEnding": "2023-09-30", + "reportedDate": "2023-10-25", + "reportedEPS": "2.2", + "estimatedEPS": "2.13", + "surprise": "0.07", + "surprisePercentage": "3.2864" + } + ] } ``` @@ -140,13 +140,13 @@ from openbb import obb obb.equity.fundamental.historical_eps(symbol = "IBM", limit=5, provider="fmp") ``` -| date | symbol | eps_actual | eps_estimated | revenue_estimated | revenue_actual | reporting_time | updated_at | period_ending | -|:-----------|:---------|-------------:|----------------:|--------------------:|-----------------:|:-----------------|:-------------|:----------------| -| 2024-01-24 | IBM | 3.87 | 3.78 | 17298500000 | 17381000000 | amc | 2024-02-29 | 2023-12-31 | -| 2024-04-17 | IBM | - | 1.59 | 14572800000 | - | bmo | 2024-02-29 | 2024-03-30 | -| 2024-07-24 | IBM | - | - | - | - | amc | 2024-02-29 | 2024-06-30 | -| 2024-10-23 | IBM | - | - | - | - | amc | 2024-02-29 | 2024-09-30 | -| 2025-01-22 | IBM | - | - | - | - | amc | 2024-02-29 | 2024-12-31 | +| date | symbol | eps_actual | eps_estimated | revenue_estimated | revenue_actual | reporting_time | updated_at | period_ending | +| :--------- | :----- | ---------: | ------------: | ----------------: | -------------: | :------------- | :--------- | :------------ | +| 2024-01-24 | IBM | 3.87 | 3.78 | 17298500000 | 17381000000 | amc | 2024-02-29 | 2023-12-31 | +| 2024-04-17 | IBM | - | 1.59 | 14572800000 | - | bmo | 2024-02-29 | 2024-03-30 | +| 2024-07-24 | IBM | - | - | - | - | amc | 2024-02-29 | 2024-06-30 | +| 2024-10-23 | IBM | - | - | - | - | amc | 2024-02-29 | 2024-09-30 | +| 2025-01-22 | IBM | - | - | - | - | amc | 2024-02-29 | 2024-12-31 | FMP is currently the only source for this endpoint. There are only two parameters, `symbol` and `limit`. The `limit` argument determines how many quarters to go back. @@ -211,9 +211,9 @@ Now we know exactly what is going to be added, and how we should structure our q So far, we have knocked out three of the outlined tasks. - - [X] Catalogue the parameters and returned fields from the chosen data provider. - - [X] Find the existing standard model that is mapped to the router endpoint. - - [X] Identify common parameters and fields to map. +- [x] Catalogue the parameters and returned fields from the chosen data provider. +- [x] Find the existing standard model that is mapped to the router endpoint. +- [x] Identify common parameters and fields to map. Let's get on with the fun stuff and start building! @@ -233,7 +233,7 @@ The first line of the file should be a docstring, followed by the import stateme ### Import Statements -Every model will be different, but most items below will be typical of nearly every data provider model. Variations will come from design choices for [HTTP requests](platform/development/contributing/http_requests.md), or other requirements. We won't get into that here though. +Every model will be different, but most items below will be typical of nearly every data provider model. Variations will come from design choices for [HTTP requests](/platform/developer_guide/http_requests), or other requirements. We won't get into that here though. ```python """AlphaVantage Historical EPS Model.""" @@ -439,7 +439,7 @@ class AVHistoricalEpsFetcher( Combining all of the code blocks above, beginning with the import statements section, makes a complete file and we have finished step 4. - - [X] Build the provider models and Fetcher class by inheriting from the standard models. +- [x] Build the provider models and Fetcher class by inheriting from the standard models. ## Map To Router @@ -473,7 +473,8 @@ alpha_vantage_provider = Provider( ``` Step 5 is complete. - - [X] Map the new provider model to the router. + +- [x] Map the new provider model to the router. ## Rebuild Static Assets @@ -494,7 +495,8 @@ If changes are only made to the static methods within the Fetcher, rebuilding is ::: Step 6 is done. - - [X] Rebuild the Python interface and static assets. + +- [x] Rebuild the Python interface and static assets. We can now run the function and test our work. @@ -509,12 +511,12 @@ obb.equity.fundamental.historical_eps( ).to_df() ``` -| date | symbol | eps_actual | eps_estimated | surprise | surprise_percent | reported_date | -|:-----------|:---------|-------------:|----------------:|-----------:|-------------------:|:----------------| -| 2023-12-31 | GOOG | 1.64 | 1.59 | 0.05 | 0.031447 | 2024-01-30 | -| 2023-12-31 | AAPL | 2.18 | 2.1 | 0.08 | 0.038095 | 2024-02-01 | -| 2023-12-31 | MSFT | 2.93 | 2.78 | 0.15 | 0.053957 | 2024-01-30 | -| 2023-12-31 | IBM | 3.87 | 3.78 | 0.09 | 0.02381 | 2024-01-24 | +| date | symbol | eps_actual | eps_estimated | surprise | surprise_percent | reported_date | +| :--------- | :----- | ---------: | ------------: | -------: | ---------------: | :------------ | +| 2023-12-31 | GOOG | 1.64 | 1.59 | 0.05 | 0.031447 | 2024-01-30 | +| 2023-12-31 | AAPL | 2.18 | 2.1 | 0.08 | 0.038095 | 2024-02-01 | +| 2023-12-31 | MSFT | 2.93 | 2.78 | 0.15 | 0.053957 | 2024-01-30 | +| 2023-12-31 | IBM | 3.87 | 3.78 | 0.09 | 0.02381 | 2024-01-24 | Checking the `annual` setting: @@ -527,12 +529,12 @@ obb.equity.fundamental.historical_eps( ).to_df() ``` -| date | symbol | eps_actual | -|:-----------|:---------|-------------:| -| 2021-09-30 | AAPL | 5.62 | -| 2022-09-30 | AAPL | 6.11 | -| 2023-09-30 | AAPL | 6.12 | -| 2023-12-31 | AAPL | 2.18 | +| date | symbol | eps_actual | +| :--------- | :----- | ---------: | +| 2021-09-30 | AAPL | 5.62 | +| 2022-09-30 | AAPL | 6.11 | +| 2023-09-30 | AAPL | 6.12 | +| 2023-12-31 | AAPL | 2.18 | We can see that the most recent `annual` data point only represent the first quarter of Apple's fiscal year, and this is something to keep in mind while working with the data. @@ -620,14 +622,15 @@ def test_av_historical_eps_fetcher(credentials=test_credentials): That's all there is to it, we can capture the cassette now. Open a terminal, navigate into the `tests` folder from above, with the `obb` environment active, and enter: -``` +```console pytest test_alpha_vantage_fetchers.py --record http --record-no-overwrite ``` A successful test will result in a file being created in the `record` subfolder. Check the file for any obvious errors. Step 7 is done. - - [X] Add unit tests. + +- [x] Add unit tests. ### Integration Tests @@ -769,8 +772,8 @@ pytest test_equity_api.py ``` Step 8 is done. - - [X] Add integration tests. +- [x] Add integration tests. All that's left now is to submit the work as a pull request for review. @@ -833,25 +836,25 @@ A pull request, in general, should have details on why the PR was created, what 1. **Why**? (1-3 sentences or a bullet point list): - - This PR is the result of a development documentation page created (not in this PR). + - This PR is the result of a development documentation page created (not in this PR). - - Closes #6104, a user feature request. + - Closes #6104, a user feature request. 2. **What**? (1-3 sentences or a bullet point list): - - Adds AlphaVantage as a provider to `obb.equity.fundamental.historical_eps()` + - Adds AlphaVantage as a provider to `obb.equity.fundamental.historical_eps()` 3. **Impact** (1-2 sentences or a bullet point list): - - Is not a breaking change. + - Is not a breaking change. - - Does not introduce any changes other than adding the provider to this endpoint. + - Does not introduce any changes other than adding the provider to this endpoint. 4. **Testing Done**: - - Created unit test and integration tests. + - Created unit test and integration tests. - - Used a variety of symbols, single and lists, to check that the EmptyDataError and symbol warnings are catching correctly. + - Used a variety of symbols, single and lists, to check that the EmptyDataError and symbol warnings are catching correctly. ### Enjoy diff --git a/website/content/platform/development/how-to/add_endpoint_to_existing_provider.md b/website/content/platform/user_guides/add_endpoint_to_existing_provider.mdx similarity index 98% rename from website/content/platform/development/how-to/add_endpoint_to_existing_provider.md rename to website/content/platform/user_guides/add_endpoint_to_existing_provider.mdx index 37edf14843fd..84fb3ab80ab3 100644 --- a/website/content/platform/development/how-to/add_endpoint_to_existing_provider.md +++ b/website/content/platform/user_guides/add_endpoint_to_existing_provider.mdx @@ -1,6 +1,6 @@ --- -title: How To Add A New Data Endpoint With An Existing Provider -sidebar_position: 2 +title: Add Command To An Existing Provider +sidebar_position: 11 description: This guide outlines the process for adding a new endpoint to an existing data provider, that does not yet have a standard model. keywords: - OpenBB Platform @@ -24,14 +24,14 @@ keywords: - how to --- -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; - + This page will walk through adding a new router endpoint to an existing data provider, and how to go about creating a new standard model. To demonstrate, we will be extending the `openbb-currency` router. The objective is to add a snapshot of currencies relative to a base currency. -The process will be very similar to adding a data provider to an existing endpoint - described on [this page](add_data_to_existing_endpoint.md) - +The process will be very similar to adding a data provider to an existing endpoint - described on [this page](add_data_to_existing_endpoint) - only here, we need to create a new router function and standard model. It's about the same amount of work, but effort should be placed in consideration of others inheriting from this model in the future. @@ -55,6 +55,8 @@ Before getting started, get a few housekeeping items in order: - If you have already done this, update your local branch: - `git fetch` - `git pull origin develop` + - `git fetch` + - `git pull origin develop` - Install the OpenBB Platform in "editable" mode. - `cd openbb_platform` - `python dev_install.py -e` @@ -66,7 +68,7 @@ Before getting started, get a few housekeeping items in order: ::: -## Let's Get Started! +## Let's Get Started Currencies, as an asset class, have different data properties than securities. For this exercise, we're really only concerned about the differences within the market data we are working with. @@ -169,7 +171,7 @@ Our `CurrencySnapshotsQueryParams` model is going to be very similar to `MarketS If the field will only sometimes accept a list of values, DO NOT define it in the standard model as a Union - `Union[str, List[str]]`. Instead, define it for the single value, `str`, and then add the property below to the provider's QueryParams model. -``` +```python __json_schema_extra__ = {"base": ["multiple_items_allowed"]} ``` @@ -220,6 +222,7 @@ This can be a consideration for the data provider models to handle, and country Like `QueryParams`, we don't want to attempt to define every potential future field. We want a core foundation for others to build on. We will define three fields as mandatory, "base_currency", "counter_currency", and "last_rate". This is enough to communicate our +We will define three fields as mandatory, "base_currency", "counter_currency", and "last_rate". This is enough to communicate our data parsing requirements for this endpoint: - Split the six-letter symbol as two symbols. @@ -271,7 +274,7 @@ Combine the three code blocks above to make a complete standard model file, and ## Build Provider Models -We're going to start with one provider, [FMP](https://site.financialmodelingprep.com/developer/docs#exchange-prices-quote), and this section will look a lot like the process outlined [here](add_data_to_existing_endpoint.md). +We're going to start with one provider, [FMP](https://site.financialmodelingprep.com/developer/docs#exchange-prices-quote), and this section will look a lot like the process outlined [here](add_data_to_existing_endpoint). Sample output data from the source is pasted below, and we can see that there are some fields which don't have anything to do with currencies. Those will be dropped. @@ -304,7 +307,7 @@ Sample output data from the source is pasted below, and we can see that there ar ] ``` -### Create File +### Create File For Provider We need to create a new file in the FMP provider extension. This will have the same name as our standard model. diff --git a/website/content/platform/development/how-to/add_obbject_extension.md b/website/content/platform/user_guides/add_obbject_extension.mdx similarity index 83% rename from website/content/platform/development/how-to/add_obbject_extension.md rename to website/content/platform/user_guides/add_obbject_extension.mdx index ef1005d0a66b..9de3170fa840 100644 --- a/website/content/platform/development/how-to/add_obbject_extension.md +++ b/website/content/platform/user_guides/add_obbject_extension.mdx @@ -1,25 +1,25 @@ --- -title: How To Add OBBject Extensions -sidebar_position: 8 +title: Build New OBBject Extension +sidebar_position: 9 description: This page provides information about how to write extensions for the OpenBB OBBject class. keywords: -- OBBject -- Python -- Development -- OpenBB Platform -- extensions -- obbject extension -- accessor -- decorator -- how-to -- contributing + - OBBject + - Python + - Development + - OpenBB Platform + - extensions + - obbject extension + - accessor + - decorator + - how-to + - contributing --- -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; - + -OpenBB provides some basic methods for interacting with common data structures that will be seen in the results attribute of the [`OBBject`](platform/development/obbject.md) +OpenBB provides some basic methods for interacting with common data structures that will be seen in the results attribute of the [`OBBject`](platform/developer_guide/obbject). If you are working with custom data, or adding new endpoints, it is possible that you will want to have your own methods for interacting with the data, and the OpenBB Platform provides a way to extend the OBBject class. The architecture for extensions was designed to be similar to extensions and accessors for Pandas, and relies on plugins through the Poetry dependency management package. @@ -80,7 +80,7 @@ With this in the file, we can install the extension by running `poetry install` ### Using the extension -Now that the extension is installed and built, we can use it! Because we are extending the `OBBject`, this will be available on any function: +Now that the extension is installed and built, we can use it! Because we are extending the `OBBject`, this will be available on any function: ```shell >>> from openbb import obb diff --git a/website/content/platform/development/how-to/add_toolkit_extension.md b/website/content/platform/user_guides/add_toolkit_extension.mdx similarity index 55% rename from website/content/platform/development/how-to/add_toolkit_extension.md rename to website/content/platform/user_guides/add_toolkit_extension.mdx index 070f56bab93e..9e8636a29637 100644 --- a/website/content/platform/development/how-to/add_toolkit_extension.md +++ b/website/content/platform/user_guides/add_toolkit_extension.mdx @@ -1,108 +1,36 @@ --- -title: How To Add Toolkit Extensions -sidebar_position: 4 +title: Build New Toolkit Extension +sidebar_position: 8 description: This guide outlines the process for adding a new toolkit extension and router path to the OpenBB Platform. keywords: -- OpenBB Platform -- Open source -- contribution -- contributing -- community -- toolkit -- code -- provider -- endpoint -- router -- openbb-provider -- openbb-core -- how to -- new -- template + - OpenBB Platform + - Open source + - contribution + - contributing + - community + - toolkit + - code + - provider + - endpoint + - router + - openbb-provider + - openbb-core + - how to + - new + - template --- -import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; - + -This page will summarize some of the differences between toolkit and provider extensions, -and then walk through adding a new toolkit extension and router path to the OpenBB Platform using a supplied template. - -## What Is A Toolkit? - -Toolkits are a generalized category of functionality for performing operations beyond the scope of any provider fetcher. -The possibilities are virtually unlimited, and each component (`equity`, `etf`, `index`, etc.) can be installed or removed independently. -A toolkit might even be extending another extension, which itself is an extension of an extension. - -An easy example to understand is, `openbb-technical`. -It has its own router where all functions are configured as a `POST` request, with data being sent by the user as a serialized JSON in the body. -Calculations are performed using the supplied data and parameters, returning a new instance of the OBBject class. - -The `pyproject.toml` defining the extension is nearly nearly identical to a Provider. -Instead of registering the plugin as a provider extension, it is an `openbb_core` extension. - -> `openbb-devtools` is an extension with no router entry points or added functionality. Its only purpose is to install a collection of Python packages. - -### Provider - -```toml -[tool.poetry.plugins."openbb_provider_extension"] -``` - -### Core - -```toml -[tool.poetry.plugins."openbb_core_extension"] -``` - -Below is the contents of the `pyproject.toml` file that defines the `openbb-technincal` extension. - -```toml -[tool.poetry] -name = "openbb-technical" -version = "1.1.3" -description = "Technical Analysis extension for OpenBB" -authors = ["OpenBB Team "] -readme = "README.md" -packages = [{ include = "openbb_technical" }] - -[tool.poetry.dependencies] -python = ">=3.8,<3.12" # scipy forces python <4.0 explicitly -scipy = "^1.10.1" -statsmodels = "^0.14.0" -scikit-learn = "^1.3.1" -pandas-ta = "^0.3.14b" -openbb-core = "^1.1.2" - -[build-system] -requires = ["poetry-core"] -build-backend = "poetry.core.masonry.api" - -[tool.poetry.plugins."openbb_core_extension"] -technical = "openbb_technical.technical_router:router" -``` - -On [this page](add_data_provider_extension), we created a data provider extension using a template ZIP file. -The structure is familiar, but let's take a look at some subtle differences. - -For convenience, a template router extension ZIP file can be downloaded [here](https://github.com/OpenBB-finance/OpenBBTerminal/files/14542427/dashboards.zip) to follow along with. - -## Folder Structure - -A couple of notable differences between a provider and toolkit extension are: - -- In the OpenBB GitHub repo, extensions are all located under: - ```console - ~/OpenBBTerminal/openbb_platform/extensions - ``` -- An additional folder housing integration tests, with the `tests` folder staying empty. -- There is a `router` file, and there can be sub-folders with additional routers. -- Utility functions don't need their own sub-folder. -- `__init__.py` files are all empty. +Before adding a new toolkit extension and router path to the OpenBB Platform using a supplied template, it is important to understand the difference between a toolkit and a provider extension. You can find more information on this [here](/platform/developer_guide/extensions). ## How To Create A Router Extension Let's create an extension which defines a new router entry point at the base of the `obb` namespace. It's going to be called, `openbb-dashboards`, and will serve as an empty router for various dashboard packages to populate **future** endpoints with. + By itself, it might not have any functions. Some other extension will name it as a dependency, using it as an entry point. We'll use the [ZIP file](https://github.com/OpenBB-finance/OpenBBTerminal/files/14542427/dashboards.zip) template as a starting point, renaming everything as the first step. @@ -110,7 +38,7 @@ We'll use the [ZIP file](https://github.com/OpenBB-finance/OpenBBTerminal/files/ ### Create Folder The folder does not have to be kept with the OpenBB code, and could be its own GitHub repo. -For demonstration purposes, we'll unpack the ZIp file template with the rest of the OpenBB extensions: +For demonstration purposes, we'll unpack the ZIP file template with the rest of the OpenBB extensions: ```console ~/OpenBBTerminal/openbb_platform/extensions/dashboards @@ -132,9 +60,9 @@ readme = "README.md" packages = [{ include = "openbb_dashboards" }] [tool.poetry.dependencies] -python = ">=3.8,<3.12" # scipy forces python <4.0 explicitly -openbb-core = "^1.1.2" -openbb-charting = "^2.0.0" +python = ">=3.8,<3.12" +openbb = "^4.1.7" +openbb-charting = "^2.0.3" [build-system] requires = ["poetry-core"] @@ -160,11 +88,13 @@ To demonstrate this extension, we'll make a simple function for creating and ret :::tip After creating a new function, remember to rebuild the Python interface and static assets. + ```python import openbb openbb.build() exit() ``` + ::: ```python diff --git a/website/content/platform/user_guides/basic_response.mdx b/website/content/platform/user_guides/basic_response.mdx new file mode 100644 index 000000000000..cd9ecd8032e8 --- /dev/null +++ b/website/content/platform/user_guides/basic_response.mdx @@ -0,0 +1,66 @@ +--- +title: Basic Response +sidebar_position: 2 +description: This page details the basic response and output that can be expected to be received from the the OpenBB Platform. +keywords: +- tutorial +- standardized output +- OBBject +- basic response +- provider +- results +- warnings +- chart +- extra +- command coverage +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +The output of every command is an object which contains the results of the request, along with additional information. It is a custom class, `OBBject`, and always returns with the fields listed below: + +```console +id: ... # UUID Tag +results: ... # Serializable results. +provider: ... # Provider name. +warnings: ... # List of warnings. +chart: ... # Chart object. +extra: ... # Extra info. +``` + +```python +from openbb import obb + +data = obb.equity.price.historical("SPY", provider="polygon") + +data +``` + +```console +OBBject + +id: 06520558-d54a-7e53-8000-7aafc8a42694 +results: [{'date': datetime.datetime(2022, 10, 5, 0, 0), 'open': 375.62, 'high': 37... +provider: polygon +warnings: None +chart: None +extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'polygon'}, 'st... +``` + +Additional class methods are helpers for converting the results to a variety of formats. + +- `to_dict()`: converts to a dictionary, accepting all standard "orientation" parameters, i.e., "records" +- `to_df()` / `to_dataframe()`: converts to a Pandas DataFrame. +- `to_numpy()`: converts to a Numpy array. +- `to_polars()`: converts to a Polars table. + +:::info +The preferred output type can be set with a user preference. + +```python +obb.user.preferences.output_type="dataframe" +``` + +::: diff --git a/website/content/platform/usage/basic_syntax.md b/website/content/platform/user_guides/basic_syntax.mdx similarity index 63% rename from website/content/platform/usage/basic_syntax.md rename to website/content/platform/user_guides/basic_syntax.mdx index 7036a9a22900..eb9db88c8a49 100644 --- a/website/content/platform/usage/basic_syntax.md +++ b/website/content/platform/user_guides/basic_syntax.mdx @@ -1,13 +1,9 @@ --- title: Basic Syntax -sidebar_position: 5 -description: This page provides comprehensive information about standardized command - syntax for an open-source platform. Topics discussed include the structure of command - syntax, use of standardized parameters, usage of provider and symbol parameters, - handling of date and limit parameters, and more. Also explored, are the methods - for selecting data sources, handling different list and ticker symbol formats, and - dealing with command responses and warnings. +sidebar_position: 1 +description: This page provides comprehensive information about standardized command syntax for an open-source platform. Topics discussed include the structure of command syntax, use of standardized parameters, usage of provider and symbol parameters, handling of date and limit parameters, and more. Also explored, are the methods for selecting data sources, handling different list and ticker symbol formats, and dealing with command responses and warnings. keywords: +- tutorial - command syntax - standardized parameters - date format @@ -29,7 +25,9 @@ import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; -The structure of command syntax is standardized across common fields. This ensures that a `date` is always a `date` and the format remains consistent throughout. Standardized parameters include, but are not limited to: +The structure of command syntax is standardized across common fields. This ensures that a `date` is always a `date` and the format remains consistent throughout. + +Standardized parameters include, but are not limited to: - [provider](#provider) - [symbol](#symbol) @@ -38,7 +36,7 @@ The structure of command syntax is standardized across common fields. This ensur - [date](#dates) - [limit](#limit) -When looking at a function's docstring, the standard parameters (shared across multiple providers) are positioned first. Provider-specific parameters positionally follow the `provider` argument. The example below is from, `obb.equity.price.quote`: +When looking at a function's docstring, the standard parameters (shared across multiple providers) are positioned first. Provider-specific parameters positionally follow the `provider` argument. The example below is from, `obb.equity.price.quote`: ```console Parameters @@ -70,7 +68,7 @@ uvicorn openbb_core.api.rest_api:app ## Provider -The `provider` parameter is the way to select the specific source of the data from the endpoint. If a [preference for the default provider](/platform/usage#user-settings) has not been defined, the default will be the first, alphabetically, installed provider. Provider values are entered in lower-case, with an underscore for multiple words - for example: +The `provider` parameter is the way to select the specific source of the data from the endpoint. If a [preference for the default provider](/platform/user_guides/settings_and_environment_variables) has not been defined, the default will be the first, alphabetically, installed provider. Provider values are entered in lower-case, with an underscore for multiple words - for example: ```python historical_prices = obb.equity.price.historical("aapl", provider="alpha_vantage") @@ -82,7 +80,7 @@ Provider coverage can be ascertained with the command below: obb.coverage.providers ``` -Refer to, [Data Providers](/platform/extensions/data_extensions), for instructions on installing data provider extensions. +Refer to, [Data Extensions](/platform/developer_guide/extensions), for instructions on installing data provider extensions. ## Symbol @@ -102,12 +100,16 @@ With providers supporting market data from multiple jurisdictions, the most comm ### One Symbol +
```python quote = obb.equity.price.quote(symbol="td", provider="fmp") ``` +
### Multiple Symbols +
+ The OpenBB Provider module enforces REST-compliant lists that can be entered in either format through the Python interface. #### Comma-Separated String @@ -154,6 +156,8 @@ response = requests.get(f"http://127.0.0.1:8000/api/v1/equity/price/quote?provid response.json() ``` +
+ ## Dates Dates are entered everywhere as a string, formatted as, "YYYY-MM-DD". If the function has only the `date` parameter, the data will be a snapshot instead of a time series. @@ -204,77 +208,6 @@ data.warnings [Warning_(category='OpenBBWarning', message="Parameter 'source' is not supported by fmp. Available for: intrinio.")] ``` -## REST API POST Requests - -Most endpoints are for data retrieval, but some [toolkit extensions](/platform/extensions/toolkit_extensions) require data to pass through the function. In these instances, it must be a POST request, and JSON. - -### Example - -This example will use a GET request to fetch daily VIX data from the Cboe data extension, and then make a POST request which passes through the data to a technical analysis function. - -First, start a development server. - -```bash -uvicorn openbb_core.api.rest_api:app --reload -``` - -This example will use Python and the Requests library, so the syntax will vary according to the preferred medium for interacting with the API. - -#### Fetch Some Data - -```python -import requests - -get_url = "http://127.0.0.1:8000/api/v1/index/price/historical?provider=cboe&symbol=vix&interval=1d" -get_response = requests.get(get_url) -data_results = get_response.json()["results"] - -data_results[-1] -``` - -```json -{'date': '2023-11-17T00:00:00', - 'open': 14.18, - 'high': 14.19, - 'low': 13.67, - 'close': 13.79, - 'volume': 0, - 'calls_volume': None, - 'puts_volume': None, - 'total_options_volume': None} -``` - -#### Send a POST Request - -Next, pass the `data_results` to a function, using the `json` field in the POST headers. For this example, realized volatiliy cones, the default parameters assume the time series data is daily and that volatility should be annualized over 252 trading days. - -The `index` parameter tells the function which field in the posted data to use as the date index. - -```python -import pandas as pd - -post_url = "http://localhost:8000/api/v1/technical/cones" -post_response = requests.post(post_url, json=data_results) -ta_results = post_response.json()["results"] - -pd.DataFrame.from_records(ta_results) -``` - -| window | realized | min | lower_25% | median | upper_75% | max | -|---------:|-----------:|-----------:|------------:|---------:|------------:|--------:| -| 3 | 0.396165 | 0.00701638 | 0.444709 | 0.72414 | 1.11563 | 8.47636 | -| 10 | 0.623199 | 0.190188 | 0.665584 | 0.852915 | 1.15491 | 4.83264 | -| 30 | 0.988435 | 0.332913 | 0.750007 | 0.921482 | 1.17072 | 2.98404 | -| 60 | 0.932594 | 0.47639 | 0.792548 | 0.964617 | 1.20171 | 2.35563 | -| 90 | 0.915137 | 0.551011 | 0.815229 | 0.965553 | 1.2128 | 2.04104 | -| 120 | 0.858644 | 0.549233 | 0.836395 | 0.983437 | 1.21097 | 1.86416 | -| 150 | 0.898628 | 0.557274 | 0.842359 | 0.991539 | 1.23165 | 1.73182 | -| 180 | 0.902293 | 0.579575 | 0.84876 | 1.00421 | 1.23584 | 1.68786 | -| 210 | 0.901717 | 0.580214 | 0.854655 | 0.996992 | 1.2271 | 1.65739 | -| 240 | 0.884282 | 0.587564 | 0.857935 | 1.00491 | 1.22141 | 1.62973 | -| 300 | 0.852533 | 0.622105 | 0.862097 | 1.00724 | 1.21705 | 1.51887 | -| 360 | 0.847416 | 0.661648 | 0.869691 | 1.03923 | 1.20488 | 1.48571 | - ## References All functions, parameters, and responses are detailed under the [Reference pages](/platform/reference). The data models for each provider source are described within the [Data Models](/platform/data_models) pages. diff --git a/website/content/platform/user_guides/dynamic_command_execution.mdx b/website/content/platform/user_guides/dynamic_command_execution.mdx new file mode 100644 index 000000000000..a0e982bb27f9 --- /dev/null +++ b/website/content/platform/user_guides/dynamic_command_execution.mdx @@ -0,0 +1,47 @@ +--- +title: Dynamic Command Execution +sidebar_position: 5 +description: This guide provides detailed instructions on how to execute commands dynamically in the OpenBB Platform. +keywords: +- OpenBB community +- OpenBB Platform +- Custom commands +- API +- Python Interface +- Dynamic Execution +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + + +Dynamic execution provides an alternate entry point to functions. This method requires formatting the query as demonstrated below. + +```python +from openbb_core.app.command_runner import CommandRunner +runner = CommandRunner() +output = await runner.run( + "/equity/fundamental/ratios", + provider_choices={ + "provider": "fmp", + }, + standard_params={ + "symbol" : "TSLA", + "period" : "quarter", + }, + extra_params={} +) +``` + +```console +>>> output +OBBject + +id: 065241b7-bd9d-7313-8000-9406d8afab75 +results: [{'symbol': 'TSLA', 'date': '2023-06-30', 'period': 'Q2', 'current_ratio':... +provider: fmp +warnings: None +chart: None +extra: {'metadata': {'arguments': {'provider_choices': {'provider': 'fmp'}, 'standa... +``` diff --git a/website/content/platform/user_guides/index.mdx b/website/content/platform/user_guides/index.mdx new file mode 100644 index 000000000000..5e7a96deb7c2 --- /dev/null +++ b/website/content/platform/user_guides/index.mdx @@ -0,0 +1,24 @@ +--- +title: User Guides +sidebar_position: 1 +description: This section provides user_guides of concepts related to usage and getting started with the OpenBB Platform. The pages include example syntax and outline what you can expect while working with the OpenBB Python interface and REST API. +keywords: +- getting started +- explanation +- guide +- syntax +- examples +- symbols +- calendars +- commitment of traders +- financial statements +- time series +- historical prices +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +This section provides more in-depth explanations and tutorials to help users learn how to work and build with the OpenBB Platform. +Pages in this section outline what to expect and include example syntax for use. diff --git a/website/content/platform/user_guides/llm_mode.mdx b/website/content/platform/user_guides/llm_mode.mdx new file mode 100644 index 000000000000..3a032e4b3b77 --- /dev/null +++ b/website/content/platform/user_guides/llm_mode.mdx @@ -0,0 +1,90 @@ +--- +title: LLM Friendly Mode +sidebar_position: 4 +description: This guide provides detailed instructions on how to enable LLM mode in the OpenBB Platform. +keywords: +- OpenBB Platform +- LLM mode +- LLM +- ChatGPT +- AI +- Financial Agents +- LLM Compatibility +- Finance AI +- Custom commands +- Python Interface +--- + +import HeadTitle from '@site/src/components/General/HeadTitle.tsx'; + + + +The OpenBB Platform provides a way to enable the Large Language Model (LLM) mode, which allows you to use LLM frameworks such as [Magentic](https://github.com/jackmpcollins/magentic), [Langchain](https://github.com/langchain-ai/langchain), [Haystack](https://github.com/deepset-ai/haystack), and more. + +This guide outlines the steps to enable LLM mode in the OpenBB Platform. + +## Enabling LLM Mode + +We first start by importing the OpenBB Platform: + +```python +from openbb import obb +``` + +The LLM mode is made possible by setting the system and user preferences to an LLM-compatible mode. + +First, we set the user preference: + +```python +obb.user.preferences.output_type="llm" +``` + +This line of code converts the `OBBject` response data results into a format that works good with LLM models. This is based on our own experience with building LLM agents for financial data. You can try other output types such as `dict`, or similar. You can also build your custom output type. + +Next, we set the system preferences: + +```python +obb.system.python_settings.docstring_sections=['description', 'examples'] +``` + +This system preference trims the docstrings of the commands so that they can fit into the LLM model's context size and also avoid redundant information. The redundant information comes from the information inside the signature of the command that is also written in the docstring. + +As our docstrings are modular we can easily choose which section of the docstrings to include. Available docstring sections are the following: + +- description +- parameters +- returns +- examples + +The next step is to limit the size of the docstrings: + +```python +obb.system.python_settings.docstring_max_length=1024 +``` + +We do this to ensure that the docstrings are not too long for the LLM model to process. The LLM model has a limit on the number of tokens it can process at once, and this setting ensures that the docstrings are within that limit. + +Finally, we can import `openbb` and rebuild the Python static assets to apply these system changes: + +```python +import openbb +openbb.build() +``` + +Now you have successfully enabled LLM mode in the OpenBB Platform. You can now use LLM frameworks to interact with the OpenBB Platform and build financial agents that can understand and respond to financial data. + +For example: + +```python +from magentic import prompt_chain, FunctionCall, OpenaiChatModel + +@prompt_chain( + "You are a helpful financial agent that can use function calling to retrieve data.\nUser Query: {query}", + functions=[obb.equity.price.quote], + model=OpenaiChatModel(model="gpt-4-turbo-preview") +) +def llm(query: str) -> FunctionCall | str: ... + +r = llm(query="What is the current stock price of AAPL?") +r +``` diff --git a/website/content/platform/usage/rest_api.md b/website/content/platform/user_guides/rest_api.mdx similarity index 94% rename from website/content/platform/usage/rest_api.md rename to website/content/platform/user_guides/rest_api.mdx index 1a5df5694a08..4ce4f3a737cd 100644 --- a/website/content/platform/usage/rest_api.md +++ b/website/content/platform/user_guides/rest_api.mdx @@ -1,8 +1,9 @@ --- title: REST API -sidebar_position: 1 +sidebar_position: 6 description: Learn how to configure and deploy the OpenBB Platform's REST API using FastAPI, including detailed guidelines on API documentation, authorization, CORS settings, and server configurations. keywords: +- tutorial - OpenBB Platform - REST API - FastAPI @@ -46,16 +47,31 @@ To learn more about how you can run the API in different scenarios refer to [uvi ## API Documentation +
+ + The Fast API app comes with a swagger documentation page. When running the API locally, navigate to [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs). The API Docs provide interactive descriptions of all available endpoints that you can call right from the documentation web page. +
+ ## Data API Keys -The API keys to your data providers are loaded from the `~/.openbb_platform/user_settings.json` file. You can find more information about the structure of the file and environment variables in the [Local Environment](api_keys#local-environment) section. + +
+ + +The API keys to your data providers are loaded from the `~/.openbb_platform/user_settings.json` file. + +You can find more information about the structure of the file and environment variables in the [Local Environment](api_keys#local-environment) section. + +
## API Authorization +
+ By default, no authorization is required. Basic authorization can be enabled with environment variables. In the `~/.openbb_platform` folder, next to the `user_settings.json`, create a new file, `.env`, if it does not yet exist. Set your Basic Auth credentials. ```.env @@ -87,8 +103,13 @@ response = requests.get(url=url, headers=headers) response.json() ``` +
+ + ## Advanced API Settings +
+ When deploying the API to the public internet, it's crucial to configure it in a way you ensure the application functions correctly and securely. Two critical aspects to consider are Cross-Origin Resource Sharing (CORS) and the configuration of the "servers" list. The configuration for these settings is managed through the `system_settings.json` file, which should be located in the same directory as your `user_settings.json`. This JSON file allows you to specify various settings that affect the behavior of the API. Here's an example structure of the `system_settings.json` file: @@ -138,3 +159,6 @@ In the example above, the settings are permissive ("\*" for origins, methods, an The servers array is used to specify the different environments where your API can be accessed. In the example, there is only one server defined, which is the local development server. For deployment to public internet, you would add an entry for the public server URL and any other environments where your API is accessible. + + +
diff --git a/website/content/platform/user_guides/settings_and_environment_variables.mdx b/website/content/platform/user_guides/settings_and_environment_variables.mdx new file mode 100644 index 000000000000..6b78e019c921 --- /dev/null +++ b/website/content/platform/user_guides/settings_and_environment_variables.mdx @@ -0,0 +1,106 @@ +--- +title: User Settings & Environment Variables +sidebar_position: 3 +description: This section details configuring the OpenBB Platform settings and environment variables. +keywords: + - OpenBB Platform + - Python client + - getting started + - OpenBB Hub + - local environment + - environment variables +--- + +import HeadTitle from "@site/src/components/General/HeadTitle.tsx"; + + + +This page outlines configuring the OpenBB Platform with user settings and environment variables. + +## User Settings + +User preferences are stored locally, `~/.openbb_platform/`, as a JSON file, `user_settings.json`. It is read upon initializing the Python client, or when the Fast API is authorized. If the file does not exist, it will be created on the first run. + +| **Preference** | **Default** | **Options** | **Description** | +| --------------------- | -------------------------------- | ------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| data_directory | /home/OpenBBUserData | Any path. | When launching the application for the first time this directory will be created. It serves as the default location where the application stores usage artifacts such as logs and exports. | +| export_directory | /home/OpenBBUserData/exports | Any path. | The OpenBB Charting Extension provides the capability to export images in various formats. This is the directory where it attempts to save such exports. | +| cache_directory | /home/OpenBBUserData/cache | Any path. | The directory where http requests and database caches are stored, for functions with caching. | +| user_styles_directory | /home/OpenBBUserData/styles/user | Any path. | The OpenBB Charting Extension supports custom stylization. This directory is the location where it looks for user-defined styles. If no user styles are found in this directory the application will proceed with the default styles. | +| charting_extension | openbb_charting | ["openbb_charting"] | Name of the charting extension to be used with the application. | +| chart_style | dark | ["dark", "light"] | The default color style to use with the OpenBB Charting Extension plots. Options include "dark" and "light". | +| plot_enable_pywry | True | [True, False] | Whether the application should enable PyWry. If PyWry is disabled the image will open in your default browser otherwise it will be displayed within your editor or in a separate PyWry window. | +| plot_pywry_width | 1400 | Any positive integer. | PyWry window width. | +| plot_pywry_height | 762 | Any positive integer. | PyWry window height. | +| plot_open_export | False | [True, False] | Controls whether the "Save As" window should pop up as soon as the image is displayed." | +| table_style | dark | ["dark", "light"] | "The default color style to use with the OpenBB Charting Extension tables. Options are "dark" and "light"" | +| request_timeout | 15 | Any positive integer. | Specifies the timeout duration for HTTP requests. | +| metadata | True | [True, False] | Enables or disables the collection of metadata which provides information about operations including arguments duration route and timestamp. Disabling this feature may improve performance in cases where contextual information is not needed or when the additional computation time and storage space are a concern. | +| output_type | OBBject | ["OBBject", "dataframe", "numpy", "dict", "chart", "polars", "llm"] | Specifies the type of data the application will output when a command or endpoint is accessed. Note that choosing data formats only available in Python such as `dataframe`, `numpy` or `polars` will render the application's API non-functional. | +| show_warnings | True | [True, False] | Enables or disables the display of warnings. | + +### Notes on Preferences + +:::note + +- If a `OpenBBUserData` folder in not in the home directory, the application will create one on first run. The user preferences with paths all default to this folder, be it exports, styles or data - this can be changed at any time to suit. +- The `OpenBBUserData` will still be created even if preferences are not pointing to it, this is because the application needs a place to store logs and other artifacts. +- One way to export files or images from the OpenBB Platform is to leverage that functionality from the OpenBB Charting Extension. The `export_directory` preference is the location where the extension will attempt to save CSV and image files. + +::: + +````json +{ + "preferences": { + "data_directory": "~/OpenBBUserData", + "export_directory": "~/OpenBBUserData/exports", + "cache_directory": "~/OpenBBUserData/cache", + "user_styles_directory": "~/OpenBBUserData/styles/user", + "charting_extension": "openbb_charting", + "chart_style": "dark", + "plot_enable_pywry": true, + "plot_pywry_width": 1400, + "plot_pywry_height": 762, + "plot_open_export": false, + "table_style": "dark", + "request_timeout": 15, + "metadata": true, + "output_type": "OBBject" +} +} + + +## Environment Variables + +Environment variables are defined in a `.env` file. If this file does not exist, create it inside the same folder `user_settings.json` is located. + +- `OPENBB_DEBUG_MODE`: enables verbosity while running the program +- `OPENBB_DEVELOP_MODE`: points hub service to .co or .dev +- `OPENBB_AUTO_BUILD`: enables automatic SDK package build on import +- `OPENBB_CHARTING_EXTENSION`: specifies which charting extension to use +- `OPENBB_API_AUTH_EXTENSION`: specifies which authentication extension to use +- `OPENBB_API_AUTH`: enables API authentication for command endpoints +- `OPENBB_API_USERNAME`: sets API username +- `OPENBB_API_PASSWORD`: sets API password + +Variables can be defined for current session only. + +```python +import os +os.environ["OPENBB_DEBUG_MODE"] = "True" +from openbb import obb +```` + +### Proxy Networks + +An environment variable can be set, in the `.env` file, to direct the Requests library to a specific address and port. + +```env +HTTP_PROXY="
" or HTTPS_PROXY="
” +``` + +For example: + +```env +HTTP_PROXY="http://10.10.10.10:8000" +```