Hdxdsys 731 update RTD data usage guide

dev-requirements.txt

@@ -1,5 +1,6 @@
 mkdocs~=1.5.2
 mkdocs-material~=9.2.6
+mkdocs-table-reader-plugin~=2.2.2
 
 pytest~=7.4.2
 pytest-cov~=4.1.0

docs/data_usage_guides/affected_people.md

@@ -0,0 +1,71 @@
+# Affected People
+
+## Refugees & Persons of Concern <a id=”refugees”></a>
+
+This dataset, compiled by the [UNHCR](https://www.unhcr.org/), offers annual
+age- and gender-disaggregated statistics on refugees and people of concern,
+categorised by their country of origin and country of asylum. The data are
+sourced primarily from governments hosting these populations, UNHCR's own
+registration data, and occasionally data published by non-governmental
+organisations.
+
+### Summary
+
+{{ read_yaml('data_usage_guides/subcategory_details/refugees_details.yaml') }}
+
+### Transformations applied
+
+* The table has been reshaped from wide to long: demographic-specific columns
+  have been cast to gender, age, and population
+* It is not possible to p-code based on the location information in the
+  original data, therefore population numbers are aggregated to the national
+  level
+* The reference period is constructed using the full range of the year
+  presented in the “Year” column of the original data
+
+### Usage notes
+
+* UNHCR is the only source for returnees in some countries and is therefore
+  included in HDX HAPI, but if data is available from the IDP endpoint, that
+  data is preferable
+* An “all” value in the `gender` and `age_range` columns indicates no
+  disaggregation
+
+## Humanitarian Needs <a id=”humanitarian-needs”></a>
+
+This Humanitarian Needs Overview (HNO) represents the shared understanding of
+OCHA Humanitarian Country Teams of people's widespread emergency needs during
+crises. It includes an estimate of the number of people by sector who require
+assistance, often referred to as People in Need (PIN), which is derived using
+the [Joint Intersectoral Analysis Framework (JIAF)](https://www.jiaf.info/).
+
+While the HNO data is
+[directly available on HDX](https://data.humdata.org/dataset/?dataseries_name=Humanitarian+Needs+Overview),
+it comes from different OCHA offices and is currently not standardised. Thus,
+HDX HAPI obtains the PIN numbers from the
+[HPC Tools API](https://api.hpc.tools/docs/v1/)-based datasets on HDX.
+
+### Summary
+
+{{ read_yaml('data_usage_guides/subcategory_details/humanitarian_needs_details.yaml') }}
+
+### Transformations applied
+
+* The table has been reshaped from wide to long: the columns of population, in
+  need, targeted, affected, and reached have been cast to a single
+  `population_status` column
+* Sector values of “ALL” have been converted to “intersectoral”
+* Gender and disabled values of “a” have been converted to “all”
+* The methodology in Yemen leads to negative population values in some admin 2
+  level areas. Where negative values appear they have been omitted from the API.
+* The reference period is obtained from the HDX dataset
+
+### Usage notes
+
+* The PIN should *not* be summed across sectors, as the same people can be
+  counted across multiple sectors. For the number of people affected across all
+  sectors, please use the PIN value where sector=intersectoral.
+* An “all” value in the `gender`, `age_range`, `disable_marker`, and
+ `population_group` columns indicates no disaggregation
+* A “population” value in the `population_status` column indicates no d
+  isaggregation

docs/data_usage_guides/coordination_and_context.md

@@ -0,0 +1,103 @@
+# Coordination & Context
+
+## 3W - Who is Doing What Where <a id=”operational-presence”></a>
+
+The [Who does What Where (3W)](https://3w.unocha.org/) is a core humanitarian
+coordination dataset that contains the geographic and sectoral spread of
+humanitarian activities and partners. It is critical to know where humanitarian
+organisations are working and what they are doing in order to ensure
+collaboration and efficient resource allocation, avoid duplication of efforts,
+identify gaps, and plan for future humanitarian response.
+
+### Summary
+
+{{ read_yaml('data_usage_guides/subcategory_details/operational_presence_details.yaml') }}
+
+### Transformations applied
+
+* For consistency and interoperability, we aggregate to an
+  [operational presence](https://humanitarian.atlassian.net/wiki/spaces/imtoolbox/pages/214499412/Who+does+What+Where+3W)
+  level (3W:OP, per org, sector, and admin2), even if the original 3W data is
+  more detailed (e.g. the source lists individual activities)
+* Countries that are not p-coded are aggregated to Admin 0
+* Organisation deduplication is a long-running challenge with this data, since
+  there are no unique identifiers, and organisation names may be spelled
+  different ways by different OCHA offices, or sometimes even within the same
+  3W. See the [`org`](org) section below for more information on how we handle
+  these details.
+* Rows without an associated sector are skipped
+* The reference period is obtained from the HDX dataset
+
+### Usage notes
+
+* Available at either Admin 0, 1, or 2 depending on the country. Please check
+  data coverage for further information
+* We cannot guarantee org consistency over time; for example, if an IMO
+  changes, the 3W might change the spelling or the whole name of an org between
+  releases
+
+## Funding <a id=”funding”></a>
+
+[OCHA Financial Tracking Service](https://fts.unocha.org/home/2024/donors/view)
+(FTS) publishes data on
+humanitarian funding flows as reported by donors and
+recipient organisations. It presents all humanitarian funding to a country
+as well as funding that is reported or that can be specifically
+mapped against funding requirements stated in Humanitarian Response Plans.
+
+### Summary
+
+{{ read_yaml('data_usage_guides/subcategory_details/funding_details.yaml') }}
+
+### Transformations applied
+
+* The present version of the API currently captures only funding associated
+  with an appeal. Funding data without associated appeals will be added in a
+  future version.
+* The reference period is taken from the start and end date provided in each
+  row from the original data
+
+## Conflict Events <a id="conflict-events"></a>
+
+[ACLED](https://acleddata.com/) collects real-time data on the locations,
+dates, actors, fatalities, and types of all reported political violence and
+protest events around the world. HDX HAPI leverages ACLED’s publicly accessible
+data, aggregated on a monthly basis and to administrative regions.
+To learn more about their methodology, please consult the
+[ACLED Codebook](https://acleddata.com/knowledge-base/codebook/).
+
+### Summary
+
+{{ read_yaml('data_usage_guides/subcategory_details/conflict_events_details.yaml') }}
+
+### Transformations applied
+
+* The data for political violence events, civilian targeting events, and
+  demonstrations are in separate resource on HDX, but are combined into a
+  single endpoint in the API
+* Any duplicate rows in the original data are removed
+* The reference period is constructed using the full range of the month
+  presented in the “Month” and “Year” columns of the original data
+
+### Usage notes
+
+* The three event categories are not mutually exclusive, see
+  [`Event Type`](enums.md#event-type) for more details
+* Data is either national or disaggregated to admin 2, see individual resources
+  for more details
+
+## National Risk <a id="national-risk"></a>
+
+The [INFORM Risk Index](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk)
+is a global, open-source risk assessment for humanitarian crises and disasters.
+It can support decisions about prevention, preparedness and response. For more
+information on the methodology, see
+[here](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk/Methodology).
+
+### Summary
+
+{{ read_yaml('data_usage_guides/subcategory_details/national_risk_details.yaml') }}
+
+### Transformations applied
+
+* The reference period is obtained from the HDX dataset

docs/data_usage_guides/enum_parameters/commodity_category_parameters.yaml

@@ -0,0 +1,8 @@
+- commodity category: cereals and tubers
+- commodity category: meat, fish and eggs
+- commodity category: milk and dairy
+- commodity category: miscellaneous food
+- commodity category: non-food
+- commodity category: oil and fats
+- commodity category: pulses and nuts
+- commodity category: vegetables and fruits

docs/data_usage_guides/enum_parameters/disabled_marker_parameters.yaml

@@ -0,0 +1,6 @@
+- disabled marker: y
+  description: 'yes'
+- disabled marker: n
+  description: 'no'
+- disabled marker: all
+  description: 'all'

docs/data_usage_guides/enum_parameters/event_type_parameters.yaml

@@ -0,0 +1,14 @@
+- event type: civilian_targeting
+  description: >
+    Civilian targeting events include violence against civilians events and
+    explosions/remote violence events in which civilians were directly
+    targeted.
+- event type: demonstration
+  description: >
+    Demonstration events include ACLED’s protests and riots event types, with
+    the exception of the mob violence sub-event type of the riots event type.
+- event type: political_violence
+  description: >
+    Political violence events include ACLED’s battles, violence against
+    civilians, and explosions/remote violence event types, as well as the mob
+    violence sub-event type of the riots event type.

docs/data_usage_guides/enum_parameters/gender_parameters.yaml

@@ -0,0 +1,12 @@
+- gender: f
+  description: 'female'
+- gender: m
+  description: 'male'
+- gender: x
+  description: 'non-binary'
+- gender: u
+  description: 'unspecified'
+- gender: o
+  description: 'other'
+- gender: all
+  description: 'all'

docs/data_usage_guides/enum_parameters/ipc_phase_parameters.yaml

@@ -0,0 +1,43 @@
+- IPC Phase: 1
+  Name: None/Minimal
+  Description: >
+    Households are able to meet essential food and non-food needs without
+    engaging in atypical and unsustainable strategies to access food and income.
+- IPC Phase: 2
+  Name: Stressed
+  Description: >
+    Households have minimally adequate food consumption but are unable to
+    afford some essential non-food expenditures without engaging in
+    stress-coping strategies.
+- IPC Phase: 3
+  Name: Crisis
+  Description: >
+    Households either have food consumption gaps that are reflected by high or
+    above-usual acute malnutrition, or are marginally able to meet minimum food
+    needs but only by depleting essential livelihood assets or through
+    crisis-coping strategies.
+- IPC Phase: 4
+  Name: Emergency
+  Description: >
+    Households either have large food consumption gaps which are reflected in
+    very high acute malnutrition and excess mortality, or are able to mitigate
+    large food consumption gaps but only by employing emergency livelihood
+    strategies and asset liquidation.
+- IPC Phase: 5
+  Name: Catastrophe/Famine
+  Description: >
+    Households have an extreme lack of food and/or other basic needs even after
+    full employment of coping strategies. Starvation, death, destitution and
+    extremely critical acute malnutrition levels are evident. (For Famine
+    Classification, an area needs to have extreme critical levels of acute
+    malnutrition and mortality.)
+- IPC Phase: 3+
+  Name: In Need of Action
+  Description: >
+    Sum of population in phases 3, 4, and 5. The population in Phase 3+ does
+    not necessarily reflect the full population in need of urgent action. This
+    is because some households may be in Phase 2 or even 1 but only because of
+    receipt of assistance, and thus, they may be in need of continued action.
+- IPC Phase: all
+  Name: Population
+  Description: Total population

docs/data_usage_guides/enum_parameters/ipc_type_parameters.yaml

@@ -0,0 +1,11 @@
+- IPC type: current
+  Definition: >
+    Food insecurity that is occurring in the current analysis period.
+- IPC type: first projection
+  Definition: >
+    Projected food insecurity occurring in the period immediately following the
+    current analysis period.
+- IPC type: second projection
+  Definition: >
+    Projected food insecurity occurring in the period immediately following the
+    first projection period.

docs/data_usage_guides/enum_parameters/population_group_parameters.yaml

@@ -0,0 +1,32 @@
+- population group: REF
+  description: 'refugees'
+- population group: ROC
+  description: 'people in a refugee-like situation'
+- population group: ASY
+  description: 'asylum seekers'
+- population group: OIP
+  description: 'other people in need of international protection'
+- population group: IDP
+  description: 'internally displaced persons '
+- population group: IOC
+  description: 'people in IDP-like situation'
+- population group: STA
+  description: 'stateless people'
+- population group: OOC
+  description: 'others of concern'
+- population group: HST
+  description: 'host community'
+- population group: RET
+  description: 'returned refugees'
+- population group: RST
+  description: 'resettled refugees'
+- population group: NAT
+  description: 'naturalized refugees'
+- population group: RDP
+  description: 'returned IDPs'
+- population group: POC
+  description: 'persons of concern'
+- population group: RRI
+  description: 'TODO: what is this'
+- population group: all
+  description: 'all'

docs/data_usage_guides/enum_parameters/population_status_parameters.yaml

@@ -0,0 +1,12 @@
+- population status: POP
+  description: 'population'
+- population status: AFF
+  description: 'affected'
+- population status: INN
+  description: 'in-need'
+- population status: TGT
+  description: 'targeted'
+- population status: REA
+  description: 'reached'
+- population status: all
+  description: 'all'

docs/data_usage_guides/enum_parameters/price_flag_parameters.yaml

@@ -0,0 +1,8 @@
+- price flag: actual
+  description: Collected with monthly frequency
+- price flag: aggregate
+  description: >
+    Collected with greater than monthly frequency (daily or weekly),
+    and averaged through a weighted mean to monthly
+- price flag: actual,aggregate
+  description:

docs/data_usage_guides/enum_parameters/price_type_parameters.yaml

@@ -0,0 +1,3 @@
+- price type: Farm Gate
+- price type: Retail
+- price type: Wholesale

docs/data_usage_guides/enum_parameters/risk_class_parameters.yaml

@@ -0,0 +1,20 @@
+- risk class: 1
+  description: very low
+  risk score min: 0.0
+  risk score max: 1.9
+- risk class: 2
+  description: low
+  risk score min: 2.0
+  risk score max: 3.4
+- risk class: 3
+  description: medium
+  risk score min: 3.5
+  risk score max: 4.9
+- risk class: 4
+  description: high
+  risk score min: 5.0
+  risk score max: 6.4
+- risk class: 5
+  description: very high
+  risk score min: 6.5
+  risk score max: 10