diff --git a/dev-requirements.txt b/dev-requirements.txt index 50017c95..2a2ee41e 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -1,5 +1,6 @@ mkdocs~=1.5.2 mkdocs-material~=9.2.6 +mkdocs-table-reader-plugin~=2.2.2 pytest~=7.4.2 pytest-cov~=4.1.0 diff --git a/docs/data_usage_guides/affected_people.md b/docs/data_usage_guides/affected_people.md new file mode 100644 index 00000000..ffa8f1e0 --- /dev/null +++ b/docs/data_usage_guides/affected_people.md @@ -0,0 +1,71 @@ +# Affected People + +## Refugees & Persons of Concern + +This dataset, compiled by the [UNHCR](https://www.unhcr.org/), offers annual +age- and gender-disaggregated statistics on refugees and people of concern, +categorised by their country of origin and country of asylum. The data are +sourced primarily from governments hosting these populations, UNHCR's own +registration data, and occasionally data published by non-governmental +organisations. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/refugees_details.yaml') }} + +### Transformations applied + +* The table has been reshaped from wide to long: demographic-specific columns + have been cast to gender, age, and population +* It is not possible to p-code based on the location information in the + original data, therefore population numbers are aggregated to the national + level +* The reference period is constructed using the full range of the year + presented in the “Year” column of the original data + +### Usage notes + +* UNHCR is the only source for returnees in some countries and is therefore + included in HDX HAPI, but if data is available from the IDP endpoint, that + data is preferable +* An “all” value in the `gender` and `age_range` columns indicates no + disaggregation + +## Humanitarian Needs + +This Humanitarian Needs Overview (HNO) represents the shared understanding of +OCHA Humanitarian Country Teams of people's widespread emergency needs during +crises. It includes an estimate of the number of people by sector who require +assistance, often referred to as People in Need (PIN), which is derived using +the [Joint Intersectoral Analysis Framework (JIAF)](https://www.jiaf.info/). + +While the HNO data is +[directly available on HDX](https://data.humdata.org/dataset/?dataseries_name=Humanitarian+Needs+Overview), +it comes from different OCHA offices and is currently not standardised. Thus, +HDX HAPI obtains the PIN numbers from the +[HPC Tools API](https://api.hpc.tools/docs/v1/)-based datasets on HDX. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/humanitarian_needs_details.yaml') }} + +### Transformations applied + +* The table has been reshaped from wide to long: the columns of population, in + need, targeted, affected, and reached have been cast to a single + `population_status` column +* Sector values of “ALL” have been converted to “intersectoral” +* Gender and disabled values of “a” have been converted to “all” +* The methodology in Yemen leads to negative population values in some admin 2 + level areas. Where negative values appear they have been omitted from the API. +* The reference period is obtained from the HDX dataset + +### Usage notes + +* The PIN should *not* be summed across sectors, as the same people can be + counted across multiple sectors. For the number of people affected across all + sectors, please use the PIN value where sector=intersectoral. +* An “all” value in the `gender`, `age_range`, `disable_marker`, and + `population_group` columns indicates no disaggregation +* A “population” value in the `population_status` column indicates no d + isaggregation diff --git a/docs/data_usage_guides/coordination_and_context.md b/docs/data_usage_guides/coordination_and_context.md new file mode 100644 index 00000000..ca9844d1 --- /dev/null +++ b/docs/data_usage_guides/coordination_and_context.md @@ -0,0 +1,103 @@ +# Coordination & Context + +## 3W - Who is Doing What Where + +The [Who does What Where (3W)](https://3w.unocha.org/) is a core humanitarian +coordination dataset that contains the geographic and sectoral spread of +humanitarian activities and partners. It is critical to know where humanitarian +organisations are working and what they are doing in order to ensure +collaboration and efficient resource allocation, avoid duplication of efforts, +identify gaps, and plan for future humanitarian response. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/operational_presence_details.yaml') }} + +### Transformations applied + +* For consistency and interoperability, we aggregate to an + [operational presence](https://humanitarian.atlassian.net/wiki/spaces/imtoolbox/pages/214499412/Who+does+What+Where+3W) + level (3W:OP, per org, sector, and admin2), even if the original 3W data is + more detailed (e.g. the source lists individual activities) +* Countries that are not p-coded are aggregated to Admin 0 +* Organisation deduplication is a long-running challenge with this data, since + there are no unique identifiers, and organisation names may be spelled + different ways by different OCHA offices, or sometimes even within the same + 3W. See the [`org`](org) section below for more information on how we handle + these details. +* Rows without an associated sector are skipped +* The reference period is obtained from the HDX dataset + +### Usage notes + +* Available at either Admin 0, 1, or 2 depending on the country. Please check + data coverage for further information +* We cannot guarantee org consistency over time; for example, if an IMO + changes, the 3W might change the spelling or the whole name of an org between + releases + +## Funding + +[OCHA Financial Tracking Service](https://fts.unocha.org/home/2024/donors/view) +(FTS) publishes data on +humanitarian funding flows as reported by donors and +recipient organisations. It presents all humanitarian funding to a country +as well as funding that is reported or that can be specifically +mapped against funding requirements stated in Humanitarian Response Plans. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/funding_details.yaml') }} + +### Transformations applied + +* The present version of the API currently captures only funding associated + with an appeal. Funding data without associated appeals will be added in a + future version. +* The reference period is taken from the start and end date provided in each + row from the original data + +## Conflict Events + +[ACLED](https://acleddata.com/) collects real-time data on the locations, +dates, actors, fatalities, and types of all reported political violence and +protest events around the world. HDX HAPI leverages ACLED’s publicly accessible +data, aggregated on a monthly basis and to administrative regions. +To learn more about their methodology, please consult the +[ACLED Codebook](https://acleddata.com/knowledge-base/codebook/). + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/conflict_events_details.yaml') }} + +### Transformations applied + +* The data for political violence events, civilian targeting events, and + demonstrations are in separate resource on HDX, but are combined into a + single endpoint in the API +* Any duplicate rows in the original data are removed +* The reference period is constructed using the full range of the month + presented in the “Month” and “Year” columns of the original data + +### Usage notes + +* The three event categories are not mutually exclusive, see + [`Event Type`](enums.md#event-type) for more details +* Data is either national or disaggregated to admin 2, see individual resources + for more details + +## National Risk + +The [INFORM Risk Index](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk) +is a global, open-source risk assessment for humanitarian crises and disasters. +It can support decisions about prevention, preparedness and response. For more +information on the methodology, see +[here](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk/Methodology). + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/national_risk_details.yaml') }} + +### Transformations applied + +* The reference period is obtained from the HDX dataset diff --git a/docs/data_usage_guides/enum_parameters/commodity_category_parameters.yaml b/docs/data_usage_guides/enum_parameters/commodity_category_parameters.yaml new file mode 100644 index 00000000..202f8302 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/commodity_category_parameters.yaml @@ -0,0 +1,8 @@ +- commodity category: cereals and tubers +- commodity category: meat, fish and eggs +- commodity category: milk and dairy +- commodity category: miscellaneous food +- commodity category: non-food +- commodity category: oil and fats +- commodity category: pulses and nuts +- commodity category: vegetables and fruits diff --git a/docs/data_usage_guides/enum_parameters/disabled_marker_parameters.yaml b/docs/data_usage_guides/enum_parameters/disabled_marker_parameters.yaml new file mode 100644 index 00000000..2147c6b7 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/disabled_marker_parameters.yaml @@ -0,0 +1,6 @@ +- disabled marker: y + description: 'yes' +- disabled marker: n + description: 'no' +- disabled marker: all + description: 'all' diff --git a/docs/data_usage_guides/enum_parameters/event_type_parameters.yaml b/docs/data_usage_guides/enum_parameters/event_type_parameters.yaml new file mode 100644 index 00000000..1e29cb5c --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/event_type_parameters.yaml @@ -0,0 +1,14 @@ +- event type: civilian_targeting + description: > + Civilian targeting events include violence against civilians events and + explosions/remote violence events in which civilians were directly + targeted. +- event type: demonstration + description: > + Demonstration events include ACLED’s protests and riots event types, with + the exception of the mob violence sub-event type of the riots event type. +- event type: political_violence + description: > + Political violence events include ACLED’s battles, violence against + civilians, and explosions/remote violence event types, as well as the mob + violence sub-event type of the riots event type. diff --git a/docs/data_usage_guides/enum_parameters/gender_parameters.yaml b/docs/data_usage_guides/enum_parameters/gender_parameters.yaml new file mode 100644 index 00000000..fbbc2b41 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/gender_parameters.yaml @@ -0,0 +1,12 @@ +- gender: f + description: 'female' +- gender: m + description: 'male' +- gender: x + description: 'non-binary' +- gender: u + description: 'unspecified' +- gender: o + description: 'other' +- gender: all + description: 'all' diff --git a/docs/data_usage_guides/enum_parameters/ipc_phase_parameters.yaml b/docs/data_usage_guides/enum_parameters/ipc_phase_parameters.yaml new file mode 100644 index 00000000..0860603b --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/ipc_phase_parameters.yaml @@ -0,0 +1,43 @@ +- IPC Phase: 1 + Name: None/Minimal + Description: > + Households are able to meet essential food and non-food needs without + engaging in atypical and unsustainable strategies to access food and income. +- IPC Phase: 2 + Name: Stressed + Description: > + Households have minimally adequate food consumption but are unable to + afford some essential non-food expenditures without engaging in + stress-coping strategies. +- IPC Phase: 3 + Name: Crisis + Description: > + Households either have food consumption gaps that are reflected by high or + above-usual acute malnutrition, or are marginally able to meet minimum food + needs but only by depleting essential livelihood assets or through + crisis-coping strategies. +- IPC Phase: 4 + Name: Emergency + Description: > + Households either have large food consumption gaps which are reflected in + very high acute malnutrition and excess mortality, or are able to mitigate + large food consumption gaps but only by employing emergency livelihood + strategies and asset liquidation. +- IPC Phase: 5 + Name: Catastrophe/Famine + Description: > + Households have an extreme lack of food and/or other basic needs even after + full employment of coping strategies. Starvation, death, destitution and + extremely critical acute malnutrition levels are evident. (For Famine + Classification, an area needs to have extreme critical levels of acute + malnutrition and mortality.) +- IPC Phase: 3+ + Name: In Need of Action + Description: > + Sum of population in phases 3, 4, and 5. The population in Phase 3+ does + not necessarily reflect the full population in need of urgent action. This + is because some households may be in Phase 2 or even 1 but only because of + receipt of assistance, and thus, they may be in need of continued action. +- IPC Phase: all + Name: Population + Description: Total population diff --git a/docs/data_usage_guides/enum_parameters/ipc_type_parameters.yaml b/docs/data_usage_guides/enum_parameters/ipc_type_parameters.yaml new file mode 100644 index 00000000..1eb96198 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/ipc_type_parameters.yaml @@ -0,0 +1,11 @@ +- IPC type: current + Definition: > + Food insecurity that is occurring in the current analysis period. +- IPC type: first projection + Definition: > + Projected food insecurity occurring in the period immediately following the + current analysis period. +- IPC type: second projection + Definition: > + Projected food insecurity occurring in the period immediately following the + first projection period. diff --git a/docs/data_usage_guides/enum_parameters/population_group_parameters.yaml b/docs/data_usage_guides/enum_parameters/population_group_parameters.yaml new file mode 100644 index 00000000..25e03bc7 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/population_group_parameters.yaml @@ -0,0 +1,32 @@ +- population group: REF + description: 'refugees' +- population group: ROC + description: 'people in a refugee-like situation' +- population group: ASY + description: 'asylum seekers' +- population group: OIP + description: 'other people in need of international protection' +- population group: IDP + description: 'internally displaced persons ' +- population group: IOC + description: 'people in IDP-like situation' +- population group: STA + description: 'stateless people' +- population group: OOC + description: 'others of concern' +- population group: HST + description: 'host community' +- population group: RET + description: 'returned refugees' +- population group: RST + description: 'resettled refugees' +- population group: NAT + description: 'naturalized refugees' +- population group: RDP + description: 'returned IDPs' +- population group: POC + description: 'persons of concern' +- population group: RRI + description: 'TODO: what is this' +- population group: all + description: 'all' diff --git a/docs/data_usage_guides/enum_parameters/population_status_parameters.yaml b/docs/data_usage_guides/enum_parameters/population_status_parameters.yaml new file mode 100644 index 00000000..2ab4a097 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/population_status_parameters.yaml @@ -0,0 +1,12 @@ +- population status: POP + description: 'population' +- population status: AFF + description: 'affected' +- population status: INN + description: 'in-need' +- population status: TGT + description: 'targeted' +- population status: REA + description: 'reached' +- population status: all + description: 'all' diff --git a/docs/data_usage_guides/enum_parameters/price_flag_parameters.yaml b/docs/data_usage_guides/enum_parameters/price_flag_parameters.yaml new file mode 100644 index 00000000..1c5e5db2 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/price_flag_parameters.yaml @@ -0,0 +1,8 @@ +- price flag: actual + description: Collected with monthly frequency +- price flag: aggregate + description: > + Collected with greater than monthly frequency (daily or weekly), + and averaged through a weighted mean to monthly +- price flag: actual,aggregate + description: diff --git a/docs/data_usage_guides/enum_parameters/price_type_parameters.yaml b/docs/data_usage_guides/enum_parameters/price_type_parameters.yaml new file mode 100644 index 00000000..941f3970 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/price_type_parameters.yaml @@ -0,0 +1,3 @@ +- price type: Farm Gate +- price type: Retail +- price type: Wholesale diff --git a/docs/data_usage_guides/enum_parameters/risk_class_parameters.yaml b/docs/data_usage_guides/enum_parameters/risk_class_parameters.yaml new file mode 100644 index 00000000..78baaf87 --- /dev/null +++ b/docs/data_usage_guides/enum_parameters/risk_class_parameters.yaml @@ -0,0 +1,20 @@ +- risk class: 1 + description: very low + risk score min: 0.0 + risk score max: 1.9 +- risk class: 2 + description: low + risk score min: 2.0 + risk score max: 3.4 +- risk class: 3 + description: medium + risk score min: 3.5 + risk score max: 4.9 +- risk class: 4 + description: high + risk score min: 5.0 + risk score max: 6.4 +- risk class: 5 + description: very high + risk score min: 6.5 + risk score max: 10 diff --git a/docs/data_usage_guides/enums.md b/docs/data_usage_guides/enums.md new file mode 100644 index 00000000..af2e7104 --- /dev/null +++ b/docs/data_usage_guides/enums.md @@ -0,0 +1,124 @@ +# Enums + +## Commodity Category + +**Used in:** +[`Food Prices`](food_security_and_nutrition.md#food-prices), +[`WFP Commodity`](metadata.md#wfp-commodity) + +The commodity categories are used in the WFP food prices data to organize +foods into food groups. + +{{ read_yaml('data_usage_guides/enum_parameters/commodity_category_parameters.yaml') }} + +## Disabled Marker + +**Used in:** [`Humanitarian Needs`](affected_people.md#humanitarian-needs) + +This marker is used to indicate any HNO PIN calculations that are disaggregated +by disability. For more information on disability classification, see the +[WHO International Classification of Functioning, Disability and Health (ICF)](https://www.who.int/standards/classifications/international-classification-of-functioning-disability-and-health). + +{{ read_yaml('data_usage_guides/enum_parameters/disabled_marker_parameters.yaml') }} + +## Event Type + +**Used in:** [`Conflict Events`](coordination_and_context.md#conflict-events) + +ACLED compiles information on political violence, demonstrations, and other +non-violent but politically important events. For their public dataset, +these events are aggregated into three, non-mutually exclusive categories, +outlined in the table below. +Please see the [ACLED Codebook](https://acleddata.com/knowledge-base/codebook/#acled-events) +for their methodology and more detailed descriptions of the sub-event types. + +{{ read_yaml('data_usage_guides/enum_parameters/event_type_parameters.yaml') }} + +## Gender + +**Used in:** +[`Baseline Population`](population_and_socio-economy.md#population), +[`Humanitarian Needs`](affected_people.md#humanitarian-needs), +[`Refugees & Persons of Concern`](affected_people.md#refugees) + +Several sub-categories in HDX-HAPI are disaggregated by gender. + +{{ read_yaml('data_usage_guides/enum_parameters/gender_parameters.yaml') }} + +## IPC Code + +**Used in:** [`Food Security`](food_security_and_nutrition.md#food-security) + +The IPC classification includes 5 different phases of increasing severity, +described in detail on page 53 of +[the IPC technical manual version 3.1](https://www.ipcinfo.org/fileadmin/user_upload/ipcinfo/manual/IPC_Technical_Manual_3_Final.pdf), +and summarised in the table below. +This enum also includes Phase 3+, to highlight the population in need of action, +and total population, used to compute fractions. + +{{ read_yaml('data_usage_guides/enum_parameters/ipc_phase_parameters.yaml') }} + +## IPC Type + +**Used in:** [`Food Security`](food_security_and_nutrition.md#food-security) + +The IPC provides different projections to aid in planning and response efforts. + +{{ read_yaml('data_usage_guides/enum_parameters/ipc_type_parameters.yaml') }} + +## Population Group + +**Used in:** +[`Humanitarian Needs`](affected_people.md#humanitarian-needs), +[`Refugees & Persons of Concern`](affected_people.md#refugees) + +Almost all population group terms come from [UNHCR's glossary](https://www.unhcr.org/refugee-statistics/methodology/data-content/), +except for "POC", "RRI", and "all". "RRI" was added as umbrella term +for returnees because unlike UNHCR, the humanitarian +needs data does not distinguish between returned refugees and returned +IDPs. + +{{ read_yaml('data_usage_guides/enum_parameters/population_group_parameters.yaml') }} + +## Population Status + +**Used in:** +[`Humanitarian Needs`](affected_people.md#humanitarian-needs), + +The population status disaggregation provides a framework for the HNO to +determine how effectively people in need are being reached. + +{{ read_yaml('data_usage_guides/enum_parameters/population_status_parameters.yaml') }} + +## Price Flag + +**Used in:** [`Food Prices`](food_security_and_nutrition.md#food-prices) + +Pre-processing characteristics of food prices. + +{{ read_yaml('data_usage_guides/enum_parameters/price_flag_parameters.yaml') }} + +## Price Type + +**Used in:** [`Food Prices`](food_security_and_nutrition.md#food-prices) + +The point in the supply chain at which the price is determined. +See FAO's +[concepts on price data](https://www.fao.org/economic/the-statistics-division-ess/methodology/methodology-systems/price-statistics-and-index-numbers-of-agricultural-production-and-prices/4-concepts-on-price-data/en/) +for more information. + +{{ read_yaml('data_usage_guides/enum_parameters/price_type_parameters.yaml') }} + +## Risk Class + +**Used in:** [`National Risk`](coordination_and_context.md#national-risk) + +In national risk, the final INFORM risk score is grouped into one of five +risk classes, with the objective of systemically identifying risk in a +consistent manner. Note that other categories and dimensions are also +grouped into classes but with different thresholds, and these are not included +in HDX HAPI. Please see the +[methodology](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk/Methodology) +for more details. + +{{ read_yaml('data_usage_guides/enum_parameters/risk_class_parameters.yaml') }} diff --git a/docs/data_usage_guides/food_security_and_nutrition.md b/docs/data_usage_guides/food_security_and_nutrition.md new file mode 100644 index 00000000..4c376c8b --- /dev/null +++ b/docs/data_usage_guides/food_security_and_nutrition.md @@ -0,0 +1,56 @@ +# Food Security & Nutrition + +## Food Security + +The [IPC Acute Food Insecurity (IPC AFI) classification](https://www.ipcinfo.org/ipcinfo-website/ipc-overview-and-classification-system/ipc-acute-food-insecurity-classification/en/) +provides strategically relevant information to decision makers that focuses on +short-term objectives to prevent, mitigate or decrease severe food insecurity. + +The Beta release of HDX HAPI only contains data from the +[Cadre Harmonisé](https://www.cadreharmonise.org/en_GB), as it is p-coded. In +a future release we will p-code and expand coverage to other IPC datasets. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/food_security_details.yaml') }} + +### Transformations applied + +* The table has been reshaped from wide to long: the population and individual +* IPC phase columns have been cast to a combination of `ipc_phase` and + `population_in_phase` +* The IPC fraction is computed in the HDX HAPI API pipeline, by dividing the + population in that phase by the total population (ipc_phase=all). +* The reference period is computed from the “reference_label” and + “reference_year” columns in the original data. In the case of a projection, + it captures date range that the projection covers, not when the projection + was made +* In the `ipc_type` column, “projected” is converted to “first projection” + +### Usage Notes + +* The total population (ipc_phase=all) is not necessarily equal to the sum of + the populations in phases 1-5. The differences are usually small (due to + rounding errors), or because there is no IPC phase data +* Due to the above, the sum of the IPC fractions from phase 1-5 may not sum to + exactly 1 + +## Food Prices + +The World Food Programme Price Database covers foods such as maize, rice, +beans, fish, and sugar for 98 countries and some 3000 markets. It is updated +weekly but primarily contains data with a monthly update frequency. For a +detailed methodology, see WFP's +[Market Analysis Guidelines](https://www.wfp.org/publications/market-analysis-guidelines). + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/food_prices_details.yaml') }} + +### Transformations applied + +* The reference period is computed by converting date from the “date” column, + originally presented as 15th day of a particular month, into a range spanning + the entire month +* The source data is not p-coded, however we have used the admin 1 and 2 names + to p-code most markets. See [Markets](markets) for more details. diff --git a/docs/data_usage_guides/metadata.md b/docs/data_usage_guides/metadata.md new file mode 100644 index 00000000..926b93f3 --- /dev/null +++ b/docs/data_usage_guides/metadata.md @@ -0,0 +1,182 @@ +# Metadata + +## HDX Metadata + +All of the data in HDX HAPI comes from publicly-available datasets on HDX, and +the HDX metadata can be used refer back to the original data to see what HAPI +has simplified or omitted. +The two primary HDX metadata tables are +[`dataset`](metadata.md#dataset) and [`resource`](metadata.md#resource), and they +contain a small subset of the metadata available through the CKAN API or +HDX library. + +The HAPI data records contain enough information to access the full records +from CKAN (see the parameters tables for more details). +All subcategory tables link back to a resource record, and all resources link +back to a dataset record. + +### Dataset + +**Used in:** [`Resource`](metadata.md#resource), all sub-category tables + +This table contains the HDX-specific metadata associated with all datasets +used to populate the HAPI sub-category tables. Every dataset has at least +one child [resource](metadata.md#resource). + +### Resource + +**Used in:** All sub-category tables + +This table contains the HDX-specific metadata associated with all resources +used to populate the HAPI sub-category tables. Every resource has one +parent [dataset](metadata.md#dataset). + +## Geographical Metadata + +HAPI supports three hierarchical levels of geographical metadata: +location (a country or country-like entity), admin 1, and admin 2. An entry in +the location table does not necessarily imply UN recognition of statehood. +The subcategory data tables link to the lowest administrative level used by +that data type; it will usually be admin2, but in some cases may be admin1 or +location. + +The names and p-codes are read in from a [global p-code list](https://data.humdata.org/dataset/global-pcodes) +taken from the common operational dataset (COD) gazetteers administrative +boundaries. +The p-code field is unique only in combination with reference_period_start, +since p-codes may be reused in different versions of geographical metadata. + +HDX HAPI utilises p-code handling provided by the +[`hdx-python-country`](https://hdx-python-country.readthedocs.io/en/latest/) +library. When bringing in data from sub-categories, p-codes are used if +present. Sometimes, these p-codes may not strictly follow the standard with +variations such as shorter or longer country or admin unit codes. +The `hdx-python-country` library has a p-code length matching algorithm to +ensure correct admin units are determined. + +This algorithm works as follows: First, a p-code is verified against the +expected p-code format. Then, a length-matching check is performed, +ensuring that for each segment (country code or admin unit code), +the length matches the predefined format for the country. For example, +a 3-letter country code could be converted to a 2-letter one or vice-versa. +Digits can be prefixed by 0's to increase the length, or have 0's dropped +from the prefix. + +## Location + +**Used in:** [`Admin 1`](metadata.md#admin1), +[`Refugees & Persons of Concern`](affected_people.md#refugees), +[`Funding`](coordination_and_context.md#funding), +[`National Risk`](coordination_and_context.md#national-risk) + +Country or country-like entities in HDX HAPI, from the CODs +[global p-code list](https://data.humdata.org/dataset/global-pcodes). + +## Admin 1 + +**Used in:** [`Admin 2](metadata.md#admin2), +[`Poverty Rate`](population_and_socio-economy.md#poverty-rate) + +Admin 1 level names and p-codes in HDX HAPI, from the CODs +[global p-code list](https://data.humdata.org/dataset/global-pcodes). + +## Admin 2 + +Admin 2 level names and p-codes in HDX HAPI, from the CODs +[global p-code list](https://data.humdata.org/dataset/global-pcodes). + +**Used in:** +[`Humanitarian Needs`](affected_people.md#humanitarian-needs), +[`3W - Who is Doing What Where`](coordination_and_context.md#operational-presence), +[`Conflict Events`](coordination_and_context.md#conflict-events), +[`Food Security`](food_security_and_nutrition.md#food-security), +[`Food Prices`](food_security_and_nutrition.md#food-prices), +[`Baseline Population`](population_and_socio-economy.md#population) + +## Sub-category Metadata + +### Org + +**Used in:** +[`3W - Who is Doing What Where`](coordination_and_context.md#operational-presence) + +The organisation table is populated from the 3W data, using the following +methodology: + +* Organisation name and acronym strings are normalised. If an acronym isn’t + available, the first 32 characters of the name are used. +* This [organisation mapping](https://docs.google.com/spreadsheets/d/e/2PACX-1vSfBWvSu3fKA743VvHtgf-pIGkYH7zhy-NP7DZgEV9_a6YU7vtCeWhbLM56aUL1iIfrfv5UBvvjVt7B/pub?gid=1040329566&single=true&output=csv) + is used for common alternative names +* Organisations must have an associated organisation type. If available, the + organisation type code is taken directly from the 3W data, otherwise the name + string is normalised and matched to the org type names. In the absence of a + direct match, phonetic matching is used for strings > 5 characters. If no + match is found, the organisation is skipped. + +### Org Type + +**Used in:** +[`Org`](metadata.md#org), +[`3W - Who is Doing What Where`](coordination_and_context.md#operational-presence) + +The table is populated using +[OCHA Digital Services organization types list](https://data.humdata.org/dataset/organization-types-beta), +with the addition of: + +* Civil Society +* Observer +* Development Programme +* Local NGO. + +Organisation types all have an associated name and code. + +### Sector + +**Used in:** +[`3W - Who is Doing What Where`](coordination_and_context.md#operational-presence), +[`Humanitarian Needs`](affected_people.md#humanitarian-needs) + +This table is populated using the Global Coordination Groups, with the +following additional entries: + +* cash +* humanitarian assistance +* multi-sector +* intersectoral. + +The sector name strings in the 3W data are normalised and then aligned to the +sector table, using the “sector_map” section of +[this configuration file](https://github.com/OCHA-DAP/hapi-pipelines/blob/main/src/hapi/pipelines/configs/core.yaml) +if needed. +In the absence of a direct match, phonetic matching is used for +strings > 5 characters. + +### Currency + +**Used in:** +[`Food Prices`](food_security_and_nutrition.md#food-prices) + +The currency table is populated using the WFP VAM Data Bridges API. + +### WFP Market + +**Used in:** +[`Food Prices`](food_security_and_nutrition.md#food-prices) + +Markets are defined as the physical locations where buyers and sellers +come together to trade goods and services. +This table is populated from the food prices data, which contains location +names down to admin 2 as well as +a latitude and longitude, but is not p-coded. Consequently, admin 1 and 2 +names must be matched to p-codes using the algorithm provided by the +[`hdx-python-country`](https://hdx-python-country.readthedocs.io/en/latest/) +library which includes phonetic name matching and +manual overrides. + +### WFP Commodity + +**Used in:** +[`Food Prices`](food_security_and_nutrition.md#food-prices) + +The commodity table tracks all food items, and their associated +commodity category, present in the food prices data. diff --git a/docs/data_usage_guides/population_and_socio-economy.md b/docs/data_usage_guides/population_and_socio-economy.md new file mode 100644 index 00000000..09aca4a5 --- /dev/null +++ b/docs/data_usage_guides/population_and_socio-economy.md @@ -0,0 +1,56 @@ +# Population & Socio-economy + +## Baseline Population + +The population statistics presented here are sourced from the +[common operational datasets](https://cod.unocha.org/) (CODs), and are +typically disaggregated by age and/or gender, and extend to administrative +levels 1 or 2. Primary data providers include the UNFPA and OCHA country +offices. These data are projections based on demographic indicators, and +therefore caution should be used when comparing to population data from other +sources. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/population_details.yaml') }} + +### Transformations applied + +* The table has been reshaped from wide to long: demographic-specific columns + have been cast to gender, age, and population +* The reference period is obtained from the HDX dataset + +### Usage Notes + +* Age disaggregation ranges are not consistent across countries +* Any aggregation to a higher administrative level (e.g., admin 1 for a + country where admin 2 is also available) has been taken directly from the + data provided, and was not computed in the API pipeline + +## Poverty Rate + +The global [Oxford Multidimensional Poverty Index](https://ophi.org.uk/global-mpi) +(MPI) measures multidimensional poverty in over 100 developing countries, +using internationally comparable datasets. The MPI assesses poverty through +three main dimensions: health, education, and living standards, each of which +is represented by specific indicators. Please see the +[OPHI methodological note](https://ophi.org.uk/publications/MN-54) for more +details. + +### Summary + +{{ read_yaml('data_usage_guides/subcategory_details/poverty_rate_details.yaml') }} + +### Transformations applied + +* For rows in the original data with two timepoints, we take each timepoint as + single entry into HDX HAPI +* The reference period is constructed using the full range of the year or year + range presented in the “year” column, pertaining to the timepoint in + question, of the original data + +### Usage Notes + +* The data is disaggregated to admin 1, but not p-coded. We have kept the + admin 1 names in the data, but link only to national level p-codes. + We plan to p-code this data in a future release. diff --git a/docs/data_usage_guides/subcategory_details/_details_template.yaml b/docs/data_usage_guides/subcategory_details/_details_template.yaml new file mode 100644 index 00000000..bc356876 --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/_details_template.yaml @@ -0,0 +1,10 @@ +- Attribute: Data provider + Details: +- Attribute: Dataset + Details: +- Attribute: Time series + Details: +- Attribute: Update frequency + Details: +- Attribute: Related tables + Details: diff --git a/docs/data_usage_guides/subcategory_details/conflict_events_details.yaml b/docs/data_usage_guides/subcategory_details/conflict_events_details.yaml new file mode 100644 index 00000000..be44497d --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/conflict_events_details.yaml @@ -0,0 +1,12 @@ +- Attribute: Data provider + Details: ACLED +- Attribute: Dataset + Details: > + [ACLED Conflict Events Data series](https://data.humdata.org/dataset/?dataseries_name=ACLED+-+Conflict+Events) +- Attribute: Time series + Details: Monthly +- Attribute: Update frequency + Details: Weekly +- Attribute: Related tables + Details: > + [`EventType`](enums.md#event-type) diff --git a/docs/data_usage_guides/subcategory_details/food_prices_details.yaml b/docs/data_usage_guides/subcategory_details/food_prices_details.yaml new file mode 100644 index 00000000..1bb168b8 --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/food_prices_details.yaml @@ -0,0 +1,17 @@ +- Attribute: Data provider + Details: The World Food Programme +- Attribute: Dataset + Details: > + [WFP Food Prices data series](https://data.humdata.org/dataset/?dataseries_name=WFP+-+Food+Prices) +- Attribute: Time series + Details: Yes, monthly +- Attribute: Update frequency + Details: Weekly +- Attribute: Related tables + Details: > + [`Commodity Category`](enums.md#commodity-category), + [`Currency`](metadata.md#currency), + [`Price Flag`](enums.md#price-flag), + [`Price Type`](enums.md#price-type), + [`WFP Commodity`](metadata.md#wfp-commodity), + [`WFP Market`](metadata.md#wfp-market) diff --git a/docs/data_usage_guides/subcategory_details/food_security_details.yaml b/docs/data_usage_guides/subcategory_details/food_security_details.yaml new file mode 100644 index 00000000..40b5c09d --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/food_security_details.yaml @@ -0,0 +1,12 @@ +- Attribute: Data provider + Details: Food Security and Nutrition Working Group, West and Central Africa +- Attribute: Dataset + Details: > + [West & Central Africa Food Security Data - Cadre Harmonise (CH) and Integrated Food Security Phase Classification (IPC) data](https://data.humdata.org/dataset/cadre-harmonise) +- Attribute: Time series + Details: Yes, with projections +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: > + [`IPCPhase`](enums.md#ipc-phase), [`IPCType`](enums.md#ipc-type) diff --git a/docs/data_usage_guides/subcategory_details/funding_details.yaml b/docs/data_usage_guides/subcategory_details/funding_details.yaml new file mode 100644 index 00000000..5da5ee52 --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/funding_details.yaml @@ -0,0 +1,11 @@ +- Attribute: Data provider + Details: OCHA FTS +- Attribute: Dataset + Details: > + [OCHA FTS - Requirements and Funding Data series](https://data.humdata.org/dataset/?dataseries_name=OCHA+FTS+-+Requirements+and+Funding+Data) +- Attribute: Time series + Details: Data contains funding since 2006, but the timepoints are not regular +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: None diff --git a/docs/data_usage_guides/subcategory_details/humanitarian_needs_details.yaml b/docs/data_usage_guides/subcategory_details/humanitarian_needs_details.yaml new file mode 100644 index 00000000..84ecac1d --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/humanitarian_needs_details.yaml @@ -0,0 +1,15 @@ +- Attribute: Data provider + Details: OCHA HPC +- Attribute: Dataset + Details: > + [Humanitarian Needs Overview tag](https://data.humdata.org/dataset/?vocab_Topics=humanitarian+needs+overview-hno) +- Attribute: Time series + Details: Past datasets are available in the HPC API, but are sparse prior to 2023 +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: > + [ `DisabledMarker` ](enums.md#disabled-marker), + [`Gender`](enums.md#gender), + [`PopulationGroup`](enums.md#population-group), + [`PopulationStatus`](enums.md#population-status) diff --git a/docs/data_usage_guides/subcategory_details/national_risk_details.yaml b/docs/data_usage_guides/subcategory_details/national_risk_details.yaml new file mode 100644 index 00000000..bb5b5373 --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/national_risk_details.yaml @@ -0,0 +1,12 @@ +- Attribute: Data provider + Details: INFORM +- Attribute: Dataset + Details: > + [INFORM Risk Index](https://data.humdata.org/dataset/inform-risk-index-2021) +- Attribute: Time series + Details: Not available for this subcategory +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: > + [`RiskClass`](enums.md#risk-class) diff --git a/docs/data_usage_guides/subcategory_details/operational_presence_details.yaml b/docs/data_usage_guides/subcategory_details/operational_presence_details.yaml new file mode 100644 index 00000000..c2a463eb --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/operational_presence_details.yaml @@ -0,0 +1,14 @@ +- Attribute: Data provider + Details: OCHA country and regional offices +- Attribute: Dataset + Details: > + [Operational Presence tag on HDX](https://data.humdata.org/dataset/?vocab_Topics=operational%20presence&q=&sort=last_modified%20desc&ext_page_size=25) +- Attribute: Time series + Details: Not available for this subcategory. While older resources are available on HDX, HDX HAPI currently only supports the latest resource. +- Attribute: Update frequency + Details: Irregular, depending on the country. Common update frequencies include quarterly and annually. Please check individual resources for more information. +- Attribute: Related tables + Details: > + [`Org`](metadata.md#org), + [`OrgType`](metadata.md#org-type), + [`Sector`](sector) diff --git a/docs/data_usage_guides/subcategory_details/population_details.yaml b/docs/data_usage_guides/subcategory_details/population_details.yaml new file mode 100644 index 00000000..c063c435 --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/population_details.yaml @@ -0,0 +1,12 @@ +- Attribute: Data provider + Details: UNFPA, OCHA country offices +- Attribute: Dataset + Details: > + [COD - subnational population statistics data series](https://data.humdata.org/dataset/?dataseries_name=COD+-+Subnational+Population+Statistics) +- Attribute: Time series + Details: No +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: > + [`Gender`](enums.md#gender) diff --git a/docs/data_usage_guides/subcategory_details/poverty_rate_details.yaml b/docs/data_usage_guides/subcategory_details/poverty_rate_details.yaml new file mode 100644 index 00000000..39096d5a --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/poverty_rate_details.yaml @@ -0,0 +1,11 @@ +- Attribute: Data provider + Details: Oxford Poverty and Human Development Initiative (OPHI) +- Attribute: Dataset + Details: > + [Oxford Poverty and Human Development Initiative - Global Multidimensional Poverty Index data series](https://data.humdata.org/dataset/?dataseries_name=Oxford+Poverty+and+Human+Development+Initiative+-+Global+Multidimensional+Poverty+Index)" +- Attribute: Time series + Details: Annual, with some combined years +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: None diff --git a/docs/data_usage_guides/subcategory_details/refugees_details.yaml b/docs/data_usage_guides/subcategory_details/refugees_details.yaml new file mode 100644 index 00000000..f6c9b59c --- /dev/null +++ b/docs/data_usage_guides/subcategory_details/refugees_details.yaml @@ -0,0 +1,13 @@ +- Attribute: Data provider + Details: UNHCR +- Attribute: Dataset + Details: > + [Data on forcibly displaced populations and stateless persons (Global)](https://data.humdata.org/dataset/unhcr-population-data-for-world)" +- Attribute: Time series + Details: Annual, dating back to 2001 +- Attribute: Update frequency + Details: Annually +- Attribute: Related tables + Details: > + [`Gender`](enums.md#gender), + [`PopulationGroup`](metadata.md#population-group)" diff --git a/docs/index.md b/docs/index.md index 0e0f660a..69516b88 100644 --- a/docs/index.md +++ b/docs/index.md @@ -2,16 +2,14 @@ --- - -The [HDX Humanitarian API](https://dev.data-humdata-org.ahconu.org/hapi)(HDX HAPI) is a way to access standardised indicators from multiple sources to automate workflows and visualisations - +The [HDX Humanitarian API](https://dev.data-humdata-org.ahconu.org/hapi)(HDX HAPI) is a way to access standardised indicators from multiple sources to automate workflows and visualisations HDX HAPI is in beta phase, and we are seeking feedback. To share your thoughts or join our slack channel, send an email to [hdx@un.org](hdx@un.org). - The initial scope of HDX HAPI will be the data included in the [HDX Data Grids](https://data.humdata.org/dashboards/overview-of-data-grids). Work is ongoing to add more data. # App Identifier + To access HDX HAPI you need to generate an API identifier. This can be done via the the [sandbox interface encode_identifier endpoint](https://stage.hapi-humdata-org.ahconu.org/docs#/Utility/get_encoded_identifier_api_v1_encode_identifier_get). Enter your application name and email address and it will return the app identifier. The key must be included as a query string parameter e.g. ``` @@ -20,7 +18,6 @@ https://stage.hapi-humdata-org.ahconu.org/api/v1/themes/3w?app_identifier={your # The Structure of HDX HAPI - ## Data Subcategory Endpoints HDX HAPI is organised around a set of key humanitarian data subcategories like **Baseline Population** and **Conflict Events**. Each of these subcategories can be queried via its endpoint. @@ -28,25 +25,30 @@ HDX HAPI is organised around a set of key humanitarian data subcategories like * #### Affected People -- [Humanitarian Needs](https://stage.hapi-humdata-org.ahconu.org/docs#/Affected%20people/get_humanitarian_needs_api_v1_affected_people_humanitarian_needs_get) -- [Refugees](https://stage.hapi-humdata-org.ahconu.org/docs#/Affected%20people/get_refugees_api_v1_affected_people_refugees_get) +- [Refugees & Persons of Concern](https://stage.hapi-humdata-org.ahconu.org/docs#/Affected%20people/get_refugees_api_v1_affected_people_refugees_get) +- [Humanitarian Needs](https://stage.hapi-humdata-org.ahconu.org/docs#/Affected%20people/get_humanitarian_needs_api_v1_affected_people_humanitarian_needs_get) -#### Coordination and Context -- [Conflict Events](https://stage.hapi-humdata-org.ahconu.org/docs#/Conflict%20Events/get_conflict_events_api_v1_coordination_context_conflict_event_get) +#### Coordination & Context + +- [3W - Who is Doing What Where](https://stage.hapi-humdata-org.ahconu.org/docs#/3W%20Operational%20Presence/get_operational_presences_api_v1_coordination_context_operational_presence_get) - [Funding](https://stage.hapi-humdata-org.ahconu.org/docs#/Funding/get_fundings_api_v1_coordination_context_funding_get) -- [National risk](https://stage.hapi-humdata-org.ahconu.org/docs#/National%20Risk/get_national_risks_api_v1_coordination_context_national_risk_get) -- [Operational Presence (3W)](https://stage.hapi-humdata-org.ahconu.org/docs#/3W%20Operational%20Presence/get_operational_presences_api_v1_coordination_context_operational_presence_get) +- [Conflict Events](https://stage.hapi-humdata-org.ahconu.org/docs#/Conflict%20Events/get_conflict_events_api_v1_coordination_context_conflict_event_get) +- [National Risk](https://stage.hapi-humdata-org.ahconu.org/docs#/National%20Risk/get_national_risks_api_v1_coordination_context_national_risk_get) + +#### Food Security & Nutrition -#### Food -- [Food Prices](https://stage.hapi-humdata-org.ahconu.org/docs#/Food%20Security%20%26%20Nutrition/get_food_prices_api_v1_food_food_price_get) - [Food Security](https://stage.hapi-humdata-org.ahconu.org/docs#/Food%20Security%20%26%20Nutrition/get_food_security_api_v1_food_food_security_get) +- [Food Prices](https://stage.hapi-humdata-org.ahconu.org/docs#/Food%20Security%20%26%20Nutrition/get_food_prices_api_v1_food_food_price_get) -#### Population Social -- [Population](https://stage.hapi-humdata-org.ahconu.org/docs#/Baseline%20Population/get_populations_api_v1_population_social_population_get) +#### Population & Socio-economy + +- [Baseline Population](https://stage.hapi-humdata-org.ahconu.org/docs#/Baseline%20Population/get_populations_api_v1_population_social_population_get) - [Poverty Rate](https://stage.hapi-humdata-org.ahconu.org/docs#/Baseline%20Population/get_poverty_rates_api_v1_population_social_poverty_rate_get) # Terms Of Use + [The HDX Terms of Service](https://data.humdata.org/faqs/terms) ## FAQS -Please [refer to the landing page](https://dev.data-humdata-org.ahconu.org/hapi) for non-technical FAQs \ No newline at end of file + +Please [refer to the landing page](https://dev.data-humdata-org.ahconu.org/hapi) for non-technical FAQs diff --git a/docs/subcategory_details.md b/docs/subcategory_details.md deleted file mode 100644 index c0b15109..00000000 --- a/docs/subcategory_details.md +++ /dev/null @@ -1,260 +0,0 @@ -# Data subcateogry details and usage notes - ---- - -This page contains detailed information about the data in HAPI, including usage notes on each subcategory of data. - -## Category: Affected People - -### Sub-category: Humanitarian Needs - -This data represents the shared understanding of OCHA Humanitarian Country Teams of people's widespread emergency needs during crises. It includes the estimated number of people who need assistance, often referred to as People in Need (PIN). This PIN data is derived from the [Joint Intersectoral Analysis Framework (JIAF)](https://www.jiaf.info/), which consolidates and analyses evidence to inform and direct strategic response planning. - -The JIAF data is available for all Humanitarian Response Plan (HRP) countries on HDX. However as the raw data comes from multiple providers (OCHA offices), and is not standardised at this time, HDX HAPI obtains the numbers from the [HPC Tools API](https://api.hpc.tools/docs/v1/). Datasets from the HPC API will soon be available on HDX. - -#### Details - - -||| -|:----|:----| -|Time series|Past datasets are available in the HPC API, but are sparse prior to 2023.| -|Update frequency|Annually| -|Data provider|OCHA HPC| -|Datasets|| -|Related endpoints and schemas|Sector, Disabled Marker, Gender, Population Group, Population Status| - -#### Usage Notes - -- The PIN should *not* be summed across sectors, as the same people can be counted across multiple sectors. For the number of people affected across all sectors, please use the PIN value where sector=intersectoral. -- Methodology in Yemen leads to negative population values in some admin 2 level areas. Where negative values appear they have been omitted from the API.https://www.jiaf.info/ - -### Sub-category: Refugees & Persons of Concern - -This dataset, compiled by the UNHCR, provides annual age- and gender-disaggregated statistics on refugees and others of concern, categorised by their country of origin and country of asylum. The data are sourced primarily from governments hosting these populations, UNHCR's own registration data, and occasionally data published by non-governmental organisations. - -#### Details -||| -|:----|:----| -|Time series|Yes, annual dating back to 2001| -|Update frequency|Annually| -|Data provider|UNHCR| -|Dataset|[Data on forcibly displaced populations and stateless persons (Global)](https://data.humdata.org/dataset/unhcr-population-data-for-world)| -|Related endpoints and schemas|Gender, Population Group| - -#### Usage Notes - -- The source data is not p-coded and only includes vague location descriptions. Since these are not consistently mappable to admin subdivisions, we aggregate to the national tables and enumslevel. -- The original data source contains several population groups such as IDPs and asylum seekers, but we only consider refugees (REF) and others of concern (OOC) -- Note that an “all” value in the` `gender` (age_range) column indicates a sum over all genders (age ranges) - -## Category: Coordination & Context - -### Sub-category: Operation Presence (3W - Who is Doing What Where) - -The [Who does What Where (3W)](https://3w.unocha.org/) is a core humanitarian coordination dataset that contains the geographic and sectoral spread of humanitarian activities and partners. It is critical to know where humanitarian organisations are working and what they are doing in order to ensure collaboration and efficient resource allocation, avoid duplication of efforts, identify gaps, and plan for future humanitarian response. - -#### Details - -||| -|:----|:----| -|Time series|Not available for this subcategory. While older resources are available on HDX, HDX HAPI currently only supports the latest resource.| -|Update frequency|Irregular, depending on the country. Common update frequencies include quarterly and annually. Please check individual resources for more information| -|Data provider|OCHA country and regional offices| -|Datasets|See the [data grids](https://data.humdata.org/dashboards/overview-of-data-grids?)| -|Related endpoints and schemas|Org, Org Type, Sector| - -#### Usage Notes - -- This data comes from multiple providers (OCHA offices), in many different formats and levels of detail -- For consistency and interoperability, we aggregate to an [operational presence](https://humanitarian.atlassian.net/wiki/spaces/imtoolbox/pages/214499412/Who+does+What+Where+3W) level (3W:OP, per org, sector, and admin2), even if the original 3W data is more detailed (e.g. the source lists individual activities) - -### Sub-category: Funding - -FTS publishes data on humanitarian funding flows as reported by donors and recipient organisations. It presents all humanitarian funding to a country and funding that is reported or that can be specifically mapped against funding requirements stated in Humanitarian Response Plans. - -#### Details - -||| -|:----|:----| -|Time series|Data contains funding for several years, but the timepoints are not regular| -|Update frequency|Annually| -|Data provider|OCHA FTS| -|Dataset|[OCHA FTS - Requirements and Funding Data series](https://data.humdata.org/dataset/?dataseries_name=OCHA+FTS+-+Requirements+and+Funding+Data)| -|Related endpoints and schemas|None| - -#### Usage Notes - -- The present version of the API currently captures only funding associated with an appeal. Funding data without associated appeals will be added in a future version. - -### Sub-category: Conflict Events - -[ACLED](https://acleddata.com/) collects real-time data on the locations, dates, actors, fatalities, and types of all reported political violence and protest events around the world. - -#### Details - -||| -|:----|:----| -|Time series|Yes, monthly| -|Update frequency|Weekly| -|Data provider|ACLED| -|Dataset|[ACLED Conflict Events Data series](https://data.humdata.org/dataset/?dataseries_name=ACLED+-+Conflict+Events)| -|Related endpoints and schemas|Event Type| - -#### Usage Notes - -- The API uses ACLED’s public aggregated data -- The data for political violence events, civilian targeting events, and demonstrations are in separate resource on HDX, but are combined into a single endpoint in the API - -### Sub-category: National Risk - -The [INFORM Risk Index](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk) is a global, open-source risk assessment for humanitarian crises and disasters. It can support decisions about prevention, preparedness and response. For more information on the methodology, see [here](https://drmkc.jrc.ec.europa.eu/inform-index/INFORM-Risk/Methodology). - -#### Details - -||| -|:----|:----| -|Time series|Not available for this subcategory| -|Update frequency|Annually| -|Data provider|INFORM| -|Dataset|[INFORM Risk Index](https://data.humdata.org/dataset/inform-risk-index-2021)| -|Related endpoints and schemas|RiskClass| - -## Category: Food Security & Nutrition - -### Sub-category: Food Security - -The [IPC Acute Food Insecurity (IPC AFI) classification](https://www.ipcinfo.org/ipcinfo-website/ipc-overview-and-classification-system/ipc-acute-food-insecurity-classification/en/) provides strategically relevant information to decision makers that focuses on short-term objectives to prevent, mitigate or decrease severe food insecurity. - -#### Details - -||| -|:----|:----| -|Time series|Yes, with projections| -|Update frequency|Annually| -|Data provider|Food Security and Nutrition Working Group, West and Central Africa| -|Dataset|[West & Central Africa Food Security Data - Cadre Harmonise (CH) and Integrated Food Security Phase Classification (IPC) data](https://data.humdata.org/dataset/cadre-harmonise)| -|Related endpoints and schemas|IPCPhase, IPCType| - -#### Usage Notes - -- The Beta release only contains data from the [Cadre Harmonisé](https://www.cadreharmonise.org/en_GB), as it is p-coded. In a future release we will p-code and expand coverage to other IPC datasets. -- The reference period refers to the time frame that the projection covers, not when the projection was made -= The IPC fraction is computed in the HDX HAPI API pipeline, buy dividing the population by the the total population (ipc_phase=all). -- The total population (ipc_phase=all) is not necessarily equal to the sum of the populations in phases 1-5. - -### Sub-category: Food Prices - -The World Food Programme Price Database covers foods such as maize, rice, beans, fish, and sugar for 98 countries and some 3000 markets. It is updated weekly but primarily contains data with a monthly update frequency. - -#### Details - -||| -|:----|:----| -|Time series|Yes, primarily monthly| -|Update frequency|Weekly| -|Data provider|The World Food Programme| -|Dataset|[WFP Food Prices data series](https://data.humdata.org/dataset/?dataseries_name=WFP+-+Food+Prices)| -|Related endpoints and schemas|Commodity, Currency, Market| - -#### Usage Notes - -- The source data is not p-coded, however we have used the admin 1 and 2 names to p-code most markets. See the Markets section for more details. - -## Category: Population & Socio-economy - -### Sub-category: Baseline Population - -This data comprises population statistics sourced from the [common operational datasets](https://cod.unocha.org/) (CODs), typical disaggregated by age and/or gender, and reaching administrative levels 1 or 2. The primary sources include the UNFPA and OCHA country offices, utilised for coordination purposes, although disparities may arise when compared to alternative sources. - -#### Details - -||| -|:----|:----| -|Time series|No| -|Update frequency|Annually| -|Data provider|UNFPA, OCHA country offices| -|Dataset|[COD - subnational population statistics data series](https://data.humdata.org/dataset/?dataseries_name=COD+-+Subnational+Population+Statistics)| -|Related endpoints and schemas|Gender| - -### Sub-category: Poverty Rate - -The global [Oxford Multidimensional Poverty Index](https://ophi.org.uk/global-mpi) (MPI) measures multidimensional poverty in over 100 developing countries, using internationally comparable datasets. -The MPI assesses poverty through three main dimensions: health, education, and living standards, each of which is represented by specific indicators.Please see the [OPHI methodological note](https://ophi.org.uk/publications/MN-54) for more details. - -#### Details - -||| -|:----|:----| -|Time series|Annual, with some combined years| -|Update frequency|Annually| -|Data provider|Oxford Poverty and Human Development Initiative (OPHI)| -|Dataset|[Oxford Poverty and Human Development Initiative - Global Multidimensional Poverty Index data series](https://data.humdata.org/dataset/?dataseries_name=Oxford+Poverty+and+Human+Development+Initiative+-+Global+Multidimensional+Poverty+Index)| -|Related endpoints and schemas|None| - - -#### Usage Notes - -The source data is very detailed. We’ve chosen the following subset of indicators, detailed below: - -|Indicator|format|Description| -|:----|:----|:----| -|mpi|fraction|The multidimensional poverty index. Derived as a product of the headcount ratio and intensity of deprivation.| -|headcount_ratio|%|The percentage of people deprived in 33% or more indicators| -|intensity_of_deprivation|%|The average proportion of indicators in which people are deprived| -|vulnerable_to_poverty|%|The percentage of people deprived in 20-33% of indicators| -|in_severe_poverty|%|The percentage of people deprived in 50% or more indicators| - -## Metadata - -### Sectors - -- The list of sectors is available through the sector endpoint -- This table is populated using the Global Coordination Groups, with the following additional entries: - - cash - - humanitarian assistance - - multi-sector - - intersectoral -The sector name strings in the 3W data are normalised and then aligned to the sector table, using the “sector_map” section of this configuration file if needed: https://github.com/OCHA-DAP/hapi-pipelines/blob/main/src/hapi/pipelines/configs/core.yaml. In the absence of a direct match, phonetic matching is used for strings > 5 characters. - -### Org - -- The organisation table is populated from the 3W data -- Organisation name and acronym strings are normalised. If an acronym isn’t available, the first 32 characters of the name are used. -- This [organisation mapping](https://docs.google.com/spreadsheets/d/e/2PACX-1vSfBWvSu3fKA743VvHtgf-pIGkYH7zhy-NP7DZgEV9_a6YU7vtCeWhbLM56aUL1iIfrfv5UBvvjVt7B/pub?gid=1040329566&single=true&output=csv) is used for common alternative names -- Organisations without a sector are not included -- Organisations can have an associated organisation type. If available, the organisation type code is taken directly from the 3W data, otherwise the name string is normalised and matched to the org type names. In the absence of a direct match, phonetic matching is used for strings > 5 characters. If no match is found, the organisation is skipped. - -### OrgType - -- The table is populated using [OCHA Digital Services organization types list](https://data.humdata.org/dataset/organization-types-beta), with the addition of: - - Civil Society - - Observer - - Development Programme - - Local NGO -Organisation types all have an associated name and code - -### IPC Code - -The IPC classification includes 5 different phases of increasing severity, described in detail on page 53 of [the IPC technical manual version 3.1](https://www.ipcinfo.org/fileadmin/user_upload/ipcinfo/manual/IPC_Technical_Manual_3_Final.pdf), and summarised below: - -|Phase|Name|Description| -|:----|:----|:----| -|1|None/Minimal|Households are able to meet essential food and non-food needs without engaging in atypical and unsustainable strategies to access food and income.| -|2|Stressed|Households have minimally adequate food consumption but are unable to afford some essential non-food expenditures without engaging in stress-coping strategies.| -|3|Crisis|Households either have food consumption gaps that are reflected by high or above-usual acute malnutrition, or are marginally able to meet minimum food needs but only by depleting essential livelihood assets or through crisis-coping strategies.| -|4|Emergency|Households either have large food consumption gaps which are reflected in very high acute malnutrition and excess mortality, or are able to mitigate large food consumption gaps but only by employing emergency livelihood strategies and asset liquidation.| -|5|Catastrophe/Famine|Households have an extreme lack of food and/or other basic needs even after full employment of coping strategies. Starvation, death, destitution and extremely critical acute malnutrition levels are evident. (For Famine Classification, an area needs to have extreme critical levels of acute malnutrition and mortality.)| -|3|In Need of Action|Sum of population in phases 3, 4, and 5. The population in Phase 3+ does not necessarily reflect the full population in need of urgent action. This is because some households may be in Phase 2 or even 1 but only because of receipt of assistance, and thus, they may be in need of continued action.| -|all|Total population|Total population| - -The above table also includes Phase 3+, to highlight the population in need of action, and total population, used to compute fractions. - -### IPC Type - -The IPC provides different projections to aid in planning and response efforts, outlined in the following table: - -|IPC type|Definition| -|:----|:----| -|current|Food insecurity that is occurring in the current analysis period.| -|first projection|Projected food insecurity occurring in the period immediately following the current analysis period.| -|second projection|Projected food insecurity occurring in the period immediately following the first projection period.| \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index 449ee77c..de4dd6b1 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -4,8 +4,26 @@ nav: - Getting Started: getting-started.md - Data Coverage: data.md - Code Examples: examples.md - - Data Usage Guide: subcategory_details.md + - Data Usage Guide: + - Affected People: data_usage_guides/affected_people.md + - Coordination & Context : data_usage_guides/coordination_and_context.md + - Food Security & Nutrition : data_usage_guides/food_security_and_nutrition.md + - Population & Socio-economy : data_usage_guides/population_and_socio-economy.md + - Metadata: data_usage_guides/metadata.md + - Enums: data_usage_guides/enums.md - Geo Data: geo.md - Contact: contact.md theme: - name: "material" \ No newline at end of file + name: "material" +markdown_extensions: + - pymdownx.highlight: + anchor_linenums: true + line_spans: __span + pygments_lang_class: true + - pymdownx.inlinehilite + - pymdownx.snippets + - pymdownx.superfences +plugins: + - search + - table-reader: + base_path: "docs_dir"