Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MaxMind Geolocation provider #1

Closed
wants to merge 51 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
aeffe67
MaxMind Geolocation provider
Apr 12, 2023
8b62ccb
Apply most of review comments
Aug 15, 2023
569a987
Cleanup & fix format
Aug 15, 2023
5ba28ed
Merge remote-tracking branch 'origin/main' into v1.25.4-security-test
Aug 15, 2023
6e3287c
Singleton maxmind config
Sep 4, 2023
04368f1
Address review comments
Sep 6, 2023
8784a98
Revert unrelated change
Sep 6, 2023
8df0173
Fix BUILD file
Sep 6, 2023
52efec3
More fixes
Sep 6, 2023
1237ed7
Try to fix factory
Sep 6, 2023
bea685b
Remove not needed ocde
Sep 6, 2023
aa124fa
Fix more things
Sep 6, 2023
e502d25
Merge remote-tracking branch 'origin/main' into v1.25.4-security-test
Sep 6, 2023
87d06bb
Address TODOs
Sep 8, 2023
5fdf71b
Fix some format
Sep 8, 2023
ac7d26d
Add owners for geoip_providers/common
Sep 8, 2023
728e668
Add title to proto file
Sep 8, 2023
b72dd29
Update api/envoy/extensions/geoip_providers/maxmind/v3/maxmind.proto
Sep 11, 2023
9eba89e
Fix proto format
Sep 11, 2023
6cd9757
Update maxmind.proto
Sep 11, 2023
ef2c242
Try to fix again...
Sep 11, 2023
3f20397
Fix CI+review comment
Sep 11, 2023
480d6a2
Here i try again
Sep 11, 2023
80ae14a
Add protos to toc tree
Sep 12, 2023
01f2030
More docs fixes
Sep 12, 2023
011085f
Fix compile time options and coverage
Sep 13, 2023
f95320e
Maybe fix WIN build
Sep 13, 2023
fee7009
Fix format
Sep 13, 2023
0f1e241
Fix platform build atrefact name
Sep 14, 2023
589223c
Fix CI checks
Sep 15, 2023
b7aa80c
Fix platform related error
Sep 15, 2023
e33798a
Add skip on windows
Sep 18, 2023
9e2e620
Merge remote-tracking branch 'origin/main' into v1.25.4-security-test
Sep 18, 2023
7e570ca
Update changelog
Sep 18, 2023
60ef4ac
Try one more thing
Sep 18, 2023
d485807
cleanup
Sep 18, 2023
9162b23
Turn off building for windows
Sep 18, 2023
c6d0ae7
Fix
Sep 18, 2023
ef299ec
Exclude windows everywhere
Sep 20, 2023
50020d2
Merge remote-tracking branch 'origin/main' into v1.25.4-security-test
Sep 20, 2023
d5fdbf7
Exclude option of building on non linux
Sep 26, 2023
3e8e2d7
Reset back the changelog until ci is fixed
Sep 26, 2023
9751802
Exclude more stuff
Sep 27, 2023
051b1bc
Merge remote-tracking branch 'origin/main'
Sep 27, 2023
6480a0a
Update changelog
Sep 28, 2023
b3d0b15
Cleanup
Sep 29, 2023
c3ccfd9
Add explanatory comment
Oct 2, 2023
b2006b7
Remove single backticks
Oct 2, 2023
b861cfc
Update example
Oct 3, 2023
631a2ef
Merge remote-tracking branch 'origin/main'
Oct 11, 2023
b304f70
Merge remote-tracking branch 'origin/main'
Oct 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -325,6 +325,9 @@ extensions/filters/http/oauth2 @derekargueta @mattklein123
/*/extensions/health_check/event_sinks/file @botengyao @yanavlasov
# IP Geolocation
/*/extensions/filters/http/geoip @nezdolik @ravenblackx
/*/extensions/geoip_providers/common @nezdolik @ravenblackx
# Maxmind geolocation provider
/*/extensions/geoip_providers/maxmind @nezdolik @ravenblackx

/*/extensions/health_checkers/common @zuercher @botengyao

Expand Down
2 changes: 2 additions & 0 deletions api/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,8 @@ proto_library(
"//envoy/extensions/formatter/cel/v3:pkg",
"//envoy/extensions/formatter/metadata/v3:pkg",
"//envoy/extensions/formatter/req_without_query/v3:pkg",
"//envoy/extensions/geoip_providers/common/v3:pkg",
"//envoy/extensions/geoip_providers/maxmind/v3:pkg",
"//envoy/extensions/health_check/event_sinks/file/v3:pkg",
"//envoy/extensions/health_checkers/redis/v3:pkg",
"//envoy/extensions/health_checkers/thrift/v3:pkg",
Expand Down
55 changes: 5 additions & 50 deletions api/envoy/extensions/filters/http/geoip/v3/geoip.proto
Original file line number Diff line number Diff line change
Expand Up @@ -21,52 +21,6 @@ option (xds.annotations.v3.file_status).work_in_progress = true;
// [#extension: envoy.filters.http.geoip]

message Geoip {
// The set of geolocation headers to add to request. If any of the configured headers is present
// in the incoming request, it will be overridden by Geoip filter.
// [#next-free-field: 10]
message GeolocationHeadersToAdd {
// If set, the header will be used to populate the country ISO code associated with the IP address.
string country = 1
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the header will be used to populate the city associated with the IP address.
string city = 2
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the header will be used to populate the region ISO code associated with the IP address.
string region = 3
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the header will be used to populate the ASN associated with the IP address.
string asn = 4
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to any type of anonymization network (e.g. VPN, public proxy etc)
// and header will be populated with the check result. Header value will be set to either "true" or "false" depending on the check result.
string is_anon = 5
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a VPN and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_vpn = 6
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a hosting provider and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_hosting = 7
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a TOR exit node and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_tor = 8
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a public proxy and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_proxy = 9
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];
}

message XffConfig {
// The number of additional ingress proxy hops from the right side of the
// :ref:`config_http_conn_man_headers_x-forwarded-for` HTTP header to trust when
Expand All @@ -82,9 +36,10 @@ message Geoip {
// [#next-free-field: 2]
XffConfig xff_config = 1;

// Configuration for geolocation headers to add to request.
GeolocationHeadersToAdd geo_headers_to_add = 2 [(validate.rules).message = {required: true}];

// Geolocation provider specific configuration.
// Geoip driver specific configuration which depends on the driver being instantiated.
// See the geoip drivers for examples:
//
// - :ref:`MaxMindConfig <envoy_v3_api_msg_extensions.geoip_providers.maxmind.v3.MaxMindConfig>`
// [#extension-category: envoy.geoip_providers]
config.core.v3.TypedExtensionConfig provider = 3 [(validate.rules).message = {required: true}];
}
9 changes: 9 additions & 0 deletions api/envoy/extensions/geoip_providers/common/v3/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.

load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")

licenses(["notice"]) # Apache 2

api_proto_package(
deps = ["@com_github_cncf_udpa//udpa/annotations:pkg"],
)
68 changes: 68 additions & 0 deletions api/envoy/extensions/geoip_providers/common/v3/common.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
syntax = "proto3";

package envoy.extensions.geoip_providers.common.v3;

import "udpa/annotations/status.proto";
import "validate/validate.proto";

option java_package = "io.envoyproxy.envoy.extensions.geoip_providers.common.v3";
option java_outer_classname = "CommonProto";
option java_multiple_files = true;
option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/geoip_providers/common/v3;commonv3";
option (udpa.annotations.file_status).package_version_status = ACTIVE;

// [#protodoc-title: Common Geolocation Provider Configuration]
// Common geolocation provider :ref:`configuration overview <config_geoip_providers_common>`.
// Common configuration shared across geolocation providers.

message CommonGeoipProviderConfig {
// The set of geolocation headers to add to request. If any of the configured headers is present
// in the incoming request, it will be overridden by the :ref:`Geoip filter <config_http_filters_geoip>`.
// [#next-free-field: 10]
message GeolocationHeadersToAdd {
// If set, the header will be used to populate the country ISO code associated with the IP address.
string country = 1
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the header will be used to populate the city associated with the IP address.
string city = 2
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the header will be used to populate the region ISO code associated with the IP address.
// The least specific subdivision will be selected as region value.
string region = 3
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the header will be used to populate the ASN associated with the IP address.
string asn = 4
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to any type of anonymization network (e.g. VPN, public proxy etc)
// and header will be populated with the check result. Header value will be set to either "true" or "false" depending on the check result.
string is_anon = 5
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a VPN and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_vpn = 6
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a hosting provider and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_hosting = 7
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a TOR exit node and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_tor = 8
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];

// If set, the IP address will be checked if it belongs to a public proxy and header will be populated with the check result.
// Header value will be set to either "true" or "false" depending on the check result.
string anon_proxy = 9
[(validate.rules).string = {well_known_regex: HTTP_HEADER_NAME ignore_empty: true}];
}

// Configuration for geolocation headers to add to request.
GeolocationHeadersToAdd geo_headers_to_add = 1 [(validate.rules).message = {required: true}];
}
13 changes: 13 additions & 0 deletions api/envoy/extensions/geoip_providers/maxmind/v3/BUILD
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# DO NOT EDIT. This file is generated by tools/proto_format/proto_sync.py.

load("@envoy_api//bazel:api_build_system.bzl", "api_proto_package")

licenses(["notice"]) # Apache 2

api_proto_package(
deps = [
"//envoy/extensions/geoip_providers/common/v3:pkg",
"@com_github_cncf_udpa//udpa/annotations:pkg",
"@com_github_cncf_udpa//xds/annotations/v3:pkg",
],
)
42 changes: 42 additions & 0 deletions api/envoy/extensions/geoip_providers/maxmind/v3/maxmind.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
syntax = "proto3";

package envoy.extensions.geoip_providers.maxmind.v3;

import "envoy/extensions/geoip_providers/common/v3/common.proto";

import "xds/annotations/v3/status.proto";

import "udpa/annotations/status.proto";
import "validate/validate.proto";

option java_package = "io.envoyproxy.envoy.extensions.geoip_providers.maxmind.v3";
option java_outer_classname = "MaxmindProto";
option java_multiple_files = true;
option go_package = "github.com/envoyproxy/go-control-plane/envoy/extensions/geoip_providers/maxmind/v3;maxmindv3";
option (udpa.annotations.file_status).package_version_status = ACTIVE;
option (xds.annotations.v3.file_status).work_in_progress = true;

// [#protodoc-title: MaxMind Geolocation Provider]
// MaxMind geolocation provider :ref:`configuration overview <config_geoip_providers_maxmind>`.
// At least one geolocation database path :ref:`city_db_path <envoy_v3_api_field_extensions.geoip_providers.maxmind.v3.MaxMindConfig.city_db_path>`,
// :ref:`isp_db_path <envoy_v3_api_field_extensions.geoip_providers.maxmind.v3.MaxMindConfig.isp_db_path>` or
// :ref:`anon_db_path <envoy_v3_api_field_extensions.geoip_providers.maxmind.v3.MaxMindConfig.anon_db_path>` must be configured.
// [#extension: envoy.geoip_providers.maxmind]

message MaxMindConfig {
// Full file path to the Maxmind city database, e.g. /etc/GeoLite2-City.mmdb.
// Database file is expected to have .mmdb extension.
string city_db_path = 1 [(validate.rules).string = {pattern: "^$|^.*\\.mmdb$"}];

// Full file path to the Maxmind ASN database, e.g. /etc/GeoLite2-ASN.mmdb.
// Database file is expected to have .mmdb extension.
string isp_db_path = 2 [(validate.rules).string = {pattern: "^$|^.*\\.mmdb$"}];

// Full file path to the Maxmind anonymous IP database, e.g. /etc/GeoIP2-Anonymous-IP.mmdb.
// Database file is expected to have .mmdb extension.
string anon_db_path = 3 [(validate.rules).string = {pattern: "^$|^.*\\.mmdb$"}];

// Common provider configuration that specifies which geolocation headers will be populated with geolocation data.
common.v3.CommonGeoipProviderConfig common_provider_config = 4
[(validate.rules).message = {required: true}];
}
2 changes: 2 additions & 0 deletions api/versioning/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,8 @@ proto_library(
"//envoy/extensions/formatter/cel/v3:pkg",
"//envoy/extensions/formatter/metadata/v3:pkg",
"//envoy/extensions/formatter/req_without_query/v3:pkg",
"//envoy/extensions/geoip_providers/common/v3:pkg",
"//envoy/extensions/geoip_providers/maxmind/v3:pkg",
"//envoy/extensions/health_check/event_sinks/file/v3:pkg",
"//envoy/extensions/health_checkers/redis/v3:pkg",
"//envoy/extensions/health_checkers/thrift/v3:pkg",
Expand Down
24 changes: 24 additions & 0 deletions bazel/foreign_cc/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -579,3 +579,27 @@ envoy_cmake(
}),
working_directory = "build/cmake",
)

envoy_cmake(
name = "maxmind",
cache_entries = {
"CMAKE_BUILD_TYPE": "Release",
"CMAKE_INSTALL_LIBDIR": "lib",
"CMAKE_CXX_COMPILER_FORCED": "on",
"BUILD_SHARED_LIBS": "off",
"BUILD_TESTING": "off",
},
defines = ["MAXMIND_STATICLIB"],
lib_source = "@com_github_maxmind_libmaxminddb//:all",
out_static_libs = ["libmaxminddb.a"],
tags = ["skip_on_windows"],
)

envoy_cc_library(
name = "maxmind_linux",
srcs = [],
deps = select({
"//bazel:linux": [":maxmind"],
"//conditions:default": [],
}),
)
11 changes: 11 additions & 0 deletions bazel/repositories.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,7 @@ def envoy_dependencies(skip_targets = []):
external_http_archive("bazel_toolchains")
external_http_archive("bazel_compdb")
external_http_archive("envoy_build_tools")
_com_github_maxmind_libmaxminddb()

# TODO(keith): Remove patch when we update rules_pkg
external_http_archive(
Expand Down Expand Up @@ -1398,3 +1399,13 @@ def _is_linux_s390x(ctxt):

def _is_linux_x86_64(ctxt):
return _is_linux(ctxt) and _is_arch(ctxt, "x86_64")

def _com_github_maxmind_libmaxminddb():
external_http_archive(
name = "com_github_maxmind_libmaxminddb",
build_file_content = BUILD_ALL_CONTENT,
)
native.bind(
name = "maxmind",
actual = "@envoy//bazel/foreign_cc:maxmind_linux",
)
15 changes: 15 additions & 0 deletions bazel/repository_locations.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -1459,6 +1459,21 @@ REPOSITORY_LOCATIONS_SPEC = dict(
license = "MIT",
license_url = "https://github.com/protocolbuffers/utf8_range/blob/{version}/LICENSE",
),
com_github_maxmind_libmaxminddb = dict(
project_name = "maxmind_libmaxminddb",
project_desc = "C library for reading MaxMind DB files",
project_url = "https://github.com/maxmind/libmaxminddb",
version = "1.7.1",
sha256 = "e8414f0dedcecbc1f6c31cb65cd81650952ab0677a4d8c49cab603b3b8fb083e",
strip_prefix = "libmaxminddb-{version}",
urls = ["https://github.com/maxmind/libmaxminddb/releases/download/{version}/libmaxminddb-{version}.tar.gz"],
use_category = ["dataplane_ext"],
extensions = ["envoy.geoip_providers.maxmind"],
release_date = "2022-09-30",
cpe = "cpe:2.3:a:maxmind:libmaxminddb:*",
license = "Apache-2.0",
license_url = "https://github.com/maxmind/libmaxminddb/blob/{version}/LICENSE",
),
)

def _compiled_protoc_deps(locations, versions):
Expand Down
3 changes: 3 additions & 0 deletions changelogs/current.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -372,6 +372,9 @@ new_features:
change: |
Added filters to update the filter state for :ref:`the HTTP requests <config_http_filters_set_filter_state>` and
:ref:`the TCP connections <config_network_filters_set_filter_state>`.
- area: geoip
change: |
Added support for :ref:`Maxmind geolocation provider <envoy_v3_api_msg_extensions.geoip_providers.maxmind.v3.MaxMindConfig>`.

deprecated:
- area: tracing
Expand Down
1 change: 1 addition & 0 deletions docs/root/api-v3/config/config.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Extensions
http/header_validators
http/original_ip_detection
http/stateful_session
geoip_provider/geoip_provider
trace/trace
internal_redirect/internal_redirect
path/match/path_matcher
Expand Down
10 changes: 10 additions & 0 deletions docs/root/api-v3/config/geoip_provider/geoip_provider.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
.. _api-v3_config_geoip_providers:

Geolocation providers
=====================

.. toctree::
:glob:
:maxdepth: 2

../../extensions/geoip_providers/*/v3/*
Loading