一篇学会 Metadata 巧妙使用动态元数据

1什么是篇学metadata

metadata就是元数据信息，他提供了基于匹配的妙使 listeners, filter chains, routes and endpoints的 额外的输入参数到过滤器，他是用动一种map的格式，通常是态元filter的名字(反向dns格式)。过滤器元数据的数据键值对在请求处理和连接发生时会别合并，后面的篇学值会覆盖前面的值。元数据有一个名称空间的妙使概念，然后是用动键值对。比如提供额外数据给httpConnectionManager的态元元数据名称空间 envoy.http_connection_manager.access_log 。另一个例子是数据每个service使用的cluster的元数据信息，他可能被多个过滤器使用。篇学对于负载均衡来说，妙使元数据提供了一种方法，用动来子集端点信息。态元关联元数据的数据endpoint，亿华云计算路由一个特定的元数据来选择端点。元数据有四种类型，分别是request类型，route类型，cluster类型，host类型。

2metadata有什么作用

tracing customtag值得来源，路由元数据匹配，负载均衡子集决策，ratelimit 动作配置，基于元数据的权限控制，本地响应映射元数据过滤，等。

3metadata数据来源

3.1 envoy.filters.http.ext_authz

当使用grpc授权服务器时，当CheckResponse包含dynamic_metadata字段时，会产生动态元数据信息。

当使用http授权服务器时，当来自授权服务器的响应头匹配 dynamic_metadata_from_headers 的配置值会产生元数据信息。动态元数据的key是匹配的头，动态元数据的值是匹配头的值。

3.2 envoy.filters.network.ext_authz

当使用grpc授权服务器时，当CheckResponse包含dynamic_metadata字段时，源码下载会产生动态元数据信息。

3.3 envoy.filters.http.header_to_metadata

配置一些规则，每条规则有header或cookie，当配置的值存在或不存在时就会触发规则，用来设置动态元数据。

比如：

http_filters:   - name: envoy.filters.http.header_to_metadata     typed_config:       "@type": type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config       request_rules:         - header: x-version           on_header_present:             metadata_namespace: envoy.lb             key: version             type: STRING           on_header_missing:             metadata_namespace: envoy.lb             key: default             value: true             type: STRING           remove: false 

上面规则显示，当x-version头存在时设置envoy.lb名称空间的key为version元数据为x-version的值，当x-version不存在时，设置envoy.lb的名称空间的key为default的元数据的值为true。

3.4 envoy.filters.http.jwt_authn

可以配置 **payload_in_**metadata ，成功验证jwt payload会写到metadata中，名称空间是envoy.filters.http.jwt_authn，例子：

envoy.filters.http.jwt_authn:   my_payload:     iss: https://example.com     sub: test@example.com     aud: https://example.com     exp: 1501281058 

可以配置 **header_in_**metadata ，成功验证的头会写到metadata中，名称空间是envoy.filters.http.jwt_authn，例子：

envoy.filters.http.jwt_authn:   my_header:     alg: JWT     kid: EF71iSaosbC5C4tC6Syq1Gm647M     alg: PS256 

3.5 envoy.filters.network.mongo_proxy

当 emit_dynamic_metadata 为true时会产生metadata，格式如下

Name Type Description key string The resource name in db.collection format. value array A list of strings representing the operations executed on the resource (insert/update/query/delete).

3.6envoy.filters.network.mysql_proxy

当发送到服务端的sql被解析后，会产生动态元数据信息，格式如下：

Name Type Description <table.db> string The resource name in table.db format. The resource name defaults to the table being accessed if the database cannot be inferred. [] list A list of strings representing the operations executed on the resource. Operations can be one of insert/update/select/drop/delete/create/alter/show.

3.7envoy.filters.network.postgres_proxy

语句被解析后，会产生动态元数据，格式如下：

Name Type Description <table.db> string The resource name in table.db format. [] list A list of strings representing the operations executed on the resource. Operations can be one of insert/update/select/drop/delete/create/alter/show.

3.8 envoy.filters.http.rbac

会产生如下元数据：

shadow_effective_policy_id string The effective shadow policy ID matching the action (if any). shadow_engine_result string The engine result for the shadow rules (i.e. either allowed or denied). access_log_hint boolean Whether the request should be logged. This metadata is shared and set under the key namespace ‘envoy.common’ (See Shared Dynamic Metadata).

3.9 envoy.filters.network.rbac

会产生如下元数据：

Name Type Description shadow_effective_policy_id string The effective shadow policy ID matching the action (if any). shadow_engine_result string The engine result for the shadow rules (i.e. either allowed or denied). access_log_hint boolean Whether the request should be logged. This metadata is shared and set under the key namespace ‘envoy.common’ (See Shared Dynamic Metadata).

3.10envoy.filters.network.zookeeper_proxy

当每个消息被解析后，会产生如下元数据：

Name Type Description   string The path associated with the request, response or event   string The opname for the request, response or event <create_type> string The string representation of the flags applied to the znode   string The size of the request message in bytes   string True if a watch is being set, false otherwise   string The version parameter, if any, given with the request   string The timeout parameter in a connect response <protocol_version> string The protocol version in a connect response   string The readonly flag in a connect response   string The zxid field in a response header   string The error field in a response header <client_state> string The state field in a watch event <event_type> string The event type in a a watch event

3.11 envoy.filters.http.ratelimit

当ratelimit服务返回 RateLimitResponse 带有dynamic_metadata时，服务器租用会产生元数据信息。

4metadata怎么使用

4.1type.tracing.v3.CustomTag

{    "tag": "...",   "literal": "{ ...}",   "environment": "{ ...}",   "request_header": "{ ...}",   "metadata": "{ ...}" } 

metadata：自定义tag，值从metadata中获取

案例：

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: httpconnectionmanager spec:   workloadSelector:     labels:       istio: ingressgateway   configPatches:   - applyTo: NETWORK_FILTER     match:       context: GATEWAY       listener:         portNumber: 8080         filterChain:           filter:             name: "envoy.filters.network.http_connection_manager"     patch:       operation: MERGE       value:               name: envoy.filters.network.http_connection_manager               typedConfig:                 @type: type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager                 route_config:                   name: test                   virtual_hosts:                   - name: test                     domains:                     - "*"                     routes:                     - name: testroute                       match:                          prefix: /                       direct_response:                         status: 200                         body:                            inline_string: "prefix"                       tracing:                           customTags:                           - metadata:                               kind:                                 request: { }                               metadataKey:                                 key: envoy.filters.http.rbac                                 path:                                 - key: istio_dry_run_allow_shadow_effective_policy_id                             tag: istio.authorization.dry_run.allow_policy.name                           - metadata:                               kind:                                 request: { }                               metadataKey:                                 key: envoy.filters.http.rbac                                 path:                                 - key: istio_dry_run_allow_shadow_engine_result                             tag: istio.authorization.dry_run.allow_policy.result                           - metadata:                               kind:                                 request: { }                               metadataKey:                                 key: envoy.filters.http.rbac                                 path:                                 - key: istio_dry_run_deny_shadow_effective_policy_id                             tag: istio.authorization.dry_run.deny_policy.name                           - metadata:                               kind:                                 request: { }                               metadataKey:                                 key: envoy.filters.http.rbac                                 path:                                 - key: istio_dry_run_deny_shadow_engine_result                             tag: istio.authorization.dry_run.deny_policy.result                           - literal:                               value: latest                             tag: istio.canonical_revision                           - literal:                               value: istio-ingressgateway                             tag: istio.canonical_service                           - literal:                               value: mesh1                             tag: istio.mesh_id                           - literal:                               value: istio-system                             tag: istio.namespace                           overallSampling:                             numerator: 100                             denominator: HUNDRED                           randomSampling:                             numerator: 1                             denominator: HUNDRED                           clientSampling:                             numerator: 100                             denominator: HUNDRED 

4.2路由设置metadata

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: httpconnectionmanager spec:   workloadSelector:     labels:       istio: ingressgateway   configPatches:   - applyTo: NETWORK_FILTER     match:       context: GATEWAY       listener:         portNumber: 8080         filterChain:           filter:             name: "envoy.filters.network.http_connection_manager"     patch:       operation: MERGE       value:               name: envoy.filters.network.http_connection_manager               typedConfig:                 @type: type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager                 route_config:                   name: test                   virtual_hosts:                   - name: test                     domains:                     - "*"                     routes:                     - name: testroute                       match:                          prefix: /                       metadata:                         filter_metadata:                           "envoy.lb":                              canary: true                       direct_response:                         status: 200                         body:                            inline_string: "prefix" 

4.3local_reply_config

{    "mappers": [],   "body_format": "{ ...}" } 

mappers:

{    "filter": "{ ...}",   "status_code": "{ ...}",   "body": "{ ...}",   "body_format_override": "{ ...}",   "headers_to_add": [] } 

filter:

{    "status_code_filter": "{ ...}",   "duration_filter": "{ ...}",   "not_health_check_filter": "{ ...}",   "traceable_filter": "{ ...}",   "runtime_filter": "{ ...}",   "and_filter": "{ ...}",   "or_filter": "{ ...}",   "header_filter": "{ ...}",   "response_flag_filter": "{ ...}",   "grpc_status_filter": "{ ...}",   "extension_filter": "{ ...}",   "metadata_filter": "{ ...}" } 

metadata_filter：

{    "matcher": "{ ...}",匹配条件   "match_if_key_not_found": "{ ...}"key不存在时是否匹配 } 

matcher:

{    "filter": "...",过滤名称   "path": [],metadata路径   "value": "{ ...}",匹配值   "invert": "..."反向匹配 } 

value:

{    "null_match": "{ ...}",null匹配   "double_match": "{ ...}",double匹配   "string_match": "{ ...}",string匹配   "bool_match": "...",bool匹配   "present_match": "...",存在性匹配   "list_match": "{ ...}"列表匹配 } 

string_match:

{    "exact": "...",   "prefix": "...",   "suffix": "...",   "safe_regex": "{ ...}",   "contains": "...",   "ignore_case": "..." } 

body_format:

{    "text_format": "...",   "json_format": "{ ...}",   "text_format_source": "{ ...}",   "omit_empty_values": "...",   "content_type": "...",   "formatters": [] } 

ef-local_reply_config.yaml

kubectl apply -f ef-local_reply_config.yaml -n istio-system

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: httpconnectionmanager spec:   workloadSelector:     labels:       istio: ingressgateway   configPatches:   - applyTo: NETWORK_FILTER     match:       context: GATEWAY       listener:         portNumber: 8080         filterChain:           filter:             name: "envoy.filters.network.http_connection_manager"     patch:       operation: MERGE       value:               name: envoy.filters.network.http_connection_manager               typedConfig:                 @type: type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager                 route_config:                   name: test                   virtual_hosts:                   - name: test                     domains:                     - "*"                     routes:                     - name: testroute                       match:                          prefix: /product                       route:                         weighted_clusters:                           clusters:                           - name: outbound|9080||productpage.istio.svc.cluster.local                             weight: 100                           total_weight: 100                           runtime_key_prefix: test                 local_reply_config:                   mappers:                   - status_code: 200                     filter:                       metadata_filter:                         matcher:                           filter: envoy.lb                           path:                            - key: canary                           value:                             string_match:                               exact: "true"                           invert: false                         match_if_key_not_found: true                     body:                        inline_string: "test"                     body_format_override:                       text_format: "%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n"                      headers_to_add:                     - header:                         key: test                         value: test                       append: true                   body_format:                     text_format: "%LOCAL_REPLY_BODY%:%RESPONSE_CODE%:path=%REQ(:path)%\n"       

4.4 envoy.filters.http.set_metadata

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: set spec:   workloadSelector:     labels:       app: productpage   configPatches:   - applyTo: HTTP_FILTER     match:       context: SIDECAR_INBOUND       listener:         portNumber: 9080         filterChain:           destinationPort: 9080           filter:             name: "envoy.filters.network.http_connection_manager"             subFilter:               name: "envoy.filters.http.router"     patch:       operation: INSERT_BEFORE       value:                    name: envoy.filters.http.set_metadata           typedConfig:             @type: type.googleapis.com/envoy.extensions.filters.http.set_metadata.v3.Config             metadata_namespace: envoy.lb             value:               canary: "true" 

4.5route match

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: httpconnectionmanager spec:   workloadSelector:     labels:       istio: ingressgateway   configPatches:   - applyTo: NETWORK_FILTER     match:       context: GATEWAY       listener:         portNumber: 8080         filterChain:           filter:             name: "envoy.filters.network.http_connection_manager"     patch:       operation: MERGE       value:               name: envoy.filters.network.http_connection_manager               typedConfig:                 @type: type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager                 route_config:                   name: test                   virtual_hosts:                   - name: test                     domains:                     - "*"                     routes:                     - name: testroute                       match:                          path: /tEst                         case_sensitive: false                         dynamic_metadata:                           filter: envoy.lb                           path:                           - key: canary                           value: "true"                           invert: false                       direct_response:                         status: 200                         body:                            inline_string: "runtime_fraction" 

4.6基于元数据的权限控制

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: apply-to spec:   workloadSelector:     labels:       app: mysqldb   configPatches:   - applyTo: NETWORK_FILTER     match:       listener:         portNumber: 3306         filterChain:           filter:             name: "envoy.filters.network.tcp_proxy"     patch:       operation: INSERT_BEFORE       value:         name: envoy.filters.network.mysql_proxy         typed_config:           "@type": type.googleapis.com/envoy.extensions.filters.network.mysql_proxy.v3.MySQLProxy           stat_prefix: mysql   - applyTo: NETWORK_FILTER     match:       listener:         portNumber: 3306         filterChain:           filter:             name: "envoy.filters.network.tcp_proxy"     patch:       operation: INSERT_BEFORE       value:         name: envoy.filters.network.rbac         typed_config:           "@type": type.googleapis.com/envoy.extensions.filters.network.rbac.v3.RBAC           stat_prefix: rbac           rules:              action: DENY              policies:                "product-viewer":                  permissions:                  - metadata:                      filter: envoy.filters.network.mysql_proxy                      path:                      - key: t1.test                      value:                        list_match:                          one_of:                            string_match:                              prefix: update                  principals:                  - any: true           enforcement_type: CONTINUOUS  

4.7负载均衡决策

略，等讲cluster再看

4.8header-To-Metadata Filter

{    "request_rules": [],   "response_rules": [] } 

request_rules:

{    "header": "...",   "cookie": "...",   "on_header_present": "{ ...}",   "on_header_missing": "{ ...}",   "remove": "..." } 

response_rules:

{    "header": "...",   "cookie": "...",   "on_header_present": "{ ...}",   "on_header_missing": "{ ...}",   "remove": "..." } 

on_header_present,on_header_missing:

{    "metadata_namespace": "...",   "key": "...",   "value": "...",   "regex_value_rewrite": "{ ...}",   "type": "...",   "encode": "..." }  apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: set spec:   workloadSelector:     labels:       app: productpage   configPatches:   - applyTo: HTTP_FILTER     match:       context: SIDECAR_INBOUND       listener:         portNumber: 9080         filterChain:           destinationPort: 9080           filter:             name: "envoy.filters.network.http_connection_manager"             subFilter:               name: "envoy.filters.http.router"     patch:       operation: INSERT_BEFORE       value:                    name: envoy.filters.http.header_to_metadata           typedConfig:             @type: type.googleapis.com/envoy.extensions.filters.http.header_to_metadata.v3.Config             request_rules:             - header: x-version               on_header_present:                 metadata_namespace: envoy.lb                 key: version                 type: STRING               on_header_missing:                 metadata_namespace: envoy.lb                 key: default                 value: true                 type: STRING               remove: false             response_rules:             - header: x-version               on_header_present:                 metadata_namespace: envoy.lb                 key: version                 type: STRING               on_header_missing:                 metadata_namespace: envoy.lb                 key: default                 value: true                 type: STRING               remove: false             

4.9ratelimit actions

apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata:   name: httpconnectionmanager spec:   workloadSelector:     labels:       istio: ingressgateway   configPatches:   - applyTo: NETWORK_FILTER     match:       context: GATEWAY       listener:         portNumber: 8080         filterChain:           filter:             name: "envoy.filters.network.http_connection_manager"     patch:       operation: MERGE       value:               name: envoy.filters.network.http_connection_manager               typedConfig:                 @type: type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager                 route_config:                   name: test                   virtual_hosts:                   - name: test                     domains:                     - "*"                     routes:                     - name: testroute                       match:                          prefix: /product                       route:                         rate_limits:                         - stage: 0                           disable_key: test                           actions:                           - metadata:                                descriptor_key: test                               default_value: test                               metadata_key:                                 key: envoy.lb                                 path:                                 - key: canary                               source: DYNAMIC                           limit:                             dynamic_metadata:                               metadata_key:                                 key: envoy.lb                                 path:                                 - key: canary                         weighted_clusters:                           clusters:                           - name: outbound|9080||productpage.istio.svc.cluster.local                             weight: 100                           total_weight: 100                           runtime_key_prefix: test 

本文转载自微信公众号「k8s实战」，可以通过以下二维码关注。转载本文请联系k8s实战公众号。