Filebeat CEL input - string library functions not recognized

[a03nikki]

For confirmed bugs, please report:

• Version: 8.6.2
• Operating System: MacOS Venture 13.2.1
• Discuss Forum URL:
• Steps to Reproduce:

1. Download latest version of Filebeat, extract, and update filebeat.yml

2. Add CEL input that points at a URL that requires username and password
   a. https://www.elastic.co/guide/en/beats/filebeat/8.6/filebeat-input-cel.html

   "Use the cel input to read messages from a file path or HTTP API with a variety of payloads using the Common Expression Language (CEL) and the mito and ext Strings CEL extension libraries."

   b. x-pack/filebeat/input/cel: new input #31233

3. Run Filebeat

   ./filebeat -v -e

4. Review the output

5. Search the code for the possible problem. It appears the ext library is not even included
   a. https://github.com/elastic/beats/blob/main/x-pack/filebeat/input/cel/input.go
   i. The import includes "github.com/google/cel-go/cel" and "github.com/google/cel-go/checker/decls" but does not include "github.com/google/cel-go/ext"
   ii. func newProgram( does not include String() as an option similar to an example I was looking at
   b. https://github.com/elastic/beats/blob/main/x-pack/filebeat/input/cel/input_test.go
   i. I only found references to functions in mito and the cel language standard. I didn't find any tests that included any of the string methods.

metricbeat.yml

filebeat.inputs:
- type: cel
  id: cel-1
  interval: 1m
  resource.url: https://api.ipify.org/?format=json
  program: |
    string(get(state.url).Body).as(body, {
        "events": [{ "message": body, "type_is_string": string(type(body) == string), "tags": body.split('i') }]
    })
  enabled: true

filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false

setup.template.settings:
  index.number_of_shards: 1
setup.kibana:

cloud.id: REDACTED
cloud.auth: REDACTED

output.elasticsearch:

processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~


logging.level: debug
logging.selectors: ["*"]

monitoring.enabled: true
monitoring.cluster_uuid: REDACTED

logs

Before adding the split function

After adding the split function with , "tags": body.split('i'):

{"log.level":"info","@timestamp":"2023-02-17T16:37:22.739-0600","log.origin":{"file.name":"instance/beat.go","file.line":491},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-02-17T16:37:22.739-0600","log.origin":{"file.name":"instance/beat.go","file.line":1071},"message":"Exiting: Failed to start crawler: starting input failed: error while initializing input: failed to check program: failed compilation: ERROR: <input>:2:101: undeclared reference to 'split' (in container '')\n |     \"events\": [{ \"message\": body, \"type_is_string\": string(type(body) == string), \"tags\": body.split('i') }]\n | ....................................................................................................^ accessing 'filebeat.inputs.0' (source:'filebeat.yml')","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: Failed to start crawler: starting input failed: error while initializing input: failed to check program: failed compilation: ERROR: <input>:2:101: undeclared reference to 'split' (in container '')
 |     "events": [{ "message": body, "type_is_string": string(type(body) == string), "tags": body.split('i') }]
 | ....................................................................................................^ accessing 'filebeat.inputs.0' (source:'filebeat.yml')

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[efd6]

You are correct, the cel/ext package is not included. This was intentional (the bug is in the documentation ­— there should not be a reference to ext). I have in mind a plan to mechanically generate a String/Bytes lib from a subset of the Go standard library strings and bytes packages if there is seen to be a need. In the mean time the regexp lib can be used in most cases where ext.strings would be used.

[efd6]

My proposal is to add CEL extension wrappers for the following strings package functions

func Compare(a, b string) int
func Contains(s, substr string) bool
func ContainsAny(s, chars string) bool
func Count(s, substr string) int
func EqualFold(s, t string) bool
func Fields(s string) []string
func HasPrefix(s, prefix string) bool
func HasSuffix(s, suffix string) bool
func Index(s, substr string) int
func IndexAny(s, chars string) int
func Join(elems []string, sep string) string
func LastIndex(s, substr string) int
func LastIndexAny(s, chars string) int
func Repeat(s string, count int) string
func Replace(s, old, new string, n int) string
func ReplaceAll(s, old, new string) string
func Split(s, sep string) []string
func SplitAfter(s, sep string) []string
func SplitAfterN(s, sep string, n int) []string
func SplitN(s, sep string, n int) []string
func Title(s string) string // Omitted: DEPRECATED in stdlib
func ToLower(s string) string
func ToTitle(s string) string
func ToUpper(s string) string
func ToValidUTF8(s, replacement string) string
func Trim(s, cutset string) string
func TrimLeft(s, cutset string) string
func TrimPrefix(s, prefix string) string
func TrimRight(s, cutset string) string
func TrimSpace(s string) string
func TrimSuffix(s, suffix string) string

and a string slicing function.