Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Functionbeat GCP pub/sub requiring storage entry point when only pub/sub is enabled #29967

Closed
chipzzz opened this issue Jan 24, 2022 · 5 comments
Labels
Functionbeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@chipzzz
Copy link

chipzzz commented Jan 24, 2022

Deploying functionbeat on GCP with entry point RunPubSub returns an error complaining about storage entry point, which I am not using.

I get the following error upon deploy

Deployment failure:
Build failed: # functions.local/app/main
src/functions.local/app/main/main.go:43:21: undefined: storage.RunPubSub; Error ID: 2f5e35a0

The problem is ,as I noticed it, Functionbeat is requiring storage entry point. I am NOT using the gcp storage functionality, I am using pub/sub functionality with RunPubSub. Why is it requiring the storage entry point? I am not able to provide multiple entry points to my knowledge.

  1. Package functionbeat using LINUX X86_64 7.13.4 up using the following command
./functionbeat  export function pubsub -c functionbeat-file.yml
  1. Go into GCP function and go through creating a cloud function steps, upload the packaed zip,
  2. add entry point "RunPubSub"
  3. Select Golang version 1.16 in order to avoid another bug (functionbeat 7.12 can't deploy -- unknown field 'VerifyConnection' in struct literal of type tls.Config #24925). <<< Selecting 1.13 will yield this error. Keep in mind the runtime of the function is GoLang 1.13 (could this be the problem?)
 ./functionbeat export function pubsub |grep runtime
    runtime: go113

My config file

functionbeat.provider.gcp.location_id: "us-central1"

functionbeat.provider.gcp.project_id: "some-project"

functionbeat.provider.gcp.storage_name: "some-bucket"

functionbeat.provider.gcp.functions:
  - name: pubsub
    enabled: true
    type: pubsub

    description: "Google Cloud Function for Pub/Sub"
    memory_size: 512MB
    timeout: 60s
    service_account_email: some-email

    trigger:
      resource: "some-topic"
processors:
- add_cloud_metadata:
    providers: ["gcp"]
- add_fields:
    target: field
    fields:
      sourcetype: test
      service: logging
- drop_event:
    when:
      not:
       regexp:
         field.sourcetype: ".*"
logging.level: info
logging.json: true
output.logstash:
  hosts: ["some-host"]
  pipelining: 0
  max_retries: 0
  ttl: 30s
setup.ilm.enabled: false
setup.template.enabled: false
path:
  config: ./
  home: ./

Including some references:
#24925
#26249
#22495
#28251
#27012
#28253
#25596
https://discuss.elastic.co/t/functionbeat-missing-package-io-fs/284975
GCP/FunctionBeat Errors [cannot find matching process for pid=1, name version already used] · Issue #25596 · elastic/beats · GitHub 1

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jan 24, 2022
@chipzzz
Copy link
Author

chipzzz commented Jan 24, 2022

@chipzzz
Copy link
Author

chipzzz commented Jan 24, 2022

Keep in mind the runtime of the function is GoLang 1.13 (could this be the problem?)
recompiled the functionbeat binary as goLang 1.16 and repackaged, same error.

@mtojek mtojek added the Team:Integrations Label for the Integrations team label Jan 25, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations (Team:Integrations)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Jan 25, 2022
@mtojek mtojek added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team and removed Team:Integrations Label for the Integrations team labels Jan 25, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@mtojek mtojek added the Team:Integrations Label for the Integrations team label Jan 25, 2022
@kvch kvch removed the Team:Integrations Label for the Integrations team label Jan 25, 2022
@jlind23 jlind23 added request-discuss Label added to request the creator to create a topic in discuss and removed request-discuss Label added to request the creator to create a topic in discuss labels Jan 25, 2022
@jlind23
Copy link
Collaborator

jlind23 commented Apr 5, 2022

We endeavor to address Functionbeat use cases via the new Serverless Forwarder project (https://github.com/elastic/elastic-serverless-forwarder)with a better integration with serverless functionality provided by major cloud providers. Enhancements to the current Functionbeat will therefore be limited.

@jlind23 jlind23 closed this as not planned Won't fix, can't repro, duplicate, stale Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Functionbeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

No branches or pull requests

5 participants