emit transaction failed: error_class=NoMethodError error="undefined method `strip\' for #<Symbol:0x000000033f370c> #33
Comments
|
Can you let me know what version of the plugin you're using? You can check either by looking at the dockerfile or by execing into the container & perhaps issuing gem list to see. Pretty sure we moved away from symbol keys a little while ago so very possibly just upgrading the plugin should do it. |
|
Thanks, tried to inspect the container a bit but couldn't find any gems. $ kubectl --namespace=kube-system describe po fluentd-elasticsearch-ip-172-20-0-131.us-west-1.compute.internal
Name: fluentd-elasticsearch-ip-172-20-0-131.us-west-1.compute.internal
Namespace: kube-system
Node: ip-172-20-0-131.us-west-1.compute.internal/172.20.0.131
Start Time: Mon, 25 Jul 2016 21:41:53 -0700
Labels: k8s-app=fluentd-logging
Status: Running
Controllers: <none>
Containers:
fluentd-elasticsearch:
Container ID: docker://fc36565027918f6eecbcf15d52552d08298674029c488c799534ab506f9bbc31
Image: gcr.io/google_containers/fluentd-elasticsearch:1.15
Image ID: docker://7948654508f0c0a9d43e45ccdc88a920717bd8d47bdd9d401ced76654aa0129b
Port:
QoS Tier:
memory: Guaranteed
cpu: Burstable
Limits:
memory: 200Mi
Requests:
memory: 200Mi
cpu: 100m
State: Running
Started: Mon, 25 Jul 2016 21:42:23 -0700
Ready: True
Restart Count: 0
Environment Variables:
Conditions:
Type Status
Ready True
Volumes:
varlog:
Type: HostPath (bare host directory volume)
Path: /var/log
varlibdockercontainers:
Type: HostPath (bare host directory volume)
Path: /var/lib/docker/containers
No events.
The container image is gcr.io/google_containers/fluentd-elasticsearch:1.15 but I'm not sure where's the Dockerfile for it (probably somewhere in the kubernetes repo?). |
|
Try |
|
Oh, just found the gem path <1:/opt/td-agent/embedded/lib/ruby/gems/2.1.0/gems# td-agent-gem list
*** LOCAL GEMS ***
actionmailer (4.2.1)
actionpack (4.2.1)
actionview (4.2.1)
activejob (4.2.1)
activemodel (4.2.1)
activerecord (4.2.1)
activesupport (4.2.1)
addressable (2.4.0, 2.3.8)
arel (6.0.3, 5.0.1.20140414130214)
aws-sdk (2.2.7)
aws-sdk-core (2.2.7)
aws-sdk-resources (2.2.7)
aws-sdk-v1 (1.66.0)
bigdecimal (1.2.4)
bson (1.12.5)
bson_ext (1.12.5)
builder (3.2.2)
bundler (1.10.6)
celluloid (0.15.2)
cool.io (1.4.2)
crass (1.0.2)
diff-lcs (1.2.5)
docker-api (1.26.0)
domain_name (0.5.25)
draper (1.4.0)
elasticsearch (1.0.15)
elasticsearch-api (1.0.15)
elasticsearch-transport (1.0.15)
erubis (2.7.0)
excon (0.45.4)
faraday (0.9.2)
fluent-logger (0.4.10)
fluent-mixin-config-placeholders (0.3.0)
fluent-mixin-plaintextformatter (0.2.6)
fluent-plugin-docker_metadata_filter (0.1.1)
fluent-plugin-elasticsearch (1.3.0)
fluent-plugin-kubernetes_metadata_filter (0.14.0)
fluent-plugin-mongo (0.7.11)
fluent-plugin-rewrite-tag-filter (1.5.3)
fluent-plugin-s3 (0.6.4)
fluent-plugin-scribe (0.10.14)
fluent-plugin-td (0.10.28)
fluent-plugin-td-monitoring (0.2.1)
fluent-plugin-webhdfs (0.4.1)
fluentd (0.12.19)
fluentd-ui (0.4.2)
font-awesome-rails (4.3.0.0)
globalid (0.3.6)
haml (4.0.6)
haml-rails (0.5.3)
hike (1.2.3)
hirb (0.7.3)
http-cookie (1.0.2)
http_parser.rb (0.6.0)
httpclient (2.5.3.3)
i18n (0.7.0)
io-console (0.4.3)
ipaddress (0.8.0)
jbuilder (2.2.6)
jmespath (1.1.3)
jquery-rails (3.1.2)
json (1.8.1)
kramdown (1.6.0)
kramdown-haml (0.0.3)
kubeclient (0.4.0)
loofah (2.1.0.rc1)
lru_redux (1.1.0)
ltsv (0.1.0)
mail (2.6.3)
mime-types (2.4.3)
mini_portile (0.6.0)
minitest (5.4.2, 4.7.5)
mixlib-cli (1.5.0)
mixlib-config (2.2.1)
mixlib-log (1.6.0)
mixlib-shellout (2.2.5)
mongo (1.12.5)
msgpack (0.5.11)
multi_json (1.11.0)
multipart-post (2.0.0)
netrc (0.11.0)
nokogiri (1.6.3.1)
ohai (6.20.0)
oj (2.14.2)
parallel (0.6.5)
psych (2.0.5)
puma (2.11.1)
rack (1.6.0)
rack-test (0.6.3)
rails (4.2.1)
rails-deprecated_sanitizer (1.0.3)
rails-dom-testing (1.0.7)
rails-html-sanitizer (1.0.2)
railties (4.2.1)
rake (10.1.0)
rdoc (4.1.0)
recursive-open-struct (0.6.1)
request_store (1.1.0)
rest-client (2.0.0.rc2)
ruby-progressbar (1.7.5)
rubyzip (1.1.7)
sass (3.2.19)
sass-rails (4.0.5)
settingslogic (2.0.9)
sigdump (0.2.3)
sprockets (2.12.3)
sprockets-rails (2.2.4)
string-scrub (0.0.5)
sucker_punch (1.0.5)
systemu (2.5.2)
td (0.13.0)
td-client (0.8.76)
td-logger (0.3.24)
test-unit (2.1.8.0)
thor (0.19.1)
thread_safe (0.3.5)
thrift (0.8.0)
tilt (1.4.1)
timers (1.1.0)
tzinfo (1.2.2)
tzinfo-data (1.2015.7)
unf (0.2.0.beta2)
uuidtools (2.1.5)
webhdfs (0.7.3)
yajl-ruby (1.2.1)
zip-zip (0.3)So it's |
|
Keys were changed to string in v0.20 so that needs updating in the docker image. |
|
I updated to the latest image (gcr.io/google_containers/fluentd-elasticsearch:1.17) and still seeing the bug. The addon is still using On a related note, I can't see logs that begin with |
|
It could be something else but I think it's to do with the record keys being symbols in the older versions of the plugin. |
|
I'm also getting a similar error with the latest version: |
|
FWIW, the log line that causes this is: |
|
@edevil oh, the log that's causing this for me is as also reported here: kubernetes/kubernetes#29640 At this point I don't know if this needs to be fixed in the Kubernetes project or here. |
|
I fixed it here: fluent/fluentd#1147 This PR would mitigate the issue while the upstream version is not propagated. |
|
Do we still need this or can we just note a compatible version of fluentd? |
|
I think it's better to note a compatible version of fluentd. |
mumoshu
added a commit
to mumoshu/kube-fluentd
that referenced
this issue
Dec 9, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I apologise in advance if I'm posting this to the wrong repo. I recently started using Kubernetes and keep seeing the following error in my logs:
Is it possibly an issue with this plugin? Thanks!
Possibly related: kubernetes/kubernetes#29640
The text was updated successfully, but these errors were encountered: