Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

can not load data in grafana #300

Closed
kuncao opened this issue Sep 16, 2017 · 3 comments
Closed

can not load data in grafana #300

kuncao opened this issue Sep 16, 2017 · 3 comments

Comments

@kuncao
Copy link

kuncao commented Sep 16, 2017

os:suse 12 sp2
NKG1000114449:/home/ksql/ksql-0.1.x/ksql-clickstream-demo/demo # ./ksql-tables-to-grafana.sh
Loading Clickstream-Demo TABLES to Confluent-Connect => Elastic => Grafana datasource
Logging to: /tmp/ksql-connect.log
Charting CLICK_USER_SESSIONS_TS
Charting USER_IP_ACTIVITY_TS
Charting CLICKSTREAM_STATUS_CODES_TS
Charting ENRICHED_ERROR_CODES_TS
Charting ERRORS_PER_MIN_ALERT_TS
Charting ERRORS_PER_MIN_TS
Charting EVENTS_PER_MIN_MAX_AVG_TS
Charting EVENTS_PER_MIN_TS
Charting PAGES_PER_MIN_TS
Navigate to http://localhost:3000/dashboard/db/click-stream-analysis
NKG1000114449:/home/ksql/ksql-0.1.x/ksql-clickstream-demo/demo # ./clickstream-analysis-dashboard.sh
Loading Grafana ClickStream Dashboard
{"slug":"click-stream-analysis","status":"success","version":5}

ksql> SELECT * FROM EVENTS_PER_MIN_TS LIMIT 5;
1505526580000 | 4^�� | 1505526580000 | 4 | 1 1505526580000 | 27^�� | 1505526580000 | 27 | 2
1505526580000 | 2^�� | 1505526580000 | 2 | 1 1505526580000 | 15^�� | 1505526580000 | 15 | 1
1505526580000 | 39^�`� | 1505526580000 | 39 | 1
LIMIT reached for the partition.
Query terminated

it seem there is no index in elasticsearch
NKG1000114449:/home/ksql/ksql-0.1.x/ksql-clickstream-demo/demo # curl -XGET 'localhost:9200/_cat/indices?v&pretty'
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

@kuncao
Copy link
Author

kuncao commented Sep 16, 2017

NKG1000114449:/home/ksql/ksql-0.1.x/ksql-clickstream-demo/demo # cat /tmp/ksql-connect.log
PROCESSING UPLOAD
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 2841 78338 --:--:-- --:--:-- --:--:-- 82714
{"acknowledged":true}==================================================================
Charting CLICK_USER_SESSIONS_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 411 100 411 0 0 55450 0 --:--:-- --:--:-- --:--:-- 58714
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"click_user_sessions_ts","index_uuid":"na","index":"click_user_sessions_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"click_user_sessions_ts","index_uuid":"na","index":"click_user_sessions_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 221 0 --:--:-- --:--:-- --:--:-- 221
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana click_user_sessions_ts
Connecting: click_user_sessions_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1619 44655 --:--:-- --:--:-- --:--:-- 48250
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 405 100 71 100 334 896 4215 --:--:-- --:--:-- --:--:-- 4227
{"id":100,"message":"Datasource added","name":"click_user_sessions_ts"}==================================================================
Charting USER_IP_ACTIVITY_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 399 100 399 0 0 52041 0 --:--:-- --:--:-- --:--:-- 57000
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"user_ip_activity_ts","index_uuid":"na","index":"user_ip_activity_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"user_ip_activity_ts","index_uuid":"na","index":"user_ip_activity_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 452 0 --:--:-- --:--:-- --:--:-- 458
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana user_ip_activity_ts
Connecting: user_ip_activity_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 2669 73598 --:--:-- --:--:-- --:--:-- 82714
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 396 100 68 100 328 848 4093 --:--:-- --:--:-- --:--:-- 4151
{"id":101,"message":"Datasource added","name":"user_ip_activity_ts"}==================================================================
Charting CLICKSTREAM_STATUS_CODES_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 431 100 431 0 0 55284 0 --:--:-- --:--:-- --:--:-- 61571
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"clickstream_status_codes_ts","index_uuid":"na","index":"clickstream_status_codes_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"clickstream_status_codes_ts","index_uuid":"na","index":"clickstream_status_codes_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 303 0 --:--:-- --:--:-- --:--:-- 305
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana clickstream_status_codes_ts
Connecting: clickstream_status_codes_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1898 52331 --:--:-- --:--:-- --:--:-- 57900
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 420 100 76 100 344 585 2649 --:--:-- --:--:-- --:--:-- 2666
{"id":102,"message":"Datasource added","name":"clickstream_status_codes_ts"}==================================================================
Charting ENRICHED_ERROR_CODES_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 415 100 415 0 0 56240 0 --:--:-- --:--:-- --:--:-- 59285
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"enriched_error_codes_ts","index_uuid":"na","index":"enriched_error_codes_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"enriched_error_codes_ts","index_uuid":"na","index":"enriched_error_codes_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 436 0 --:--:-- --:--:-- --:--:-- 440
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana enriched_error_codes_ts
Connecting: enriched_error_codes_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1723 47533 --:--:-- --:--:-- --:--:-- 48250
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 408 100 72 100 336 600 2802 --:--:-- --:--:-- --:--:-- 2800
{"id":103,"message":"Datasource added","name":"enriched_error_codes_ts"}==================================================================
Charting ERRORS_PER_MIN_ALERT_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 415 100 415 0 0 60725 0 --:--:-- --:--:-- --:--:-- 69166
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"errors_per_min_alert_ts","index_uuid":"na","index":"errors_per_min_alert_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"errors_per_min_alert_ts","index_uuid":"na","index":"errors_per_min_alert_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 372 0 --:--:-- --:--:-- --:--:-- 375
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana errors_per_min_alert_ts
Connecting: errors_per_min_alert_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 2479 68350 --:--:-- --:--:-- --:--:-- 72375
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 408 100 72 100 336 968 4520 --:--:-- --:--:-- --:--:-- 4540
{"id":104,"message":"Datasource added","name":"errors_per_min_alert_ts"}==================================================================
Charting ERRORS_PER_MIN_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 391 100 391 0 0 57584 0 --:--:-- --:--:-- --:--:-- 65166
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"errors_per_min_ts","index_uuid":"na","index":"errors_per_min_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"errors_per_min_ts","index_uuid":"na","index":"errors_per_min_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 414 0 --:--:-- --:--:-- --:--:-- 417
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana errors_per_min_ts
Connecting: errors_per_min_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1864 51407 --:--:-- --:--:-- --:--:-- 52636
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 390 100 66 100 324 1024 5031 --:--:-- --:--:-- --:--:-- 5062
{"id":105,"message":"Datasource added","name":"errors_per_min_ts"}==================================================================
Charting EVENTS_PER_MIN_MAX_AVG_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 423 100 423 0 0 61824 0 --:--:-- --:--:-- --:--:-- 70500
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"events_per_min_max_avg_ts","index_uuid":"na","index":"events_per_min_max_avg_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"events_per_min_max_avg_ts","index_uuid":"na","index":"events_per_min_max_avg_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 481 0 --:--:-- --:--:-- --:--:-- 485
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana events_per_min_max_avg_ts
Connecting: events_per_min_max_avg_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1820 50195 --:--:-- --:--:-- --:--:-- 52636
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 414 100 74 100 340 947 4353 --:--:-- --:--:-- --:--:-- 4358
{"id":106,"message":"Datasource added","name":"events_per_min_max_avg_ts"}==================================================================
Charting EVENTS_PER_MIN_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 391 100 391 0 0 60386 0 --:--:-- --:--:-- --:--:-- 65166
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"events_per_min_ts","index_uuid":"na","index":"events_per_min_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"events_per_min_ts","index_uuid":"na","index":"events_per_min_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 261 0 --:--:-- --:--:-- --:--:-- 261
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana events_per_min_ts
Connecting: events_per_min_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1735 47847 --:--:-- --:--:-- --:--:-- 52636
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 390 100 66 100 324 727 3572 --:--:-- --:--:-- --:--:-- 3600
{"id":107,"message":"Datasource added","name":"events_per_min_ts"}==================================================================
Charting PAGES_PER_MIN_TS
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 387 100 387 0 0 63840 0 --:--:-- --:--:-- --:--:-- 77400
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"pages_per_min_ts","index_uuid":"na","index":"pages_per_min_ts"}],"type":"index_not_found_exception","reason":"no such index","resource.type":"index_or_alias","resource.id":"pages_per_min_ts","index_uuid":"na","index":"pages_per_min_ts"},"status":404} % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 33 100 33 0 0 323 0 --:--:-- --:--:-- --:--:-- 326
{"message":"Data source deleted"}\n\nConnecting KSQL->Elastic->Grafana pages_per_min_ts
Connecting: pages_per_min_ts
Loading Elastic Dynamic Template to ensure _TS fields are used for TimeStamp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 600 100 21 100 579 1822 50247 --:--:-- --:--:-- --:--:-- 52636
{"acknowledged":true}Adding Elastic Source\n\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed to connect to localhost port 8083: Connection refused

Adding Grafana Source
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 387 100 65 100 322 1106 5483 --:--:-- --:--:-- --:--:-- 5551
{"id":108,"message":"Datasource added","name":"pages_per_min_ts"}

@kuncao
Copy link
Author

kuncao commented Sep 16, 2017

please help to check what's wrong?
thanks a lot

@kuncao
Copy link
Author

kuncao commented Sep 16, 2017

according curl: (7) Failed to connect to localhost port 8083: Connection refused message
netstat -anp|grep 8083
there is no record
restart confluent
./confluent start
zookeeper is already running. Try restarting if needed
kafka is already running. Try restarting if needed
schema-registry is already running. Try restarting if needed
kafka-rest is already running. Try restarting if needed
Starting connect
connect is [UP]----note here , kafka connect is up
check 8083 port
netstat -tulpn | grep 8083
tcp 0 0 :::8083 :::* LISTEN 125511/java

run ./ksql-tables-to-grafana.sh again
the data can be show in grafana

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant