Skip to content
This repository has been archived by the owner on Aug 23, 2019. It is now read-only.

observe traffic and expose statistics #243

Merged
merged 26 commits into from
Feb 27, 2018
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
494d7fd
starting to observe
pgte Feb 16, 2018
758cf2a
WIP: stats: observer to stats
pgte Feb 19, 2018
0afd916
stats have transport now
pgte Feb 19, 2018
0f5bfaa
stats: peer list and transport list
pgte Feb 19, 2018
20d6934
stats: stop
pgte Feb 19, 2018
7210efc
trying to set peer info
pgte Feb 19, 2018
0eb8c26
propagating peer info
pgte Feb 19, 2018
84e766b
stats: first tests
pgte Feb 20, 2018
fa6e377
stats: transports, protocols and peers
pgte Feb 20, 2018
a1d9014
stats: transport-specific stats
pgte Feb 20, 2018
3d0ad42
stats: not relying on the order / existence of tests
pgte Feb 20, 2018
7ee4fc5
stats: peer-specific stats
pgte Feb 20, 2018
6ea2061
setPeerInfo only if there's a peerInfo
pgte Feb 20, 2018
e692fa2
stats: tests now are isolated
pgte Feb 20, 2018
905680a
stats: tests: removed console.log
pgte Feb 20, 2018
f6db42d
stats: tests not hardcoding the port
pgte Feb 20, 2018
ae88175
stats: documented API
pgte Feb 20, 2018
39f01af
removed console log
pgte Feb 20, 2018
84b00f9
stats: emit update event every time stats get updated
pgte Feb 21, 2018
eb2cf59
docs: moved stats api place in readme
pgte Feb 21, 2018
fb0d209
docs: fixed typo
pgte Feb 21, 2018
a2cd482
setting the peer info directly without having to cache the method
pgte Feb 21, 2018
2ed0cf8
docs: consistency for list markdown
pgte Feb 21, 2018
5b3cb3b
docs: dcumented options
pgte Feb 21, 2018
6c0ff32
stats: retaning old peers in case they reconnect
pgte Feb 21, 2018
efc7c92
stats: fix: now accounts for transport, and separates protocol-specif…
pgte Feb 22, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 16 additions & 5 deletions src/stats/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@ module.exports = (observer, _options) => {
const globalStats = new Stat(initialCounters, options)
const peerStats = new Map()
const transportStats = new Map()
const protocolStats = new Map()

observer.on('message', (peerId, transportTag, protocol, direction, bufferLength) => {
observer.on('message', (peerId, transportTag, protocolTag, direction, bufferLength) => {
// if (!peerId) {
// throw new Error('message should have peer id')
// }
console.log('m', peerId, transportTag, protocol, direction, bufferLength)
console.log('m', peerId, transportTag, protocolTag, direction, bufferLength)
const event = directionToEvent[direction]

// global stats
Expand All @@ -54,6 +55,14 @@ module.exports = (observer, _options) => {
transportStats.set(transportTag, transport)
}
transport.push(event, bufferLength)

// protocol stats
let protocol = protocolStats.get(protocolTag)
if (!protocol) {
protocol = new Stat(initialCounters, options)
protocolStats.set(protocolTag, transport)
}
protocol.push(event, bufferLength)
})

observer.on('peer:closed', (peerId) => {
Expand All @@ -63,10 +72,12 @@ module.exports = (observer, _options) => {
return {
stop: stop,
global: globalStats,
peers: Array.from(peerStats.keys()),
peers: () => Array.from(peerStats.keys()),
forPeer: (peerId) => peerStats.get(peerId),
transports: Array.from(transportStats.keys()),
forTransport: (transport) => transportStats.get(transport)
transports: () => Array.from(transportStats.keys()),
forTransport: (transport) => transportStats.get(transport),
protocols: () => Array.from(protocolStats.keys()),
forProtocol: (protocol) => protocolStats.get(protocol)
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does one "clear" previous stats from a peer?


function stop () {
Expand Down
44 changes: 36 additions & 8 deletions test/stats.node.js
Original file line number Diff line number Diff line change
Expand Up @@ -82,19 +82,47 @@ describe('Stats', () => {
})
})

it('waits a bit', (done) => setTimeout(done, 1900))
it('waits a bit', (done) => setTimeout(done, 1000))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A test case that just waits? Should fix the tests to not depend on the ordering...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@victorbjelkholm good call! Fixed by moving the preparation code to a before hook.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pgte better to get rid of the after/before hooks we have and have function calls in the test cases that returns what you need. If all tests where shaped like this, we can run tests in parallel and just run one isolated test case if wanted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In my case I need something to run once, not for every test.
In this case, I want to setup some switches, exchange some protocol-specific data, wait for the stats to compute, and then I can infer all the statistics are there.
@victorbjelkholm Any recommendations?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@victorbjelkholm changed to isolated tests for stats, would love your feedback.


it('A has some stats', () => {
const snapshot = switchA.stats.global.snapshot
console.log('%j', snapshot)
it('both nodes have some global stats', () => {
let snapshot = switchA.stats.global.snapshot
expect(snapshot.dataReceived.toFixed()).to.equal('51')
expect(snapshot.dataSent.toFixed()).to.equal('49')
})

it('B has some stats', () => {
const snapshot = switchB.stats.global.snapshot
console.log('%j', snapshot)
snapshot = switchB.stats.global.snapshot
expect(snapshot.dataReceived.toFixed()).to.equal('51')
expect(snapshot.dataSent.toFixed()).to.equal('49')
})

it('both nodes know some transports', () => {
const expectedTransports = [
'/mplex/6.7.0',
'/secio/1.0.0'
]
expect(switchA.stats.transports().sort()).to.deep.equal(expectedTransports)
expect(switchB.stats.transports().sort()).to.deep.equal(expectedTransports)
})

it('both nodes know some protocols', () => {
const expectedProtocols = [
'/echo/1.0.0',
'/mplex/6.7.0'
]
expect(switchA.stats.protocols().sort()).to.deep.equal(expectedProtocols)
expect(switchB.stats.protocols().sort()).to.deep.equal(expectedProtocols)
})

it('both nodes know about each other', () => {
expect(switchA.stats.peers().sort()).to.deep.equal([switchB._peerInfo.id.toB58String()])
expect(switchB.stats.peers().sort()).to.deep.equal([switchA._peerInfo.id.toB58String()])
})

it('both have transport-specific stats', () => {
let snapshot = switchA.stats.forTransport('/secio/1.0.0').snapshot
expect(snapshot.dataReceived.toFixed()).to.equal('92')
expect(snapshot.dataSent.toFixed()).to.equal('90')
snapshot = switchB.stats.forTransport('/secio/1.0.0').snapshot
expect(snapshot.dataReceived.toFixed()).to.equal('92')
expect(snapshot.dataSent.toFixed()).to.equal('90')
})
})