Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bitswap rounds #438

Merged
merged 64 commits into from
Dec 18, 2014
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
64 commits
Select commit Hold shift + click to select a range
c4a935c
go complains about lack of buildable file
jbenet Dec 18, 2014
12b296e
create wantlist object
whyrusleeping Dec 10, 2014
5b6a5e8
implement bitswap roundWorker
whyrusleeping Dec 10, 2014
57e7dd7
extracted ledgerset from strategy, cleaned up a few comments from the PR
whyrusleeping Dec 10, 2014
3778eed
dont spawn so many goroutines when rebroadcasting wantlist
whyrusleeping Dec 10, 2014
e7bba82
add priorities to GetBlocks requests, and add waitgroup to sendWantLi…
whyrusleeping Dec 10, 2014
50aa37f
blockstore.ErrNotFound, and proper wantlist sorting
whyrusleeping Dec 10, 2014
da68475
fix go version
Dec 11, 2014
59f0ffb
remove noisy statement
Dec 13, 2014
946d2a9
add locks to wantlist to avoid race condition
whyrusleeping Dec 14, 2014
cfbe92b
rewrite sendWantlistToProviders
whyrusleeping Dec 15, 2014
029e305
tasklist queue for bitswap tasks
whyrusleeping Dec 16, 2014
90a3096
renaming and removing empty strategy file
whyrusleeping Dec 16, 2014
2240272
change Provide RPC to not wait for an ACK, improves performance of 'A…
whyrusleeping Dec 16, 2014
6389bfd
some cleanup before CR
whyrusleeping Dec 16, 2014
86c438b
refactor() message API
Dec 16, 2014
29ef238
remove dead code
Dec 16, 2014
1bced71
queue-like naming
Dec 16, 2014
f028c44
name findOrCreate
Dec 16, 2014
f533678
avoid attaching context to object when it's not necessary.
Dec 16, 2014
69dd260
refactor peerSet
whyrusleeping Dec 16, 2014
c0a18d9
fix(test): nil Blockstore
Dec 17, 2014
5603b2e
style: line wrapping
Dec 17, 2014
8bef1dc
fix: move to callsite so public callers don't experience the internal…
Dec 17, 2014
a495a01
style constify variables
Dec 17, 2014
2ea8ed8
refactor: change Tasks to Outbox
Dec 17, 2014
8c05c44
refactor: avoid loop reuse
Dec 17, 2014
f66d94a
fix: move the check into the function.
Dec 17, 2014
10e970c
refactor: context first in argument list
Dec 17, 2014
4609678
doc: comment
Dec 17, 2014
93fde86
refactor: taskKey := p.Key() + block.Key()
Dec 17, 2014
286723d
unexport task and taskList
Dec 17, 2014
b41fef2
refactor: remove peerKey type
Dec 17, 2014
cc2a731
add comment to fix race
Dec 17, 2014
7280aac
perf: avoid lots of communication by signaling once at end of method
Dec 17, 2014
198aa19
it's not a queue yet but it's okay to name it as such
Dec 17, 2014
181ff4e
tq.Cancel -> tq.Remove
Dec 17, 2014
e11f099
privatize Task
Dec 17, 2014
1d71595
doc: add comment to Envelope
Dec 17, 2014
fedcebf
refactor: re-use wantlist.Entry type wherever it makes sense
Dec 17, 2014
7fdbae1
refactor: separate responsibilties
Dec 17, 2014
39d7193
mv comment
Dec 17, 2014
962a947
refactor: remove ledgerMap type
Dec 17, 2014
d069ae1
refactor: put mutex next to the things it protects
Dec 17, 2014
bef6222
refactor: wantlist splits into WL and ThreadSafe WL
Dec 17, 2014
5bd0b95
rename to strategy.LedgerManager to decision.Engine
Dec 17, 2014
1d23e94
rm empty file
Dec 17, 2014
acc7148
rename to peerRequestQueue
Dec 17, 2014
8d4d5b8
fix: don't sort the output of Entries()
Dec 17, 2014
1b1260b
rm unused method
Dec 17, 2014
9c301a2
add comment
Dec 17, 2014
6e7c46a
unexport functions
Dec 17, 2014
2b60b64
fix: check blockstore before adding task
Dec 17, 2014
e36d656
log unusual event
Dec 17, 2014
45faa4d
fix: set peerset size
Dec 17, 2014
1976488
doc: peerset fixme
Dec 17, 2014
0545c4d
refactor: *Entry -> Entry
Dec 17, 2014
4bcfe09
extract constants
Dec 17, 2014
175513e
refactor(bs/decision.Engine): pass in Entry
Dec 17, 2014
ce2d0a2
fix: add lock to taskQueue
Dec 17, 2014
bd3ee73
doc: some comments about the future of the decision engine
Dec 17, 2014
8100582
fix: batches of blocks have equal priority
Dec 17, 2014
9328fba
clean peerset constructor names
whyrusleeping Dec 17, 2014
9fafec1
do not run epic tests in parallel
Dec 18, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion blocks/blockstore/blockstore.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,16 @@ import (
"errors"

ds "github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/go-datastore"

mh "github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/go-multihash"

blocks "github.com/jbenet/go-ipfs/blocks"
u "github.com/jbenet/go-ipfs/util"
)

var ValueTypeMismatch = errors.New("The retrieved value is not a Block")

var ErrNotFound = errors.New("blockstore: block not found")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


// Blockstore wraps a ThreadSafeDatastore
type Blockstore interface {
DeleteBlock(u.Key) error
Expand All @@ -34,6 +36,9 @@ type blockstore struct {

func (bs *blockstore) Get(k u.Key) (*blocks.Block, error) {
maybeData, err := bs.datastore.Get(k.DsKey())
if err == ds.ErrNotFound {
return nil, ErrNotFound
}
if err != nil {
return nil, err
}
Expand Down
4 changes: 1 addition & 3 deletions blockservice/blockservice.go
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ import (
"fmt"

context "github.com/jbenet/go-ipfs/Godeps/_workspace/src/code.google.com/p/go.net/context"
ds "github.com/jbenet/go-ipfs/Godeps/_workspace/src/github.com/jbenet/go-datastore"

blocks "github.com/jbenet/go-ipfs/blocks"
"github.com/jbenet/go-ipfs/blocks/blockstore"
exchange "github.com/jbenet/go-ipfs/exchange"
Expand Down Expand Up @@ -67,7 +65,7 @@ func (s *BlockService) GetBlock(ctx context.Context, k u.Key) (*blocks.Block, er
return block, nil
// TODO be careful checking ErrNotFound. If the underlying
// implementation changes, this will break.
} else if err == ds.ErrNotFound && s.Exchange != nil {
} else if err == blockstore.ErrNotFound && s.Exchange != nil {
log.Debug("Blockservice: Searching bitswap.")
blk, err := s.Exchange.GetBlock(ctx, k)
if err != nil {
Expand Down
16 changes: 0 additions & 16 deletions epictest/addcat_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,6 @@ import (
const kSeed = 1

func Test100MBInstantaneous(t *testing.T) {
t.Log("a sanity check")

t.Parallel()

conf := Config{
NetworkLatency: 0,
RoutingLatency: 0,
Expand All @@ -41,43 +37,31 @@ func Test100MBInstantaneous(t *testing.T) {

func TestDegenerateSlowBlockstore(t *testing.T) {
SkipUnlessEpic(t)
t.Parallel()

conf := Config{BlockstoreLatency: 50 * time.Millisecond}

if err := AddCatPowers(conf, 128); err != nil {
t.Fatal(err)
}
}

func TestDegenerateSlowNetwork(t *testing.T) {
SkipUnlessEpic(t)
t.Parallel()

conf := Config{NetworkLatency: 400 * time.Millisecond}

if err := AddCatPowers(conf, 128); err != nil {
t.Fatal(err)
}
}

func TestDegenerateSlowRouting(t *testing.T) {
SkipUnlessEpic(t)
t.Parallel()

conf := Config{RoutingLatency: 400 * time.Millisecond}

if err := AddCatPowers(conf, 128); err != nil {
t.Fatal(err)
}
}

func Test100MBMacbookCoastToCoast(t *testing.T) {
SkipUnlessEpic(t)
t.Parallel()

conf := Config{}.Network_NYtoSF().Blockstore_SlowSSD2014().Routing_Slow()

if err := AddCatBytes(RandomBytes(100*1024*1024), conf); err != nil {
t.Fatal(err)
}
Expand Down
Loading