Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make Powergate CGO free & use testnet/3 #210

Merged
merged 18 commits into from
Mar 27, 2020
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
**/docker-compose.yaml
**/docker-compose-embedded.yaml
**/Dockerfile
extern
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bye externaka git-submodules.

**/Dockerfile
8 changes: 1 addition & 7 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ on:
push:
branches:
- master
- jsign/pg5
pull_request:
branches:
- master
Expand All @@ -11,13 +12,6 @@ jobs:
name: Test
runs-on: self-hosted
steps:
- name: Set up Rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: nightly
- name: Install OpenCL
run: sudo apt-get update && sudo apt-get install -y mesa-opencl-icd ocl-icd-opencl-dev jq make
Comment on lines -14 to -20
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No Rust nor OpenCL for building the git-submodule.
Still self-hosted since running test is still heavy. Instead of running the devnet code, is running the devnet container.

- name: Check out code
uses: actions/checkout@v1
- name: Test
Expand Down
3 changes: 0 additions & 3 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +0,0 @@
[submodule "extern/filecoin-ffi"]
path = extern/filecoin-ffi
url = https://github.com/filecoin-project/filecoin-ffi.git
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wohoo!

38 changes: 12 additions & 26 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,32 +1,18 @@
SUBMODULES=.update-modules
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to see without diffs.
Now it's super simple. Also closes #212.


FFI_PATH:=./extern/filecoin-ffi/
FFI_DEPS:=libfilecoin.a filecoin.pc filecoin.h
FFI_DEPS:=$(addprefix $(FFI_PATH),$(FFI_DEPS))

$(FFI_DEPS): .filecoin-build ;
clean:
rm -f powd pow
.PHONY: clean

.filecoin-build: $(FFI_PATH)
$(MAKE) -C $(FFI_PATH) $(FFI_DEPS:$(FFI_PATH)%=%)
@touch $@
build-cli:
go build -o pow exe/cli/main.go
.PHONY: build-cli

.update-modules:
git submodule update --init --recursive
@touch $@
build-server:
go build -o powd exe/server/main.go
.PHONY: build-server

build: .update-modules .filecoin-build
build: build-cli build-server
.PHONY: build

clean:
rm -f .filecoin-build
rm -f .update-modules
git submodule deinit --all -f

PARAMCACHE_PATH:=/var/tmp/powergate/filecoin-proof-parameters
test: build
mkdir -p $(PARAMCACHE_PATH)
cat build/proof-params/parameters.json | jq 'keys[]' | xargs touch
mv -n v20* $(PARAMCACHE_PATH)
rm v20* || true
PARAMCACHE_PATH=$(PARAMCACHE_PATH) go test -short -p 1 -count 1 ./...
test:
go test -short -p 1 ./...
.PHONY: test
28 changes: 12 additions & 16 deletions api/server/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ import (
"time"

"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/api/apistruct"
"github.com/improbable-eng/grpc-web/go/grpcweb"
"github.com/ipfs/go-datastore"
badger "github.com/ipfs/go-ds-badger2"
Expand Down Expand Up @@ -93,6 +92,7 @@ type Config struct {
IpfsApiAddr ma.Multiaddr
LotusAddress ma.Multiaddr
LotusAuthToken string
LotusMasterAddr string
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Embedded bool
GrpcHostNetwork string
GrpcHostAddress string
Expand All @@ -104,25 +104,21 @@ type Config struct {

// NewServer starts and returns a new server with the given configuration.
func NewServer(conf Config) (*Server, error) {
var c *apistruct.FullNodeStruct
var cls func()
var err error
var masterAddr address.Address
c, cls, err := lotus.New(conf.LotusAddress, conf.LotusAuthToken)
if err != nil {
return nil, fmt.Errorf("connecting to lotus node: %s", err)
}

if conf.Embedded {
c, cls, err = lotus.NewEmbedded()
if err != nil {
return nil, fmt.Errorf("creating the embedded network: %s", err)
}
masterAddr, err = c.WalletDefaultAddress(context.Background())
if err != nil {
return nil, fmt.Errorf("getting default address: %s", err)
if masterAddr, err = c.WalletDefaultAddress(context.Background()); err != nil {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In embedded mode we get the master address as the default, like in the old days.
This because that whole devnet is ephemeral.

return nil, fmt.Errorf("getting default wallet addr as masteraddr: %s", err)
}

} else {
c, cls, err = lotus.New(conf.LotusAddress, conf.LotusAuthToken)
}
if err != nil {
return nil, err
if masterAddr, err = address.NewFromString(conf.LotusMasterAddr); err != nil {
return nil, fmt.Errorf("parsing masteraddr: %s", err)
}
}

fchost, err := fchost.New()
Expand Down Expand Up @@ -174,7 +170,7 @@ func NewServer(conf Config) (*Server, error) {
lchain := lotuschain.New(c)
var ms ffs.MinerSelector
if conf.Embedded {
ms = fixed.New([]fixed.Miner{{Addr: "t0300", EpochPrice: 4000000}})
ms = fixed.New([]fixed.Miner{{Addr: "t01000", EpochPrice: 1000000}})
Comment on lines -177 to +173
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Necessary consequences of switching to testnet/3. Similar changes are in the wild.

} else {
ms = reptop.New(rm, ai)
}
Expand Down
32 changes: 16 additions & 16 deletions api/stub/asks.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,29 +19,29 @@ func (a *Asks) Get(ctx context.Context) (*ask.Index, error) {
Miner: "miner1",
Price: 5001,
MinPieceSize: 1004,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 1,
Expiry: 100,
Comment on lines -22 to +23
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Timestamp and Expiry are now of type ChainEpoch, which ultimately is an integer.
That integer refers to the height at which things happen (or will).

},
"miner2": ask.StorageAsk{
Miner: "miner2",
Price: 5002,
MinPieceSize: 1004,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 2,
Expiry: 100,
},
"miner3": ask.StorageAsk{
Miner: "miner3",
Price: 5003,
MinPieceSize: 1004,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 3,
Expiry: 100,
},
"miner4": ask.StorageAsk{
Miner: "miner4",
Price: 5004,
MinPieceSize: 1004,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 4,
Expiry: 100,
},
}
return &ask.Index{
Expand All @@ -59,29 +59,29 @@ func (a *Asks) Query(ctx context.Context, query ask.Query) ([]ask.StorageAsk, er
Miner: "miner1",
Price: 1245,
MinPieceSize: 1024,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 1,
Expiry: 100,
},
ask.StorageAsk{
Miner: "miner2",
Price: 3420,
MinPieceSize: 2048,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 2,
Expiry: 200,
},
ask.StorageAsk{
Miner: "miner3",
Price: 1245,
MinPieceSize: 1024,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 3,
Expiry: 100,
},
ask.StorageAsk{
Miner: "miner4",
Price: 1245,
MinPieceSize: 1024,
Timestamp: uint64(time.Now().Unix()),
Expiry: uint64(time.Now().Unix()),
Timestamp: 4,
Expiry: 200,
},
}
return asks, nil
Expand Down
7 changes: 0 additions & 7 deletions build/parameters.go

This file was deleted.

103 changes: 0 additions & 103 deletions build/proof-params/parameters.json

This file was deleted.

15 changes: 5 additions & 10 deletions chainsync/chainsync.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,18 @@ import (
"context"
"fmt"

"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
"github.com/textileio/lotus-client/api/apistruct"
"github.com/textileio/lotus-client/chain/store"
)

// API provides an abstraction to a Filecoin full-node
type API interface {
ChainGetPath(context.Context, types.TipSetKey, types.TipSetKey) ([]*store.HeadChange, error)
ChainGetGenesis(context.Context) (*types.TipSet, error)
}
Comment on lines -12 to -15
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to drop these "abstractions" from modules. In the end they are always referring to concrete Lotus APIs.
Originally, I thought it would be useful to just narrow down which APIs where used here.

Feels defining this API might be very much related in plans of supporting multiple clients or similar. It will take some works to build some reasonable-coherent generic API. But this is another story.


// ChainSync provides methods to resolve chain syncing situations
type ChainSync struct {
api API
api *apistruct.FullNodeStruct
}

// New returns a new ChainSync
func New(api API) *ChainSync {
func New(api *apistruct.FullNodeStruct) *ChainSync {
return &ChainSync{
api: api,
}
Expand All @@ -42,7 +37,7 @@ func (cs *ChainSync) Precedes(ctx context.Context, from, to types.TipSetKey) (bo

// ResolveBase returns the base TipSetKey that both left and right TipSetKey share,
// plus a Revert/Apply set of operations to get from last to new.
func ResolveBase(ctx context.Context, api API, left *types.TipSetKey, right types.TipSetKey) (*types.TipSetKey, []*types.TipSet, error) {
func ResolveBase(ctx context.Context, api *apistruct.FullNodeStruct, left *types.TipSetKey, right types.TipSetKey) (*types.TipSetKey, []*types.TipSet, error) {
var path []*types.TipSet
if left == nil {
genesis, err := api.ChainGetGenesis(ctx)
Expand Down
7 changes: 3 additions & 4 deletions chainsync/chainsync_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,13 @@ func TestMain(m *testing.M) {
}

func TestPrecede(t *testing.T) {
dnet, _, _, close := tests.CreateLocalDevnet(t, 1)
defer close()
client, _, _ := tests.CreateLocalDevnet(t, 1)
ctx := context.Background()

h, err := dnet.Client.ChainHead(ctx)
h, err := client.ChainHead(ctx)
checkErr(t, err)

csync := New(dnet.Client)
csync := New(client)
head := h.Key()
prevhead := types.NewTipSetKey(h.Blocks()[0].Parents...)
yes, err := csync.Precedes(ctx, prevhead, head)
Expand Down
Loading