Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft pull request: Add Vapor and Linux support #642

Closed
wants to merge 38 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
280316c
exclude ios vc by default
ronaldmannak Oct 21, 2022
1850b46
Merge branch 'skywinder:develop' into develop
ronaldmannak Oct 30, 2022
3461cb8
Import FoundationNetworking for Linux
ronaldmannak Oct 30, 2022
773ba92
URLsession to FoundationNetworking
ronaldmannak Oct 30, 2022
f14a899
FoundationNetworking preceeding Foundation
ronaldmannak Oct 30, 2022
33479ee
Add Linux alternative to SecRandomCopyBytes
ronaldmannak Oct 30, 2022
c8fc0a7
Fix return random
ronaldmannak Oct 30, 2022
9898f0d
Fix count
ronaldmannak Oct 30, 2022
5f90f67
Import Vapor
ronaldmannak Oct 30, 2022
8460d92
Add [UInt8].random
ronaldmannak Oct 30, 2022
b7b020d
Fix [UInt8] to Data
ronaldmannak Oct 30, 2022
d4955d4
Excluded WebKit extension for Linux
ronaldmannak Oct 30, 2022
855ad8e
Disabled CoreImage for Linux
ronaldmannak Oct 30, 2022
936ff2a
Comment out random
ronaldmannak Oct 30, 2022
268564f
Rename random to randomData
ronaldmannak Oct 30, 2022
8d7c7df
fix randomData
ronaldmannak Oct 30, 2022
8486802
Add Ubuntu CI workflow
ronaldmannak Oct 30, 2022
44039d2
Rename swift action to ubuntu
ronaldmannak Oct 30, 2022
9e341a3
Add dev/random
ronaldmannak Oct 30, 2022
93a89c1
Use URandom
ronaldmannak Oct 30, 2022
ec86b96
Swift to temp random
ronaldmannak Oct 30, 2022
60b181b
Add try?
ronaldmannak Oct 31, 2022
5976670
Remove force unwraps in unit test
ronaldmannak Oct 31, 2022
84528e2
Force actions to run
ronaldmannak Oct 31, 2022
cdf4ad4
Update Ubuntu actions
ronaldmannak Oct 31, 2022
616168f
Run tests in single step
ronaldmannak Oct 31, 2022
bea91b5
Install ganache
ronaldmannak Oct 31, 2022
ab991d5
Install Node
ronaldmannak Oct 31, 2022
bb0a410
ping localhost
ronaldmannak Nov 1, 2022
71117e3
install nc
ronaldmannak Nov 1, 2022
3cdb528
remote tests only
ronaldmannak Nov 1, 2022
1f51faf
fix yaml error
ronaldmannak Nov 1, 2022
96191e9
Revert to default swift test
ronaldmannak Nov 1, 2022
b7b77a4
fix localhost typo
ronaldmannak Nov 2, 2022
f3c7125
Intentional wrong port. Should fail
ronaldmannak Nov 2, 2022
71ec91d
revert to correct port
ronaldmannak Nov 2, 2022
ce46ea7
Update Ubunti CI script
ronaldmannak Nov 3, 2022
7052ac7
The inevitable “I love YAML” commit
ronaldmannak Nov 3, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/ubuntu.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# This workflow will build a Swift project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-swift

name: Ubuntu

on:
push:
branches:
- master
- develop
- hotfix
- unstable
paths:
- Packag*.swift
- web3swift.podspec
- Cartfile
- Sources/**
- 'Tests/**'
- 'web3swift*/**'
- '.github/workflows/**'
pull_request:
branches:
- master
- develop
- unstable

jobs:
build:

runs-on: ubuntu-latest
container: swift:5.7-focal

steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
with:
node-version: 18
- name: Resolve dependencies
run: swift package resolve
- name: Build
run: swift build
- name: Install ganache
run: npm install ganache --global
- name: Start ganache in background
run: ganache &
- name: Wait till ganache starts
run: sleep 1
- name: install nc
run: apt-get update && apt-get install -y netcat
- name: Ping
run: nc -vz 127.0.0.1 8545
- name: Run tests
run: swift test
6 changes: 3 additions & 3 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@

import PackageDescription

#if os(macOS)
#if os(iOS)
let excludeFiles: String = []
#else
let excludeFiles = [
"./Browser/BrowserViewController.swift" // Because of inheriting iOS only class failed to build on macOS.
]
#elseif os(iOS)
let excludeFiles: String = []
#endif

let package = Package(
Expand Down
3 changes: 3 additions & 0 deletions Sources/Core/EthereumNetwork/Request/APIRequest+Methods.swift
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
// Created by Yaroslav Yashin on 12.07.2022.
//

#if canImport(FoundationNetworking)
import FoundationNetworking
#endif
JeneaVranceanu marked this conversation as resolved.
Show resolved Hide resolved
import Foundation
import BigInt

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
// Created by Yaroslav Yashin on 05.06.2022.
//

#if canImport(FoundationNetworking)
import FoundationNetworking
#endif
JeneaVranceanu marked this conversation as resolved.
Show resolved Hide resolved
import Foundation

@available(iOS, obsoleted: 15.0, message: "Use the built-in API instead")
Expand Down
5 changes: 5 additions & 0 deletions Sources/Core/Structure/SECP256k1.swift
Original file line number Diff line number Diff line change
Expand Up @@ -336,6 +336,10 @@ extension SECP256K1 {
}

internal static func randomBytes(length: Int) -> Data? {
#if os(Linux)
// return Data(URandom().bytes(count: length))
return try? Data.random(length: length)
#else
Comment on lines +339 to +342
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like it's good time to change this sepc256k implementation to those that bitcoin maintains, that did @JeneaVranceanu proposed in the past. I'd bet it's cross platform.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be cross-platform as it's written in C but it's not a solution here as the code you are commenting on is our custom extension function and it's just the question of how to get secure random bytes on Linux.

I'd not complicate this PR with the secp256k1 update.

for _ in 0...1024 {
var data = Data(repeating: 0, count: length)
let result = data.withUnsafeMutableBytes { (mutableRBBytes) -> Int32? in
Expand All @@ -353,6 +357,7 @@ extension SECP256K1 {
}
}
return nil
#endif
}

internal static func toByteArray<T>(_ value: T) -> [UInt8] {
Expand Down
3 changes: 3 additions & 0 deletions Sources/Core/Structure/Web3ProviderProtocol.swift
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
// Created by Yaroslav Yashin on 11.07.2022.
//

#if canImport(FoundationNetworking)
import FoundationNetworking
#endif
Comment on lines +8 to +10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this proposal I see that URL doesn't included within FoundationNetwork, is it necessary here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me double check if this one is necessary

import Foundation

public protocol Web3Provider {
Expand Down
6 changes: 6 additions & 0 deletions Sources/Core/Utility/Data+Extension.swift
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@

import Foundation


extension Data {
init<T>(fromArray values: [T]) {
let values = values
Expand Down Expand Up @@ -41,6 +42,10 @@ extension Data {
}

public static func randomBytes(length: Int) -> Data? {
#if os(Linux)
// return Data(URandom().bytes(count: length))
return try? Data.random(length: length)
#else
Comment on lines +45 to +48
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is necessary, why btw? But if it is, it's better to implement it in a way as backward capability implemented above.

Means implement exact same methods as existed one, but marked it with @available(obsolete) on all Apple platforms.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is SecRandomCopyBytes isn't available on Linux. I was trying to avoid spinning up a new process and use dev/random, but now believe it might be inevitable.

for _ in 0...1024 {
var data = Data(repeating: 0, count: length)
let result = data.withUnsafeMutableBytes { (body: UnsafeMutableRawBufferPointer) -> Int32? in
Expand All @@ -56,6 +61,7 @@ extension Data {
}
}
return nil
#endif
}

public func bitsInRange(_ startingBit: Int, _ length: Int) -> UInt64? { // return max of 8 bytes for simplicity, non-public
Expand Down
75 changes: 75 additions & 0 deletions Sources/Core/Utility/URandom.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
//
// File.swift
//
//
// Created by Ronald Mannak on 10/29/22.
//

import Foundation
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we importing libc here?

#if canImport(libc)
import libc


///// URandom represents a file connection to /dev/urandom on Unix systems.
///// /dev/urandom is a cryptographically secure random generator provided by the OS.
//public final class URandom: RandomProtocol {
// public enum Error: Swift.Error {
// case open(Int32)
// case read(Int32)
// }
//
// private let file: UnsafeMutablePointer<FILE>
//
// /// Initialize URandom
// public init(path: String) throws {
// guard let file = fopen(path, "rb") else {
// // The Random protocol doesn't allow init to fail, so we have to
// // check whether /dev/urandom was successfully opened here
// throw Error.open(errno)
// }
// self.file = file
// }
//
// deinit {
// fclose(file)
// }
//
// private func read(numBytes: Int) throws -> [Int8] {
//
//
// // Initialize an empty array with space for numBytes bytes
// var bytes = [Int8](repeating: 0, count: numBytes)
// guard fread(&bytes, 1, numBytes, file) == numBytes else {
// // If the requested number of random bytes couldn't be read,
// // we need to throw an error
// throw Error.read(errno)
// }
//
// return bytes
// }
//
// /// Get a random array of Bytes
// public func bytes(count: Int) throws -> Bytes {
// return try read(numBytes: count).map({ Byte(bitPattern: $0) })
// }
//}
//
//extension URandom: EmptyInitializable {
// public convenience init() throws {
// try self.init(path: "/dev/urandom")
// }
//}

#endif


extension Data {
/// Returns cryptographically secure random data.
/// Not safe. Needs to be replaced. See https://forums.swift.org/t/random-data-uint8-random-or-secrandomcopybytes/56165
///
/// - Parameter length: Length of the data in bytes.
/// - Returns: Generated data of the specified length.
public static func random(length: Int) throws -> Data {
return Data((0 ..< length).map { _ in UInt8.random(in: UInt8.min ... UInt8.max) })
}
}
2 changes: 2 additions & 0 deletions Sources/web3swift/Browser/Bridge.swift
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
// Copyright © 2017 Samaritan. All rights reserved.
//

#if !os(Linux)
import WebKit
Comment on lines +9 to 10
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess this file should be excluded from the sources on the Linux platform in a manner that it is now for macOS one. Means it should be ignored in Package.swift.


/// Bridge for WKWebView and JavaScript
Expand Down Expand Up @@ -248,3 +249,4 @@ fileprivate extension WKWebView {
evaluateJavaScript(jsString, completionHandler: completionHandler)
}
}
#endif
6 changes: 6 additions & 0 deletions Sources/web3swift/Utils/EIP/EIP67Code.swift
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@
//

import Foundation
#if !os(Linux)
import CoreImage
#endif
Comment on lines +7 to +9
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've got to investigate what the hell happening here, and why ERC spec file required CoreImage as an import?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's for the creation of the QR code, which is convenient to have on iOS and macOS I assume.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, so we really could consider just to crop this file for a Linux platform until we figured out how we could implement it properly.

import BigInt
import Core

Expand Down Expand Up @@ -77,11 +79,14 @@ extension Web3 {
return mainPart
}

#if !os(Linux)
public func toImage(scale: Double = 1.0) -> CIImage {
return EIP67CodeGenerator.createImage(from: self, scale: scale)
}
#endif
}

#if !os(Linux)
public struct EIP67CodeGenerator {

public static func createImage(from: EIP67Code, scale: Double = 1.0) -> CIImage {
Expand All @@ -94,6 +99,7 @@ extension Web3 {
return image
}
}
#endif

public struct EIP67CodeParser {
public static func parse(_ data: Data) -> EIP67Code? {
Expand Down
3 changes: 3 additions & 0 deletions Sources/web3swift/Web3/Web3+HttpProvider.swift
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
// Copyright © 2018 Alex Vlasov. All rights reserved.
//

#if canImport(FoundationNetworking)
import FoundationNetworking
#endif
Comment on lines +6 to +8
Copy link
Collaborator

@yaroslavyaroslav yaroslavyaroslav Nov 15, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And this one should be refactored to not use URLSession in it (it was a rough one implementation within 3.0.0 release, so it's could be changed).

import Foundation
import BigInt
import Core
Expand Down
2 changes: 2 additions & 0 deletions Tests/web3swiftTests/localTests/EIP67Tests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ class EIP67Tests: LocalTestCase {
print(encoding)
}

#if !os(Linux)
func testEIP67codeGeneration() throws {
var eip67Data = Web3.EIP67Code.init(address: EthereumAddress("0xe22b8979739D724343bd002F9f432F5990879901")!)
eip67Data.gasLimit = BigUInt(21000)
Expand All @@ -29,6 +30,7 @@ class EIP67Tests: LocalTestCase {
let encoding = eip67Data.toImage(scale: 5.0)
XCTAssert(encoding != CIImage())
}
#endif

func testEIP67decoding() throws {
var eip67Data = Web3.EIP67Code.init(address: EthereumAddress("0xe22b8979739D724343bd002F9f432F5990879901")!)
Expand Down
22 changes: 15 additions & 7 deletions Tests/web3swiftTests/localTests/LocalTestCase.swift
Original file line number Diff line number Diff line change
Expand Up @@ -8,31 +8,39 @@ import web3swift
// SuperClass that all local tests should be derived from
// while this class does show up in the navigator, it has no associated tests
class LocalTestCase: XCTestCase {

enum TestError: Error {
case testError
}

static let url = URL(string: "http://127.0.0.1:8545")!

override func setUp() async throws {
let web3 = try! await Web3.new(LocalTestCase.url)
let web3 = try await Web3.new(LocalTestCase.url)

let block = try! await web3.eth.blockNumber()
let block = try await web3.eth.blockNumber()
if block >= 25 { return }

print("\n ****** Preloading Ganache (\(25 - block) blocks) *****\n")

let allAddresses = try! await web3.eth.ownedAccounts()
let allAddresses = try await web3.eth.ownedAccounts()
let sendToAddress = allAddresses[0]
let contract = web3.contract(Web3.Utils.coldWalletABI, at: sendToAddress, abiVersion: 2)
let value = Utilities.parseToBigUInt("1.0", units: .eth)!
guard let contract = web3.contract(Web3.Utils.coldWalletABI, at: sendToAddress, abiVersion: 2),
let value = Utilities.parseToBigUInt("1.0", units: .eth) else {
throw TestError.testError
}

let from = allAddresses[0]
let writeTX = contract!.createWriteOperation("fallback")!
guard let writeTX = contract.createWriteOperation("fallback") else {
throw TestError.testError
}
writeTX.transaction.from = from
writeTX.transaction.value = value
writeTX.transaction.gasLimitPolicy = .manual(78423)
writeTX.transaction.gasPricePolicy = .manual(20000000000)

for _ in block..<25 {
let _ = try! await writeTX.writeToChain(password: "")
let _ = try await writeTX.writeToChain(password: "")
}
}
}