Skip to content

Initialize project #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 5, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Editor configuration, see http://editorconfig.org
root = true

[*]
charset = utf-8
indent_style = space
indent_size = 2
insert_final_newline = true
trim_trailing_whitespace = true

[*.md]
max_line_length = off
trim_trailing_whitespace = false
35 changes: 35 additions & 0 deletions .eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"root": true,
"ignorePatterns": ["**/*"],
"plugins": ["@nrwl/nx"],
"overrides": [
{
"files": ["*.ts", "*.tsx", "*.js", "*.jsx"],
"rules": {
"@nrwl/nx/enforce-module-boundaries": [
"error",
{
"enforceBuildableLibDependency": true,
"allow": [],
"depConstraints": [
{
"sourceTag": "*",
"onlyDependOnLibsWithTags": ["*"]
}
]
}
]
}
},
{
"files": ["*.ts", "*.tsx"],
"extends": ["plugin:@nrwl/nx/typescript"],
"rules": {}
},
{
"files": ["*.js", "*.jsx"],
"extends": ["plugin:@nrwl/nx/javascript"],
"rules": {}
}
]
}
23 changes: 23 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

## Type of change

Please delete options that are not relevant.

- [ ] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
- [ ] This change requires a documentation update

# Checklist:

- [ ] My code follows the style guidelines of this project
- [ ] I have performed a self-review of my own code
- [ ] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation
- [ ] My changes generate no new warnings
- [ ] Any dependent changes have been merged and published in downstream modules
39 changes: 39 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# See http://help.github.com/ignore-files/ for more about ignoring files.

# compiled output
/dist
/tmp
/out-tsc

# dependencies
node_modules

# IDEs and editors
/.idea
.project
.classpath
.c9/
*.launch
.settings/
*.sublime-workspace

# IDE - VSCode
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json

# misc
/.sass-cache
/connect.lock
/coverage
/libpeerconnection.log
npm-debug.log
yarn-error.log
testem.log
/typings

# System Files
.DS_Store
Thumbs.db
4 changes: 4 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Add files here to ignore them from prettier formatting

/dist
/coverage
3 changes: 3 additions & 0 deletions .prettierrc
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"singleQuote": true
}
27 changes: 27 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# CONTRIBUTING

Contributions are welcome, and are accepted via pull requests. Please review
these guidelines before submitting any pull requests.

## Process

1. Fork the project.
1. Create a new branch.
1. Code, test, commit and push.
1. Open a pull request detailing your changes. Make sure to follow the
[template][1].

## Guidelines

* Please follow the [TypeScript Style Guide by Google][2].
* Send a coherent commit history, making sure each individual commit in your
pull request is meaningful.
* Please follow the [Conventional Commits specification][3].
* You may need to [rebase][4] to avoid merge conflicts.
* Please remember that we follow [SemVer][5].

[1]:.github/PULL_REQUEST_TEMPLATE.md
[2]:https://google.github.io/styleguide/tsguide.html
[3]:https://www.conventionalcommits.org/en/v1.0.0
[4]:https://git-scm.com/book/en/v2/Git-Branching-Rebasing
[5]:http://semver.org
186 changes: 185 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,185 @@
# tfjs-node-helpers
# tfjs-node-helpers

## Introduction

This library was created to simplify the work with [TensorFlow.js][1] in
[Node][2].

Currently, this library provides the helpers for the
[binary classification task][3].
The long-term plan is to implement helpers for other tasks, for example,
[regression][4] and [multiclass classification][5]), as well as to cover
different [machine learning approaches][6].

## Installation

Before you start using the helpers in your project, you need to install the
[@ronas-it/tfjs-node-helpers][7] package:

```bash
> npm install @ronas-it/tfjs-node-helpers --save
```

## Usage

### Feature extraction

Before training any model, you need to extract the valuable information
from your dataset. This information is usually called *features*.
This library provides a few helpers to streamline the process of feature extraction.

First, you need to define the feature extractors. In the example below we
extract the `gender` feature from the dataset item. For that we create a
`GenderFeatureExtractor` class extending the `FeatureExtractor` base class
provided by the library. Please note that feature extractors also encode the
extracted information as a number in the range between `0` and `1`, so that it
can be consumed when training the model.

```typescript
type DatasetItem = {
id: number;
gender: string;
age: number;
annual_salary: number;
owns_the_car: number;
};

class GenderFeatureExtractor extends FeatureExtractor<DatasetItem, FeatureType> {
public featureType = FeatureType.GENDER;

public extract(item: DatasetItem): Feature<FeatureType> {
return new Feature({
type: this.featureType,
label: item.gender,
value: (item.gender === 'Male') ? 1 : 0
});
}
}
```

That's it! Now we can use the defined feature extractor to extract valuable
information from our dataset.

### Binary classification

This library provides two classes to train and evaluate binary classification
models:

1. `BinaryClassificationTrainer` – used for training and testing.
1. `BinaryClassifier` – used for evaluation.

#### Creating the trainer

Before training the model, you need to create an instance of the
`BinaryClassificationTrainer` class first and provide a few parameters:

- `batchSize` – the number of training samples in each batch.
- `epochs` – the maximum number of iterations that we should train the model.
- `patience` – the number of iterations after which the trainer will stop if
there is no improvement.
- `hiddenLayers` – a list of hidden layers. You can also provide the custom
model by using the optional `model` parameter instead.
- `inputFeatureExtractors` – a list of feature extractors to extract information
that should be fed into the model as inputs.
- `outputFeatureExtractor` – the feature extractor to extract information that
we want to predict.

An example can be found below:

```typescript
const trainer = new BinaryClassificationTrainer({
batchSize: BATCH_SIZE,
epochs: EPOCHS,
patience: PATIENCE,
hiddenLayers: [
layers.dense({ units: 128, activation: 'mish' }),
layers.dense({ units: 128, activation: 'mish' })
],
inputFeatureExtractors: [
new AgeFeatureExtractor(),
new AnnualSalaryFeatureExtractor(),
new GenderFeatureExtractor()
],
outputFeatureExtractor: new OwnsTheCarFeatureExtractor()
});
```

#### Training and testing

To train the model, you need to call the `trainAndTest` method of the
instantiated `BinaryClassificationTrainer`.

You can pass the `data` parameter, and in this case trainer will extract
features from the provided dataset first. If you want something more customized,
then you can create the datasets for training, validation and testing manually,
and pass them as the `trainingDataset`, `validationDataset` and `testingDataset`
parameters.

You can also print the testing results by setting the `printResults` to `true`.

An example can be found below:

```typescript
await trainer.trainAndTest({
data,
printResults: true
});
```

#### Saving the model

To save the trained model, you need to call the `save` method of the
instantiated `BinaryClassificationTrainer` and pass the path where the model
should be saved:

```typescript
await trainer.save(join(__dirname, './trained_model'));
```

#### Creating the classifier

Before evaluating the model, you need to create an instance of the
`BinaryClassifier` class:

```typescript
const classifier = new BinaryClassifier();
```

#### Loading the trained model

To load the trained model, you need to call the `load` method of the
instantiated `BinaryClassifier` class and pass the path where the model json
file is located:

```typescript
await classifier.load(join(__dirname, './trained_model/model.json'));
```

#### Evaluation

To evaluate the trained model, you need to load it first, and then call the
`predict` method of the instantiated `BinaryClassifier` class and pass an array
of encoded inputs which will be fed into the model:

```typescript
const ownsTheCar = await classifier.predict([0.2, 0.76, 0]);
```

## Contributing

Thank you for considering contributing to `tfjs-node-helpers` library! The
contribution guide can be found in the [Contributing guide][8].

## License

`tfjs-node-helpers` is licensed under the [MIT license][9].

[1]:https://www.tensorflow.org/js
[2]:https://nodejs.org
[3]:https://en.wikipedia.org/wiki/Binary_classification
[4]:https://en.wikipedia.org/wiki/Regression_analysis
[5]:https://en.wikipedia.org/wiki/Multiclass_classification
[6]:https://en.wikipedia.org/wiki/Machine_learning#Approaches
[7]:https://www.npmjs.com/package/@ronas-it/tfjs-node-helpers
[8]:CONTRIBUTING.md
[9]:LICENSE
5 changes: 5 additions & 0 deletions jest.config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { getJestProjects } from '@nrwl/jest';

export default {
projects: getJestProjects(),
};
3 changes: 3 additions & 0 deletions jest.preset.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
const nxPreset = require('@nrwl/jest/preset').default;

module.exports = { ...nxPreset };
32 changes: 32 additions & 0 deletions nx.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"$schema": "./node_modules/nx/schemas/nx-schema.json",
"npmScope": "@ronas-it/tfjs-node-helpers",
"affected": {
"defaultBase": "main"
},
"implicitDependencies": {
"package.json": {
"dependencies": "*",
"devDependencies": "*"
},
".eslintrc.json": "*"
},
"tasksRunnerOptions": {
"default": {
"runner": "nx/tasks-runners/default",
"options": {
"cacheableOperations": ["build", "lint", "test", "e2e"]
}
}
},
"targetDefaults": {
"build": {
"dependsOn": ["^build"]
}
},
"workspaceLayout": {
"appsDir": "packages",
"libsDir": "packages"
},
"defaultProject": "tfjs-node-helpers-example"
}
Loading