Releases: lightly-ai/lightly
Fix Imports
Fix Imports
Fixes a bug introduced in the last release.
Possible in v1.1.1 but not in v1.1.0:
import lightly
dataset = lightly.data.LightlyDataset(input_dir='my/dataset')Models
Self-supervised Active Learning
Lightly gets support for Active-Learning
We're excited to offer our new active-learning functionality! Use the strong representations learned in a self-supervised fashion together with model predictions to further improve the data selection process.
This release introduces breaking changes with respect to the API calls.
Active-Learning
The self-supervised representations together with the model predictions provide a great basis for deciding which samples should be annotated and which ones are redundant.
This release brings a completely new interface with which you can add active-learning to your ML project with just a few lines of code.:
- ApiWorkflowClient:
lightly.api.api_workflow_client.ApiWorkflowClient
The ApiWorkflowClient is used to connect to our API. The API handles the selection of the images based on embeddings and active- learning scores. To initialize the ApiWorkflowClient, you will need the datasetId and the token from the Lightly Platform. - ActiveLearningAgent:
lightly.active_learning.agents.agent.ActiveLearningAgent
The ActiveLearningAgent builds the client interface of our active-learning framework. It helps with indicating which images are preselected and which ones to sample from. Furthermore, one can query it to get a new batch of images. To initialize an ActiveLearningAgent you need an ApiWorkflowClient. - SamplerConfig:
lightly.active_learning.config.sampler_config.SamplerConfig
The SamplerConfig allows the configuration of a sampling request. In particular, you can set the number of samples, the name of the resulting selection, and the SamplingMethod. Currently, you can set the SamplingMethod to one of the following:- Random: Selects samples uniformly at random.
- Coreset: Selects samples that are diverse.
- Coral: Combines Coreset with scores to do active-learning.
- Scorer:
lightly.active_learning.scorers.scorer.Scorer
The Scorer takes as input the predictions of a pre-trained model on the set of unlabeled images. It evaluates different scores based on how certain the model is about the images and passes them to the API so the sampler can use them with Coral.
Check out our documentation to learn more!
API (breaking)
With the refactoring of our API, we are switching to using a generated Python client. This leads to clearer and unified endpoints, fewer errors, and better error messages. Unfortunately, this means that previous versions of the package are no longer compatible with our new API.
Note that this only affects all API calls. Using the package for self-supervised learning is unaffected.
Models
SimSiam and Refactoring of Models and Dataset
SimSiam and Refactoring of Models and Dataset
This release contains breaking changes. The models SimCLR and MoCo, the LightlyDataset, and the BaseCollateFunction were refactored. These changes were necessary to make the code base better understandable.
SimSiam (@busycalibrating)
An implementation of the SimSiam self-supervised framework is introduced. It relies on a siamese network architecture and aims to maximize similarity between two augmentations of one image.
Refactoring: LightlyDataset
The LightlyDataset is refactored such that the constructor now always expects an input directory input_dir which indicates where the images are stored. To use a LightlyDataset with any PyTorch dataset, the class method LightlyDataset.from_torch_dataset can be used.
1.0.7 (incompatible)
>>> dataset = LightlyDataset(from_folder='path/to/data')
>>>
>>> dataset = LightlyDataset(root='./', name='cifar10', download=True)1.0.8
>>> dataset = LightlyDataset(input_dir='path/to/data')
>>>
>>> torch_dataset = torchvision.datasets.CIFAR10(root='./', download=True)
>>> dataset = LightlyDataset.from_torch_dataset(torch_dataset)Refactoring: BaseCollateFunction
The BaseCollateFunction now returns a tuple of augmented image batches along with the labels and filenames (aug0, aug1), labels, filenames where aug0 and aug1 are both of shape bsz x channels x H x W.
Refactoring: SimCLR, MoCo and NTXentLoss
In accordance with the changes of the BaseCollateFunction, SimCLR and MoCo will expect the augmented images seperately now instead of as a single batch. Similarly, the NTXentLoss now requires a separate batch of representations as inputs.
1.0.7 (incompatible)
>>> # batch size is 128
>>> batch, labels, filenames = next(iter(dataloader))
>>> batch.shape
torch.Size([256, 3, 32, 32])
>>> # number of features is 64
>>> y = simclr(batch)
>>> y.shape
torch.Size([256, 64])
>>> loss = ntx_ent_loss(y)1.0.8
>>> # batch size is 128
>>> (batch0, batch1), labels, filenames = next(iter(dataloader))
>>> batch0.shape
torch.Size([128, 3, 32, 32])
>>> batch1.shape
torch.Size([128, 3, 32, 32])
>>> # number of features is 64
>>> y0, y1 = simclr(batch0, batch1)
>>> y0.shape
torch.Size([128, 64])
>>> y1.shape
torch.Size([128, 64])
>>> loss = ntx_ent_loss(y0, y1)Documentation Updates
A tutorial about how to use the SimSiam model is added along with some minor changes and improvements.
Minor Changes
Private functions are hidden from autocompletion.
Models
Video Reader Backend
Video Reader Backend
Torchvision Video Reader Compatability
If available, the video loader can use the torchvision video reader backend to load frames quicker.
The new sequential video loader also allows much faster iteration through frames when they are processed in order.
Continous Testing
Integration of continuous unit testing with a badge in the README.
New SimCLR Tutorial
SimCLR with only a few lines of code - tutorial on a clothing dataset.
Models
Custom Backbones Made Easy
Custom Backbones Made Easy
Decoupling Self-Supervised Models from ResNet
The implementation of SimCLR and MoCo have been changed such that they can now be constructed from an arbitrary backbone network.
Furthermore, the backbone of the self-supervised models is now called backbone instead of features.
Big Documentation Update
The documentation has received a lot of love and improvements to make working with lightly easier.
There is also a new tutorial on how to train MoCo on Cifar-10.
Minor Changes
The LightlyDataset can now be passed a list of indices marking relevant samples. Non-relevant samples will be ignored during further processing of the data.
Models
Customizable Checkpoint Callbacks, Batch Shuffling and More
Customizable Checkpoint Callbacks, Batch Shuffling and More
Fixed download speed for image datasets.
lightly-magic can now be used with trainer.max_epochs=0.
Fixed the pytorch-lightning warning: "Passing a ModelCheckpoint instance to Trainer(checkpoint_callbacks=...) is deprecated since v1.1 and will no longer be supported in v1.3."
Customizable Checkpoint Callback
Checkpoint callbacks are now customizable (even from the command-line):
# save the 5 best models
lightly-train input_dir='data/' checkpoint_callback.save_top_k=5
# don't save the model of the last epoch
lightly-train input_dir='data/' checkpoint_callback.save_last=FalseBatch Shuffling
Added batch shuffling to MoCo and SplitBatchNorm to simulate multi-gpu behaviour.
Image Resizing
Images can be resized before uploading them to the web-app:
# no resizing (default)
lightly-upload input_dir='data/' dataset_id='XYZ' token='123' resize=-1
# resize such that shortest edge of the image is 128
lightly-upload input_dir='data/' dataset_id='XYZ' token='123' resize=128
# resize images to (128, 128)
lightly-upload input_dir='data/' dataset_id='XYZ' token='123' resize=[128,128]Models
Video File Support and Minor Changes
Video File Support and Minor Changes
Refactoring of lightly.api.upload.py and lightly.api.utils.py.
Moved the checkpoint loading from the CLI to lightly.models.simclrandlightly.models.moco` respectively.
Minor bug fixes.
Video File Support
Lightly can now directly work with video files! No need to extract the frames first. Check the docs to see how!
Models
Bug Fixes and On-Premise Documentation
Bug Fixes and On-Premise Documentation
Fixed the efficiency display during embedding.
Added trainer.precision key to config. Train at half-precision with:
lightly-train input_dir=my/input/dir trainer.precision=16New On-Premise Documentation
The documentation received a whole new part on how to use the Lightly on-premise docker solution.
Models
Bug Fixes and Improvements
Bug Fixes and Improvements
Refactoring of lightly.api to remove circular imports.
Rewriting of import statements to ensure compatability with Python 3.6.
Handled the warning from pytorch_lightning during training.
Models
MoCo and New Documentation
MoCo and New Documentation
New Model: MoCo
lightly.models.moco.ResNetMoCo implements the momentum encoder architecture for self-supervised visual representation learning.
lightly.loss.memory_bank.MemoryBankWrapper allows the training of self-supervised models with a queue of negative samples.
New Documentation
The URL for the documentation has changed to https://docs.lightly.ai.
New section on how lightly works.
New tutorials have been added, check out the Pizza Tutorial to learn how to train a pizza classifier.
Further Changes:
Refactoring of lightly.api.
Default collate functions which implement the SimCLR and MoCo (v1) transfomations.
Collate functions work with tuple as input_size.
New tests and tox environments.
Removed sklearn dependency for PCA.