Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to train with Async SGD? #190

Closed
zuowang opened this issue Oct 11, 2016 · 2 comments
Closed

How to train with Async SGD? #190

zuowang opened this issue Oct 11, 2016 · 2 comments

Comments

@zuowang
Copy link

zuowang commented Oct 11, 2016

I am confused on how to train with Async SGD.
I found there is a command line argument --async_count, but it's not used at all.
Could you give me an example?

@backyes
Copy link
Contributor

backyes commented Oct 11, 2016

check setting(...) in model config file.

python/paddle/trainer_config_helper/optimizers.py
338 def settings(batch_size,
339 learning_rate=1e-3,
340 learning_method=None,
341 regularization=None,
342 is_async=False,
343 model_average=None,
344 gradient_clipping_threshold=None
345 ):

368 :param is_async: Is Async-SGD or not. Default value is False.
369 :type is_async: bool

So, set is_asyn=True enable async-sgd. For most model, is_asyn=True is preferred

@zuowang
Copy link
Author

zuowang commented Oct 12, 2016

Thanks!

@zuowang zuowang closed this as completed Oct 12, 2016
thisjiang pushed a commit to thisjiang/Paddle that referenced this issue Oct 28, 2021
gglin001 pushed a commit to graphcore/Paddle-fork that referenced this issue Dec 8, 2021
* refactor pass

* fix ops and add unitests

* paddle inference update

* paddle inference tests

* python pass update
wangxicoding pushed a commit to wangxicoding/Paddle that referenced this issue Dec 9, 2021
zhoutianzi666 pushed a commit to zhoutianzi666/Paddle that referenced this issue May 23, 2022
…em-optim-on

Fix mkldnn int8 inference config settings doc
zmxdream pushed a commit to zmxdream/Paddle that referenced this issue Dec 24, 2022
* check node&feature format when loading

* check node&feature format when loading (2£ (2)
danleifeng pushed a commit to danleifeng/Paddle that referenced this issue Dec 29, 2022
* check node&feature format when loading

* check node&feature format when loading (2£ (2)

Co-authored-by: root <root@yq01-inf-hic-k8s-a100-ab2-0009.yq01.baidu.com>
zmxdream added a commit that referenced this issue Jan 30, 2023
* add set slot_num for psgpuwraper (#177)

* add set slot_num_for_pull_feature for psgpuwarper

* Add get_epoch_finish python interface (#182)

* add get_epoch_finish interface

* add return

* delete return

* add unzip op (#183)

* fix miss key for error dataset (#186)

* fix miss key for error dataset

* fix miss key for error dataset

Co-authored-by: yangjunchao <yangjunchao@baidu.com>

* add excluded_train_pair and infer_node_type (#187)

* support return of degree (#188)

* fix task stuck in barrier (#189)

Co-authored-by: yangjunchao <yangjunchao@baidu.com>

* check node/feature format when loading (#190)

* check node&feature format when loading

* check node&feature format when loading (2£ (2)

* degrade log (#191)

* [PGLBOX]fix conflict

* [PGLBOX]fix conflict

* [PGLBOX]replace LodTensor with phi::DenseTensor

* [PGLBOX]fix gpu_primitives.h include path

* [PGLBOX]from platform::PADDLE_CUDA_NUM_THREADS to phi::PADDLE_CUDA_NUM_THREADS

* [PGLBOX]fix unzip example code

* [PGLBOX]fix unzip example code

* [PGLBOX]fix unzip example code

* [PGLBOX]fix unzip example code

* [PGLBOX]fix unzip ut

* [PGLBOX]fix unzip ut

* [PGLBOX]fix code style

* [PGLBOX]fix code style

* [PGLBOX]fix code style

* fix code style

* fix code style

* fix unzip ut

* fix unzip ut

* fix unzip ut

* fix unzip

* fix code stype

* add ut

* add c++ ut & fix train_mode_ set

* fix load into memory

* fix c++ ut

* fix c++ ut

* fix c++ ut

* fix c++ ut

* fix code style

* fix collective

* fix unzip_op.cc

* fix barrier

* fix code style

* fix barrier

* fix barrier

* fix code styple

* fix unzip

* add unzip.py

* add unzip.py

* fix unzip.py

---------

Co-authored-by: chao9527 <33347532+chao9527@users.noreply.github.com>
Co-authored-by: Siming Dai <908660116@qq.com>
Co-authored-by: huwei02 <53012141+huwei02@users.noreply.github.com>
Co-authored-by: yangjunchao <yangjunchao@baidu.com>
lizexu123 pushed a commit to lizexu123/Paddle that referenced this issue Feb 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants