Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In q-learning chapter 6: why sample_env() only once in an iteration? #60

Open
BAXMAY opened this issue Oct 18, 2019 · 0 comments
Open

Comments

@BAXMAY
Copy link

BAXMAY commented Oct 18, 2019

In 01_frozenlake_q_learning.py, every iteration you call sample_env() only one time.

while True:
    iter_no += 1
    s, a, r, next_s = agent.sample_env()
    agent.value_update(s, a, r, next_s)
    reward = 0.0
    ...

I think this can be improve by calling sample_env() for many time in one iteration. like this:

while True:
    iter_no += 1
    for _ in range(1000):
        s, a, r, next_s = agent.sample_env()
        agent.value_update(s, a, r, next_s)
    reward = 0.0
    ...

It can solve in much less iterations.

Is this valid or I misunderstand the concept of q-learning?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant