How can I change observation states' values in OpenAI gym's cartpole environment?

Question

I am learning with the OpenAI gym's cart pole environment.
I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$
-infty, infty$] to some finite upper and lower limits. (By the way, these states are velocity and pole velocity at the tip).
How can I change these limits in the actual gym's environment?
Any other suggestions are also welcome.

rert588 · Answer

I don't recommend changing the rules of the environment.
What you could do:
Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation.
e.g. Say I do have a continuous state space with one variable in range $[-infty,infty]$
The buckets can be as follows:
0). x < -1000
1). -1000 $le$ x $<$ -500
2). -500 $le$ x $<$ -100
3). -100 $le$ x $<$ -50
4). -50 $le$ x $<$ 0
5). 0 $le$ x $<$ 50
6). 50 $le$ x $<$ 100
7). 100 $le$ x $<$ 500
8). 500 $le$ x $<$ 1000
9). x > 1000
Therefore in this example scenario there are 9 buckets. Hence, the observations can be in range [0, 9] discretely.

How can I change observation states' values in OpenAI gym's cartpole environment?

One Answer

Add your own answers!

Ask a Question