A state in reinforcement learning represents the current situation of an agent within its environment.
It contains the information that the agent can use to understand what is happening around it and decide what to do next.
For example, imagine a robot moving through a warehouse.
The robot's state might include:
- Its current location
- Nearby obstacles
- Battery level
- Distance from the target
- Current movement direction
The agent uses this information to select an action.
The basic reinforcement learning process is:
State → Action → Environment → Reward → New State → Next Action
Why is a State Important?
An agent needs information about its current situation before choosing an action.
Without useful state information, the agent may not know which action makes sense.
For example, consider a game character standing near an obstacle.
If the state tells the agent that an obstacle is directly ahead, it may choose to move left or right.
If the obstacle is not represented in the state, the agent may move forward and collide with it.
Therefore, the state provides important context for decision-making.
What Does a State Represent?
A state can contain different types of information.
The exact information depends on the reinforcement learning problem.
For example, in a game, the state might contain:
- Player position
- Enemy positions
- Health level
- Available items
- Current score
- Remaining time
In a robot, it might contain:
- Location
- Speed
- Direction
- Sensor readings
- Battery level
- Distance from objects
In a financial simulation, it might contain:
- Current prices
- Portfolio information
- Available cash
- Previous actions
- Market indicators
The state should provide information that helps the agent make useful decisions.
State and Environment
The environment is the world where the agent operates.
The state describes the agent's current situation within that environment.
For example:
Environment: A maze
State: Agent's current position and nearby walls
Action: Move left, right, forward, or backward
Reward: Positive reward for reaching the exit
The environment changes after the agent performs an action.
This usually produces a new state.
How Does an Agent Use a State?
The agent observes the current state.
It then uses its learned policy or decision-making process to select an action.
For example:
Current State
↓
Agent observes information
↓
Agent chooses an action
↓
Environment responds
↓
Reward received
↓
New State
The agent repeats this process many times.
Through these interactions, it learns which actions work well in different states.
Simple Example of a State
Imagine an agent playing a simple grid game.
The game board contains:
The current situation might be represented as:
Agent → Position (2,3)
Target → Position (5,3)
Wall → Position (3,3)
This information forms part of the current state.
The agent sees that a wall blocks the direct path.
It may choose to move around the wall.
After moving, the environment produces a new state.
The agent then makes another decision.
State Changes After an Action
A state is not necessarily fixed.
It can change after every action.
For example:
State 1 → Agent at Position A
Action → Move Right
State 2 → Agent at Position B
Action → Move Down
State 3 → Agent at Position C
Each new state gives the agent updated information.
The agent uses this information to continue interacting with the environment.
State and Reward
A state and a reward are different concepts.
The state describes the situation.
The reward provides feedback about the result of an action.
For example:
State:
Robot is 5 meters from the target.
Action:
Move forward.
Reward:
+1
New State:
Robot is 4 meters from the target.
The reward tells the agent that the action produced a useful result.
The new state tells the agent what the situation looks like now.
Both are important for learning.
State, Action, and Reward
These three concepts work together.
State
Describes the current situation.
Action
Describes what the agent chooses to do.
Reward
Provides feedback about the outcome.
For example:
State → Robot near obstacle
Action → Turn left
Reward → +2
New State → Robot safely moves around obstacle
After many interactions, the agent can learn that turning left in similar situations may be useful.
What is a State Space?
The state space is the collection of all possible states that an agent can encounter.
For example, imagine a simple game with only four possible positions.
The state space could contain:
Position 1
Position 2
Position 3
Position 4
A more complex environment may have millions or even an extremely large number of possible states.
For example, a self-driving system has many possible combinations of:
- Vehicle position
- Speed
- Road conditions
- Nearby vehicles
- Pedestrians
- Traffic signals
- Weather conditions
This creates a very large state space.
Discrete States
A discrete state has a limited or countable set of possible values.
For example, a board game may have states such as:
Player at Square 1
Player at Square 2
Player at Square 3
Another example is an elevator system:
Floor 1
Floor 2
Floor 3
Floor 4
These states can be represented as distinct values.
Continuous States
A continuous state can contain values that change across a range.
For example, a robot may have:
Speed = 2.73 m/s
Position = 14.82 meters
Temperature = 28.6°C
These values can change continuously.
Many real-world reinforcement learning problems involve continuous states.
State Representation
The way information is stored is called the state representation.
A state can be represented using:
- Numbers
- Vectors
- Arrays
- Images
- Sensor readings
- Text
- Combinations of different inputs
For example, a simple robot state could be represented as:
state = [10.5, 2.3, 0.8, 15.0]
These values might represent:
- Position
- Speed
- Battery level
- Distance from target
The agent processes this information to choose an action.
State in Image-Based Reinforcement Learning
Some environments provide visual information.
For example, a game-playing agent may receive a screenshot instead of simple numerical values.
The image itself can represent the current state.
The agent processes the image and identifies useful patterns.
It can then choose an action based on what it sees.
For example:
Game Screen → State Representation → Action → Reward
This approach is common in reinforcement learning tasks involving games and visual environments.
State and Partial Information
Sometimes the agent cannot observe everything about the environment.
This creates a partially observable environment.
For example, imagine a robot moving through a building.
Its sensors may only detect objects within a certain range.
The robot cannot see through walls.
Therefore, its current observation may not contain the complete state of the environment.
In such cases, the agent may use previous observations or memory to make better decisions.
State vs Observation
These terms are related but not always identical.
A state describes the actual situation of the environment.
An observation is the information the agent receives about that situation.
For example:
Actual Environment State:
A car is approaching from behind a building.
Agent Observation:
The car is not currently visible.
The agent may therefore have only partial information about the true state.
This distinction becomes important in partially observable reinforcement learning.
State and Policy
A policy tells the agent how to choose actions based on states.
For example:
If State = Near obstacle
→ Turn left
If State = Open path
→ Move forward
If State = Target reached
→ Stop
A trained policy can be much more complex than these simple rules.
It learns which actions are useful for different states based on experience.
The general relationship is:
State → Policy → Action
How Does the Agent Learn Better Actions?
The agent repeatedly interacts with the environment.
Suppose an agent is learning to navigate a maze.
At first, it may choose actions randomly.
Some actions lead to negative rewards.
Other actions move it closer to the goal.
Over many attempts, the agent learns which actions tend to produce better rewards from different states.
The process can look like:
Observe State → Choose Action → Receive Reward → Reach New State → Update Learning
This process continues until the agent develops a useful strategy.
Example: Robot Navigation
Consider a robot that must reach a charging station.
The robot's state might include:
Current position
Distance from charging station
Battery level
Nearby obstacles
Movement direction
Suppose the robot is close to an obstacle.
It chooses:
Action → Turn right
The robot safely moves around the obstacle.
It receives:
Reward → +2
The robot enters a new state.
After many similar experiences, it may learn that turning right in certain situations produces good results.
Example: Game Playing
Consider an AI agent playing a racing game.
The state might include:
Car position
Car speed
Track position
Nearby vehicles
Distance from finish line
The agent can choose actions such as:
- Accelerate
- Brake
- Turn left
- Turn right
The environment provides rewards based on performance.
For example:
Stay on track → Positive reward
Move toward finish → Positive reward
Crash → Negative reward
Finish race → Large positive reward
The agent learns which actions work best in different states.
Why Good State Representation Matters
The quality of the state representation can strongly affect learning.
If important information is missing, the agent may struggle to choose good actions.
For example, suppose a robot must avoid obstacles.
If its state only contains its location but not nearby obstacle information, it may not have enough information to make safe decisions.
A useful state representation should provide the information needed for the task.
State in the Reinforcement Learning Loop
The complete process can be summarized as:
1. Observe the State
The agent receives information about its current situation.
2. Choose an Action
The agent selects an action based on its current policy.
3. Interact With the Environment
The environment changes because of the action.
4. Receive a Reward
The agent receives feedback.
5. Observe the New State
The agent sees the updated situation.
6. Learn
The agent updates its strategy based on the experience.
This process repeats many times.
Final Thoughts
A state in reinforcement learning represents the current situation of an agent within its environment.
It provides information that helps the agent decide what action to take.
After an action, the environment produces a reward and usually moves the agent into a new state. By repeating this process, the agent learns which actions produce better long-term results.
In simple terms: A state tells the agent "Where am I, and what is happening right now?" The agent uses that information to choose an action, receives a reward, observes the new state, and gradually learns better decisions.