A reward signal is a numerical value that tells an agent how good or bad its recent action was in a reinforcement learning environment.
The agent uses rewards as feedback while learning. It tries to choose actions that lead to higher rewards over time.
For example, imagine a robot learning to reach a charging station.
If the robot moves closer to the station, it may receive a small positive reward.
If it reaches the station, it may receive a large positive reward.
If it moves in the wrong direction, it may receive a negative reward.
The basic learning cycle is:
State → Action → Environment → Reward → New State → Better Decision
Why is a Reward Signal Important?
An agent does not always know which action is correct.
The reward signal provides feedback after the agent interacts with the environment.
It helps the agent understand which behaviors are useful and which behaviors should be avoided.
For example:
- Reaching a goal → Positive reward
- Moving toward a goal → Small positive reward
- Moving away from a goal → Negative reward
- Breaking a rule → Large negative reward
Over many interactions, the agent learns which actions tend to produce better results.
How Does an Agent Learn From Rewards?
The agent observes its current state.
It then selects an action.
The environment responds to that action.
The agent receives a reward and observes the new state.
This process repeats many times.
For example:
Robot at Position A
↓
Robot moves forward
↓
Robot gets closer to the target
↓
Reward = +1
↓
Robot learns that moving forward can be useful
The agent does not learn from a single reward alone.
It improves by observing patterns across many experiences.
What is a Positive Reward?
A positive reward tells the agent that an action or outcome was useful.
For example, consider a game where an agent must collect coins.
Collecting a coin could produce:
Reward = +10
The positive reward increases the value of actions that helped the agent reach that outcome.
Over time, the agent may learn to choose actions that increase its chances of collecting more coins.
Positive rewards do not always need to be large.
Even a small positive reward can provide useful feedback.
What is a Negative Reward?
A negative reward tells the agent that an action or outcome was undesirable.
For example, if a robot hits an obstacle:
Reward = -10
The agent receives feedback that this behavior should generally be avoided.
Negative rewards are also called penalties in many reinforcement learning systems.
They help the agent learn from unwanted outcomes.
Are Rewards Always Positive or Negative?
No.
A reward can also be zero.
For example:
Reward = 0
This may mean that the current action did not produce a meaningful immediate outcome.
Rewards can also have different scales.
For example:
Small progress → +1
Successful action → +10
Major failure → -10
The exact reward values depend on how the reinforcement learning problem is designed.
Immediate Rewards and Long-Term Rewards
One important idea in reinforcement learning is that the best action is not always the one that gives the largest immediate reward.
An agent often needs to consider future rewards.
Imagine a game where the agent can choose between two actions.
One action gives:
Immediate reward = +5
Another action gives:
Immediate reward = 0
But the second action may lead to a much larger reward later.
The agent should learn to consider the long-term consequences of its actions.
This is why reinforcement learning focuses on maximizing cumulative reward, not simply maximizing the next reward.
What is Cumulative Reward?
Cumulative reward is the total reward an agent expects to receive over a sequence of actions.
For example:
Action 1 → +2
Action 2 → +3
Action 3 → +5
Action 4 → -1
The total reward is:
+2 + 3 + 5 - 1 = 9
The agent aims to learn behavior that produces high total rewards over time.
This is important because some useful actions may not provide an immediate reward.
What is a Reward Function?
A reward function defines how rewards are assigned.
It takes the current situation and action outcome into account.
For example, a simple robot task might use:
Reach target → +100
Move closer → +1
Hit obstacle → -50
Each unnecessary step → -1
This reward function encourages the robot to reach the target efficiently while avoiding obstacles.
The reward function is therefore an important part of reinforcement learning system design.
Why Does Reward Design Matter?
The agent learns what the reward system encourages.
If the reward function is poorly designed, the agent may learn unexpected behavior.
For example, suppose a game rewards an agent for collecting objects.
If the reward system only rewards the number of objects collected, the agent might ignore safety or efficiency.
A better reward system could consider multiple goals:
Collect object → +10
Avoid obstacle → +2
Hit obstacle → -20
Complete task → +100
The reward structure should reflect the actual goal of the task.
Reward Shaping
Reward shaping means adding carefully designed intermediate rewards to help an agent learn.
Suppose a robot must reach a destination.
Giving a reward only when it reaches the destination may make learning difficult.
Instead, the system could provide smaller rewards for progress.
For example:
Move closer → +1
Move farther away → -1
Reach destination → +100
This provides more frequent feedback.
The agent can then learn which actions move it toward the goal.
Reward Signal in a Game
Games provide an easy example.
Imagine an AI agent playing a racing game.
The reward system could be:
Move toward finish → +1
Complete lap → +20
Finish race → +100
Crash → -20
Go off track → -5
The agent plays the game repeatedly.
It observes which actions lead to higher rewards.
Over time, it can learn to:
- Stay on the track
- Avoid crashes
- Take useful turns
- Maintain speed
- Reach the finish line
The reward signal guides this learning process.
Reward Signal in Robotics
Reward signals are also useful in robotics.
Imagine a robotic arm learning to pick up an object.
The reward could be designed as:
Move toward object → +1
Touch object → +5
Grip object correctly → +20
Drop object → -10
Complete pickup → +50
The robot performs many attempts.
It learns from the feedback received after each attempt.
Eventually, it can discover actions that increase the chance of successful pickups.
Reward Signal in Autonomous Systems
Reward signals can also guide autonomous systems.
For example, an autonomous navigation system may consider:
- Reaching the destination
- Avoiding obstacles
- Reducing unnecessary movement
- Following navigation rules
The reward function can assign positive and negative values based on these outcomes.
The agent then learns a policy that attempts to maximize long-term reward.
Reward and Policy
A policy defines how an agent chooses actions.
The reward signal helps the agent improve this policy.
For example:
State → Policy → Action → Reward
If an action produces a useful reward, the agent can increase its preference for similar actions in similar situations.
If an action repeatedly produces poor rewards, the agent can reduce its preference for that behavior.
This process gradually improves decision-making.
Reward and Value
Reinforcement learning also uses the idea of value.
A value represents how useful a state or action is expected to be based on future rewards.
For example, an agent may be in a state that provides little immediate reward.
However, that state may lead to a very large reward later.
The agent can learn that the current state has high long-term value.
This helps it make better decisions than simply looking at immediate rewards.
Reward Signal and Exploration
An agent also needs to discover new actions.
This is called exploration.
For example, an agent may already know one route that provides a reasonable reward.
But another route might provide an even higher reward.
The agent sometimes needs to try new actions to discover better strategies.
This creates the well-known reinforcement learning challenge:
Exploration vs. Exploitation
Exploration means trying new actions.
Exploitation means using actions that are already known to work well.
The reward signal helps the agent evaluate these choices.
How Does Learning Improve Over Time?
At the beginning, an agent may make many poor decisions.
It may receive negative rewards frequently.
After repeated interactions, it starts identifying patterns.
For example:
Early Training
Random actions → Mixed rewards → Many mistakes
Later Training
Better actions → Higher rewards → Fewer mistakes
More Training
Learned strategy → More consistent rewards → Better decisions
The exact learning process depends on the reinforcement learning algorithm.
Reward Signal Example
Consider a simple maze.
The agent needs to reach the exit.
The reward system could be:
Move toward exit → +1
Move away from exit → -1
Hit wall → -2
Reach exit → +100
At first, the agent may move randomly.
It may hit walls and receive negative rewards.
After many attempts, it learns which movements are useful.
Eventually, it can discover a path that produces a high total reward.
The learning process can be represented as:
Observe Maze → Choose Action → Receive Reward → Update Learning → Choose Better Action
Why Reward Signals Can Be Difficult to Design
Reward design is not always simple.
A reward function may accidentally encourage unwanted behavior.
For example, if an agent is rewarded only for speed, it may learn