Question

why friction doesn't roll a ball backward if no force is pushing or pulling it

Answer 1

this is because friction only opposes a body which is in motion as in this case the ball initially is at rest so there is just static friction (friction on body which are at rest) acting on it
hope this helps

Answer 2

because from definition of friction we know that friction is a force which only acts when an object is moving, then a backward force is applied which applies force on object in backward direction.
case 1.
& if no force is acting then the object must be in rest therefore no force is applied.
case 2.
& if no force is acting then the object must have uniform velocity