traffic will be dropped if flows between different zones, unless someone explicitly allows the traffic to pass/inspect

Create a policy map for icmp reply
policy-map type inspect pol-icmp-reply
class type inspect cls-icmp-reply
inspect
exit
class class-default
exit

For router to send icmp echo to the outside network and get reply:
create a zone pair to pair up router and outside zone

zone-pair security zp-self-out source self destination out-zone
service-policy type inspect pol-icmp-reply
exit

For outside network to be able to ping my router I need to pair up outside network with my router (self-zone)
zone-pair security zp-out-self source out-zone destination self
service-policy type inspect pol-icmp-reply
inspect
exit

Tested the policy-maps and they work.