netstat -g -- made sure I was subscribing
tcpdump -enni eth1 -- made sure I was receiving the data
From some of my previous run-ins with similar issues, I decided to check rp_filter and sure enough it was set to 1.
cat /proc/sys/net/ipv4/eth1/rp_filter returned 1
Although sysctl -p and /etc/sysctl.conf showed that it should be set to 0. This might be a bug which I will have to look into at a later time.
RPF check was failing for the source of the multicast traffic coming in on eth1.
What is RPF check? RPF check will make sure that the interface where you are receiving the traffic is the outgoing interface for the source of the traffic. One of the main reasons for RPF check is to prevent address spoofing and DDOS attacks.
So for example if I was receiving packets from 220.127.116.11 on eth1 then my outgoing interface to reach 18.104.22.168 should be set to eth1, however in this scenario since I didn't have a specific route for 22.214.171.124 pointed out of eth1 I was taking the default for 126.96.36.199 which was pointed to eth0.
echo 0 > /cat/proc/sys/net/ipv4/eth1/rp_filter fixed the issue right away. Note that this will not survive a reboot and the correct way to set this would be in /etc/sysctl.conf to make it persistent.
Many more articles to come so ....
Please subscribe/comment/+1 if you like my posts as it keeps me motivated to write more and spread the knowledge.