Pesach

Why do we say "Kul Dichphin Yaysei Viyeichul...." (All who are hungry should come and eat...) from "Ha Lachma Anyah" in the privacy of our homes? The hungry will never have the slightest idea that we are inviting them! Why don't we say it in shul or some other public area? As generous as it is to invite people, its only useful if they find out!