There seems to be a fair amount of anti-soy information floating around, but none of it really has any basis in fact, IMO. If you check around here you'll find many discussions on this issue. Many, many people have used tofu as a staple food for many, many years and not had any kinds of problems, if you're allergic to soy then I suppose it would be like any other food allergen. I have used tofu now for about 15 years, I probably use a pound, maybe, a week, if that, and a little bit of soy yogurt here and there and I use some soy milk in making sauces and such, but I certainly haven't noticed any adverse effects.
I just think people should back their claims up with science, not just repeat everything they hear. . .