They may have made more money collectively, meaning every sequel's earnings in a franchise put together. But I doubt Bully made less then say a game like Manhunt or Max Payne that deals with more mature themes simply because a game like Bully appeals to more age groups and therefore more people buying it, which in turn leads to more incentive for them to to make another. Not to mention that pretty much anybody who has an interest in a GTA game, might have the same interest for a game like Bully.
Regardless, when you compare it to other games in Rockstar's roster, Bully is one of the most likely to see another entry. Games like The Warriors, Table tennis, and Beaterator were pretty much one hit wonders that stand alone, not to mention a sequel for one of these wouldn't make a whole lot of sense. But games like Manhunt, that already have sequels have much less of a chance of seeing another one compared to Bully. Partly because Bully follows, almost to a T, the same formula as GTA, Rockstar's biggest and most selling franchise, and also because if the've already played around with something to such an extent there going to feel the need to try something new. Or, at least give an older franchise a much needed breath of fresh air.
Bottom line is, try to keep a little optimism. Don't just dismiss it over a gut feeling.