Panels can be a lot of fun if the other writers there come prepared to discuss the topic (instead of complain that they don't know why they were put on the panel and then decide to go off on tangents about something else entirely). You don't have to agree with the other panelists, but it helps if you are all talking about the same thing! That way, you can each build on what the others have said before you, and the discussion will have some substance. If they have different, or even opposite viewpoints on the question at hand, it can make for lively and interesting discussions. Face-to-face, even people who disagree tend to be much, much more civil than they may be on the internet, so these disagreements rarely result in any rancor.
If people do go off on tangents that have nothing to do with the subject you were allegedly there to discuss, don't be shy about bringing the discussion back on track when it comes time for you to speak again. And if they are not including you in the conversation, don't be afraid to jump in as soon as someone pauses for breath. If you are engaging and want to talk about the thing the audience came there to hear about, generally they will like you a lot more for asserting yourself in that way, rather than allowing yourself to be cowed by the people who talk the loudest and the longest. Also, notice who isn't talking, who is being ignored or doesn't have the courage to speak up, and try to bring them into the conversation by asking them what they think. Even if it doesn't elicit much of a reaction (but it may) it will give everyone a break from the "monologist" on the panel.