This episode revolves around the Prince's desire to get some elocution lessons from two rather annoying actors, whom Blackadder merrily tortures by saying 'Macbeth' as frequently as possible. Eventually, he is able to dispose of the actors entirely because Baldrick mistakes a play they are rehearsing ('The Murder of Prince Romero and his Enormously Bosomed Wife', if I recall correctly) for a genuine plot to kill the Prince. The Prince has no trouble believing this because he doesn't fully understand the concept of acting - and this is where Classics comes in. The Prince is convinced that, during the production of Julius Caesar he saw not long ago, Brutus actually killed Caesar, even though, as Blackadder points out, Caesar got up for the curtain call afterwards. Julius Caesar seems to have made an impression on the Prince - when he asks for lessons, he decides he must dress up in a toga, because that is what actors wear (and a silly moustache).
The reason for the toga is, of course, the Shakespeare play he has recently seen, but it is interesting that this, and Julius Caesar, are chosen to represent theatre in general. The toga itself is useful because it is so very different from Georgian period costume. Medieval or Elizabethan costume would be different as well, but to the modern eye, the difference between a medieval tunic, an Elizabethan doublet and hose and, er, whatever it is the Prince wears, although clear, is not so great. A toga, however, is clearly a very different and completely ridiculous thing for a Georgian to wear - the visual impact is greater, and therefore the joke works better.
Another factor is that the Prince needs to give a speech, which is partly why this is a useful play to use. Shakespeare's 'Friends, Romans, Countrymen' is, of course, very well known, but importantly, it is also written and performed as a speech, delivered to a crowd. The other famous speeches the actors mention ('to be or not to be' from Hamlet, for example) tend to be, in fact, monologues, delivered by a character to the audience in an ostensibly private setting . The speeches in Julius Caesar, however, are really speeches; the importance of public speaking in Classical literature is carried through into Shakespeare's play, and it is that which makes this play useful for Blackadder.
The other references to Classics are briefer. Pitt the Younger, as Prime Minister, calls on a Member of Parliament to test him on his Latin vocab in 'Dish and Dishonesty'. In addition to being historically accurate, since all schools for someone of Pitt's rank taught Latin at that time, this enhances the episode's depiction of Pitt's school as embodying all the worst and most cliched aspects of British public boys' schools - other jokes include references to a hot crumpet burning his cheeks with shame, and he doesn't mean the cheeks on his face. Finally, in 'Nob and Nobility', Blackadder refers to Baldrick's trousers as Pandora's Box - because if they are opened (or removed) all the evils in the world will come out. He doesn't make reference to the idea that only Hope was left in the box - where this detail would logically lead is probably best left to everyone's imagination!
(Blackadder the Third is, by the way, the series that plays with history the most - the first series claims to be an alternate history, and can do what it likes, the second messes around a bit but stays broadly within the reign of Elizabeth I and the fourth is really pretty accurate. In the third, on the other hand, we have the French Revolution (1790s), the Regency (1811-1820), Pitt the Younger (1783), Samuel Johnson's dictionary (1755) and the Prince's marriage (1795) all happening at once!).