Why the Ministry of Magic Was Angry at Harry for Using a Patronus in Front of a Muggle
In Harry Potter and the Order of the Phoenix, the Ministry of Magic was particularly concerned about Harry using a Patronus in front of Dudley Dursley because it could expose the magical world to a Muggle. The Patronus Charm is a powerful form of magic, and its use in front of a non-magical person could lead to serious consequences, including potential breaches of the International Statute of Wizarding Secrecy.
Regulations and Consequences
The Ministry's strict regulations about revealing magic to Muggles still applied, even though Dudley was aware of some aspects of Harry's magical life. According to the Book of Hidden Wonders, if news of Harry's Patronus spread, it could create panic or unwanted attention, leading to further scrutiny of the magical community. Additionally, Harry's actions could be seen as reckless, demonstrating a lack of awareness of the potential implications of his magic being witnessed by a Muggle. This was especially sensitive given the Ministry's overall tension with Harry and the Order at that time as they were trying to maintain control over the narrative surrounding Voldemort's return.
The Minister's Agenda
Arthur Weasley, the Minister of Magic, Fudge, wanted Harry to be silenced and discredited. He ignored the fact that Dudley was already aware of Harry's magical life. Harry was put under trial because he was banned from using magic outside school according to the Decree for the Reasonable Restriction of Underage Sorcery. The decree states that underaged witches and wizards aren't allowed to use magic in front of any Muggle, even family members or in a Muggle-inhabited region, unless they are in a life-threatening situation. This was the main charge the Ministry was trying to nail Harry with.
Legal and Political Consequences
Legally speaking, there are two reasons why Harry casting the Patronus Charm in the Fifth Book was a big no-no. The first was that he did so in the presence of a Muggle, thereby breaking the International Statute of Secrecy. However, as you mentioned, Dudley was already aware of the existence of the magical world, so no harm no foul. The second reason was that Harry was underaged. Per the decree, underaged witches and wizards aren't allowed to use magic outside of school, unless they are in a life-threatening situation, which Harry was not.
However, in all actuality, the Ministry was just looking for an excuse to come after Harry. Fudge didn't believe or at least didn't want to believe his and Dumbledore's claims that Voldemort had returned to the land of the living and wanted them discredited. He thought Harry was a mentally unsound attention-seeking liar, and Dumbledore was just trying to stir up shit so he could sneak his way into stealing Fudge's job. It's worth noting that Dumbledore had been offered the position of Minister for Magic multiple times and refused every time, not because he desired office but because he didn't wish to engage in the political drama of a public role.
In conclusion, Harry's actions, while well-intentioned, were caught up in a web of political, magical, and legal complications that ultimately served to highlight the tensions between Harry and the Ministry of Magic.