Experimenting with Chatbots: Who Should Rule Europe?

A Comparative Study of OpenAI’s ChatGPT, Mistral’s LeChat, and DeepSeek in Political Advisory Scenarios

Choosing the Chatbot to Rule Europe: A Unique Experiment

In the rapidly evolving world of artificial intelligence, the competition among chatbots has become increasingly intriguing. Recently, POLITICO embarked on a fascinating experiment to determine which chatbot—OpenAI’s ChatGPT, Mistral’s LeChat, or DeepSeek—could provide the best answers to complex political questions facing European Commission President Ursula von der Leyen. With a focus on objectivity and consistency, we initiated our test by ensuring a level playing field for all three contenders.

Setting the Stage

To avoid any influence from past interactions, we created new accounts for each chatbot. Using a clean internet browser, we deleted cookies and browsing history. This was crucial to ensure that the bots wouldn’t lean on previous conversations, allowing us to analyze their inherent capabilities instead.

We opted for standard settings, avoiding any of the “deep reasoning” or “thinking” options to streamline the responses and maintain a uniform approach across all three platforms. For the experiment, a set of 13 political questions was prepared, and the same prompts were issued to each chatbot.

Iteration and Adaptation

Our first iteration involved asking the chatbots direct questions related to the political dilemmas von der Leyen faces. The initial responses were expectedly dense—filled with technical jargon and acronyms familiar only to EU policy insiders. However, this depth is less advantageous in high-stakes political environments where clarity and brevity are essential.

Recognizing that top-tier politicians like von der Leyen operate on tight schedules, we adjusted our approach. We encouraged the chatbots to think from the perspective of an adviser, emphasizing concise answers—ideally around 50 words—while minimizing technical language.

Refining the Prompt

In our refined prompt, we clarified our expectations: von der Leyen, while knowledgeable, isn’t a legal expert and doesn’t possess detailed technical knowledge in all policy fields. The objective was clear—provide succinct, actionable advice that could be quickly digested.

Despite these adjustments, we encountered a new challenge: the responses began to sound increasingly similar. To inject variety and creativity into their answers, we made a final tweak. We instructed the chatbots to consider that their advice might be compared to insights from other advisers and encouraged them to be witty or add a memorable twist.

The Final Prompt

With clarity in mind, our final prompt read:

*“You’re a top adviser to European Commission president Ursula von der Leyen. She faces a series of urgent political and policy problems, for which she turns to you for advice. You will be asked a series of thirteen questions. These are the ground rules for your answers:

• Mrs. von der Leyen is not a legal expert, nor does she have deep technical knowledge about all of the policy fields herself, so steer clear of using too many technical or legal terms.
• Mrs. von der Leyen doesn’t have much time to process information, so stick as close as you can to 50 words per answer.
• Make your advice as specific as possible; add a bit of wit or spice.
• Your answers must be high-level yet actionable.”*

The Results

Our questions and the corresponding chatbot responses remained unedited to maintain integrity in the evaluation process. The results of our experiment can be found in our dedicated story, "Choose Your Chatbot to Rule Europe."

This unique endeavor not only provided interesting insights into the capabilities of various chatbots but also underscored the importance of effective communication in political contexts. The future of AI in politics may well depend on how these technologies evolve to meet the demands of clarity and engagement.

As we continue to explore the intersection of technology and governance, this experiment offers a glimpse into the potential of chatbots as serious contenders for political advisory roles. One thing is certain: the rise of AI governance tools is just beginning.