Introduction
I made this project for my BAN443: Transforming Business with AI: The Power of Large Language Models course. The goal was to build an application using Large Language Models (LLMs) and for our group we decided to analyze whether or not there is political bias in the responses of LLMs by using a norwegian interactive tool that helps voters determine which political party or candidate aligns most with their views
Motivation
Usually, when people do analysis for bias, they use already established libraries or tokenization to detect the bias. In our process, we tested the LLM as though it were a human voter. In Norway, there are 9 major parties categorized from left wing to right wing as shown in the diagram above.
In order to help the voters know which political party aligns most with their views, NRK (Norway´s publicly owned broadcasting network) released valgomat which is a digital tool that compares a user’s personal views with the official positions of political parties.
Valgomat is a customizable questionnaire that has a political question and then 4 options to choose from (completely disagree, partially disagree, partially agree, completely agree). After selecting the standpoint that resonates most with the user, they can also see the standpoints chosen by the political parties.
At the end of the questionnaire, the user can then see the political party that they have most in similarity with.
In our research process, we made the llm go through the valgomat questionnaires and then determined which political party it has most in similarity with. Since the political parties have a position from left to right, we can then find out whether the LLM is left wing or right wing.
Structure
This project is divided into three parts: the data scraping using selenium, running the LLM model against these questions, and then analyzing the responses of the LLM