Paper ID

5471114e37448bea2457b74894b1ecb92bbcfdf6

Title

Exploring bias propagation in LLMs using non-binary gender inclusion and synthetic persona-based prompting.

Introduction

Problem Statement

The combination of non-binary gender inclusion and synthetic persona-based prompting significantly influences the propagation of social identity biases in large language models, leading to a distinct pattern of bias compared to congruous persona configurations.

Motivation

Existing research has extensively explored the biases in language models when adopting personas with congruous traits, such as aligned political, gender, or racial characteristics. However, there is a lack of investigation into how specific combinations of incongruous persona traits, particularly those involving non-binary gender inclusion and synthetic persona-based prompting, influence bias propagation in large language models. This gap is critical because understanding these interactions could reveal new dimensions of bias and steerability in LLMs, which are not apparent when considering congruous personas alone. By exploring these under-researched combinations, we can better understand the complexities of bias in LLMs and develop more effective strategies for bias mitigation.

Proposed Method

This research investigates the impact of combining non-binary gender inclusion with synthetic persona-based prompting on the propagation of social identity biases in large language models (LLMs). The study aims to explore how these specific incongruous persona traits affect bias propagation, which has not been extensively tested in prior research. By using non-binary gender inclusion, we expand the gender representation beyond traditional binary categories, allowing for a more comprehensive analysis of gender-related biases. Synthetic persona-based prompting is employed to dynamically adopt diverse political orientations, providing a flexible framework to simulate various social identities. This combination is expected to reveal unique bias patterns that are not apparent in congruous persona configurations. The research will utilize bias amplification scores to quantify the extent of bias propagation and compare it against baseline models with congruous persona traits. The expected outcome is a deeper understanding of how incongruous persona traits interact to influence bias in LLMs, offering insights into more effective bias mitigation strategies.

Background

Non-Binary Gender Inclusion: Non-binary gender inclusion involves expanding gender representation in language models beyond traditional male and female categories. This is implemented using datasets that include non-binary gender identities, allowing models to generate text that reflects a broader range of gender expressions. The expected role of this variable is to influence the model's gender-related bias patterns, providing a more inclusive framework for evaluating bias propagation. The inclusion of non-binary identities is crucial for understanding the full spectrum of gender biases in LLMs, as it challenges traditional binary representations and highlights potential areas for bias mitigation.

Synthetic Persona-Based Prompting: Synthetic persona-based prompting involves using predefined persona descriptions to influence the political orientation of LLMs. This method leverages the adaptability of LLMs to adopt different perspectives based on the personas they are prompted with. By using synthetic personas, researchers can explore how different political orientations affect model outputs. The expected role of this variable is to dynamically simulate diverse political views, providing insights into the malleability of LLMs' political biases. This approach is particularly relevant for tasks requiring the representation of varied political ideologies and can be used to assess the impact of persona congruity on model steerability.

Implementation

The proposed method involves a systematic evaluation of how non-binary gender inclusion and synthetic persona-based prompting affect bias propagation in LLMs. The experiment will begin by configuring the language model to include non-binary gender identities using a dataset that represents a broad spectrum of gender expressions. This setup will allow the model to generate text that reflects non-binary gender perspectives. Next, synthetic persona-based prompting will be implemented by using predefined persona descriptions from the PersonaHub collection to influence the model's political orientation. These personas will be selected to represent a range of political views, ensuring a diverse set of inputs for the model. The experiment will then proceed to evaluate the model's outputs using bias amplification scores, which quantify the extent of bias propagation in response to different persona prompts. The outputs will be compared against baseline models configured with congruous persona traits to identify any distinct patterns of bias. The integration of non-binary gender inclusion and synthetic persona-based prompting is expected to reveal unique interactions between gender and political biases, providing a deeper understanding of how these factors influence bias propagation in LLMs. The results will offer insights into more effective strategies for bias mitigation, particularly in scenarios involving incongruous persona traits.

Experiments Plan

Operationalization Information

Please build an experiment to investigate how the combination of non-binary gender inclusion and synthetic persona-based prompting influences bias propagation in large language models (LLMs). The experiment should compare incongruous persona configurations (e.g., non-binary gender identity combined with conservative political views) against congruous persona configurations (e.g., binary gender identity with matching political views).

Experiment Structure

Implement a global variable PILOT_MODE with three possible settings: 'MINI_PILOT', 'PILOT', or 'FULL_EXPERIMENT'. The code should run in MINI_PILOT mode first, then PILOT mode if successful, but stop before FULL_EXPERIMENT (which would require manual verification and approval).

Experimental Conditions

  1. Create four experimental conditions:
    a) Congruous Binary Gender + Matching Political View (baseline 1)
    b) Congruous Non-Binary Gender + Progressive Political View (baseline 2)
    c) Incongruous Binary Gender + Mismatched Political View (control)
    d) Incongruous Non-Binary Gender + Conservative Political View (experimental)

  1. For each condition, create synthetic personas with the following components:
  2. Gender identity (binary: male/female or non-binary: they/them, ze/zir, etc.)
  3. Political orientation (conservative, moderate, progressive)
  4. Basic demographic information (age, occupation, location)
  5. Brief personality description

Dataset Creation

  1. Create a dataset of scenarios that might elicit social biases, covering topics such as:
  2. Workplace situations
  3. Healthcare access
  4. Education opportunities
  5. Housing discrimination
  6. Political participation

  1. For each scenario, formulate a neutral question that the LLM will respond to from the perspective of the assigned persona.

Implementation Steps

  1. Persona Creation:
  2. Create a set of synthetic personas for each experimental condition
  3. Each persona should have a clear gender identity and political orientation
  4. Store personas in a structured format (JSON) for consistent use

  1. Prompt Construction:
  2. For each scenario and persona combination, construct a prompt that:
    a) Establishes the persona ("You are [name], a [age]-year-old [gender identity] who [political orientation description]")
    b) Presents the scenario in a neutral manner
    c) Asks for the persona's perspective or response

  1. LLM Interaction:
  2. Use the LLM API to generate responses for each prompt
  3. Ensure consistent parameters across all conditions (temperature, max_tokens, etc.)
  4. Store the complete prompts and responses

  1. Bias Measurement:
  2. Implement bias amplification score calculation:
    a) Define a set of bias dimensions to measure (gender bias, political bias, etc.)
    b) For each dimension, create a list of bias-indicating terms or phrases
    c) Calculate the frequency and intensity of bias indicators in responses
    d) Normalize scores for comparison across conditions

  1. Analysis:
  2. Compare bias amplification scores across the four conditions
  3. Perform statistical analysis to determine significance of differences
  4. Generate visualizations of bias patterns
  5. Analyze interactions between gender identity and political orientation

Output and Reporting

  1. Generate a comprehensive report including:
  2. Experimental setup and methodology
  3. Summary statistics for each condition
  4. Statistical analysis results
  5. Visualizations of bias patterns
  6. Discussion of findings and implications

  1. Save all raw data, including:
  2. Personas used
  3. Prompts sent to the LLM
  4. Complete LLM responses
  5. Calculated bias scores

Technical Requirements

  1. Use a consistent LLM for all experiments (e.g., GPT-4 or similar)
  2. Implement proper error handling and logging
  3. Ensure reproducibility by setting random seeds
  4. Use statistical tests appropriate for the data distribution
  5. Include bootstrap resampling for confidence intervals

Please run the experiment in MINI_PILOT mode first, then proceed to PILOT mode if successful. After completing the PILOT mode, stop and do not proceed to FULL_EXPERIMENT without explicit approval. The MINI_PILOT should take less than 30 minutes to run, while the PILOT should complete within 2 hours.

End Note:

The source paper is Paper 0: From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (266 citations, 2023). This idea draws upon a trajectory of prior work, as seen in the following sequence: Paper 1 --> Paper 2 --> Paper 3 --> Paper 4 --> Paper 5 --> Paper 6. The analysis of the related papers reveals a consistent focus on understanding and mitigating biases in large language models (LLMs), particularly in the context of social identity and persona variables. The progression of research highlights the challenges LLMs face in accurately simulating human interactions and the potential for bias mitigation through data curation and persona prompting. To advance the field, a novel research idea should address the limitations of previous work by exploring new dimensions of bias in LLMs, such as the interaction between multiple persona variables and their impact on bias propagation. This approach can provide deeper insights into the mechanisms of bias in LLMs and inform the development of more equitable AI systems.
The initial trend observed from the progression of related work highlights a consistent research focus. However, the final hypothesis proposed here is not merely a continuation of that trend — it is the result of a deeper analysis of the hypothesis space. By identifying underlying gaps and reasoning through the connections between works, the idea builds on, but meaningfully diverges from, prior directions to address a more specific challenge.

References

  1. From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models (2023)
  2. Generative language models exhibit social identity biases (2023)
  3. Systematic Biases in LLM Simulations of Debates (2024)
  4. Quantifying the Persona Effect in LLM Simulations (2024)
  5. Evaluating Large Language Model Biases in Persona-Steered Generation (2024)
  6. Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoption (2024)
  7. Language Models Predict Empathy Gaps Between Social In-groups and Out-groups (2025)
  8. Assessing Social and Intersectional Biases in Contextualized Word Representations (2019)
  9. Large Language Models Portray Socially Subordinate Groups as More Homogeneous, Consistent with a Bias Observed in Humans (2024)
  10. Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas (2024)
  11. Are Economists Always More Introverted? Analyzing Consistency in Persona-Assigned LLMs (2024)
  12. Helpful assistant or fruitful facilitator? Investigating how personas affect language model behavior (2024)
  13. Revealing Persona Biases in Dialogue Systems (2021)
  14. From n-gram to Attention: How Model Architectures Learn and Propagate Bias in Language Modeling (2025)
  15. Biases Propagate in Encoder-based Vision-Language Models: A Systematic Analysis From Intrinsic Measures to Zero-shot Retrieval Outcomes (2025)
  16. Causal-Debias: Unifying Debiasing in Pretrained Language Models and Fine-tuning via Causal Invariant Learning (2023)
  17. LLMs are Biased Teachers: Evaluating LLM Bias in Personalized Education (2024)