Teaching AI to reason about privacy norms

Privacy questions often turn on context. The same information can be appropriate to share in one setting and improper in another, depending on who sends it, who receives it and why. Fan and colleagues build GoldCoin around that insight, using a social theory of information flow to help language models reason about privacy-law scenarios.

From legal rules to information flows

GoldCoin uses contextual integrity, a framework for judging whether information moves appropriately within a particular social context. It represents a situation through features such as sender, recipient, subject, information type and transmission purpose. The researchers connect those features to privacy norms in the US Health Insurance Portability and Accountability Act (HIPAA), then create synthetic scenarios grounded in relevant legal provisions.

The framework is designed to help models identify the context around a possible privacy violation, retrieve an applicable norm and judge whether the information flow is permitted or forbidden. The team evaluates its approach on real court cases and compares fine-tuned models with baselines across judicial tasks (framework and experiments, proceedings pages 3324–3338).

Context improves legal judgments

The authors report that GoldCoin improves models’ recognition of privacy risks in real cases and outperforms baselines across the evaluated tasks. Their analysis also shows that removing contextual-feature extraction reduces legal-norm retrieval accuracy among the open models they tested. That result supports the central design idea: a legal assessment may depend on the flow of information and the roles of participants, not simply on whether a datum appears sensitive (abstract and results, proceedings pages 3321 and 3332–3338).

The work is an evaluation of model reasoning against selected cases and HIPAA provisions. It does not establish that a model can provide authoritative legal advice or replace a lawyer’s analysis.

Privacy governance and AI systems

For organizations that handle personal data, GoldCoin highlights why privacy controls need to account for purpose, relationships and rules governing disclosure. A system that checks only for familiar identifiers may miss context-dependent information flows. The benchmark offers a structured way to ask whether a model can connect a scenario to a relevant norm.

On applicability, Mistral-7B reached 97.66% accuracy and Llama-2-13B 99.53%. Against zero-shot baselines, GOLDCOIN raised compliance Macro-F1 by 17.87% for MPT-7B, 12.45% for Llama-2-7B, 17.96% for Mistral-7B and 8.12% for Llama-2-13B. Human review labeled compliance correctly in 99.03% of cases and classified synthetic-case applicability at 100%. These benchmark and HIPAA-specific scores do not show that every privacy rule or jurisdiction is handled correctly (Tables 1–3; §5).

Study limits

The authors focus on HIPAA and do not model interactions among multiple legal norms, although real cases can involve cross-references. Their work does not include few-shot prompting and does not use retrieval-augmented generation or vector search to retrieve legal norms (limitations, proceedings pages 3339–3340). Performance on these tests may not generalize to other privacy regimes, such as the General Data Protection Regulation, or to different legal datasets.

Possible implications for practice

The study suggests that future privacy tools could represent how information moves between people and organizations, alongside the data involved. That may provide a richer basis for evaluation than a list of sensitive fields alone. Broader evidence across laws and legal settings would clarify how far this approach can travel.

Bibliography & sources

1. Wei Fan, Haoran Li, Zheye Deng, Weiqi Wang, and Yangqiu Song. “GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory.” In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 3321–3343 (2024). https://doi.org/10.18653/v1/2024.emnlp-main.195.