AI and Consumer Rights in India Working Paper

This working paper examines whether India's Consumer Protection Act, 2019, adequately addresses harm caused by defective AI products and services, and whether it proportionately allocates liability across the AI value chain.

The paper finds that the Act's broad definitions of product liability, harm, and deficiency appear technology-agnostic and potentially applicable to AI-related incidents—including personal injury, psychological harm, biased outputs, and loss of control. The Act defines harm broadly to include damage to any property, other than the product itself; personal injury, illness, or death; mental agony or emotional distress attendant to personal injury or illness or damage to property. Deficiency includes fault or imperfection in the service, and also includes negligence, which causes loss or injury to the consumer.

However, significant gaps remain. Proving causation between AI defects and consumer harm presents a technical challenge, as AI failures often stem from design choices rather than discrete defects. The paper notes that testing for 'defectiveness' and a 'causal' link in the way that consumer protection regulators may be able to do for more traditional products or services will be difficult in this context. It cites a US ruling where Meta and Google were found liable for addictive social media platforms, and notes that AI hallucinations are a feature of AI design and not a bug, raising questions about how authorities might scrutinize AI design rather than effects.

Additionally, the Act's framework assumes distinct roles for manufacturers, sellers, and service providers, yet the AI value chain involves overlapping responsibilities among data providers, model developers, deployers, and users that do not neatly map to these categories. The paper explains that "data providers and model developers could both be considered 'manufacturers' since they're involved in creating the AI product. Entities fine-tuning such models for different use cases may be covered as 'sellers,' and if they are themselves providing the services, then also under 'service providers.' However, the underlying assumption in such a framework is that the role and value added by different entities in a product/service can be defined and distinguished. This is not the case with AI, where responsibility is fragmented and overlapping."

The paper concludes that current liability frameworks lack proportionate mechanisms to effectively address complex, multi-stakeholder AI harms. While the Act may cover AI entities, enforcement requires clarification on sector-specific overlaps. It recommends that the AI Governance and Economic Group, Technology and Policy Expert Committee, and Central Consumer Protection Authority should clarify overlaps with sector-specific laws, and that the AI Safety Institute should build technical capacity of consumer forums while the CCPA spreads awareness around AI and consumer protection. The paper also notes that complementary legal regimes may be needed depending on context, personalization level, and personal information collected.

Improvements to AI systems:

1. Causality-Aware Explainability Module: Implement a system that explicitly tracks and documents design decisions, training data provenance, and model behavior under varied inputs. This enables post-hoc causal tracing from consumer harm back to specific design choices or data biases, addressing the paper’s noted difficulty in proving causation. The improved system can generate auditable causal chains (e.g., harm occurred because training data underrepresented demographic X, leading to biased output Y) that regulators and courts can use to assess defectiveness.

2. Role-Aware Liability Mapping: Build a dynamic attribution framework that assigns responsibility across the AI value chain (data providers, model developers, fine-tuners, deployers, users) based on actual contribution to a harmful outcome, rather than assuming static categories like manufacturer or seller. The improved system can output a liability score for each stakeholder, weighted by factors such as control over training data, fine-tuning modifications, deployment context, and user interaction. This helps proportionate allocation under the Consumer Protection Act.

3. Design-Versus-Effect Distinction Engine: Develop a diagnostic tool that separates inherent AI features (e.g., hallucinations as a probabilistic design trade-off) from defective outputs (e.g., hallucinations in high-stakes medical advice). The system can classify each incident into a spectrum—ranging from expected model behavior to actionable deficiency—based on industry standards, user expectations, and risk context. This prevents over-penalizing AI for intrinsic limitations while enabling enforcement against genuine flaws.

4. Sector-Specific Compliance Advisor: Integrate a module that cross-references the Act’s provisions with sectoral regulations (e.g., healthcare, finance, transport) to identify overlaps, conflicts, or gaps. The improved system can proactively flag when an AI product’s deployment triggers multiple legal regimes, and recommend which authority (e.g., CCPA vs. sectoral regulator) has primary jurisdiction, reducing ambiguity for developers and consumers.

5. Consumer Harm Simulation and Pre-Deployment Testing: Add a simulation layer that models potential harms (psychological, property, personal injury) across diverse user demographics and usage scenarios before launch. The system can generate harm heatmaps showing where liability is most likely to arise, and suggest design mitigations (e.g., disclaimers, confidence thresholds, human-in-the-loop checks) to lower risk. This shifts enforcement from reactive to preventive, aligning with the Act’s intent.

6. Transparent Personalization and Data-Use Logging: For AI services that personalize outputs or collect personal information, implement a logging system that records what data was used, how it influenced the output, and whether consent was obtained. The improved system can produce a consumer-facing explanation of data-driven decisions, aiding in claims of mental agony or emotional distress by demonstrating whether harm resulted from misuse of personal data versus inherent AI behavior.

What the improved AI system can do:

• Provide regulators with clear, evidence-based causal links between AI design/operation and consumer harm, enabling fair enforcement under the Act.

• Allocate liability proportionately across multiple stakeholders, avoiding the current fragmented and overlapping confusion.

• Distinguish between acceptable AI limitations and actionable defects, reducing frivolous claims while protecting consumers.

• Navigate complex multi-sector legal landscapes, ensuring compliance without overstepping jurisdictional boundaries.

• Preemptively identify high-risk use cases and mitigate them before harm occurs, lowering litigation and improving consumer trust.

• Offer transparent, auditable records of personalization and data use, supporting claims of emotional distress or privacy-related harm.

As AI systems proliferate in consumer facing applications, questions about liability for AI related harms remain unresolved. This working paper examines whether India's Consumer Protection Act, 2019, adequately addresses harm caused by defective AI products and services, and whether it proportionately allocates liability across the AI value chain. The Act's broad definitions of product liability, harm, and deficiency appear technology agnostic and potentially applicable to AI related incidents including personal injury, psychological harm, biased outputs, and loss of control. However, significant gaps remain. Proving causation between AI defects and consumer harm presents a technical challenge, as AI failures often stem from design choices rather than discrete defects. Additionally, the Act's framework assumes distinct roles for manufacturers, sellers, and service providers, yet the AI value chain involves overlapping responsibilities among data providers, model developers, deployers, and users that do not neatly map to these categories. Current liability frameworks lack proportionate mechanisms to effectively address complex, multistakeholder AI harms. While the Act may cover AI entities, enforcement requires clarification on sector specific overlaps.