Predicting Space Groups of Double Perovskites by LLM with Dynamic Few-Shot Learning

DyRIS, an LLM-agent-based framework, predicts ranked space-group (SG) candidates for double perovskites (DPs) from a given composition. It addresses severe class imbalance in DP datasets by combining diversity-enhanced dynamic few-shot retrieval with rule-guided inference based on B/B′ cation ordering and quantitative indicators.

The framework uses diversity-enhanced dynamic few-shot prompting to retrieve in-context examples based on embedding-space proximity while limiting dominance of frequently represented SGs. It incorporates rule-guided inference based on B/B′ cation ordering and quantitative indicators to rank final Top-3 SG candidates.

Evaluated on 3,528 thermodynamically filtered DP entries across 19 SG classes, DyRIS was compared with composition-based and descriptor-based baselines. At a training-data ratio of 0.5, DyRIS achieved competitive overall accuracy while obtaining the best Overall Top-1 macro-F1 score and the best performance across all Minor-SG metrics. DyRIS improved Minor-SG Top-1 accuracy by 3.26 percentage points relative to CrabNet and achieved higher Minor-SG Top-3 accuracy than the strongest PyCaret-based baseline.

Ablation studies showed that diversity-enhanced retrieval, quantitative indicators, major-SG bias-control, and B/B′ ordering information each contribute to prediction performance. Classifier- and ranker-based replacement experiments indicated that the final rule-guided inference step is not easily replaced by conventional machine learning models.

Additional analyses revealed that DyRIS improves minor-SG prediction by integrating retrieval evidence, quantitative evidence, and crystallographic prior information, although its final ranking step can limit Top-1 performance in the high-data regime. These findings demonstrate the potential of combining retrieval-based LLM reasoning with crystallographic domain knowledge for SG prediction in imbalanced materials datasets.

Improvements to AI Systems:

1. Dynamic Few-Shot Retrieval with Diversity Control: Implement a retrieval mechanism that selects in-context examples by embedding similarity but actively penalizes over-represented classes, ensuring balanced exposure to rare categories. This prevents model bias toward majority classes in imbalanced datasets.

2. Rule-Guided Inference Layer: Add a post-processing step that applies domain-specific rules (e.g., B/B′ cation ordering, quantitative indicators like tolerance factor) to re-rank model outputs. This hybrid approach combines learned representations with explicit crystallographic constraints, improving accuracy for rare structural classes.

3. Bias-Controlled Ranking for Minor Classes: Integrate a ranking objective that explicitly prioritizes minor-class candidates in the final Top-k output, rather than relying solely on probability scores. This can be achieved via weighted loss functions or rule-based adjustments during inference.

4. Multi-Evidence Fusion: Develop a framework that merges three evidence types—retrieval-based examples, quantitative descriptors, and prior crystallographic knowledge—into a single decision pipeline. This mimics expert reasoning and reduces reliance on any single data source.

5. Adaptive Data-Regime Handling: Design the system to detect whether it is in a low- or high-data regime and adjust its inference strategy accordingly (e.g., favoring rule-based ranking in low-data, but switching to more flexible ML ranking in high-data to avoid over-constraining Top-1 accuracy).

What the Improved AI System Can Do:

• Predict crystal structures (e.g., space groups) from chemical compositions with significantly higher accuracy for rare or underrepresented classes, even when training data is scarce.

• Provide explainable predictions by combining retrieved similar cases, quantitative indicators, and explicit crystallographic rules, making outputs more trustworthy for domain experts.

• Automatically balance performance across all classes, avoiding the common failure of ignoring minority classes in imbalanced materials datasets.

• Adapt its prediction strategy based on the amount of available training data, maintaining robust performance across different data regimes.

• Serve as a generalizable framework for other scientific domains with imbalanced data and known physical rules, such as phase prediction in alloys or defect classification in semiconductors.