A new framework uses the Toulmin model of argumentation to decompose image-based medical diagnoses into claims, grounds, and warrants. A MedGemma agent provides the medical knowledge necessary to link extracted biomarkers to final diagnostic claims. This structured approach moves beyond standard XAI to provide a more interpretable audit trail for clinicians.