Using an LLM to translate, not to judge

A meme can look harmless if its image and caption are read independently. GEMF starts from the opposite assumption: the useful signal often appears in the relationship between both, together with tone, implication and cultural context.

Gemini does not output the final class. It converts visual and pragmatic meaning into a structured textual representation. Supervised encoders then combine it with OCR, EEG and Ekman emotion features. This keeps the generative model in an evidence-producing role rather than treating it as an unaccountable judge.

The project learns from soft labels because annotator disagreement can describe genuine ambiguity. That choice also makes calibration and thresholding first-class modelling decisions. The repository documents where this worked and where hard-label thresholds did not transfer reliably.