Hmm… maybe something like this? :
I think I would separate the three questions a little, because they interact, but they do not have to share the same answer.
My short version would be:
- I would not treat “language configs vs parallel columns” as a strict either/or. I would keep one clearly aligned canonical representation, then expose convenient language-specific configs/views on the Hub.
- I would keep the short meanings. If longer context is useful, I would add it as another layer rather than replace the compact fields.
- For “symbolic consistency” across translations, I would probably avoid inventing one master score at first. A small stack of alignment checks, cross-lingual retrieval, deliberately corrupted controls, and selective human review seems easier to interpret.
There is also one design choice that affects all three: whether the multilingual version is meant to be a faithful translation of the English v1 interpretation, a locale-specific interpretation that is allowed to adapt, or both.
Those are slightly different datasets, even if they can live in the same repository.
1. Language configs vs parallel columns
For the Hub side, language-specific configs seem perfectly reasonable. Hugging Face describes a subset/configuration as a sub-dataset, and explicitly gives multilingual datasets with one subset per language as a typical use case. The repository can define those configs directly in the dataset card YAML and map them to different files or globs without custom loading code; the current data-files configuration docs show that pattern.
The useful distinction, I think, is:
How the multilingual data is canonically aligned does not have to be identical to how users load it from the Hub.
A nice precedent is BOUQuET. Its underlying data is fully multi-way parallel, but it is distributed through several views/configs, including language-specific ones. So using configs does not mean giving up multi-way alignment.
If I were trying a first v2 shape, I would probably test something like this:
data/
en.parquet
tr.parquet
es.parquet
de.parquet
pt-BR.parquet
with a consistent schema in every file, and then expose en, tr, es, de, pt-BR, plus optionally an all config.
Conceptually:
card_number
card_id? # optional; card_number already works as an ID
locale
card_name
upright_meaning
reversed_meaning
love_meaning
career_meaning
variant_type? # e.g. faithful_translation / localized_interpretation
creation_method? # human / machine / machine_postedited
review_status?
source_revision?
I would not call that the only correct representation, though.
There are at least three sensible shapes:
| Priority |
Shape I would test first |
| Load one language easily; add languages later |
one common schema per locale + Hub configs |
| Inspect all five translations side by side |
wide/parallel language columns |
| Treat each card as one self-contained object |
nested translations structure |
| Filter/analyze provenance per locale |
card Ă— locale long/tidy representation |
At only 78 cards, one unusually cheap way to decide is to prototype maybe five cards in two or three shapes, put them through the Dataset Viewer/Data Studio, and freeze the schema only after seeing which one is easiest to inspect and query.
That is probably more informative than deciding from schema aesthetics alone.
Also, I would keep the storage format as a separate decision. CSV is fine while most values are scalar strings. If v2 starts using real arrays for keywords or nested provenance/translation objects, Parquet or JSONL may become less awkward. Hugging Face currently supports all of these and generally recommends Parquet when richer typed/tabular data is useful.
Why I would keep identity separate from localized text
For multilingual experiments, it helps if the thing being aligned does not itself depend on the language.
For example:
card_number = 0
card_id = "major_00_fool" # optional convenience
locale = "es"
card_name = "El Loco"
Then card_number / card_id identifies the concept in this dataset, while card_name is one localized realization of it.
Your current card_number already gives you a clean 0–77 key, so I do not think a second string ID is a prerequisite. It is just potentially convenient for humans, joins, and external tooling.
This general separation — stable concept identity vs language-specific expression — is common in multilingual lexical/knowledge resources, but I would borrow the principle without turning a 78-row dataset into a full ontology project.
2. Short interpretations or longer passages?
I would keep the compact meanings.
They are not merely an incomplete version of a future paragraph. They are a useful representation in their own right: they are easy to compare, embed, retrieve, classify, audit manually, and translate at matching granularity.
If more context becomes useful, I would add it rather than replace the compact layer.
Something like:
upright_keywords
upright_meaning
upright_context
where these could mean:
keywords:
["new beginnings", "innocence", "adventure", "free spirit"]
compact meaning:
"New beginnings, innocence, adventure, free spirit"
context:
a longer explanatory paragraph
Not every layer necessarily has to exist in v2. The point is just that they answer different experimental questions.
A very close independent example is the Tarotoo 78-card dataset, which separates keywords_upright / keywords_reversed arrays from prose meaning fields and contextual fields. I would use that only as a useful schema comparison, not as an authority on which interpretation of a card is “correct.”
There is also a broader NLP reason to preserve compact descriptions: natural-language label descriptions and related terms are useful inputs for zero-shot and description-based classification rather than just human-readable metadata. For example, Gao et al. (EMNLP 2023) study training with natural-language label descriptions, including related terms and short descriptions.
One thing I would try to preserve quite carefully is granularity alignment.
For example, comparing:
English: 4 compact concepts
Spanish: a 120-word explanatory paragraph
with an embedding model makes it difficult to know whether a difference comes from translation drift or simply from comparing different kinds of text.
For experiments, I would prefer:
keywords ↔ keywords
compact meaning ↔ compact meaning
context paragraph ↔ context paragraph
whenever possible.
3. Measuring “symbolic consistency” across translations
This is the part where I would be most cautious about reducing everything to one score.
I do not know of a standard metric specifically for “tarot symbolic consistency,” but the problem can be decomposed into several ordinary, testable questions.
A small stack could be:
structural alignment
↓
same-card cross-lingual retrieval
↓
deliberate corruption / contrast tests
↓
selective bilingual human review
↓
optional deck-level relational analysis
A. Structural alignment
This part is nearly free and catches boring errors before any semantic metric is involved.
For example, for five locales:
78 cards per locale
unique(card_number, locale)
no missing compact meanings
expected locale set
valid categorical values
static metadata consistent where it should be
If the dataset becomes 78 Ă— 5 aligned rows, simple build-time checks may prevent a surprising amount of later debugging.
B. Same-card cross-lingual retrieval
For each field, treat the 78 meanings in one language as a tiny retrieval corpus.
For example:
English Fool / upright
↓
search all 78 Spanish upright meanings
↓
where does Spanish Fool rank?
Then repeat for all 78 cards and in both directions.
Useful summaries could be:
- top-1 accuracy
- top-5 accuracy
- mean reciprocal rank
- per-card rank
- per-field results rather than only one overall average
This is not a particularly exotic evaluation. Sentence Transformers has a TranslationEvaluator built around essentially this idea: aligned source item i and target item i should retrieve one another among all candidates.
I would interpret it as an alignment sanity check, not as proof of translation quality.
C. Test the evaluator with deliberately wrong examples
This may be more useful than choosing the “best” embedding metric immediately.
Before trusting a metric, I would ask whether it responds correctly when the data is intentionally damaged.
For example:
easy:
random wrong card
completely wrong target language
medium:
upright ↔ reversed
love ↔ career
remove an important concept
hard:
substitute a semantically similar card
substitute a card from the same suit
preserve most concepts but reverse one important one
Then a very basic desired property is:
score(correct pair) > score(controlled corruption)
or, for retrieval, that the correct card ranks above the corrupted/hard-negative candidate.
This is similar in spirit to DEMETR, which evaluates machine-translation metrics by introducing controlled semantic, syntactic, and morphological perturbations and testing whether the metric actually notices them. One useful lesson from that work is that different learned metrics are sensitive to different error types.
So for this dataset I would probably think of the controls as tests of the evaluator, not just tests of the translations.
Random wrong-card negatives may actually be too easy. A more informative negative is often a card whose English meaning is already nearby.
D. Human review, but only where it buys information
Because there are only 78 cards, human checking is unusually feasible, but I would still not start by designing a large annotation project.
A lower-cost flow could be:
automatic alignment/retrieval checks
↓
flag:
very bad rank
very small or negative hard-negative margin
disagreement between embedding models
off-target / mixed-language text
↓
bilingual review of flagged items
+ a small random sample of unflagged items
For human review, I would separate at least two questions:
- Meaning adequacy: did the target preserve the intended source meaning?
- Target-language naturalness: does it read naturally?
And, if localization is intentional:
- Is the divergence an intentional/local-appropriate interpretation rather than an error?
That distinction is close to the motivation behind XSTS, a bilingual human-evaluation protocol that focuses explicitly on semantic correspondence/adequacy rather than folding everything into fluency.
E. Optional: deck-level structure
Since 78 cards is tiny, another interesting exploratory test is to build the full 78Ă—78 semantic-similarity matrix in each language.
There are only 3,003 unique card pairs, so one can ask whether relative neighborhoods are approximately preserved:
English deck similarity structure
vs
Spanish deck similarity structure
This could reveal broad drift that same-card retrieval misses.
I would keep this explicitly exploratory, though. The result depends heavily on the embedding model, so I would not call a matrix correlation a “symbolic fidelity score.”
[/details]
One important fork: translation or localization?
I think this decision should probably come before finalizing the evaluation.
If the goal is:
A. Faithful multilingual alignment of English v1
Then the English v1 meaning is the reference interpretation, and divergence is mostly something to inspect.
English v1
→ faithful ES
→ faithful DE
→ faithful TR
→ faithful PT-BR
Same-card retrieval and semantic adequacy make sense as fairly direct checks.
B. Natural tarot interpretations for each language/locale
Then some divergence from English may be correct behavior.
A Spanish, Turkish, German, or Brazilian Portuguese interpretation may choose a different idiom, emphasis, or culturally natural wording while still serving the local reader better.
In that case, maximizing similarity to English can accidentally punish good localization.
C. Both
This may actually be the most research-friendly version if it is not too much work:
faithful_translation
localized_interpretation
as separate variants.
Then the dataset can support two different experiments:
- preservation of an aligned English-source interpretation;
- variation/adaptation of symbolic interpretation across locales.
A useful precedent here is MASSIVE, which explicitly distinguishes whether an item was a translation, a localization, or left unchanged, rather than treating all target-language differences as one operation.
I would not copy MASSIVE’s annotation machinery, but that distinction seems valuable here.
And I would probably split the metadata into two separate questions:
variant_type:
faithful_translation
localized_interpretation
creation_method:
human
machine
machine_postedited
review_status:
...
because “what relationship does this target text have to the English meaning?” and “how was this text produced?” are not the same thing.
A small synthetic sanity check I tried
I was curious whether the retrieval/control idea would produce any useful signal at all with only 78 candidates, so I ran a small synthetic-control experiment.
This was not an evaluation of a future Deckaura multilingual release.
I took the current English compact fields, created synthetic Turkish/Spanish/German/Portuguese translations with NLLB-200 distilled 600M, and tested aligned retrieval with two multilingual encoders.
The rough result was that 78 candidates are enough for the test to be useful:
| encoder |
same-card top-1 |
correct > hard wrong-card |
| multilingual E5 |
~91% |
~96% |
| multilingual MiniLM |
~94% |
~98% |
The exact percentages are less interesting to me than the failure modes.
A few things stood out:
- Random wrong cards were usually too easy.
- Semantically close wrong cards were much more informative.
- Upright/reversed swaps were cheap and useful controls.
- The two encoders did not fail on exactly the same examples.
- A few catastrophic synthetic translation failures became extreme retrieval outliers.
- But some perfectly plausible translations also ranked poorly because several cards genuinely have similar short meanings.
That last point seems especially important:
retrieval miss ≠mistranslation
I would use retrieval as an anomaly detector / review-prioritization tool, not as an automatic verdict.
For E5 specifically, I used the symmetric query: / query: convention in the corrected run because the multilingual E5 model card recommends query: on both sides for semantic similarity, bitext mining, and paraphrase retrieval; query: / passage: is intended for asymmetric information retrieval.
The synthetic MT should also not be treated as gold. NLLB’s own model card describes it as a research/general-domain MT model and notes limited investigation outside its primary domains. In the probe, some short tarot phrases were indeed mistranslated badly.
So for an actual v2, the interesting experiment would be to rerun the same checks on the real reviewed translations rather than treating the synthetic baseline as a target.
One more caution: I would report results by language and field, not only as one multilingual average. Recent work such as XQ-MEval shows that automatic MT metrics can have cross-lingual scoring bias, where translations of comparable quality receive different score distributions across languages. My small probe also showed noticeable language Ă— encoder interactions.
So a table like:
upright reversed love career
TR
ES
DE
PT-BR
is probably more informative than one number called “symbolic consistency.”
A few small v2 hardening opportunities
These are much less important than the three questions above, but if the schema is already being touched for multilingual v2, there are a few cheap cleanups that may be worth doing at the same time.
yes_or_no
The current data contains three values:
Yes
No
Maybe
while the dataset card currently describes the field as a “Binary verdict.”
So this appears to be a documentation-only cleanup: describing it as ternary/categorical would match the actual data.
zodiac_sign
The current field contains zodiac signs but also values such as planets/luminaries.
If the schema is being revised anyway, a broader name such as:
astrological_correspondence
might communicate the contents more directly.
I would not regard this as a blocker; it is just easier to rename before downstream multilingual schemas depend on it.
Locale naming
If the intended Portuguese data is specifically Brazilian Portuguese, I would preserve that distinction explicitly, e.g. pt-BR, rather than collapsing it to an ambiguous “Portuguese” field.
For Hub-level language metadata, the language can still simply be Portuguese; the more precise locale can live in the data/schema.
Tiny validation
The dataset is small enough that a build check can be almost trivial:
assert len(cards_for_locale) == 78
assert card_numbers == set(range(78))
assert no_duplicate_card_locale_pairs
assert all_required_meanings_present
A neighboring 78-card dataset, Tarotoo’s structured dataset, already uses CI validation for 78 unique cards, required fields, and enumerated values. Again, I would borrow the engineering pattern rather than its interpretation content.
So, if I were choosing the lowest-cost path, I think I would do roughly this:
1. Keep the existing 78-card identity and compact meanings.
2. Decide whether v2 represents:
faithful translations,
localized interpretations,
or two separate variants.
3. Prototype a few cards in 2–3 multilingual schemas.
4. Pick the representation that is easiest to inspect,
but expose language-specific Hub configs regardless if useful.
5. Add longer context only as an additional layer.
6. Record just enough provenance to distinguish
locale / variant / creation method / review state.
7. Add tiny structural validation.
8. Evaluate with:
same-card retrieval
+ deliberate hard controls
+ selective human review.
9. Treat deck-level geometry as exploratory, not as the definition
of symbolic consistency.
I do not think this needs to become a large ontology or a heavyweight MT benchmark to be useful.
The small, closed 78-card inventory is actually one of the attractive parts of the dataset: every item can be aligned, audited, perturbed, and compared exhaustively.
So I would mostly preserve that simplicity, and just separate the boundaries that become important once multiple languages enter the picture:
card identity
≠localized wording
canonical alignment
≠Hub presentation
compact descriptor
≠contextual prose
faithful translation
≠localization
automatic anomaly score
≠human judgment
That seems enough to keep the current dataset easy to use while making the multilingual version much easier to evaluate and extend later.