EU financial regulation as data
I wanted a parallel corpus of EU financial law to train and test retrieval on. I assumed the hard part would be aligning the translations. It turned out I did not have to align anything: EUR-Lex numbers every language version of an act with the same structural ids, so the same paragraph carries the same id in all 24. What follows is what I found once I stopped assuming that and started checking it.
Parallel legal corpora are normally built by matching sentences across translations. It is approximate work, and it fails hardest on exactly the provisions people care about: the long ones, where a single English sentence becomes three in German and the numbering drifts.
EUR-Lex does not need any of that. It assigns structural ids per act, not per
translation. Article 16 is art_16 in all twenty-four versions, and its seventh
paragraph is 016.007 whether you asked for English, Greek or Croatian. Two rows
that share an id are the same provision by construction, not by inference.
The interesting part is not that the alignment is good. It is that there is no alignment step at all. The Publications Office did it in 2014 and the ids have been sitting in the markup ever since.
That has a second consequence which turned out to matter more than the corpus itself. If the ids align, then the Dutch text of a provision is a free label for whatever you are extracting from the English one. No annotation, no gold set. I used that as a test harness for the rest of the project.
My first check covered English, Dutch, German and French across eight acts. Thirty-two act-language pairs, all exactly aligned, 2,442 paragraphs each. I wrote that number down and nearly shipped it as the headline.
Running the same check across all 24 languages took the claim apart in a more useful way:
| Check | Pairs | Share |
|---|---|---|
| Article ids identical to English | 181 / 181 | 100% |
| Article and paragraph ids identical | 179 / 181 | 98.9% |
The two failures were UCITS and AIFMD in Croatian, and they had not diverged slightly. They had returned zero articles. Silently. A parser that looked perfect on four languages was producing an empty document for two real ones and reporting no error at all.
The cause is a date. Croatia joined the EU in 2013. UCITS was adopted in 2009 and AIFMD in
2011, so neither has a Croatian version from the day it was passed — they appear instead as
a special edition, translated later and published in the older EUR-Lex layout. That layout
uses unprefixed class names, has no eli-subdivision wrappers, and carries no
structural ids whatsoever.
An accession date is visible in the HTML. The enlargement of the European Union is, among other things, a change of file format.
Those two acts now parse through a fallback that reads the old layout and returns the right 119 and 71 articles. They still have no paragraph ids, because the format genuinely does not have any, so they align by article and not below it. That is a real limit rather than a rounding error, and the dataset card says so.
The same sweep turned up a second gap I would not have guessed: MiFID II, UCITS and AIFMD have no Irish version at all. Irish was an official language with a derogation on translation volume that ran until 2022, so acts adopted before then were never published in it.
EU acts cite each other constantly, and a citation graph is most of the value of having the text at all. Extracting citations in English is easy. Doing it in Bulgarian, Finnish and Maltese is not, and I had no labelled data for any of them.
I did not need any. Because the ids align, the same article in Dutch must cite the same acts as the English version. Any disagreement is a bug in my extractor, not a difference in the law. That is a complete regression suite for free, and it found four distinct bugs:
| What broke | Because |
|---|---|
Verordeningen (EU) nr. 1093/2010 | my stem matched German Verordnung but not Dutch Verordening |
регламенти (ЕС) № 1093/2010 | Bulgarian writes the number marker as №, which my pattern could not match |
(EU) 2019/2033 | read as act 2019 of year 2033 — an impossible CELEX |
Ez a rendelet … 2011/61/EU irányelv | Hungarian says “This Regulation” constantly, capturing every directive cited after it |
The third one is worth a sentence. EU acts changed their numbering convention in 2015:
before it was No 600/2014, number then year, and after it is
2022/2554, year then number. Both are just two numbers separated by a slash. The
thing that actually distinguishes them is whether the “No” marker is present — which is why
a rule about punctuation ends up being a rule about legal drafting history.
Hungarian is the one that is still short, at 86%. It writes 1093/2010/EU rendelet
— number before year, act word after the number — which collides with the directive form used
everywhere else. I know why it fails and have not fixed it, which seems worth saying plainly.
With citations extracted across eight acts, the graph has 3,662 edges. The obvious question is which provision the rest of the corpus leans on hardest. Counting only citations that come from a different act, the answer is not any of the famous conduct rules:
| Provision | Citations |
|---|---|
| ESMA Regulation Article 19 — binding mediation between authorities | 15 |
| ESMA Regulation Article 15 — implementing technical standards | 14 |
| ESMA Regulation Article 16 — guidelines and recommendations | 10 |
| EBA Regulation Article 16 — guidelines and recommendations | 7 |
Every one of them is a rule about how rules get made. MiFID II, MiCA and DORA spend their cross-references pointing back at the supervisor's power to issue standards, publish guidelines and settle disagreements — not at each other's substantive obligations. If you are building anything that reasons over this corpus, the practical reading is that the Level 1 text is a frame and the binding detail lives one hop away.
Getting there required fixing something duller. “Articles 10 to 13” is four citations, and I was counting it as one. Expanding enumerations took the graph from 2,709 edges to 3,662 — a third of it had been missing, and the graph looked entirely plausible without them.
Parsing 192 documents took about half an hour, so I did what most people do and reached for
a faster parser. I installed lxml, expecting the usual large win over Python's built-in
html.parser. It was slower — 15.0s against 9.65s on the largest
act. That result made no sense until I accepted what it was telling me, which is that the
parser was never the bottleneck.
It was two loops. Building an article's plain text created a brand-new parse of that article's HTML, so a 149-article regulation was parsed 150 times. And resolving each article's chapter and section headings searched the entire document once per ancestor, about 450 full scans per act. Both became dictionary lookups.
| Attempt | Per document | All 192 |
|---|---|---|
| Baseline | 9.65s | ~31 min |
| Swap in lxml | 15.0s | ~48 min |
| Fix the two loops, keep html.parser | 0.82s | ~2.6 min |
11.8× faster, and lxml is not a dependency. I keep the failed attempt in the table because it is the more useful half of the result: the instinct to buy speed with a better library, or a bigger machine, would have run the same bad loops slightly differently. Nothing about renting a GPU improves an O(articles × document) scan.
Getting text out of EUR-Lex is well covered. kevin91nl/eurlex parses any CELEX document into a DataFrame. tseidl/eurlex-builder goes considerably further — a research pipeline emitting DuckDB and Parquet, handling six HTML eras plus scanned PDFs, granularity down to the lettered point, and document relations pulled from Cellar's own metadata. It has a DOI and a paper behind it. If you want breadth, that is the one to reach for, and the Croatian layout I treated as a discovery above is one of six eras it already handles.
What neither produces is the official translations aligned to each other. eurlex-builder normalises to English and machine-translates non-English sources, which is right for policy analysis but makes the multilingual text model output rather than the human translation the Union published. That is the gap this fills: all 24 versions, aligned exactly, plus citations that resolve to the target article rather than only the target document.