Can Mistral AI read a page laid out in multiple columns in the right order?
Mistral AI read the two-column page down each column in turn rather than across the page. On the three pages, the left and right columns stayed in order without interleaving.
1 of 1 test case passed
Every test case in this scenario has a result.
The test case
Each test case is judged on its own: Pass, Fail, Not gradable, or Untested. The scenario result above counts this row.
Two-column page — natural reading orderPassEvidence
# The Vellamy Society *Bulletin* Founded 1898 Number 212 · March 2026 ## Spring lecture programme Three lectures are arranged for the spring, all at the Assembly Rooms on Vardenne Quay at half past seven. On 11 March Dr Aileen Prosser will speak on the salt industry of the upper estuary; on 8 April Mr Colm Tarbert will describe the 1953 flood as it was recorded at Kellow; and on 13 May the Society's archivist, Miss Ruth Sandiman, will introduce the Harrowby field books, which the Society acquired last autumn. Members may bring one guest without charge. Tea is served from seven o'clock, and the room is open from half past six for those who wish to look at the exhibits. Anyone needing a lift from the town should telephone the honorary secretary, who keeps a list of members willing to offer one. ## The Harrowby field books The eleven field books kept by Thomas Harrowby between 1871 and 1904 came to the Society in October, the gift of his great-granddaughter, and are now with the county record office for conservation. They are working notebooks rather than journals: tide times, wind, the state of the banks at Thrimby, the names of the men who worked each reach, and here and there a sketch of a channel that has since moved half a mile. The eighth book, which covers the winter of 1889, is the one most often asked for, because it records the breach at Kellow in some detail and gives the height of the water against the wall of the old lifeboat house. The Society hopes to publish a transcript of the eighth book, and would welcome help with it. What is wanted is not palaeography - the hand is a clear one - but local knowledge: field names, the older names for the reaches, and the identification of men named only by a surname and an initial. Members who could give an evening a month to this should write to the archivist. ## The summer excursion The summer excursion will be to Marnhead on Saturday 20 June, to see the salt pans and what is left of the tramway on the north side of the harbour. Mr Tarbert has agreed to lead the walk, which is about three miles over rough ground and is not suitable for anyone who cannot manage a stile. The coach leaves the Assembly Rooms at half past nine and is back by six. The cost, including the coach and the entry to the harbour office, is fourteen pounds, and members should book with the excursion secretary before 1 June. ## New members and gifts The Society welcomes seven new members since the autumn bulletin: three from Vardenne, two from Braithe, one from Kellow and one, to the committee's particular pleasure, from Winnipeg. The library has received the bound volumes of the Braithe Advertiser for 1928 to 1934 from the estate of the late Mr E. Lissamer, and an album of photographs of the 1947 regatta from Mrs Prosser of Thrimby. Both have been accessioned and may be consulted on the usual notice. ## Notes and queries A member asks whether anyone can identify the building in the postcard reproduced in the autumn bulletin. The committee's view is that it is the old ferry office at Vardenne, pulled down about 1961, but the roof line does not match the photograph in the Society's own collection and the question is left open. Replies to the archivist, please, and not to the editor, who is away until April. ## Subscriptions Subscriptions for the year beginning 1 April are unchanged: eighteen pounds for a member, twenty-six pounds for two members at one address, and six pounds for members under twenty-five. Those who pay by standing order need do nothing. The Society remains grateful to the twenty-one members who add a donation to their subscription. ## The old lifeboat house
Copied from Proof 1 · Output file (Markdown), lines 1–39 · an excerpt; the link below opens the whole file
Open the Markdown file ↗Part of this file is printed above, under What the tool returned. Show the whole file
# The Vellamy Society *Bulletin* Founded 1898 Number 212 · March 2026 ## Spring lecture programme Three lectures are arranged for the spring, all at the Assembly Rooms on Vardenne Quay at half past seven. On 11 March Dr Aileen Prosser will speak on the salt industry of the upper estuary; on 8 April Mr Colm Tarbert will describe the 1953 flood as it was recorded at Kellow; and on 13 May the Society's archivist, Miss Ruth Sandiman, will introduce the Harrowby field books, which the Society acquired last autumn. Members may bring one guest without charge. Tea is served from seven o'clock, and the room is open from half past six for those who wish to look at the exhibits. Anyone needing a lift from the town should telephone the honorary secretary, who keeps a list of members willing to offer one. ## The Harrowby field books The eleven field books kept by Thomas Harrowby between 1871 and 1904 came to the Society in October, the gift of his great-granddaughter, and are now with the county record office for conservation. They are working notebooks rather than journals: tide times, wind, the state of the banks at Thrimby, the names of the men who worked each reach, and here and there a sketch of a channel that has since moved half a mile. The eighth book, which covers the winter of 1889, is the one most often asked for, because it records the breach at Kellow in some detail and gives the height of the water against the wall of the old lifeboat house. The Society hopes to publish a transcript of the eighth book, and would welcome help with it. What is wanted is not palaeography - the hand is a clear one - but local knowledge: field names, the older names for the reaches, and the identification of men named only by a surname and an initial. Members who could give an evening a month to this should write to the archivist. ## The summer excursion The summer excursion will be to Marnhead on Saturday 20 June, to see the salt pans and what is left of the tramway on the north side of the harbour. Mr Tarbert has agreed to lead the walk, which is about three miles over rough ground and is not suitable for anyone who cannot manage a stile. The coach leaves the Assembly Rooms at half past nine and is back by six. The cost, including the coach and the entry to the harbour office, is fourteen pounds, and members should book with the excursion secretary before 1 June. ## New members and gifts The Society welcomes seven new members since the autumn bulletin: three from Vardenne, two from Braithe, one from Kellow and one, to the committee's particular pleasure, from Winnipeg. The library has received the bound volumes of the Braithe Advertiser for 1928 to 1934 from the estate of the late Mr E. Lissamer, and an album of photographs of the 1947 regatta from Mrs Prosser of Thrimby. Both have been accessioned and may be consulted on the usual notice. ## Notes and queries A member asks whether anyone can identify the building in the postcard reproduced in the autumn bulletin. The committee's view is that it is the old ferry office at Vardenne, pulled down about 1961, but the roof line does not match the photograph in the Society's own collection and the question is left open. Replies to the archivist, please, and not to the editor, who is away until April. ## Subscriptions Subscriptions for the year beginning 1 April are unchanged: eighteen pounds for a member, twenty-six pounds for two members at one address, and six pounds for members under twenty-five. Those who pay by standing order need do nothing. The Society remains grateful to the twenty-one members who add a donation to their subscription. ## The old lifeboat house The Society's letter to the district council about the lifeboat house was answered in January. The building is not listed and the council does not propose to seek listing, but the officer who replied confirmed that any application to demolish it would come before the conservation panel, on which the Society has a seat. The nineteen members who wrote in support of the letter are thanked. Correspondence to the Honorary Secretary, 4 Kellow Lane, Vardenne BR7 3AT · 01632 960377 Page 1 The Vellamy Society · Number 212 · March 2026 The committee's view remains that the interest of the place is the winch house and the slipway rather than the shed itself, which was re-roofed in the 1960s and has lost its doors. A measured survey would settle what is left of the original fabric, and the committee is asking whether any member has the skill and the time to make one. ### **From the archive: the breach of 1889** The eighth Harrowby field book gives the night of 12 December 1889 in more detail than any other source the Society holds. Harrowby was at Kellow from the early evening and his entries are timed: the wall was overtopping by nine, the first gap was made at about half past ten, and by midnight he put it at sixty yards and taking. He gives the height of the water against the lifeboat house wall as four feet nine inches, which is nine inches above the mark cut there after the flood of 1874. What the book does not say is how the gap was closed, and the parish records are no better. The rate book shows a payment to a Marnhead contractor in the spring of 1890 and nothing else at all. If any member knows of a family paper that mentions the work, the archivist would be glad to hear of it. The Society intends to print the whole of the December entries in the summer bulletin, with a note on the tide and the wind of that week. The transcript is finished and is being checked against the original at the record office. ### **The photograph collection** Cataloguing has reached the end of the Vardenne boxes: eleven hundred prints and about four hundred negatives, of which rather more than half are now identified at least to a street or a reach. The catalogue is a card index and will stay one until the Society can be reasonably sure that whatever replaces it could be maintained by whoever is doing the work in ten years' time. Copies can be supplied to members for a charge covering the print and the postage. The Society does not hold the copyright in every image and will say so where that is the case; where the photographer is unknown, prints are supplied on the usual undertaking. ### **The estuary in 1926** A hundred years ago this spring the harbour board was deciding whether to dredge the bar, and the papers of the argument are worth reading. The board's engineer wanted a channel of fourteen feet and got eleven; the fishermen wanted the bar left alone and were not consulted until the decision had been taken; and the compromise was arrived at in a meeting of which no minutes were kept at all. The channel silted again within four years. The Society holds copies of the engineer's two reports and of the petition against them, which carries a hundred and ninety-one signatures and is the best list we have of the men working out of Vardenne in that year. Members are welcome to consult it, and a transcript of the signatures is being prepared. ### **Mr E. Lissamer** Members will be sorry to learn of the death in November of Mr Edward Lissamer, who joined the Society in 1974 and served on the committee for eleven years, five of them as honorary treasurer. He was a patient and exact man who kept the accounts in a hand nobody could misread, and he had a memory for the estuary's boats that the Society will not replace. His library has come to us under his will and will be listed in the next bulletin. ### **Visits and walks** Four walks are arranged for the summer, each of about two hours and none of them strenuous. On 6 June, the Vardenne quays with Mr Tarbert; on 4 July, the training wall and the east bank, tide permitting, with Miss Sandiman; on 1 August, the Kellow marshes, led jointly with the county wildlife trust; and on 5 September, the town's chapels. All start at half past ten from the Assembly Rooms unless the programme card says otherwise. Members are asked to book for the marshes walk, which is limited to twenty and needs boots. The others need no booking and no charge is made, though the collection at the end goes to the room fund. ### **Committee news** The annual general meeting will be held at the Assembly Rooms on Tuesday 19 May at seven o'clock. The officers all offer themselves for re-election except the honorary treasurer, who does not, and nominations for that office should reach the secretary in writing a fortnight before the meeting. The accounts to 31 March will be circulated with the summer bulletin. The committee has agreed to meet the increase in the Correspondence to the Honorary Secretary, 4 Kellow Lane, Vardenne BR7 3AT · 01632 960377 Page 2 The Vellamy Society · Number 212 · March 2026 room hire from the reserve rather than from subscriptions, and to review that at the annual general meeting. ### Reviews *A History of the Marnhead Salt Trade*, by J. Cavenshaw (Vardenne, 2025, 214 pp). This is a careful book and a slightly frustrating one. Cavenshaw has worked through the harbour board's papers properly and his chapter on the tramway is the best account we have. But he is not much interested in the pans themselves, and a reader who wants to know how salt was actually made at Marnhead will have to go back to Sandiman. Recommended, with that reservation. ### Letters Sir - your correspondent's note in the autumn bulletin on the naming of Croyde Gut may interest members who know the Ordnance sheets of 1897 and 1923. The name appears on neither, and the reach is called Crowder's Gut on both. Whether Croyde is a correction or a corruption I cannot say, but the older form is at least the surveyed one. - R. Vale, Thrimby. ### Wanted: the Vardenne ferry The Society is trying to establish when the Vardenne ferry stopped running. The date usually given is 1961, which is the year the ferry office was pulled down, but the office may well have stood empty for some time before that. A member recalls being taken across as a child in the summer of 1958 and is sure the boat was still working then; another remembers it ending just after the war. A timetable, a ticket, or a photograph with a date on the back would settle it. The same member has given the Society a copy of the 1957 timetable for the Marnhead bus, which connected with the ferry and may therefore date its last summer more exactly than anything else we hold. It is on the notice board at the Assembly Rooms for anyone who wants to look at it. ### The chapels survey The survey of the town's chapels, begun in 2019 and interrupted twice, is now complete for the four buildings still standing and for two of the three that are not. What remains is the Bethel on Kellow Lane, of which no plan and only one photograph have been found. The survey will be printed as an occasional paper in the autumn, and the members who took part are asked to check their entries before then. The Society is grateful to the trustees of two of the chapels for access to their records, and to the county archivist for the loan of a plan that had been catalogued under the wrong parish for sixty years. ### Membership Membership stood at two hundred and six at the end of January, four more than a year ago and the first increase since 2019. Twenty-two members live outside the county and six outside the country; the Society posts the bulletin to all of them and would rather do that than lose them. Anyone who would prefer it by e-mail should tell the secretary, who keeps a list and is happy for it to grow. ### Notices The Society's thanks go to the members who helped at the December sale, which raised a hundred and twelve pounds for the room fund, and to Mrs Tarbert for the loan of the tables. The room fund stood at three hundred and eighty pounds at the end of January. The committee thanks the members who have given to it, and reminds everyone that the fund exists to keep the Assembly Rooms available to the Society rather than to accumulate. Copy for the summer bulletin should reach the editor by 15 May. Short notes are as welcome as articles, and a query that has been in the back of a member's mind for years is often the most useful thing in an issue. Correspondence to the Honorary Secretary, 4 Kellow Lane, Vardenne BR7 3AT · 01632 960377 Page 3
These are scenario-level criteria. Each test case's Expected and Found are listed separately.
- Whether text is emitted in the natural reading order down each column.
- Whether content from different columns is kept separate rather than interleaved.
- Whether the page's reading sequence is preserved even when all text is present.
| ✓ | Found: Natural reading order is preserved, with each left-column fragment followed by its matching right-column fragment. |
On all three pages, each gutter-spanning sentence stays in order, with the left-column text followed by the right. The cited output shows no column interleaving.
Page-break furniture split. At a page break, the footer, page number, and running header appear between two halves of one sentence. The sentence order still matches the requested reading order.
Gutter splits stay ordered. Sentences crossing a gutter are split into two paragraphs instead of stitched into one. Their order stays consistent at all three gutters.
Configuration and setup
How this tool was set up for the run and what the test needed in place. Each row is a fact from the run's records; a fact the records do not hold is left out, not guessed.
How this scenario is graded
How we decide Pass, Fail and Not gradable. The same rules apply to every tool tested on this scenario.
Each test case gets one result per tool: Pass, Fail or Not gradable. A test case we haven't run yet shows Untested. There are no partial results.
- Pass: the tool did everything the test expected, and nothing it said contradicts the correct answer.
- Fail: at least one expected behaviour clearly didn't happen; the row says which and quotes the tool.
- Not gradable: our evidence couldn't settle the outcome (for example a record we needed is missing). It is never counted as a fail, and the row says what's missing.
Where this sits in the benchmark
This page is one cell of a larger study: one tool, one scenario. Only this benchmark's frame appears here.
| Level | Name | Scope |
|---|---|---|
| Benchmark | Converting a complex PDF into clean Markdown with a hosted API → | 12 scenarios · 13 tools |
| Capability | Reading Order & Layout → | |
| Scenario | A page laid out in multiple columns → | |
| Tool | Mistral AI → |
Global scenario definition → · Global capability definition → · Mistral AI product page →
History of this result
What has happened to this result since it was first published. Runs and grades are never overwritten: a retest or a re-grade publishes a new result and keeps the earlier one readable.
Act on this result
Nothing filed here edits the run or the grade. A challenge opens a review, and a review can produce a new run or a re-grade — which becomes the current result and leaves this one in the history.
You run the same kind of test against your own setup and get the same behaviour.
Agree →Yours behaves differently. Tell us what you got, with a screenshot if you have one.
Disagree →Something here is wrong — a reference value, a transcription, a grade.
Report an issue →Tell us what changed and we schedule a rerun of the failing test case. The old result stays as history.
Vendor notice →The same record is available as structured data through the AI Demos MCP server, with the counts, the coverage and every per-test-case reason carried as fields.
Verify the proof files
These files support this result. Open a file to inspect the original evidence.
File fingerprints (SHA-256)
A fingerprint identifies the exact file used for this result.