Converting a PDF to Markdown before feeding it to a language model preserves the document's structure — headings, lists, tables, emphasis — in a form models handle natively. Plain text extraction discards that structure, and losing it measurably degrades retrieval quality, because the model can no longer tell a heading from a sentence or a table row from a paragraph.
The conversion is the easy part. The decisions around it — what to preserve, how to chunk, what to do about tables — determine whether your retrieval works.
Why Markdown rather than plain text?
| Format | Preserves structure | Token overhead | Model handling |
|---|---|---|---|
| Plain text | No | Lowest | Structure must be inferred, often wrongly |
| Markdown | Yes | Low | Native — models are trained on it |
| HTML | Yes | High | Works, but tags consume tokens |
| JSON with layout | Yes, precisely | Highest | Best for programmatic use, worst for context windows |
Markdown sits at the useful point: structure survives, token cost stays low, and models were trained on enormous quantities of it.
The practical difference shows up in retrieval. A chunk that begins ## Payment Terms carries its own context. The same chunk as plain text begins with a sentence fragment, and the retriever has no signal about what section it came from.
What should the conversion preserve?
Headings. The most valuable single element. They give each chunk a label and let you build a document outline for hierarchical retrieval.
Tables. As Markdown pipe tables where the structure is simple. Complex tables — merged cells, nested headers — often convert better as a short prose summary plus the raw data, because a mangled Markdown table is worse than no table.
Lists. Ordered and unordered both matter. A numbered procedure that loses its numbering loses its meaning.
Emphasis. Bold often marks defined terms, warnings, or key figures. Cheap to preserve, useful at retrieval time.
Page boundaries. Not visible in the output, but worth keeping as metadata so you can cite a page number in an answer.
What to discard: headers and footers repeated on every page, page numbers, decorative rules. These pollute chunks and add nothing.
How do I convert a PDF to Markdown?
curl -X POST https://www.ashdocs.com/api/v1/tools/pdf-to-markdown \
-H "X-API-Key: ash_live_..." \
-F "file=@contract.pdf"
{
"markdown": "# Service Agreement\n\n## 1. Scope\n\nThe Provider shall...",
"pages": 14,
"metadata": { "title": "Service Agreement", "has_tables": true }
}
For scanned documents, OCR must run first. A scanned PDF contains images of text, not text. Converting it directly yields nothing. Check whether the document has a text layer before choosing a path — if a text extraction returns almost nothing on a document that visibly has text, it is scanned.
How should I chunk the output?
The decision that most affects retrieval quality, and the one most often made by accident.
Chunk on heading boundaries, not fixed character counts. A 1,000-character window cuts mid-sentence and mid-concept. A section boundary is a semantic boundary, which is what the retriever is trying to match against.
Keep chunks between roughly 200 and 800 tokens. Below that, insufficient context. Above it, the chunk covers several topics and matches poorly on all of them.
Prepend the heading path to each chunk:
Document: Service Agreement
Section: 4. Payment Terms > 4.2 Late Payment
Invoices unpaid after 30 days accrue interest at...
That prefix costs a few tokens and substantially improves retrieval, because the embedding now carries the section's context.
Overlap by 10–15%. Enough to catch concepts spanning a boundary, not so much that you duplicate storage and return near-identical chunks.
Keep tables whole. A table split across chunks is useless in both halves. If a table exceeds your size limit, keep it whole anyway and accept the outlier.
What usually goes wrong
Multi-column layouts interleave. Academic papers and newsletters read left-to-right across both columns, producing alternating fragments. Layout-aware extraction handles this; naive text extraction does not. Check a two-column document early.
Headers and footers appear in every chunk. A company name and page number repeated 200 times pollutes your embeddings. Strip repeating elements before chunking.
Scanned pages return empty. No text layer. Run OCR first.
Tables become unreadable. Complex tables rarely survive as Markdown. Consider extracting them separately as structured data and referencing them, rather than forcing them into pipe syntax.
Ligatures break search. Some PDFs encode "fi" and "fl" as single glyphs, so a search for "workflow" misses "workflow". Normalise Unicode after extraction.
Footnotes appear mid-sentence. They are positioned at the page bottom but belong to a specific sentence. Either move them to the end of their section or drop them, but do not leave them interrupting the flow.
Frequently asked questions
Why convert PDF to Markdown instead of plain text for RAG?
Markdown preserves headings, lists, tables and emphasis, which plain text discards. That structure gives each chunk semantic context and measurably improves retrieval quality, at a very small token cost.
Can I convert a scanned PDF to Markdown?
Not directly — a scanned PDF contains images rather than text. Run OCR first to produce a text layer, then convert.
What chunk size works best for RAG?
Roughly 200 to 800 tokens, split on heading boundaries rather than fixed character counts, with 10 to 15 percent overlap and the section heading prepended to each chunk.
How do I handle tables when converting PDFs for LLMs?
Simple tables convert cleanly to Markdown pipe syntax. Complex tables with merged cells are usually better extracted separately as structured data, with a short prose summary in the Markdown.
Does converting to Markdown lose information?
It discards visual formatting — fonts, colours, exact positioning — and keeps semantic structure. For retrieval that is the correct trade, since models reason over meaning rather than appearance.
In short
Convert to Markdown rather than plain text. Preserve headings, lists and simple tables; strip repeating headers and footers. Chunk on section boundaries at 200–800 tokens with the heading path prepended. Run OCR first on anything scanned, and check a multi-column document early.