#doc_parse

#What This Library Is

• container and package structure
• source-native document models
• relationships, parts, manifests, pages, shapes, cells, spans, and metadata
• inspect reports and inventory APIs
• validation and safety boundaries
• parser-level foundations that can be reused outside markitdown-mb

#What This Library Is Not

• a traditional compiler-style AST parser for one grammar
• a full document-to-Markdown converter
• a CLI product
• a Word, PowerPoint, Excel, browser, EPUB reading-system, or PDF visual layout engine
• an OCR system
• a full-spec claim for every supported format
• the owner of final Markdown, asset, metadata, or layout policy

#Relationship To markitdown-mb

#What You Can Build On Top Of It

• custom document-to-Markdown converters
• RAG and knowledge-base ingestion pipelines
• document inventory and audit tools
• Office package inspectors
• EPUB spine / manifest / metadata analyzers
• PDF text-page inspection tools
• CSV / TSV / JSON / YAML / XML validation utilities
• private IR builders
• document diff / provenance tooling
• lightweight search indexing preprocessors
• format-specific validators
• MoonBit-native content engineering tools

#Package Map

#Shared Foundation

• foundation Small shared string, path, and text-cleanup helpers used by parser packages. This is not the full markitdown-mb product core.

#Container And Package Foundations

• doc_parse/zip ZIP container reader, entry inventory, path-safety, and validation layer.

• doc_parse/ooxml OOXML package foundation for parts, relationships, content types, media, docProps, structured inventory, error classification, and strict validation. This is the shared base for DOCX, PPTX, and XLSX parsing.

• doc_parse/epub EPUB package foundation for container.xml, OPF rootfiles, manifest, spine, nav / NCX discovery, cover candidates, metadata, inspect, and validation.

#PDF Foundation

• doc_parse/pdf/api
• doc_parse/pdf/raw
• doc_parse/pdf/model
• doc_parse/pdf/text

#OOXML Semantic Sublayers

• doc_parse/docx WordprocessingML source-native body, paragraph, table, style, numbering, note, relationship, media, and inspect foundation.

• doc_parse/pptx PresentationML source-native slide, shape tree, paragraph/run, explicit table, note, media, hyperlink, and inspect foundation.

• doc_parse/xlsx SpreadsheetML workbook, sheet, cell, shared string, style, merged range, formula trace, inspect, and validation foundation.

#Text, Markup, And Structured Data

• doc_parse/text Plain-text bytes/string open, BOM/newline handling, structural model, and inspect surface.

• doc_parse/markdown Lightweight Markdown scanner foundation for raw block inventory, frontmatter, fenced code, and validation.

• doc_parse/csv
• doc_parse/tsv Delimited table parsers with table model, inspect, and ragged-row validation.

• doc_parse/json JSON parser foundation with AST/model, inspect, and malformed-input classification.

• doc_parse/yaml YAML-subset parser foundation with subset model, inspect, and fail-closed unsupported-feature boundaries.

• doc_parse/xml Safe XML tokenizer/parser/model/inspect/validation foundation with explicit no-XXE and no-DTD-expansion boundary.

• doc_parse/html Lightweight HTML DOM-ish parser foundation with tolerant tokenization, raw-node inventory, inspect, validation, and explicit no-fetch / no-script-execution boundaries.

#API Stability Model

#Stable Candidate API

• open_*
• parse_*
• scan_*
• read_*
• list_*
• find_*
• inspect_*
• validate_*
• classify_*

#Compatibility Surface

#Diagnostic / Profile Helpers

#Internal Exposed Surface

#Usage Examples

#Inspect an OOXML package

let pkg = @ooxml.open_ooxml_package(bytes)
let inventory = @ooxml.inspect_ooxml_inventory(pkg)

println("parts=" + inventory.part_count.to_string())
println("relationships=" + inventory.relationship_part_count.to_string())
println("media=" + inventory.media_asset_count.to_string())

#Read DOCX source-native structure

let pkg = @ooxml.open_ooxml_package(bytes)
let doc = @docx.parse_docx_document_from_package(pkg)
let report = @docx.inspect_docx_document(doc)

println("paragraphs=" + report.paragraph_count.to_string())
println("tables=" + report.table_count.to_string())

#Inspect an EPUB spine

let pkg = @epub.open_epub_package(bytes, "book.epub")
let rootfile = @epub.get_epub_primary_rootfile(pkg)
let spine = @epub.list_epub_spine_reading_order(pkg)
let nav = @epub.find_epub_nav_item(pkg)

println("rootfile=" + rootfile.full_path)
println("spine_items=" + spine.length().to_string())

#Inspect a native text-PDF

let summary = @pdf.open_pdf_summary("paper.pdf")

println("pages=" + summary.model.page_count.to_string())
println("issues=" + summary.inspect.issues.length().to_string())

#Parse structured data

let doc = @json.parse_json_document(bytes)
let report = @json.inspect_json_document(doc)

println("nodes=" + report.node_count.to_string())

#Safety Boundaries

• no remote fetch
• no script execution
• no browser rendering
• no EPUB reading-system behavior
• no OCR by default
• no entity expansion / XXE behavior in XML
• path traversal checks for container formats
• external relationships are classified, not fetched
• unsupported features are reported or fail closed where appropriate
• parser-level validation is separate from final product conversion

#What Is Included

• doc_parse/** parser packages
• foundation support package used by parser packages
• source-native models
• inspect and inventory APIs
• validation APIs
• error classifiers
• parser-level tests and wbtests that do not require the full product corpus
• the PDF vendor subtree required by doc_parse/pdf
• focused documentation for the extracted library

#What Is Intentionally Not Included

• convert/**
• core/** product IR / emitter / metadata logic
• cli/**
• Markdown output pipeline
• asset export
• metadata sidecar product features
• full product regression corpus
• samples/main_process
• samples/real_world
• Microsoft MarkItDown overlap comparison corpus
• product-path benchmark harness
• cold-start benchmark harness

#Testing

moon fmt
moon info
moon check
moon test

moon test doc_parse/ooxml/tests --target native
moon test doc_parse/epub/test --target native
moon test doc_parse/pdf/api/test --target native

#Library Performance Snapshot

cd ../markitdown
./samples/bench.sh --suite doc-parse --kind library --iterations 10 --warmup 2

• parent repo remains the benchmark source of truth
• table below uses the latest local avg_ms values from the benchmark summary
• these are local observations, not cross-machine guarantees
• scope: direct package API only; excludes CLI startup, Markdown emission, asset export, metadata sidecars, and product-path conversion overhead

#Known Limits

• This is a parser foundation, not a complete document conversion product.
• It exposes source-native models and inspect signals, not final Markdown.
• PDF support targets native text-PDF; OCR/scanned-PDF product behavior is out of scope here.
• OOXML packages are parsed at package and source-native semantic layers; this is not a full Word / PowerPoint / Excel engine.
• HTML parsing is lightweight and safe; it is not a browser engine.
• EPUB parsing covers package/spine/nav foundations; it is not a reading system.
• XML parsing is safety-oriented and does not claim full XML processor behavior.
• Format support is conservative and does not imply blanket full-spec coverage.

#Future Direction

• making doc_parse fully independent from the parent repository lifecycle
• stabilizing compatibility surfaces into narrower public facades
• expanding parser-level examples
• adding a small library-only benchmark harness
• publishing more focused package-level guides
• adding downstream integration smoke tests
• gradually moving more validation from product-level samples into parser-level contract tests