Search, reports and document generation - from an inverted index to a PDF

How full-text search works inside - the inverted index, stemming and scoring - where a search should run, and how reports are built once, kept as snapshots, sent on a schedule and written as Excel and PDF files without giving away small groups.

In short

Search and reporting look like two jobs, but they share one idea: find the right records quickly, then present them so people can act. Full-text search works by building an inverted index - a map from every word to the documents that contain it - and scoring the matches. Reports work by turning records into one document shape of headline numbers, tables and charts, which can be shown on a screen, kept as a snapshot, sent on a schedule or written as an Excel or PDF file. In {{brand}}, Help is searched in the browser with MiniSearch, records are searched live in the database, and every file is written from the same report document. Wherever a number describes people, a small-group rule stops it from revealing one person.

After this lesson you can Explain how an inverted index and relevance scoring work, choose where a search should run - browser, database or search engine - design a report pipeline with snapshots, comparisons and schedules, write correct Excel and PDF files from .NET, and keep group totals from exposing one person.

Words you need

• Inverted index A map from each term to its posting list - the documents (and positions) that contain it. Finding a word is one lookup, not a scan of every document.
• Tokens, normalizing and stemming Cutting text into words (tokens), then making them comparable: lower case, accents removed, small stop words such as the and how dropped. Stemming cuts a word down to its stem so related forms meet: a simple stemmer turns share, shares, shared and sharing all into shar.
• Relevance score and field boost A number for how well a document matches. TF-IDF and its successor BM25 reward words that are frequent in this document but rare in the others. A field boost adds weight to where a word was found: a hit in the title counts more than the same hit in the body.
• Fuzzy and prefix match Fuzzy forgives a typo by allowing a small edit distance - the number of single-letter insertions, deletions or substitutions that turn one word into the other (bord is one insertion away from board). Prefix matches a word still being typed (boa finds board).
• Debounce Waiting until typing pauses before searching, so a fast typist starts one search, not ten.
• Report document and provider One neutral report document - headline numbers, then sections of stats, tables, charts and text - that screens, files and emails all read. A report provider is a small class in the module that owns the data: it builds one report for one person and returns nothing for what that person may not see.
• Snapshot A report stored whole as it was when it ran, so it reads the same later and the next run can be compared with it.
• Lease A time-limited lock on a job. A worker takes the lease and runs the job; if it dies, the lease expires and another worker can take over.
• Layout engine The part of a PDF library that measures text, breaks lines and paginates. You describe rows, columns and tables; the engine decides what fits on each page.
• Text layer (PDF) A PDF stores no sentences, only glyphs placed at positions. A reader such as PdfPig rebuilds words from them. A scanned page is just a picture with no text layer, so it needs OCR.
• Small-group rule (k-anonymity) Show a group's value only when it stands for at least k people, where k is a minimum chosen for each figure. No figure then stands for just one or two people.
• Differencing attack Working out one person's number by subtracting two totals that are each safe on their own - an organization's total minus every team shown.

How full-text search works inside

Every full-text engine - from a few kilobytes of JavaScript to an Elasticsearch cluster - does two things: it indexes documents once, and it answers queries from that index. Learn the two flows and you can read any search design.

• 1. Split into fields Title, keywords and body are indexed separately, so each can carry its own boost later.
• 2. Tokenize Payment timeouts on checkout becomes the tokens payment, timeouts, on, checkout.
• 3. Normalize Lower case, fold accents (şirkət becomes sirket) and drop stop words (on).
• 4. Stem timeouts becomes timeout, so the singular and the plural meet.
• 5. Write posting lists For each term: which documents hold it and how often. For each document: its length, which the score needs.

• 1. Process the query the same way The same tokenizer, folding and stemmer as the documents - otherwise timeouts never meets timeout.
• 2. Look up the terms Read each term's posting list; with fuzzy or prefix on, also the lists of terms close to it.
• 3. Combine AND keeps documents that hold every word; OR keeps those that hold any.
• 4. Score Each document gets a relevance score, multiplied by the boost of the field where each word was found.
• 5. Sort and return Best first, cut to a page, with stored fields such as the title and link, so the list needs no second read.

Three tiny documents and the index built from them, with the stop word a dropped and every word stemmed:

The query sharing board becomes shar and board. With AND, only D1 holds both. With OR, D1 comes first because it holds two terms, then D2 and D3 with one each. No document was read to find this - only two posting lists.

How results are scored

BM25 is the scoring formula behind Lucene, and so behind Elasticsearch and OpenSearch; MiniSearch uses a close variant. For every query word it combines three facts: how often the word appears in the document (term frequency), how rare it is across all documents (inverse document frequency), and how long the document is.

A worked example. The index holds 1,000 documents. The word webhook appears in 5 of them, board in 300. IDF(webhook) = ln(1 + 995.5 / 5.5), about 5.2. IDF(board) = ln(1 + 700.5 / 300.5), about 1.2. In a document of average length with k1 = 1.2, one mention of webhook scores about 5.2, while five mentions of board score only about 2.1 (1.2 times a repeat factor of 1.8), because repeats help less and less. One rare word outweighs five common ones.

PostgreSQL ranks differently PostgreSQL's ts_rank and ts_rank_cd score each document on its own - how often and how close together the words appear, and in which weighted field - with no statistics about the other documents, so there is no IDF. That is fine for ordering a short list of candidates you already found, and one reason very large collections move to a search engine.

Three places a search can run

• Write The application saves a row in its database.
• Announce It publishes a change event, or a job polls for changed rows.
• Index A worker turns the row into a search document - including who may see it.
• Refresh The engine makes new documents searchable after a short refresh interval (one second by default in Elasticsearch).
• Search Until then, and whenever a step fails, search answers from an old copy: a deleted record or a removed permission can still appear.

The options in the market

The three green cards are three separate decisions, one per kind of content: records found by name, passages ranked for a question, and static help text searched in the browser.

• Substring match in the database {{brand}} uses it to find records by name. In SQL it is LIKE '%term%' or a substring function such as strpos.; Works on any database; Exact for names, keys and codes; Permissions in the same query; No stemming and no relevance score of its own; '%term%' cannot use an ordinary B-tree index
• PostgreSQL full-text search {{brand}} uses it to rank passages and similar cards.; Stemming, stop words and weighted fields; Runs where the data and permissions already are; Can rank text passed in at query time, with nothing stored; No typo tolerance; No corpus-wide statistics (no IDF); One language per query: english stems English only
• MiniSearch in the browser {{brand}} uses it for Help.; A few kilobytes, MIT licence; BM25 scoring, fuzzy, prefix and field boosts; Instant results without a server; The whole index sits in each browser; Only for content every reader of the app may see
• Elasticsearch / OpenSearch The heavy standard; OpenSearch is the Apache-2.0 fork.; Scales to billions of documents; Analyzers for many languages, synonyms, facets, aggregations; A cluster to run, tune and secure; The index lags the database; deletes and permission changes must be synced
• Meilisearch / Typesense Search-as-you-type engines.; Typo tolerance and good defaults out of the box; Simple to run; Another service and another copy of the data; Less control over ranking and aggregations

The second big decision is how reports are made. Every module can write its own export, or one engine can build them all.

• Each module writes its own export The simplest start.; Simple at first; No shared design to agree on; The screen, the file and the email drift apart; Every module repeats permissions, history and scheduling
• A BI tool such as Metabase or Power BI Dashboards over a copy of the data.; Fast dashboards with no code; Analysts can explore on their own; A second copy of the data with its own permissions; Reports cannot reuse the application's access rules
• A reporting server such as SSRS or JasperReports Paginated, printable layouts.; Rich paginated layouts; Mature scheduling and export; A heavy, separate runtime to run and secure; Another place to keep permissions in step
• One document shape, one provider per module, one engine Reports are built once and shown everywhere.; One place for permissions, history, schedules and files; A new report gets every output without new engine code; You build and maintain the engine yourself; The shared shape limits what a report can show

What {{brand}} chose, and why

The trade-off accepted: record search has no typo tolerance, synonyms or facets - a search for paymnt finds nothing. That is acceptable because people search for names they know, and results arrive as they type, so a typo is seen and fixed at once. A search engine would be the right call for millions of documents, long prose, typo-tolerant product catalogues or log search - and then every indexed document would have to carry its permissions.

Permissions decide the architecture The hard part of server-side search is not ranking - it is who may see what. A separate index must copy every permission and update it on every change. Searching live, as the person, inside each module gets permissions right by construction.

Client-side search with MiniSearch

MiniSearch is a full-text engine of a few kilobytes for JavaScript, MIT licensed. You give it documents with fields; it keeps an inverted index in memory - a radix tree, so prefix and fuzzy lookups are fast - and scores matches with BM25. Everything else - which words count, how typos are forgiven, what must match - is configuration.

• Debounce the input Wait for a short pause in typing, so a fast typist starts one search, not ten.
• Load the search code lazily The first search loads the library and builds the index; every later search reuses it, so the feature costs nothing at the app's start. {{brand}} does this for Help.
• Process terms exactly like the documents Each word is folded (case, accents, the dotted and dotless i), small words are dropped and endings are stemmed - the same rule as at indexing time.
• Match with AND, prefix and fuzzy Every word must match (AND); the last word matches as a prefix; longer words forgive a typo.
• Score with field boosts Title counts most, then keywords, headings, summary and body. Results are ranked best first and cut to a short list.
• Index bigger text in a second step Large text, such as the sections inside long guides, can be indexed in a second step once it has loaded, and listed below the first results.

• Prefix only on the last word: earlier words are finished, and a prefix on them floods the list.
• Fuzzy only on longer words: a typo allowed in a three-letter word matches half the dictionary.
• AND for short queries: share board should find sharing a board, not every page that mentions boards.
• Keep the last results on screen while the next ones load, so the list does not blink.

Fold text the same way everywhere Lower-casing alone is not enough across languages: toLocaleLowerCase in a Turkish locale turns I into ı, and İ lower-cases to i plus a combining dot, so results would depend on the reader's language. Plain lower-casing also leaves ə, ß and ø unchanged. Normalize to NFD, strip the combining marks, map the few special letters yourself - and use the folded form only for matching, never for showing.

Learn from searches that found nothing Log the searches that found nothing, so your content can grow where people looked in vain. Strip anything that looks like an email, a number or a token before storing, and list a term only when several different people searched for it.

Server-side search: ask every owner, as the person asking

Records live in the database, change every second, and each one has its own rules about who may open it. So {{brand}} does not copy them into an index. The command palette sends the typed term to the server, and the server asks every module to search its own records, as the person asking, in the workspace they stand in.

• Type The browser waits for a short pause and for a couple of characters, and keeps the last results visible while the next ones arrive.
• Check the term The server trims it and refuses one that is too short or too long - before any database work.
• Ask who is on One check finds which modules this person has switched on; the others are never asked.
• Search in parallel Every module searches at the same time, each with its own database session, only among records this person may open.
• Rank inside each module Most modules rank by name closeness: exact name first, then names that start with the term, then a later word that starts with it, then anywhere.
• Interleave fairly Interleave result kinds round by round - a board, a card, a board, a card - so one kind cannot fill the list. This is a standard fan-out and merge technique.
• Merge The module with the closest match comes first; a module that fails is left out while the rest still answer.

Escape LIKE wildcards In LIKE, % and _ are wildcards. A person who types 50%_off would otherwise match almost every row. Escape them - and the escape character itself - before building the pattern, and always pass the term as a parameter, never as SQL text.

Each module searches in parallel, each in its own scope: a database session such as EF Core's DbContext is not safe to share between threads. A module that throws is logged and left out, so one broken part never empties the whole list.

Never send what was pasted A pasted cURL command or web address may hold a token. Send only plain typed text as a search term, so a secret never lands in a server's access log.

PostgreSQL full-text search in practice

PostgreSQL turns text into a tsvector - a sorted list of stems with their positions and weights - and a question into a tsquery. The @@ operator says whether they match, and the ranking functions say how well. A text search configuration such as english decides the stemmer and the stop words.

Ranking text you did not store to_tsvector works on parameters as well as columns: send a few dozen candidate passages with the question and get one score per passage back. {{brand}} ranks the assistant's passages this way, at question time, so no index holds a copy of anyone's data. plainto_tsquery joins words with AND; to let any word count, replace its & with |.

The report provider pattern: build once, show everywhere

Without a pattern, every module writes its own Word export, its own spreadsheet and its own screen - and they drift until the file and the screen disagree. The report provider pattern splits the work in three: one document shape, one provider per report in the module that owns the data, and one engine that runs, keeps, compares, summarizes, exports and schedules any of them.

• Find and gate The engine finds the provider by its key and checks that its module is switched on for this person.
• Resolve the context Who asks and which workspace are decided on the server, never taken from the request; the client chooses only the target, the period and filters.
• Build The provider reads through its module's own access rules. Anything this person may not see returns not found - never an empty report.
• Compare The previous period's headline numbers are laid over the new ones, so each number shows its change.
• Summarize (optional) An AI model writes an executive summary; if it fails or the allowance is used up, the report still returns.
• Keep (optional) The whole document is stored as a snapshot with a fingerprint.
• Show or export The screen, Excel, PDF, Word, CSV or Markdown - all from the same document.

Snapshots, comparisons and schedules

A snapshot is the whole report document stored as JSON when it ran, with its headline numbers copied beside it so a list of past runs is cheap to show. A fingerprint - a hash of exactly what was stored - lets a copy someone was sent be matched to the kept one. Links to records are better worked out for each reader when the report is shown, so someone who lost access reads the words without a way in.

• This run It covers 14 days: 1 to 14 October.
• The period before The same length, ending where this one starts: 17 to 30 September.
• Find it The latest kept run of the same report, owner, workspace and target whose start and end fall within a small tolerance, in proportion to the period length, of that window.
• Lay it over Its headline values are matched to the new ones by stat key: each number gets its previous value, its change and its change in percent.
• Colour it The stat's tone decides good or bad: failures going up is red, tests passing going up is green.

• Store the wish Daily, weekly on Monday or monthly on the 1st, at a time on the owner's own clock with their time zone, covering the last N days.
• Compute the next run The next moment, strictly after now, that shows that wall-clock time in that zone - so 07:00 stays 07:00 when summer time starts.
• Wake up A background worker looks for due schedules on a regular tick. A lease makes sure only one worker runs each schedule, so a due schedule is never run twice.
• Act as the owner Each run uses a scope that acts as the schedule's owner, so every query sees exactly what they would see. A schedule whose owner can no longer act is paused, with the reason.
• Build and keep The same run as by hand, stored as a snapshot, in the owner's language.
• Mail Headline numbers, the summary and a link; a file only when asked for and small enough. A report holding restricted data goes only to recipients allowed to read it right now.
• Move on Compute the next run and record what happened - including who was passed over.

Keep the wall clock, not the offset Store 07:00 Europe/Berlin, not 05:00 UTC: a schedule kept in UTC moves by an hour twice a year. Decide on purpose what a time the clocks skip in spring does (run at the first minute that exists) and one they show twice in autumn does (run the first time).

Check again at every read A report is built as one person and then travels: it is kept, reopened, exported and mailed. A careful design stores the rules about who may read it with the copy and asks them again for each reader at the moment they read. A rule nobody can answer should mean nobody reads - fail closed. And a copy that lists people one by one should not be kept longer than it is needed.

Generating Excel and PDF files

An .xlsx file (and a .docx) is a ZIP of XML parts - the Office Open XML standard. You can write that XML through Microsoft's Open XML SDK, or use a library that gives you workbooks, sheets, cells and styles. A PDF is different: it places every glyph at an exact position and has no idea of a table or a page break. Something must measure the text and decide what fits - that is the layout engine.

• One sheet per table With a name Excel accepts: at most 31 characters, none of \ / ? * [ ] :, and unique in the workbook.
• Typed cells Numbers stay numbers, with a number format that shows the unit; moments are real dates in UTC, and the header says so.
• A real Excel table Banded rows, filter buttons and a totals row under the counts and hours.
• Readable layout Freeze the header row; fit the columns to their content, and wrap text past a readable width.
• Guard against formulas Text that starts with = + - or @ is stored as text, never run as a formula.
• Say what was left out A table cut at the row ceiling says how many rows it dropped.

Formula injection A cell that starts with =, +, - or @ can run as a formula when someone opens or edits the file - a crafted =HYPERLINK can send data to an outside address. Store such text as text: a quote prefix in Excel, a leading apostrophe in CSV.

• ClosedXML A friendly API over Open XML.; MIT licence; Tables, styles, number formats, formulas, pictures; Reads files too - good for imports; Builds the whole workbook in memory; Slower than a streaming writer for huge sheets
• EPPlus Feature-rich and well known.; Charts, pivot tables, conditional formats; A paid licence for commercial use since version 5
• Open XML SDK directly Microsoft's own low-level library.; MIT licence, full control; A streaming writer for very large files; Shared strings and styles by index; Easy to write a file Excel refuses to open
• CSV only Plain text, one table.; Trivial, streams any size; Every tool reads it; No types, formats or sheets; Encoding and formula-injection traps

QuestPDF describes a document in C#, much like a UI layout: a document holds pages; a page has a header, content and footer; content is built from containers - Column, Row, Table, Text, Image. Its layout engine then measures everything and paginates: a table's header row repeats on each new page, a heading is kept with its first lines, and the footer can say page X of Y.

• Describe Document.Create, then a Page with its size, margins and default text style.
• Fill the slots Header, Content and Footer; the header and footer repeat on every page.
• Compose Column stacks blocks, Row puts them side by side, Table has relative column widths and a header that repeats.
• Measure The engine asks each element how much space it needs and how much is left on the page.
• Paginate What does not fit moves to the next page. EnsureSpace keeps a heading with what follows; ShowEntire keeps a chart whole.
• Render Text with embedded fonts, pictures and vector SVG, plus metadata such as the title and language.

• QuestPDF Layout in C#, rendered in your process.; A real layout engine: pagination, repeating headers, page numbers; No browser to install next to the server; Fast and predictable output; Paid licence above a revenue threshold; Not HTML: a design is rebuilt in its API
• Headless Chromium HTML and CSS printed by Playwright or Puppeteer.; Reuse the HTML and CSS you already have; Looks like the web page; A whole browser to install next to your app - hundreds of megabytes; Slower and hungrier; page breaks harder to control
• iText The long-standing PDF toolkit.; Very complete: forms, signatures, PDF/A; AGPL, or a commercial licence
• PdfSharp / MigraDoc Older open-source pair.; MIT licence; MigraDoc adds paragraphs and tables; Less layout power; Fewer modern features

Fonts decide whether letters appear A PDF embeds the fonts it uses. If the machine making the file has no font with Cyrillic or Azerbaijani letters, those characters print as empty boxes. So the writer must have such a font, and a missing font or letter should fall back to the next font instead of failing the download.

Licences are part of the choice ClosedXML, the Open XML SDK and MiniSearch are MIT. QuestPDF is free under its Community licence for individuals, non-profits, open-source projects and companies under USD 1M gross yearly revenue, and paid above that; the licence is set in code before the first document. EPPlus and iText need a paid licence for most commercial use. Read a library's licence before it reaches production.

Word and PDF writers in {{brand}} take the same calls - title, heading, paragraph, table, chart - so a document is written once and the reader picks the file. Charts are drawn once as pictures by a chart library (ScottPlot) for Word, PDF and Excel, while the web draws the same data live. Very long tables are cut, with a line saying how many rows were left out.

Files also flow the other way: someone uploads a PDF and a program needs its text. Inside, a PDF is a set of numbered objects, and each page has a content stream of drawing commands, roughly "use font F1 at 11 points, move to x 72, y 700, show glyphs 72 101 108 108 111". There are no paragraphs, lines or even spaces. A library must map glyph codes back to Unicode through the font's encoding (its ToUnicode table), then guess word breaks from the gaps between letters and line breaks from changes in height. PdfPig, written in plain C#, does this: PdfDocument.Open, then page.Text for the page's text or page.GetWords() for words with their bounding boxes.

• PdfPig A managed library that reads PDFs.; Apache 2.0 licence and plain managed code - no native tool to ship; Gives letters and words with positions, so you can rebuild lines and columns; Still a pre-1.0 library; reading order for columns and tables is a heuristic; No OCR: a scanned page yields nothing
• iText The long-standing PDF toolkit.; Very complete: forms, signatures, text strategies; AGPL, or a commercial licence
• pdftotext (Poppler) A command-line tool run as a separate process.; Fast, with a good layout mode that keeps columns apart; A native program to install and start, and a licence (GPL) to respect; Process start-up and error handling to manage
• OCR (Tesseract or a cloud service) Read the page as an image.; The only way to read a scan; Slow, with recognition errors; A cloud service sends the document to a third party

{{brand}} reads uploaded PDFs with PdfPig because it adds nothing to install next to the server, its licence is permissive, and it also lets the tests read back the PDFs the product generates to prove they say what they should. The trade-off is the heuristics: text is returned in the order the file draws it, which is not always the order a person reads. Two rules matter for any reader. A PDF from a stranger is untrusted input, so bound its size, the pages you read, the text you keep and the time you spend, and treat a file past a bound as unreadable. And say plainly when a file has no text layer, instead of returning an empty answer as if all were well.

Privacy in totals: small groups and differencing

A total feels anonymous, but it is not automatically. In a team of two, each person can work out the other's hours from the team total. k-anonymity is the classic answer: every value you publish must stand for at least k people. A common choice is k = 3, 5 or 10; the right k depends on how sensitive the figure is. {{brand}} applies such a minimum and goes further against the subtler differencing attacks.

• Count the people Count the distinct people behind the value - on every day of the period.
• Too few Fewer than k: leave the value out and say why - for example, Not enough people in October.
• Side by side Place each person in one group only, starting from the smallest group; drop a group that falls under k and try again.
• Complements For each shown group, the people the total holds outside it must be none or k or more.
• Periods Publish values made of people only for closed periods, so two readings never differ by one person's day.
• Readers If the figure is kept or sent, carry the rule about who may read it with it, and ask again for each reader.

Absent, not zero A hidden value must be absent. A zero is a lie - and it can be subtracted like any other number. Say why it is missing, so the reader trusts the rest.

Try it yourself: a search box and a PDF in an hour

Free tools only: Node.js for the search, the .NET SDK for the PDF. Build both halves of this lesson and watch each rule change the result.

• Pick 20 documents For example the sections of a README: an id, a title and a body each.
• Index with MiniSearch Index title and body, store the title, boost the title three times.
• Search three ways An exact word, the same word with a typo (fuzzy), and half a word (prefix). Compare the results with the expected ones below.
• Add term processing A processTerm that folds accents, drops stop words and stems endings. Then try sharing board with AND and watch which results move.
• Print the results A console app with QuestPDF writing a table of 200 rows, so the header must repeat on page 2. Set the Community licence line first, or the app throws. Run it with dotnet run --project TryPdf.
• Break it on purpose Write a CSV with a cell =1+1 and open it in a spreadsheet: it shows 2, not the text. That is why the formula guard exists.

What you should see when you run try-search.ts:

Common mistakes

• Searching a copy that ignores permissions: an index built for everyone will one day return a record someone may not open. Search live, as the person - or copy permissions into every indexed document and keep them in sync.
• Processing documents and queries differently: a stemmer at index time but not at query time means timeouts never finds timeout. Use one term-processing function for both.
• Numbers written as text in spreadsheets: 12.5 h as a string cannot be summed or sorted. Store the number and show the unit with a number format.
• Trusting a total because it is a total: a group of two, or a total minus the shown groups, reveals one person. Apply a minimum group size and check the complements.
• Comparing periods that do not match: a 14-day run beside a 7-day one gives a fake trend. Compare only with a run of the same length that ended where this one starts.
• Keeping schedules in UTC: 07:00 becomes 08:00 after the clocks change. Keep the wall-clock time and the time zone, and work out each run from them.

Interview questions

• What is an inverted index? A map from each term to its posting list - the documents, and often positions, that contain it. A query looks up a few lists and intersects (AND) or unions (OR) them, so no document is scanned.
• How does BM25 rank documents, and how does PostgreSQL's ts_rank differ? BM25 adds, for each query word, a score from term frequency with diminishing returns (k1), inverse document frequency, and a length normalization (b): rare words, repeated in short fields, win. ts_rank scores each document on its own - frequency, proximity and field weights - with no statistics about the rest of the collection, so there is no IDF; it is good for ranking candidates, weaker as a global relevance model.
• When would you choose PostgreSQL full-text search over Elasticsearch? When the data and its permissions already live in PostgreSQL, the volume is moderate, and you want no second copy to sync. Elasticsearch wins for huge volumes, typo tolerance, facets, many languages and logs - at the cost of sync lag and copied permissions.
• How do you keep search results inside a user's permissions? Either search live inside the module that owns the records, as the signed-in user and within their workspace, or copy the access rules into every indexed document and update them on every change. Never trust an owner or workspace id sent by the client, because the client could ask for someone else's data.
• What is the report provider pattern, and why use it? One document shape, one provider per report in the module that owns the data, and one engine that runs, stores, compares, exports and schedules any of them. A new report gets a screen, files, history and schedules without new engine code, and a provider returns nothing for what the caller may not see.
• Why store a report snapshot instead of running the report again? History must read exactly as it was sent, even after the data changes, and the next run needs something to compare with. A fingerprint proves which copy is which, and a careful system re-checks who may read a copy at every read.
• How do you schedule every Monday at 07:00 correctly? Keep the wall-clock time and the time zone, compute the next occurrence in that zone, and decide what happens in DST gaps and overlaps. Run it under a lease so a schedule never fires twice, as the owner, then compute the next run.
• What is k-anonymity, and how can totals still leak? Each published value must stand for at least k people. Totals still leak by differencing: subtracting overlapping groups, comparing two to-date values, or a total minus the shown groups - so check complements, count each person in one group, and use closed periods.
• What is formula injection, and how do you prevent it? Text starting with =, +, - or @ in a CSV or Excel cell can run as a formula when the file is opened, for example to send data out with HYPERLINK. Store such text as text - a quote prefix or a leading apostrophe - and keep numbers as typed cells.
• Why generate PDFs from a layout library instead of printing HTML in a headless browser? A layout library has a real layout engine for pagination and repeating headers and needs no browser next to your app. Headless Chromium reuses your CSS but adds hundreds of megabytes and makes page breaks hard to control.

Related

• /help/technology/ai-chunking
• /help/technology/data
• /help/technology/secure-links-privacy