Quant Engineering: The Series Map

Part 1 of Quant Engineering — the series map.

Nothing in this series is trading advice. It is engineering: how to build and validate the machinery that measures trading strategies, and how that machinery fails. Whether any strategy is worth trading is a question this series deliberately never answers.

The gap this series fills

Search for content about NIFTY options backtesting and you will find three things: marketing pages for retail backtesting platforms, YouTube videos of equity curves with the settings conveniently cropped out, and academic papers that assume you have tick-level order book data from a vendor charging more per month than most people's trading capital.

What you will not find is the engineering. Nobody writes about what happens when your options dataset turns out to contain only the front week of each expiry. Nobody explains why a platform's "expiry index 1" and your engine's expiries[1] are different contracts, or how that one assumption silently deletes 96 percent of your trades while every run reports success. Nobody walks through what a stop-loss trigger actually means when all you have is a one-minute candle whose range covers both your stop and your target.

I know nobody writes about these things because I went looking for every one of them, found nothing, and had to work each one out the hard way. This series is the writeup I wish had existed.

The domain is deliberately narrow: NIFTY index options, minute-bar data, and the engines that backtest them. Narrowness is the point. Generic backtesting content — the kind that demonstrates a moving-average crossover on daily Apple bars — has been written a thousand times and transfers poorly to the Indian index options market, which has its own strike grids, its own expiry calendar politics, its own cost stack, and until recently the largest derivatives volume in the world by contract count. The engineering problems here are specific, and specific problems deserve specific writing.

Where this comes from

Everything in this series is sourced from two real projects, built back to back.

The first was a validation project: I had an archive of more than ten thousand strategy configurations exported from a retail options backtesting platform, each with its own recorded profit number, and I wanted to know whether an independent engine could reproduce those numbers. That project failed in the most instructive way possible. It failed because of the data, not the code — and by the time I understood why the recorded numbers could never validate a backtest (they turned out to be live-trading figures over each run's own private date window, not backtest outputs at all), I had learned more about options data reality than any success would have taught me.

The second was a from-scratch engine build: a self-hosted backtesting engine that interprets those same platform-style strategy configurations — multi-leg option baskets with entry clocks, strike-selection rules, per-leg stop-losses, trailing stops, re-entry counts, and forced square-off times — and runs them against a parquet store of NIFTY minute bars covering roughly four and a half years and over eleven hundred trading days. That engine works, runs a full trading day in about a fifth of a second, and every one of its design decisions had to be made explicitly: what does the engine clock tick on, when do indicators update, what fills at what price, what happens when a stop and a target land inside the same candle.

Between them, the two projects produced a long list of decisions, bugs, dead ends, and validation traps. That list is this series' backlog. When a post here says "this failure cost me a week," it is not a rhetorical device.

What "quant engineering" means here

The phrase gets used loosely, so let me pin down what this series does and does not cover.

In scope: everything between raw market data and a number you can defend. Data pipelines and storage layout. Expiry calendars and contract mechanics. The event-loop semantics of a backtesting engine. Options pricing mathematics used as instrumentation — implied volatility solvers, Greeks for strike selection — rather than as alpha. Cost modeling for the Indian market. P&L attribution. The statistics of not fooling yourself: walk-forward analysis, Monte Carlo resampling, multiple-testing corrections.

Out of scope: strategy recommendations, live-trading signals, broker comparisons, anything that could be mistaken for a claim that some configuration makes money. Posts will frequently use realistic strategies — the 9:20 short straddle is the fruit fly of Indian options research, and it appears here constantly — but always as a test subject for the engineering, never as advice. When a post shows a return number, it exists to demonstrate a measurement technique or a pitfall, and it will carry its caveats in the same breath.

This framing is not legal cover; it is the honest shape of the work. The uncomfortable truth I took away from both projects is that most retail backtest results are artifacts of engine semantics, not market edge. Two defensible engines given the same strategy description can disagree by thirty percent annually purely on interpretation: leg-level versus combined stop-loss, close fills versus next-open fills, spot versus futures reference for strike selection. Until you control the engineering, you are not measuring the market — you are measuring your simulator. That is the thesis of the entire library.

The nine tracks

The series is organized into nine tracks. Posts are numbered globally (this is part 1) but each also carries a track and a position within it, so you can read one track straight through.

Market Data Pipelines covers everything the engine reads: what a minute bar actually promises, why spot, futures, and options are three different datasets with three different coverage windows, how to derive an expiry calendar from the contracts that actually traded instead of trusting a weekday rule, and the audit that tells you which strategy questions your data is structurally unable to answer. This track exists because both projects were nearly sunk by data assumptions, not code bugs.

Options Mechanics is contract-level truth: expiry indexing conventions and the off-by-one that ruins everything, what "ATM" means precisely enough to implement, strike grids and their history, chain reconstruction from per-contract bars, settlement behavior on expiry day, and market-structure oddities like ban periods and shortened sessions. The NIFTY-specific details live here — including the fact that the weekly expiry day itself has moved over the years, which breaks any hardcoded calendar.

Backtest Engine Core is the simulator itself: the engine clock and minute-close evaluation semantics, order lifecycle and fill models, stop-loss and target triggers evaluated against candle highs and lows, trailing-stop and re-entry machinery, forced square-off, and the taxonomy of lookahead bugs — the ways tomorrow's information leaks into today's simulated decision without anything crashing.

Greeks and Volatility treats pricing math as tooling. The engine does not need Black–Scholes to price contracts the market already priced; it needs it to invert premiums into implied volatility, flag corrupt bars, and compute the deltas that strike-selection rules ask for. Expect a lot of material on making solvers converge on real, ugly minute data — sub-rupee premiums ten minutes before expiry do not care about your textbook Newton–Raphson.

Strategy Modeling makes strategies precise enough to test: the canonical intraday archetypes modeled honestly, every platform toggle (wait-and-trade, re-entry, trailing) given exact semantics, and the design of a config-driven strategy schema so thousands of strategies stay definable as data rather than code. This track leans directly on the archive of ten thousand real configurations and the dialect-translation work they forced.

P&L Attribution is where gross fantasy meets net reality: the full Indian cost stack from brokerage to STT to stamp duty modeled per fill, slippage models that are not fantasy, decomposing P&L into direction, theta, vega, and execution — and the flagship validation trap, the reason live P&L can never serve as backtest ground truth.

Risk and Margin covers what a P&L-only backtest ignores: SPAN and exposure margin approximation, sizing frameworks, drawdown mathematics on intraday equity curves, portfolio effects when many strategies crowd the same underlying, and stress replay against specific historical days.

Validation and Statistics is the conscience of the library: overfitting mechanics, walk-forward discipline, Monte Carlo resampling of trade sequences, and the multiple-testing corrections you owe whenever you searched across configurations. With ten thousand sibling configs, the best one is an outlier by construction; this track quantifies how much of any result is luck.

Engine Performance closes the loop: profiling where backtest time actually goes, parquet partition design driven by read patterns, caching derived artifacts like expiry calendars and chain snapshots, and the parallelism that turns a research question over ten thousand configs from a weekend into a lunch break.

How the library is built

A few structural promises, so you know what you are subscribing to.

Every post is a minimum ten-minute read. No listicles, no three-paragraph "tips." Each post takes one engineering decision and walks it end to end: the naive version, the failure story, the corrected version with worked NIFTY numbers, the edge cases the fix must survive, and a checklist you can run against your own engine today. If a topic cannot honestly fill that shape, it gets merged into a neighbor rather than published thin. A verification script enforces the word floor mechanically, because a promise without a gate is a hope.

The plan is 1,100+ posts. That is not a typo, and it is not a weekend's ambition — it is a multi-year library built in small batches, the same way this site's AWS series grew past a thousand parts. The series page shows the full track map with planned volumes, and a curriculum of twenty-four foundation modules sketches the spine. Posts publish on a drip, typically one per day while a batch lasts.

Numbers are real. Strike grids at 50-point steps near the money, realistic premiums, actual coverage windows, lot sizes correct for the period under discussion (and the lot size has changed repeatedly, which is itself a backtesting trap that gets its own post). Where a story involves a commercial platform's behavior, the platform stays unnamed and the lesson stays general.

Failures are included. The most valuable material from both projects came from things that went wrong. The series would be dishonest — and much shorter — without them.

Code is adaptable, not copy-paste. Snippets appear in nearly every post, in Python with pandas and parquet idioms, trimmed to the decision being discussed rather than dumped wholesale. The goal is that you can rebuild any mechanism in your own engine's style, not that you can clone mine — an engine you pasted together is an engine whose semantics you cannot defend, which defeats the entire point of the library.

Two reading orders work. The global part numbers interleave tracks deliberately, the way the problems actually arrive when you build — a data problem, then a mechanics problem, then an engine problem. If you prefer depth-first, every post's header names its track and position within it, and the series page groups the library by track so you can read, say, all of Validation and Statistics straight through. Neither order is wrong; the interleaved one reflects how the scars were earned.

Who this is for

You will get the most from this series if you are an engineer who trades or a trader who codes: comfortable with Python and pandas, curious about options, and suspicious of every backtest you have ever seen, including your own. You do not need a mathematics degree; the pricing math appears when needed and is always in service of an engineering decision. You do not need expensive data; the whole series is built on the kind of minute-bar dataset a determined retail researcher can realistically obtain, and a recurring theme is exactly what such datasets can and cannot support.

If you are here hoping to find a profitable strategy, this is the wrong library, and the Validation track will spend considerable energy explaining why anyone offering you a backtested strategy deserves your sharpest questions.

Where to start

The next post begins the Market Data Pipelines track with the least glamorous, most consequential topic in the whole field: what your backtest actually reads, row by row — three datasets, their exact coverage windows, and the audit that tells you what your data cannot answer. Everything else in the library stands on top of it.

After that, the early posts cover the expiry-indexing off-by-one that quietly destroyed my first validation attempt, the front-week-only data trap, minute-close evaluation semantics, and why live P&L can never validate an engine. Those five posts are the distilled tuition of both projects; if you read nothing else, read those.

The full track map, the 24-module curriculum, and the running list of published parts live on the series page. See you in part 2.